Commit Graph

2608 Commits

Author SHA1 Message Date
HaoranYi e14933c54d
move bank test fn to its test_utils mod (#24171) 2022-04-11 10:42:24 -05:00
Justin Starry 8eef3d9713
Add tests to the blockhash queue (#24238) 2022-04-11 19:36:24 +08:00
Christian Kamm a058f348a2 Address review comments 2022-04-08 14:37:55 -05:00
Christian Kamm 2ed29771f2 Unittest for cost tracker after process_and_record_transactions 2022-04-08 14:37:55 -05:00
Christian Kamm 924b8ea1eb Adjustments to cost_tracker updates
- don't store pending tx signatures and costs in CostTracker
- apply tx costs to global state immediately again
- go from commit_or_cancel to update_or_remove, where the cost tracker
  is either updated with the true costs for successful tx, or the costs
  of a retryable tx is removed
- move the function into qos_service and hold the cost tracker lock for
  the whole loop
2022-04-08 14:37:55 -05:00
Tao Zhu 9e07272af8 - Only commit successfully executed transactions' cost to cost_tracker;
- In-fly transactions are pended in cost_tracker until being committed
  or cancelled;
2022-04-08 14:37:55 -05:00
Dmitri Makarov 03ed334ebb Double the chunk size for sending the program binary data in tx 2022-04-08 09:06:40 -07:00
Jeff Washington (jwash) 210f6a6fab
move hash calculation out of acct bg svc (#23689)
* move hash calculation out of acct bg svc

* pr feedback
2022-04-08 10:42:03 -05:00
Jeff Washington (jwash) c27150b1a3
reserialize_bank_fields_with_hash (#23916)
* reserialize_bank_with_new_accounts_hash

* Update runtime/src/serde_snapshot.rs

Co-authored-by: Brooks Prumo <brooks@prumo.org>

* Update runtime/src/serde_snapshot/tests.rs

Co-authored-by: Brooks Prumo <brooks@prumo.org>

* Update runtime/src/serde_snapshot/tests.rs

Co-authored-by: Brooks Prumo <brooks@prumo.org>

* pr feedback

Co-authored-by: Brooks Prumo <brooks@prumo.org>
2022-04-07 14:05:57 -05:00
Jeff Washington (jwash) 48d1af01c8
add metrics around rewards (#24160) 2022-04-07 11:44:26 -05:00
Jeff Washington (jwash) f7b2951c79
move around some index metrics to reduce locks (#24161) 2022-04-07 09:43:19 -05:00
HaoranYi 42c094739d
add test cfg attribute (#24154) 2022-04-07 09:05:20 -05:00
Jeff Washington (jwash) 550ca7bf92
compare contents of serialized banks instead of exact file format (#24141)
* compare contents of serialized banks instead of exact file format

* Update runtime/src/snapshot_utils.rs

Co-authored-by: Brooks Prumo <brooks@prumo.org>

* Update runtime/src/snapshot_utils.rs

Co-authored-by: Brooks Prumo <brooks@prumo.org>

* pr feedback

* get rid of clone

* pr feedback

Co-authored-by: Brooks Prumo <brooks@prumo.org>
2022-04-06 21:55:44 -05:00
Jeff Washington (jwash) fddd162645
reserialize bank in ahv by first writing to temp file in abs (#23947) 2022-04-06 21:39:26 -05:00
Brooks Prumo c322842257
Replace channel with Mutex<Option> for AccountsPackage (#24013) 2022-04-06 05:47:19 -05:00
carllin 4ea59d8cb4
Set drop callback on first root bank (#23999) 2022-04-05 13:02:33 -05:00
Jeff Washington (jwash) 4a11fa072f
hash_account_with_rent_epoch (#24104) 2022-04-05 08:10:31 -05:00
Jeff Washington (jwash) 6a7f6585ce
persist historical_roots (#24029) 2022-04-04 13:13:11 -05:00
Jeff Washington (jwash) 132f08486a
remove basically duplicate function (#24107) 2022-04-04 12:55:05 -05:00
Tao Zhu 997db7637c do simple math without floats 2022-04-04 12:32:00 -05:00
Jeff Washington (jwash) f8f3edac3c
update comment (#24108) 2022-04-04 11:06:01 -05:00
Jeff Washington (jwash) 2820b64eb3
roots_original -> historical_roots (#24063) 2022-04-04 09:12:12 -05:00
behzad nouri ef3e3dce7a
hides implementation details of vote-accounts from public interface (#24087) 2022-04-04 13:20:26 +00:00
behzad nouri fa7eb7f30c
improves Stakes::activate_epoch performance (#24068)
Tested with mainnet stakes obtained from the ledger at 5 recent epoch
boundaries, this code is ~30% faster than current master.

Current code:
  epoch: 289, elapsed: 82901us
  epoch: 290, elapsed: 80525us
  epoch: 291, elapsed: 79122us
  epoch: 292, elapsed: 79961us
  epoch: 293, elapsed: 78965us

This commit:
  epoch: 289, elapsed: 61710us
  epoch: 290, elapsed: 55721us
  epoch: 291, elapsed: 55886us
  epoch: 292, elapsed: 55399us
  epoch: 293, elapsed: 56803us
2022-04-02 22:48:51 +00:00
Jeff Washington (jwash) 0ca5a0ec68
prior_roots -> historical_roots (#24064) 2022-04-02 12:01:43 -05:00
Jeff Washington (jwash) ec97d6d078
rename remove_old_roots (#24059) 2022-04-02 12:01:13 -05:00
Jeff Washington (jwash) 3ca4fffa78
root -> alive_root (#24062) 2022-04-02 12:00:52 -05:00
Justin Starry 792bbf75ab
Support sending versioned txs in AsyncClient (#23982) 2022-04-02 11:12:02 +08:00
Alexander Meißner 1b45c509c3
Refactor: Use `InstructionContext::get_instruction_data()` (#24014)
* Adds transaction_context and instruction_context where invoke_context.get_keyed_accounts() is used.

* Use instruction_context.get_instruction_data() instead of an explicit parameter.

* Removes instruction_data parameter from Executor::execute().

* Removes instruction_data parameter from ProcessInstructionWithContext.
2022-04-01 15:48:05 +02:00
Jeff Washington (jwash) 31997f8251
hash calc scanning takes config (#24016) 2022-03-31 14:26:37 -05:00
Jeff Washington (jwash) 9c8dad33c7
add epoch_schedule and rent_collector to hash calc (#24012) 2022-03-31 10:51:18 -05:00
Jeff Washington (jwash) da001d54e5
calculate_accounts_hash_helper uses config (#24003) 2022-03-31 09:29:45 -05:00
Justin Starry cb5e67d327
Use Rent sysvar directly for stake split instruction (#24008)
* Use Rent sysvar directly for stake split ix

* Add feature to gate rent sysvar change

* fix tests

* cargo clippy
2022-03-31 16:46:35 +08:00
Jeff Washington (jwash) 125f9634fd
add hash calc config.use_write_cache (#24005) 2022-03-30 17:19:34 -05:00
Jeff Washington (jwash) 82c5230bc2
AccountsPackage::new less brittle (#23968) 2022-03-30 14:06:15 -05:00
HaoranYi 1fb82d7924
fix typo in comments (#24004) 2022-03-30 09:47:51 -05:00
Jeff Washington (jwash) af9344fe22
SortedStorages refactoring (#23998) 2022-03-30 09:19:03 -05:00
Jeff Washington (jwash) 5636570d6d
add roots_original to roots tracker (#23849) 2022-03-30 08:52:45 -05:00
Jeff Washington (jwash) da844d7be5
refactoring of SortedStorages tests to make other changes easier (#23990) 2022-03-29 22:06:48 -05:00
Jeff Washington (jwash) 5a613e9b6e
use CalcAccountsHashConfig in calculate_accounts_hash (#23987) 2022-03-29 22:05:47 -05:00
Alexander Meißner 794645d092
Adds check_number_of_instruction_accounts() to all builtin programs except for the address-lookup-table. (#23984) 2022-03-29 19:06:50 +02:00
Brooks Prumo 31b707b625
Specify if archive size datapoint is for full or incremental snapshots (#23941) 2022-03-26 12:29:13 -05:00
Jeff Washington (jwash) c24de17278
remove index hash calculation as an option (#23928) 2022-03-25 15:32:53 -05:00
Jeff Washington (jwash) ec78702bc8
RollingBitField::get_all_less_than (#23919) 2022-03-25 15:20:22 -05:00
Jeff Washington (jwash) acfd22712b
RollingBitFIeld to its own file (#23917) 2022-03-25 10:37:00 -05:00
Jeff Washington (jwash) 55d61023f7
document 'accounts' hash (#23907) 2022-03-24 15:58:52 -05:00
Alexander Meißner 140c8dd01f
Refactor: Replaces KeyedAccount in_get_sysvar_with_account_check (#23905)
* Replaces all use sites of get_sysvar_with_account_check by get_sysvar_with_account_check2.

* Removes get_sysvar_with_account_check.

* Renames get_sysvar_with_account_check2 to get_sysvar_with_account_check.
2022-03-24 19:30:42 +01:00
Jeff Washington (jwash) 37c36ce3fa
pass stats separately from CalcAccountsHashConfig (#23892) 2022-03-24 12:48:47 -05:00
Jeff Washington (jwash) 82328fd9d8
move max_clean_root deeper in flush cache (#23869) 2022-03-24 12:45:49 -05:00
steviez c31db81ac4
Use VoteAccountsHashMap type alias in all applicable spots (#23904) 2022-03-24 12:09:48 -05:00
Jeff Washington (jwash) a22a2384bf
fix ci test error (#23908) 2022-03-24 11:30:20 -05:00
Jeff Washington (jwash) 5b916961b5
HashCalc uses self.accounts_cache (#23890) 2022-03-24 10:34:28 -05:00
Jeff Washington (jwash) f2aea3b7c7
flush_slot_cache takes [Slot] (#23865) 2022-03-24 10:24:36 -05:00
Jeff Washington (jwash) 9d3b17c635
HashCalc uses self.accounts_index (#23888) 2022-03-24 10:06:32 -05:00
Jeff Washington (jwash) 396b49a7c1
Start saving/loading prior_roots(_with_hash) to snapshot (#23844)
* Start saving/loading prior_roots(_with_hash) to snapshot

* Update runtime/src/accounts_index.rs

Co-authored-by: Michael Vines <mvines@gmail.com>

* Update runtime/src/accounts_index.rs

Co-authored-by: Michael Vines <mvines@gmail.com>

* update comment

Co-authored-by: Michael Vines <mvines@gmail.com>
2022-03-24 10:06:24 -05:00
Jeff Washington (jwash) b22165ad69
hash calc uses self.filler_account_suffix (#23887) 2022-03-24 09:58:06 -05:00
Jeff Washington (jwash) 9022931689
calc hash uses self.num_hash_scan_passes (#23883) 2022-03-24 09:44:42 -05:00
Jeff Washington (jwash) e3eb002f66
Log storage size stats at hash calc (#23843) 2022-03-24 09:40:35 -05:00
Jeff Washington (jwash) f1a411c897
add epoch_schedule and rent_collector to hash calc (#23857) 2022-03-24 09:39:22 -05:00
Jeff Washington (jwash) db5d68f01f
HashCalc uses self.accounts_hash_cache_path (#23882) 2022-03-24 09:31:55 -05:00
HaoranYi 90009f330b
small refactor to shorten the lock on slot_under_contention hashset (#23891)
* small refactor to shorten the lock on slot_under_contention hashset

* adding comments

* comments
2022-03-24 08:20:56 -05:00
Alexander Meißner 91c2729856
Replaces keyed_account get_signers() by InstructionContext::get_signers(). (#23863) 2022-03-24 12:57:51 +01:00
Jeff Washington (jwash) 5a892af2fe
disable 'check_hash' on accounts hash calc (#23873) 2022-03-23 21:03:31 -05:00
Jeff Washington (jwash) 3e22d4b286
calc hash uses self.thread_pool_clean (#23881) 2022-03-23 20:52:38 -05:00
Jack May 486f7b7673
use array access function (#23895) 2022-03-23 17:03:01 -07:00
Jeff Washington (jwash) dc3863ef14
flush_slot_cache_with_clean (#23868) 2022-03-23 14:09:56 -05:00
Jeff Washington (jwash) 260f899eda
write cache: hashmap to set (#23866) 2022-03-23 14:05:45 -05:00
Jeff Washington (jwash) 9e61fe7583
add AccountsHashConfig to manage parameters (#23850) 2022-03-23 13:44:23 -05:00
Jeff Washington (jwash) b1280b670a
calculate_accounts_hash_without_index takes &self (#23846)
* calculate_accounts_hash_without_index takes &self

* Update runtime/src/snapshot_package.rs

Co-authored-by: Brooks Prumo <brooks@prumo.org>

Co-authored-by: Brooks Prumo <brooks@prumo.org>
2022-03-23 11:57:32 -05:00
Jeff Washington (jwash) 7b89222fde
don't start extra threads for shrink/clean/hash (#23858) 2022-03-23 11:53:37 -05:00
Jeff Washington (jwash) 493a8e2348
remove random flushing of write cache (#23845) 2022-03-23 08:45:44 -05:00
Jeff Washington (jwash) bc35e1c5f5
snapshot code needs all storages for hash calc (#23840) 2022-03-22 21:27:54 -05:00
Alexander Meißner 9f0ca6d88a
Refactor: Remove `trait` from nonce keyed account (#23811)
* Removes the trait `NonceKeyedAccount`.
2022-03-23 02:09:30 +01:00
Jon Cinque 7af48465fa
transaction-status: Add return data to meta (#23688)
* transaction-status: Add return data to meta

* Add return data to simulation results

* Use pretty-hex for printing return data

* Update arg name, make TransactionRecord struct

* Rename TransactionRecord -> ExecutionRecord
2022-03-22 23:17:05 +01:00
Jeff Washington (jwash) 1089a38aaf
AcctIdx: rework scan and write to disk (#23794) 2022-03-22 11:54:12 -05:00
Jeff Washington (jwash) 89ba3ff139
log fail to evict (#23815) 2022-03-22 09:19:38 -05:00
Will Hickey c4ecfa5716
Bump version to v1.11 (#23807)
* Revert crossbeam_epoch to stable. 0.9.8 only works with nightly
* Remove unneeded unit expression
2022-03-21 17:40:50 -05:00
Jeff Washington (jwash) 24f6855f86
AcctIdx: only remove a fixed number of items per write lock (#23795) 2022-03-21 16:55:04 -05:00
Brooks Prumo cb06126388
Set accounts_data_len on feature activation (#23730) 2022-03-21 12:28:26 -05:00
Jeff Washington (jwash) 965ab9186d
AcctIdx: fix infinite loop (#23806) 2022-03-21 10:58:36 -05:00
Justin Starry 15357480ec
Refactor instruction compilation and update message account key ordering (#23729)
* Refactor: Make instruction compilation usable for other message versions

* apply trents feedback

* Fix tests

* Fix bpf compatiblity
2022-03-21 20:53:32 +08:00
Jeff Washington (jwash) 258db77100
AcctIdx: factor 'scan' out of flush_internal (#23777) 2022-03-20 22:00:38 -05:00
carllin f34434f96b
Drop lock (#23765) 2022-03-20 21:27:24 -04:00
Jeff Washington (jwash) dd69f3baf5
throttle index adding to allow disk flushing to keep up and reduce startup ram usage (#23773) 2022-03-20 19:56:20 -05:00
Brooks Prumo 335c4b668b
Fix bug in bank/sysvar_cache tests (#23780) 2022-03-19 21:38:18 -05:00
Jeff Washington (jwash) df29276eb0
AcctIdx: remove -> evict (#23775) 2022-03-18 17:13:21 -05:00
Jeff Washington (jwash) a419374fa4
factor out function (#23742) 2022-03-18 14:10:52 -05:00
Tao Zhu 56428be629 Not exposing inner cost_table to encapsulating implementation details,
making future change easier.
2022-03-18 12:58:43 -05:00
Jeff Washington (jwash) 998e7d18f9
AcctIdx: never retry a bucket flush (#23732) 2022-03-18 12:20:42 -05:00
Tao Zhu 0ed23899e7 directly use compute_budget MAX_UNITS and DEFAULT_UNITS 2022-03-18 08:53:11 -05:00
Tao Zhu a4cacf3389 add deterministic default cost 2022-03-18 08:53:11 -05:00
Jeff Washington (jwash) 857576d76f
AcctIdx: move write to disk outside in mem write lock (#23731) 2022-03-17 23:09:41 -05:00
Brooks Prumo 7ff8c80e25
Add accounts_data_len to bank snapshot (#23714) 2022-03-17 20:14:54 -05:00
Jeff Washington (jwash) 664deb2157
AcctIdx: get rid of unused is_dirty (#23733) 2022-03-17 16:29:36 -05:00
Will Hickey 2f58c9e501
Bump version to 1.10.4 (#23743) 2022-03-17 14:02:13 -05:00
Jeff Washington (jwash) 66b1f55351
AcctIdx: cheaper check for should evict while flushing (#23705) 2022-03-17 08:46:32 -05:00
Jeff Washington (jwash) 3a46f45650
AcctIdx: add stats for flushing and estimated memory (#23709) 2022-03-17 08:46:00 -05:00
Justin Starry b4350a2522
Make solana-address-lookup-table-program crate bpf compatible (#23700) 2022-03-17 08:21:07 +08:00
Jeff Washington (jwash) be0aeea01a
AcctIdx: check for range holds outside lock (#23706) 2022-03-16 17:44:59 -05:00
Jeff Washington (jwash) caddb851be
acct idx flush keeps slot_list read lock open (#23704) 2022-03-16 17:29:04 -05:00