c6a751d658
Detect and report bank drop signal queue full and disconnect events ( #24112 )
...
* nonblocking send when when droping banks
* detect and report drop signal queue full/disconnect events
* comments
* use counter for reporting bank_drop_queue events
* reduce log
* use datapoint to report stats
* logging instead of reporting bank drop signal full
* fix a corner case for reporting
* fix build
2022-04-18 11:19:11 -05:00
578d59c802
Remove the code that handles cost update for separate pr
2022-04-17 19:26:24 -05:00
85281de981
move number utility functions to sdk; log error if cost adjust overflows i64
2022-04-17 19:26:24 -05:00
9dadfb2e2c
Add checked_add_signed() to apply cost adjustment to cost_tracker
2022-04-17 19:26:24 -05:00
094da35b91
Address review comments:
...
1. use was_executed to correctly identify transactions requires cost adjustment;
2. add function to specifically handle executino cost adjustment without have to copy accounts
2022-04-17 19:26:24 -05:00
29ca21ed78
undo transaction cost from cost_tracker if it was not executed successfully
2022-04-17 19:26:24 -05:00
f33ad34531
Add feature_set_override parameter to mock_process_instruction() ( #24386 )
2022-04-15 13:43:04 -05:00
b4fd9124bf
log secondary index contents on startup ( #24348 )
2022-04-15 13:30:03 -05:00
8c9430359e
add early exit in get_corrected_rent_epoch_on_load ( #24331 )
2022-04-15 13:28:16 -05:00
ba7a2efa66
SlotInfoInEpoch ( #24332 )
2022-04-15 13:27:41 -05:00
4ed647d8ec
Test that tick slot hashes update the recent blockhash queue ( #24242 )
2022-04-16 00:30:20 +08:00
0e7b0597db
check for rewrites skipped in closure ( #24330 )
2022-04-14 13:46:18 -05:00
d20b4c9958
rename function param to avoid conflict ( #24342 )
2022-04-14 11:56:32 -05:00
a91b0c8ea3
dashmap -> rwlock<hashmap> for rewrites ( #24327 )
2022-04-14 11:55:58 -05:00
e3ef0741be
simplify bank drop calls ( #24142 )
...
* simplify bank drop calls
* clippy: import
* Update ledger/src/blockstore_processor.rs
Co-authored-by: Brooks Prumo <brooks@prumo.org>
* Update runtime/src/accounts_background_service.rs
Co-authored-by: Brooks Prumo <brooks@prumo.org>
* Update runtime/src/bank.rs
Co-authored-by: Brooks Prumo <brooks@prumo.org>
* cleanup
* format
* use msb of bank_id to indicates that we are dropping
* clippy
* restore bank id
* clippy
* revert is-serialized_with_abs flag
* assert
* clippy
* whitespace
* fix bank drop callback check
* more fix
* remove msb dropping implementation
* fix
Co-authored-by: Brooks Prumo <brooks@prumo.org>
2022-04-14 08:43:54 -05:00
255ef66a27
move feature activation log to correct fn for hash ( #24326 )
2022-04-13 17:43:33 -05:00
6a474f29cd
default to disk index ( #24251 )
2022-04-13 09:24:50 -05:00
b6b8783323
add maybe_update_rent_epoch_on_load ( #24294 )
2022-04-13 08:55:24 -05:00
929753a11f
typo ( #24291 )
2022-04-12 16:46:59 -05:00
69e9ad5571
update comment ( #24288 )
2022-04-12 12:37:46 -05:00
2d4d639635
add expected_rent_collection ( #24028 )
...
* add expected_rent_collection
* update some comments for clarity and resolve a todo
* add test for 'LeaveAloneNoRent'
2022-04-12 11:32:23 -05:00
bdbca3362e
increase test timeout ( #24277 )
2022-04-12 09:54:57 -05:00
1bc49d219d
IndexLimitMb option adds 'Unspecified' state ( #24249 )
2022-04-12 09:38:09 -05:00
605036c117
move test fn into its own mod ( #24212 )
...
* move test fn into its own mod
* pub
2022-04-12 09:36:05 -05:00
f3aa80d3f8
typo ( #24257 )
2022-04-12 09:08:35 -05:00
9b8850f99e
test-validator: Add `--max-compute-units` flag ( #24130 )
...
* test-validator: Add `--max-compute-units` flag
* Add `RuntimeConfig` for tweaking runtime behavior
* Actually add the file
* Move RuntimeConfig to runtime
2022-04-12 02:28:10 +02:00
4fd184c131
Use Release/Acquire instead of SeqCst for is_bank_drop_callback_enabled ( #24134 )
2022-04-11 19:19:17 -05:00
6fbe2b936c
fix comment ( #24254 )
2022-04-11 18:53:45 -05:00
c0019edf00
document WaitableCondvar ( #24252 )
2022-04-11 14:45:23 -05:00
9ac2245970
remove clone ( #24244 )
2022-04-11 13:15:00 -05:00
60b2155bd3
Add accounts-filler-size command line option ( #23896 )
2022-04-11 13:10:09 -05:00
e14933c54d
move bank test fn to its test_utils mod ( #24171 )
2022-04-11 10:42:24 -05:00
8eef3d9713
Add tests to the blockhash queue ( #24238 )
2022-04-11 19:36:24 +08:00
a058f348a2
Address review comments
2022-04-08 14:37:55 -05:00
2ed29771f2
Unittest for cost tracker after process_and_record_transactions
2022-04-08 14:37:55 -05:00
924b8ea1eb
Adjustments to cost_tracker updates
...
- don't store pending tx signatures and costs in CostTracker
- apply tx costs to global state immediately again
- go from commit_or_cancel to update_or_remove, where the cost tracker
is either updated with the true costs for successful tx, or the costs
of a retryable tx is removed
- move the function into qos_service and hold the cost tracker lock for
the whole loop
2022-04-08 14:37:55 -05:00
9e07272af8
- Only commit successfully executed transactions' cost to cost_tracker;
...
- In-fly transactions are pended in cost_tracker until being committed
or cancelled;
2022-04-08 14:37:55 -05:00
03ed334ebb
Double the chunk size for sending the program binary data in tx
2022-04-08 09:06:40 -07:00
210f6a6fab
move hash calculation out of acct bg svc ( #23689 )
...
* move hash calculation out of acct bg svc
* pr feedback
2022-04-08 10:42:03 -05:00
c27150b1a3
reserialize_bank_fields_with_hash ( #23916 )
...
* reserialize_bank_with_new_accounts_hash
* Update runtime/src/serde_snapshot.rs
Co-authored-by: Brooks Prumo <brooks@prumo.org>
* Update runtime/src/serde_snapshot/tests.rs
Co-authored-by: Brooks Prumo <brooks@prumo.org>
* Update runtime/src/serde_snapshot/tests.rs
Co-authored-by: Brooks Prumo <brooks@prumo.org>
* pr feedback
Co-authored-by: Brooks Prumo <brooks@prumo.org>
2022-04-07 14:05:57 -05:00
48d1af01c8
add metrics around rewards ( #24160 )
2022-04-07 11:44:26 -05:00
f7b2951c79
move around some index metrics to reduce locks ( #24161 )
2022-04-07 09:43:19 -05:00
42c094739d
add test cfg attribute ( #24154 )
2022-04-07 09:05:20 -05:00
550ca7bf92
compare contents of serialized banks instead of exact file format ( #24141 )
...
* compare contents of serialized banks instead of exact file format
* Update runtime/src/snapshot_utils.rs
Co-authored-by: Brooks Prumo <brooks@prumo.org>
* Update runtime/src/snapshot_utils.rs
Co-authored-by: Brooks Prumo <brooks@prumo.org>
* pr feedback
* get rid of clone
* pr feedback
Co-authored-by: Brooks Prumo <brooks@prumo.org>
2022-04-06 21:55:44 -05:00
fddd162645
reserialize bank in ahv by first writing to temp file in abs ( #23947 )
2022-04-06 21:39:26 -05:00
c322842257
Replace channel with Mutex<Option> for AccountsPackage ( #24013 )
2022-04-06 05:47:19 -05:00
4ea59d8cb4
Set drop callback on first root bank ( #23999 )
2022-04-05 13:02:33 -05:00
4a11fa072f
hash_account_with_rent_epoch ( #24104 )
2022-04-05 08:10:31 -05:00
6a7f6585ce
persist historical_roots ( #24029 )
2022-04-04 13:13:11 -05:00
132f08486a
remove basically duplicate function ( #24107 )
2022-04-04 12:55:05 -05:00
997db7637c
do simple math without floats
2022-04-04 12:32:00 -05:00
f8f3edac3c
update comment ( #24108 )
2022-04-04 11:06:01 -05:00
2820b64eb3
roots_original -> historical_roots ( #24063 )
2022-04-04 09:12:12 -05:00
ef3e3dce7a
hides implementation details of vote-accounts from public interface ( #24087 )
2022-04-04 13:20:26 +00:00
fa7eb7f30c
improves Stakes::activate_epoch performance ( #24068 )
...
Tested with mainnet stakes obtained from the ledger at 5 recent epoch
boundaries, this code is ~30% faster than current master.
Current code:
epoch: 289, elapsed: 82901us
epoch: 290, elapsed: 80525us
epoch: 291, elapsed: 79122us
epoch: 292, elapsed: 79961us
epoch: 293, elapsed: 78965us
This commit:
epoch: 289, elapsed: 61710us
epoch: 290, elapsed: 55721us
epoch: 291, elapsed: 55886us
epoch: 292, elapsed: 55399us
epoch: 293, elapsed: 56803us
2022-04-02 22:48:51 +00:00
0ca5a0ec68
prior_roots -> historical_roots ( #24064 )
2022-04-02 12:01:43 -05:00
ec97d6d078
rename remove_old_roots ( #24059 )
2022-04-02 12:01:13 -05:00
3ca4fffa78
root -> alive_root ( #24062 )
2022-04-02 12:00:52 -05:00
792bbf75ab
Support sending versioned txs in AsyncClient ( #23982 )
2022-04-02 11:12:02 +08:00
1b45c509c3
Refactor: Use `InstructionContext::get_instruction_data()` ( #24014 )
...
* Adds transaction_context and instruction_context where invoke_context.get_keyed_accounts() is used.
* Use instruction_context.get_instruction_data() instead of an explicit parameter.
* Removes instruction_data parameter from Executor::execute().
* Removes instruction_data parameter from ProcessInstructionWithContext.
2022-04-01 15:48:05 +02:00
31997f8251
hash calc scanning takes config ( #24016 )
2022-03-31 14:26:37 -05:00
9c8dad33c7
add epoch_schedule and rent_collector to hash calc ( #24012 )
2022-03-31 10:51:18 -05:00
da001d54e5
calculate_accounts_hash_helper uses config ( #24003 )
2022-03-31 09:29:45 -05:00
cb5e67d327
Use Rent sysvar directly for stake split instruction ( #24008 )
...
* Use Rent sysvar directly for stake split ix
* Add feature to gate rent sysvar change
* fix tests
* cargo clippy
2022-03-31 16:46:35 +08:00
125f9634fd
add hash calc config.use_write_cache ( #24005 )
2022-03-30 17:19:34 -05:00
82c5230bc2
AccountsPackage::new less brittle ( #23968 )
2022-03-30 14:06:15 -05:00
1fb82d7924
fix typo in comments ( #24004 )
2022-03-30 09:47:51 -05:00
af9344fe22
SortedStorages refactoring ( #23998 )
2022-03-30 09:19:03 -05:00
5636570d6d
add roots_original to roots tracker ( #23849 )
2022-03-30 08:52:45 -05:00
da844d7be5
refactoring of SortedStorages tests to make other changes easier ( #23990 )
2022-03-29 22:06:48 -05:00
5a613e9b6e
use CalcAccountsHashConfig in calculate_accounts_hash ( #23987 )
2022-03-29 22:05:47 -05:00
794645d092
Adds check_number_of_instruction_accounts() to all builtin programs except for the address-lookup-table. ( #23984 )
2022-03-29 19:06:50 +02:00
31b707b625
Specify if archive size datapoint is for full or incremental snapshots ( #23941 )
2022-03-26 12:29:13 -05:00
c24de17278
remove index hash calculation as an option ( #23928 )
2022-03-25 15:32:53 -05:00
ec78702bc8
RollingBitField::get_all_less_than ( #23919 )
2022-03-25 15:20:22 -05:00
acfd22712b
RollingBitFIeld to its own file ( #23917 )
2022-03-25 10:37:00 -05:00
55d61023f7
document 'accounts' hash ( #23907 )
2022-03-24 15:58:52 -05:00
140c8dd01f
Refactor: Replaces KeyedAccount in_get_sysvar_with_account_check ( #23905 )
...
* Replaces all use sites of get_sysvar_with_account_check by get_sysvar_with_account_check2.
* Removes get_sysvar_with_account_check.
* Renames get_sysvar_with_account_check2 to get_sysvar_with_account_check.
2022-03-24 19:30:42 +01:00
37c36ce3fa
pass stats separately from CalcAccountsHashConfig ( #23892 )
2022-03-24 12:48:47 -05:00
82328fd9d8
move max_clean_root deeper in flush cache ( #23869 )
2022-03-24 12:45:49 -05:00
c31db81ac4
Use VoteAccountsHashMap type alias in all applicable spots ( #23904 )
2022-03-24 12:09:48 -05:00
a22a2384bf
fix ci test error ( #23908 )
2022-03-24 11:30:20 -05:00
5b916961b5
HashCalc uses self.accounts_cache ( #23890 )
2022-03-24 10:34:28 -05:00
f2aea3b7c7
flush_slot_cache takes [Slot] ( #23865 )
2022-03-24 10:24:36 -05:00
9d3b17c635
HashCalc uses self.accounts_index ( #23888 )
2022-03-24 10:06:32 -05:00
396b49a7c1
Start saving/loading prior_roots(_with_hash) to snapshot ( #23844 )
...
* Start saving/loading prior_roots(_with_hash) to snapshot
* Update runtime/src/accounts_index.rs
Co-authored-by: Michael Vines <mvines@gmail.com>
* Update runtime/src/accounts_index.rs
Co-authored-by: Michael Vines <mvines@gmail.com>
* update comment
Co-authored-by: Michael Vines <mvines@gmail.com>
2022-03-24 10:06:24 -05:00
b22165ad69
hash calc uses self.filler_account_suffix ( #23887 )
2022-03-24 09:58:06 -05:00
9022931689
calc hash uses self.num_hash_scan_passes ( #23883 )
2022-03-24 09:44:42 -05:00
e3eb002f66
Log storage size stats at hash calc ( #23843 )
2022-03-24 09:40:35 -05:00
f1a411c897
add epoch_schedule and rent_collector to hash calc ( #23857 )
2022-03-24 09:39:22 -05:00
db5d68f01f
HashCalc uses self.accounts_hash_cache_path ( #23882 )
2022-03-24 09:31:55 -05:00
90009f330b
small refactor to shorten the lock on slot_under_contention hashset ( #23891 )
...
* small refactor to shorten the lock on slot_under_contention hashset
* adding comments
* comments
2022-03-24 08:20:56 -05:00
91c2729856
Replaces keyed_account get_signers() by InstructionContext::get_signers(). ( #23863 )
2022-03-24 12:57:51 +01:00
5a892af2fe
disable 'check_hash' on accounts hash calc ( #23873 )
2022-03-23 21:03:31 -05:00
3e22d4b286
calc hash uses self.thread_pool_clean ( #23881 )
2022-03-23 20:52:38 -05:00
486f7b7673
use array access function ( #23895 )
2022-03-23 17:03:01 -07:00
dc3863ef14
flush_slot_cache_with_clean ( #23868 )
2022-03-23 14:09:56 -05:00
260f899eda
write cache: hashmap to set ( #23866 )
2022-03-23 14:05:45 -05:00
9e61fe7583
add AccountsHashConfig to manage parameters ( #23850 )
2022-03-23 13:44:23 -05:00
b1280b670a
calculate_accounts_hash_without_index takes &self ( #23846 )
...
* calculate_accounts_hash_without_index takes &self
* Update runtime/src/snapshot_package.rs
Co-authored-by: Brooks Prumo <brooks@prumo.org>
Co-authored-by: Brooks Prumo <brooks@prumo.org>
2022-03-23 11:57:32 -05:00
7b89222fde
don't start extra threads for shrink/clean/hash ( #23858 )
2022-03-23 11:53:37 -05:00
493a8e2348
remove random flushing of write cache ( #23845 )
2022-03-23 08:45:44 -05:00
bc35e1c5f5
snapshot code needs all storages for hash calc ( #23840 )
2022-03-22 21:27:54 -05:00
9f0ca6d88a
Refactor: Remove `trait` from nonce keyed account ( #23811 )
...
* Removes the trait `NonceKeyedAccount`.
2022-03-23 02:09:30 +01:00
7af48465fa
transaction-status: Add return data to meta ( #23688 )
...
* transaction-status: Add return data to meta
* Add return data to simulation results
* Use pretty-hex for printing return data
* Update arg name, make TransactionRecord struct
* Rename TransactionRecord -> ExecutionRecord
2022-03-22 23:17:05 +01:00
1089a38aaf
AcctIdx: rework scan and write to disk ( #23794 )
2022-03-22 11:54:12 -05:00
89ba3ff139
log fail to evict ( #23815 )
2022-03-22 09:19:38 -05:00
24f6855f86
AcctIdx: only remove a fixed number of items per write lock ( #23795 )
2022-03-21 16:55:04 -05:00
cb06126388
Set accounts_data_len on feature activation ( #23730 )
2022-03-21 12:28:26 -05:00
965ab9186d
AcctIdx: fix infinite loop ( #23806 )
2022-03-21 10:58:36 -05:00
15357480ec
Refactor instruction compilation and update message account key ordering ( #23729 )
...
* Refactor: Make instruction compilation usable for other message versions
* apply trents feedback
* Fix tests
* Fix bpf compatiblity
2022-03-21 20:53:32 +08:00
258db77100
AcctIdx: factor 'scan' out of flush_internal ( #23777 )
2022-03-20 22:00:38 -05:00
f34434f96b
Drop lock ( #23765 )
2022-03-20 21:27:24 -04:00
dd69f3baf5
throttle index adding to allow disk flushing to keep up and reduce startup ram usage ( #23773 )
2022-03-20 19:56:20 -05:00
335c4b668b
Fix bug in bank/sysvar_cache tests ( #23780 )
2022-03-19 21:38:18 -05:00
df29276eb0
AcctIdx: remove -> evict ( #23775 )
2022-03-18 17:13:21 -05:00
a419374fa4
factor out function ( #23742 )
2022-03-18 14:10:52 -05:00
56428be629
Not exposing inner cost_table to encapsulating implementation details,
...
making future change easier.
2022-03-18 12:58:43 -05:00
998e7d18f9
AcctIdx: never retry a bucket flush ( #23732 )
2022-03-18 12:20:42 -05:00
0ed23899e7
directly use compute_budget MAX_UNITS and DEFAULT_UNITS
2022-03-18 08:53:11 -05:00
a4cacf3389
add deterministic default cost
2022-03-18 08:53:11 -05:00
857576d76f
AcctIdx: move write to disk outside in mem write lock ( #23731 )
2022-03-17 23:09:41 -05:00
7ff8c80e25
Add accounts_data_len to bank snapshot ( #23714 )
2022-03-17 20:14:54 -05:00
664deb2157
AcctIdx: get rid of unused is_dirty ( #23733 )
2022-03-17 16:29:36 -05:00
66b1f55351
AcctIdx: cheaper check for should evict while flushing ( #23705 )
2022-03-17 08:46:32 -05:00
3a46f45650
AcctIdx: add stats for flushing and estimated memory ( #23709 )
2022-03-17 08:46:00 -05:00
b4350a2522
Make solana-address-lookup-table-program crate bpf compatible ( #23700 )
2022-03-17 08:21:07 +08:00
be0aeea01a
AcctIdx: check for range holds outside lock ( #23706 )
2022-03-16 17:44:59 -05:00
caddb851be
acct idx flush keeps slot_list read lock open ( #23704 )
2022-03-16 17:29:04 -05:00
fa7926580a
minor VoteAccount refactoring ( #23686 )
2022-03-16 09:50:12 -05:00
3252dc7203
uses structural sharing for stake-delegations hash-map ( #23585 )
...
StakeDelegations is using Arc to implement copy-on-write semantics:
https://github.com/solana-labs/solana/blob/58c0db970/runtime/src/stake_delegations.rs#L14-L16
However a single delegation change will still clone the entire hash-map,
resulting in excessive memory use as observed in:
https://github.com/solana-labs/solana/issues/23061#issuecomment-1063444072
This commit instead uses immutable hash-map implementing structural
sharing:
> which means that if two data structures are mostly copies of each
> other, most of the memory they take up will be shared between them.
https://docs.rs/im/latest/im/
2022-03-16 12:58:05 +00:00
584ac80b1e
Revert: `KeyedAccount` refactoings in builtin programs ( #23649 )
...
* Revert "Replaces KeyedAccount by BorrowedAccount in the BPF loader. (#23056 )"
6c56eb9663#23217 )"
ee7e411d68#23214 )"
1a68f81f89#23302 )"
a14c7c37ee#23348 )"
e2fa6a0f7a#23375 )"
ee3fc39f1c
2022-03-16 11:30:01 +01:00
2d3501dff9
make upsert infallible op
2022-03-15 17:05:41 -05:00
b5a99b9b09
avoid data copies in writing to cache ( #23674 )
2022-03-15 16:42:26 -05:00
8c4f010b8d
fix logging: only walk remote dir if it exist ( #23663 )
2022-03-15 08:56:22 -05:00
8c8f9694e0
Refactor: Sanitized transaction creation ( #23558 )
...
* Refactor: SanitizedTransaction::try_create optionally computes hash
* Refactor: Add SimpleAddressLoader
2022-03-15 12:02:22 +08:00
f05ac7a899
move to get_slots_in_epoch ( #23657 )
2022-03-14 22:47:45 -05:00
102dd68a03
Rename AccountsDb plugins to Geyser plugins ( #23604 )
2022-03-14 19:18:46 -06:00
c68c0e881e
consolidate use-jit flag ( #23652 )
2022-03-14 15:00:00 -07:00
0c684721d8
Separate remotely downloaded snapshot archives ( #23510 )
...
* seperate remotely downloaded snapshot archives
* add str const for snapshot download dir
* only walk remote sub directory
* move directory creation outside of loop
* move is_remote to traits
* clippy simplify
* clippy
* clippy
* add unittest
* fix local cluster tests
* look for remote snapshot archive in remote foler
* create remote dir in tests
* use snapshot download dir constant
* extract build_remote_dir fn
* fix build
* code review - walking snapshot archived dirs explicitly
* fix build
* fix build
* fix comments
* Update runtime/src/snapshot_utils.rs
Co-authored-by: Brooks Prumo <brooks@prumo.org>
* clippy
* borrow to avoid copy
Co-authored-by: Brooks Prumo <brooks@prumo.org>
2022-03-14 14:03:59 -05:00
a3242b6b86
remove unnecessary namespaces ( #23646 )
2022-03-14 12:28:55 -05:00
11be3fb7fa
Add FlushGuard to ensure `flushing_active` is used safely ( #23571 )
2022-03-14 10:35:00 -05:00
e8a8f4e9e2
fix filler acct slots in epoch ( #23536 )
2022-03-14 09:57:44 -05:00
7758c32035
Banking Stage drops transactions that'll exceed the total account data size limit ( #23537 )
2022-03-13 15:58:57 +00:00
ccff006948
Add more logs to `purge_old_snapshot_archives` ( #23531 )
2022-03-11 15:01:48 -06:00
3c6840050c
Ensure blocks do not exceed the max accounts data size during Replay Stage ( #23422 )
2022-03-10 10:24:31 -06:00
17b00ad3a4
Add quic-client module ( #23166 )
...
* Add quic-client module to send transactions via quic, abstracted behind the TpuConnection trait (along with a legacy UDP implementation of TpuConnection) and change thin-client to use TpuConnection
2022-03-09 21:33:05 -05:00
1fe0d6eeeb
Set ordering flushing_active.swap() to AcqRel ( #23567 )
2022-03-10 01:22:42 +00:00
1eddb6d1e9
Move ArchiveFormat into own module ( #23562 )
2022-03-09 16:09:34 -06:00
9bbccbe27c
Use `AsRef<Path>` instead of `PathBuf` for parameters ( #23560 )
2022-03-09 16:08:33 -06:00
ba54b30101
clippy ( #23563 )
2022-03-09 15:33:50 -06:00
5a0cd05866
Revert "- estimate a program cost as 2 standard deviation above mean"
...
This reverts commit a25ac1c988
2022-03-08 17:18:44 -08:00
9acbfa5eb1
Revert "use EMA in place of Welford"
...
This reverts commit 6587dbfa47
2022-03-08 17:18:44 -08:00
c878c9e2cb
Revert "1. Persist to blockstore less frequently;"
...
This reverts commit 7aa1fb4e24
2022-03-08 17:18:44 -08:00
cc4d75a16f
fix typo ( #23535 )
2022-03-08 18:28:00 -06:00
5599bd9442
Remove BankFromArchiveTimings from ledger/
2022-03-08 08:11:50 -08:00
3114c199bd
Add RPC support for versioned transactions ( #22530 )
...
* Add RPC support for versioned transactions
* fix doc tests
* Add rpc test for versioned txs
* Switch to preflight bank
2022-03-08 15:20:34 +08:00
ddbf5c782f
Fix broken doc links in solana-runtime ( #23504 )
2022-03-07 22:21:40 +00:00
181fffb916
rename status filename to be consistent ( #23501 )
2022-03-07 17:34:35 +00:00
463cd564cf
fix typos ( #23495 )
...
* fix typos
2022-03-05 20:46:46 -06:00
38db1dead4
Refactor RPC tests ( #23483 )
2022-03-04 18:20:11 +08:00
b28acd2d4d
Reduce fn visibility
2022-03-04 09:52:46 +01:00
7f608965ef
new counter data point for unexpected rent_epoch ( #23449 )
2022-03-03 09:09:31 -06:00
ddfd4f86f3
exit early from acct bg svc if no shrink candidates ( #23459 )
2022-03-03 08:47:39 -06:00
a99fd09c16
allow index update to change storage slot # ( #23311 )
2022-03-03 08:40:48 -06:00
634f4eb37d
(LedgerStore) Use different path for different blockstore storage type. ( #23236 )
...
#### Summary of Changes
To avoid mixing the use of different shred storage types, each shred storage type
will have its blockstore in a different directory.
This PR still keeps the RocksFifo setting hidden. The default ShredStorageType and
blockstore directory are still RocksLevel and `rocksdb`.
Will follow-up with PRs on making FIFO option public in ledger-tool and validator.
#### Test Plan
* Added a new test to verify the existence of `rocksdb-fifo` directory when FIFO compaction is used.
* Updated existing test to verify the current setting still store ledger under `rocksdb` directory.
* Manually ran ledger_cleanup_test with both level and fifo compaction and verified the resulting ledger.
* Ran a validator with this PR.
2022-03-02 18:30:22 -08:00
97d40ba3da
Resized accounts must be rent exempt
2022-03-02 13:02:02 -08:00
e88da2ec0a
minor test cleanup ( #23447 )
...
* minor test cleanup
* fmt
2022-03-02 12:10:05 -06:00
e630eb73d7
rework test to avoid multi-epoch gap ( #23393 )
2022-03-02 11:12:10 -06:00
ef8b7d9c62
ledger tool halt at slot verify hash ( #23424 )
2022-03-02 11:11:18 -06:00
d909b7c80b
log hash and lamport result of calculate_accounts_hash_without_index ( #23425 )
2022-03-02 11:10:40 -06:00
7b7fdb42d9
minor test refactoring ( #23423 )
2022-03-02 08:31:14 -06:00
c8cb940b4e
tweak 2 rent tests to narrow epoch ranges ( #23420 )
2022-03-02 08:22:33 -06:00
454e82683e
refactor rent test ( #23392 )
2022-03-01 08:47:09 -06:00
6b2683f7da
refactor test ( #23384 )
2022-03-01 08:22:10 -06:00
19448ba078
Allow sub-rent-exempt-minimum transfers to `1nc1nerator` ( #23382 )
...
* Add failing test
* Allow small burns to incinerator
* Use check_id method
2022-02-28 23:56:34 -07:00
f0a235d16f
add comment ( #23378 )
2022-02-28 15:18:42 -06:00
4eeb9f4648
acct idx comments ( #23377 )
2022-02-28 15:00:53 -06:00
611d745241
refactor rent_due for normal case of exempt ( #23350 )
2022-02-28 09:42:42 -06:00
30dafc7135
list -> slot_list ( #23355 )
2022-02-27 23:08:58 -06:00
36484f4f08
Prevent new RentPaying state created by paying fees ( #23358 )
...
* Add failing test
* Check fee-payer rent-state change on load
* Add more test cases
* Review comments
2022-02-26 12:10:01 -07:00
0ad4757159
plumbing for 'other_slot' in 'update_index' ( #23330 )
2022-02-25 12:58:08 -06:00
5e0086c1ee
followup safety checks for #23295
2022-02-24 17:50:58 -07:00
2b0f16e7c3
slot per account in StorableAccounts ( #23313 )
2022-02-24 10:34:38 -06:00
017170c99d
DiskIdx: new items are not dirty by default ( #23123 )
2022-02-24 10:17:35 -06:00
c0d0724be0
clippy ( #23310 )
2022-02-23 18:07:00 -06:00
7ee549e5ae
fix params for store_accounts_frozen ( #23312 )
2022-02-23 17:40:51 -06:00
227df52213
update comment ( #23314 )
2022-02-23 17:33:06 -06:00
cafc18c3f9
update_index uses ReadableAccount to reduce params ( #23305 )
2022-02-23 17:01:23 -06:00
a245efe83d
trait for acct data with slot per item ( #23285 )
2022-02-23 15:22:29 -06:00
c81dd602c4
Hack fix for ICE as seen in CI
2022-02-23 11:17:39 -07:00
2996f1f783
add comments ( #23275 )
2022-02-23 10:53:11 -06:00
d0e85c293f
Fix rustfmt check ( #23296 )
2022-02-23 16:38:53 +08:00
20d031e2b8
Refactor ExecuteTimings w/ enum-indexed array ( #23085 )
2022-02-22 14:46:56 -08:00
88e1192c6b
Refactor new() and default() for CostTracker ( #23267 )
2022-02-22 10:44:23 -06:00
8b32e80ee2
DiskIdx: rename 'remove' to 'evict' ( #23188 )
2022-02-22 09:40:25 -06:00
7ebf398ed7
AcctIdx: env var "SOLANA_TEST_ACCOUNTS_INDEX_MEMORY_LIMIT_MB" ( #23194 )
...
* AcctIdx: env var "SOLANA_TEST_ACCOUNTS_INDEX_MEMORY_LIMIT_MB"
* ignore env var when starting as validator
* Update runtime/src/bucket_map_holder.rs
Co-authored-by: Trent Nelson <trent.a.b.nelson@gmail.com>
Co-authored-by: Trent Nelson <trent.a.b.nelson@gmail.com>
2022-02-22 09:40:12 -06:00
bcda74f42f
Fix builtin handling on epoch boundaries ( #23256 )
2022-02-22 20:54:08 +08:00
72c68695b5
Do not keep oldest full snapshot
2022-02-21 17:10:31 -07:00
7e08ae1d0c
Revert "Add simulation detection countermeasure ( #22880 )" ( #23261 )
...
This reverts commit c42b80f099
2022-02-21 21:15:37 +00:00
04d23a1597
fix memory ordering in append_vec ( #23215 )
2022-02-20 20:30:49 -06:00
70ebab2c82
Add rustfmt.toml and `cargo fmt` ( #23238 )
...
* fmt
* formatted
Co-authored-by: Lucas B <buffalu@jito.network>
2022-02-19 13:32:29 +08:00
1719d2349f
Skip adding builtins if they will be removed ( #23233 )
...
* Add failing test for precompile transition
* Skip adding builtins if they will be removed
* cargo clean
* nits
* fix abi check
* remove workaround
Co-authored-by: Jack May <jack@solana.com>
2022-02-18 18:36:59 -08:00
ee7e411d68
Replaces `KeyedAccount` by `BorrowedAccount` in `system_instruction_processor`. ( #23217 )
...
* Adds InstructionContext::check_number_of_instruction_accounts() and InstructionContext::get_instruction_account_key().
Bases check_sysvar_account() on instuction account indices.
* Adds instruction_account_indices enums to system_instruction_processor.
* Reorders parameters and adds InstructionContext.
* Replaces KeyedAccount by BorrowedAccount in system_instruction_processor (part 1).
* Replaces KeyedAccount by BorrowedAccount in system_instruction_processor (part 2).
* Replaces KeyedAccount by BorrowedAccount in system_instruction_processor (part 3).
* Replaces KeyedAccount by BorrowedAccount in system_instruction_processor (part 4).
* Replaces KeyedAccount by BorrowedAccount in system_instruction_processor (part 5).
* Code cleanup
2022-02-19 02:06:33 +01:00
970f543ef6
Precompiles owned by the native loader ( #23237 )
2022-02-18 15:41:59 -08:00
1a68f81f89
Replaces `KeyedAccount` by `BorrowedAccount` in `nonce_keyed_account`. ( #23214 )
...
* Adds get_sysvar_with_account_check2 for ABIv2.
* Replaces get_signers() and get_sysvar_with_account_check() in system_instruction_processor.
* Replaces KeyedAccount by BorrowedAccount in nonce_keyed_account.
2022-02-17 17:36:55 +01:00
da00b39f4f
Cleanup: get_program_key() and get_loader_key() in TransactionContext ( #23191 )
...
* Moves TransactionContext::get_program_key() to InstructionContext::get_program_key().
* Removes TransactionContext::get_loader_key().
* Test full program and loader executable account chain in BPF loader.
2022-02-17 10:16:28 +01:00
a102453bae
accounts_index: Add SPL Token account indexing for token-2022 accounts ( #23043 )
2022-02-16 16:23:25 -08:00
813725dfec
Fix ed25519 builtin program handling ( #23182 )
...
* Fix ed25519 builtin program handling
* Fix tests
* Add integration tests for processing transactions with ed25519 ixs
* Fix another test
* fix formatting
2022-02-16 18:48:55 +00:00
c2435363f3
DiskIdx: fix race condition with holding ranges in mem ( #23158 )
2022-02-16 09:25:00 -06:00
577fa4ec0c
Generate full snapshots 4x faster to keep incremental snapshots nice and small
2022-02-15 21:09:21 -08:00
43393ea45d
assert on AcctInfo.store_id with cached store ( #21952 )
2022-02-15 15:22:36 -06:00
c42b80f099
Add simulation detection countermeasure ( #22880 )
...
* Add simulation detection countermeasures
* Add program and test using TestValidator
* Remove incinerator deposit
* Remove incinerator
* Update Cargo.lock
* Add more features to simulation bank
* Update Cargo.lock per rebase
Co-authored-by: Jon Cinque <jon.cinque@gmail.com>
2022-02-15 13:09:59 +01:00
20bc37c9c9
Removes KeyedAccount in tests of system_instruction_processor. ( #23118 )
2022-02-15 00:00:49 +01:00
3d9874b95a
Add fees to tx-wide caps ( #22081 )
2022-02-11 16:23:16 -08:00
0a1ab945bc
Move cap_accounts_data_len feature gate only around new error ( #23048 )
2022-02-10 11:57:00 -06:00
47e4291303
add Debug to RentResult ( #23044 )
2022-02-09 18:07:36 -06:00
226a71f073
Refactor: Use instruction_trace instead of instruction_context_stack ( #22988 )
...
* Moves stack_height of the instruction_trace into InstructionContext.
* Uses nesting_level instead of stack_height internally in transaction_context.rs
* Uses the instruction_trace instead of the instruction_context_stack internally.
* Adds feature gate: record_instruction_in_transaction_context_push.
2022-02-09 20:04:49 +01:00
205cd4609b
bank::get_rent_collector_from_parent ( #23016 )
2022-02-09 12:31:58 -06:00
f334931903
rent: refactor test function ( #23014 )
2022-02-08 22:11:47 -06:00
fb47c1f0a3
consolidate Rent construction in test code ( #23018 )
2022-02-08 21:51:52 -06:00
869cfc9a1c
Return the accounts data len delta after processing messages ( #22986 )
2022-02-08 19:24:47 -06:00
7aa1fb4e24
1. Persist to blockstore less frequently;
...
2. reduce alpha for EMA to 1 percent to have roughly 200 data points for estimatio
2022-02-08 16:18:23 -06:00
6587dbfa47
use EMA in place of Welford
2022-02-08 16:18:23 -06:00
a25ac1c988
- estimate a program cost as 2 standard deviation above mean
...
- replaced get_average / get_mode with get_default to assign max units to unknown program
2022-02-08 16:18:23 -06:00
86cf226395
eliminate separate max root member ( #22943 )
2022-02-08 09:31:32 -06:00
f0f4042680
Put accounts data len updates behind feature gate ( #22918 )
2022-02-08 08:51:16 -06:00
b2e475b5c4
clarify that RollingBitField max is exclusive ( #22947 )
2022-02-07 13:40:31 -06:00
a160fc30f2
make max_roots_inclusive clear ( #22942 )
2022-02-07 13:26:53 -06:00
ba215e94f6
Refactor: Add AccountKeys struct for static and dynamic message keys ( #22960 )
2022-02-05 20:00:31 +08:00
660f6981c6
Cleanup: TransactionContext ( #22910 )
...
* Adds BorrowedAccount::check_sysvar().
* Adds BorrowedAccount::get_data_mut().
* Implements account resizing in BorrowedAccount.
* Exposes is_signer() and is_writable() in InstructionContext.
* Removes AccountMeta and get_instruction_accounts_metas().
* Makes throwing errors in BorrowedAccount optional.
* Removes result return values from BorrowedAccount.
2022-02-03 17:19:42 +01:00
c16cf9cf8a
Refactor: Use `SysvarCache` in all builtin programs ( #22864 )
...
* Replaces from_keyed_account() by SysvarCache in stake instruction.
* Replaces from_keyed_account() by SysvarCache in system instruction processor.
* Removes from_keyed_account().
Moves check_sysvar_keyed_account() into sysvar_cache.rs
* Removes tests which test for incorrectly serialized sysvars.
2022-02-03 13:03:50 +01:00
60af1a4cce
Refactor: Add trait for loading addresses ( #22903 )
2022-02-03 11:00:12 +00:00
cc94a93b56
Safer invoke context ( #22898 )
...
* Safer invoke context
* feedback and rebase with master
2022-02-03 02:34:51 -08:00
bd1850df25
Return actual committed transactions from process_transactions() ( #22802 )
2022-02-03 03:56:36 -05:00
ab02dba96f
Add get_processed_sibling_instruction syscall ( #22859 )
2022-02-02 16:45:57 -08:00
75563f6c7b
Reject close of active vote accounts ( #22651 )
...
* 10461 Reject close of vote accounts unless it earned no credits in the previous epoch. This is checked by comparing current epoch (from clock sysvar) with the most recent epoch with credits in vote state.
2022-02-02 14:16:24 -06:00
fb0e71946f
Cleanup use declaration ( #22875 )
2022-02-01 23:34:11 +00:00
22f6db2e4f
Cleanup serde snapshot common.rs ( #22854 )
2022-02-01 05:39:53 -06:00
551c24da57
nit: consistent name ( #22857 )
2022-02-01 07:46:04 +00:00
d60dac9749
add comment ( #22846 )
2022-01-31 11:40:19 -06:00
fb95fa68a2
cleanup api ( #22845 )
2022-01-31 11:39:23 -06:00
bc800a8d5a
Refactor: Unify `SysvarCache` ( #22843 )
...
* Unifies SysvarCache filling in the runtime and tests.
Removes new_mock_with_sysvars_and_features()
Removes mock_process_instruction_with_sysvars().
Replaces from_keyed_account() by SysvarCache in vote processor.
* Replaces from_keyed_account() by SysvarCache in BPF loader.
2022-01-31 17:53:50 +01:00
94a5aee484
Add new_from_parent() timings ( #22744 )
2022-01-27 20:35:13 -05:00
89c42ebcbe
refactor rent collection ( #22803 )
2022-01-27 18:48:04 -06:00
350207b05f
impl ReadableAccount for LoadedAccount ( #22793 )
2022-01-27 16:59:30 -06:00
18d69c8d2b
SortedStoragesIter to remove random access requirement ( #22713 )
2022-01-27 10:25:13 -06:00
d9c259a231
Set the correct root in block commitment cache initialization ( #22750 )
...
* Set the correct root in block commitment cache initialization
* clean up test
* bump
2022-01-27 00:48:00 +08:00
071e97053f
Perf: Reduce write locks on blockhash queue ( #22729 )
...
* Perf: Reduce write locks on blockhash queue
* Add comment about thread safety
* Add comment about write starvation
2022-01-26 16:24:27 +08:00
8b90084ebc
fix comment typos ( #22723 )
2022-01-25 11:10:42 -06:00
HaoranYi
Tao Zhu
Brooks Prumo
Jeff Washington (jwash)
Justin Starry
Jon Cinque
Giorgio Gambino
Christian Kamm
Dmitri Makarov
carllin
behzad nouri
Alexander Meißner
steviez
Jack May
Tyera Eulberg
ryleung-solana
Michael Vines
Brian Anderson
Yueh-Hsuan Chiang
Trent Nelson
Gavin Chan
Brooks Prumo
Jeff Washington (jwash)
buffalu
Will Hickey