be0bdd2261
explicitly ignores struct fields which do not impact PartialEq impl ( #24624 )
...
Unless struct fields are explicitly ignored in PartialEq implementation,
there are no compile-time checks that if more fields are added to the
struct, PartialEq implementation is accordingly updated.
2022-04-24 14:33:42 +00:00
8a062273de
Move error counters to be reported by leader only at end of slot ( #24581 )
...
* Add error counters to leader metrics only
* Add dependencies
2022-04-23 18:10:47 -05:00
8bfde0940f
cleanup ( #24598 )
...
* cleanup
* cargo
2022-04-23 08:28:49 -05:00
14ee36a2af
cleanup in vote/stake accounts ( #24608 )
...
* cleanup in vote/stake accounts
* reorder comparisons in accounts_equal
2022-04-23 08:28:21 -05:00
84e3342612
Process blockstore after starting the TVU
2022-04-22 21:17:49 -07:00
83e041299a
Run real snapshot packager while processing blockstore at validator startup
2022-04-22 21:17:49 -07:00
b101e00ffa
chore: bump bytemuck from 1.8.0 to 1.9.1 ( #24604 )
...
* chore: bump bytemuck from 1.8.0 to 1.9.1
Bumps [bytemuck](https://github.com/Lokathor/bytemuck ) from 1.8.0 to 1.9.1.
- [Release notes](https://github.com/Lokathor/bytemuck/releases )
- [Changelog](https://github.com/Lokathor/bytemuck/blob/main/changelog.md )
- [Commits](https://github.com/Lokathor/bytemuck/compare/v1.8.0...v1.9.1 )
---
updated-dependencies:
- dependency-name: bytemuck
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2022-04-22 20:50:47 -06:00
ce7d2964c9
metrics on ancient slots ( #24596 )
2022-04-22 15:01:51 -05:00
08627e3f67
chore: bump ouroboros from 0.14.2 to 0.15.0 ( #24575 )
...
* chore: bump ouroboros from 0.14.2 to 0.15.0
Bumps [ouroboros](https://github.com/joshua-maros/ouroboros ) from 0.14.2 to 0.15.0.
- [Release notes](https://github.com/joshua-maros/ouroboros/releases )
- [Commits](https://github.com/joshua-maros/ouroboros/compare/0.14.2...0.15.0 )
---
updated-dependencies:
- dependency-name: ouroboros
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <you@example.com>
2022-04-22 13:56:20 -06:00
0d51596224
sim: Override slot hashes account on simulation bank ( #24543 )
...
* sim: Override slot hashes during simulation
* Add simulation test program
* Address feedback
* Add AccountOverrides explicit type
* Cargo fmt
2022-04-22 12:32:31 +02:00
a0c4be8cd7
add StorableAccountsMovingSlots ( #24555 )
2022-04-21 15:48:20 -05:00
25e9199397
cleanup FlushStats ( #24556 )
2022-04-21 14:05:01 -05:00
3f7b8d3d2c
provide default impl of 'is_empty' ( #24554 )
2022-04-21 12:49:44 -05:00
d524ae293f
collects metrics for cached vote/stake accounts consistency
...
Tracking mismatches between cached vote/stake accounts and accounts-db
in preparation of activating feature for updating rewards at
epoch-boundary using cached vote/stake accounts.
2022-04-21 15:28:41 +00:00
108aa23d90
enforces type-safety in Stakes<StakeAccount> using phantom data
...
StakeAccount<Delegation> can only wrap a stake-state which is a
Delegation; whereas StakeAccount<()> wraps any account with stake state.
As a result, StakeAccount::<Delegation>::delegation() will return
Delegation instead of Option<Delegation>.
2022-04-21 15:28:41 +00:00
f937fcbd95
updates rewards at epoch boundary using cached accounts
...
Loading vote and stake accounts from accounts-db takes a significant
portion of time updating rewards at epoch boundary.
This commit bypasses accounts-db and instead uses vote and stake
accounts cached in bank stakes:
https://github.com/solana-labs/solana/blob/d2702201c/runtime/src/stakes.rs#L148-L152
These cached accounts are synchronized with accounts-db after each
transaction, and so there should not be any change in the resulting
computation:
https://github.com/solana-labs/solana/blob/d2702201c/runtime/src/bank.rs#L4526
Nevertheless, to avoid any chances of introducing a consensus issue, the
switch to cached account is feature gated.
2022-04-21 15:28:41 +00:00
b4491ff4ba
adds StakesEnum type representing Stakes<StakeAccount|Delegation>
...
For backward compatibility, we can only serialize and deserialize
Stakes<Delegation>. However Bank caches Stakes<StakeAccount>. This type
mismatch incurs a conversion cost at epoch boundary when updating
EpochStakes.
This commit adds StakesEnum which allows EpochStakes to include either a
Stakes<StakeAccount> or Stakes<Delegation> and so bypass the conversion
cost between the two at the epoch boundary.
2022-04-21 15:28:41 +00:00
454ef38e43
caches StakeAccount instead of Delegation in Stakes
...
The commit makes values in stake_delegations map in Stakes struct
generic. Stakes<Delegation> is equivalent to the old code and is used
for backward compatibility in BankFieldsTo{Serialize,Deserialize}.
But banks cache Stakes<StakeAccount> which includes the entire stake
account and StakeState deserialized from account. Doing so, will remove
the need to load stake account from accounts-db when working with
stake-delegations.
2022-04-21 15:28:41 +00:00
8c4d6357fe
adds StakeAccount type embodying an Account and a StakeState
...
The added type does sanity checks on the Account and stores deserialized
StakeState. Following commits will use this type instead of Delegation
in Stakes.
2022-04-21 15:28:41 +00:00
d5127abf46
Only add hashes for completed blocks to recent blockhashes ( #24389 )
...
* Only add hashes for completed blocks to recent blockhashes
* feedback
2022-04-21 21:05:29 +08:00
a21fc3f303
Apply transaction actual execution units to cost_tracker ( #24311 )
...
* Pass the sum of consumed compute units to cost_tracker
* cost model tracks builtins and bpf programs separately, enabling adjust block cost by actual bpf programs execution costs
* Copied nightly-only experimental `checked_add_(un)signed` implementation to sdk
* Add function to update cost tracker with execution cost adjustment
* Review suggestion - using enum instead of struct for CommitTransactionDetails
Co-authored-by: Justin Starry <justin.m.starry@gmail.com>
* review - rename variable to distinguish accumulated_consumed_units from individual compute_units_consumed
* not to use signed integer operations
* Review - using saturating_add_assign!(), and checked_*().unwrap_or()
* Review - using Ordering enum to cmp
* replace checked_ with saturating_
* review - remove unnecessary Option<>
* Review - add function to report number of non-zero units account to metrics
2022-04-21 07:38:07 +00:00
79923c3b58
Refactor: Rename BlockhashQueue fields and methods for clarity ( #24426 )
2022-04-21 11:57:17 +08:00
52f43a986d
skip rewrites in bank ( #23985 )
2022-04-20 14:08:46 -05:00
b1c89cf6d4
append_vec::remaining_bytes ( #24531 )
2022-04-20 13:44:57 -05:00
813d06529d
stats for rewrites ( #24529 )
2022-04-20 10:43:50 -05:00
05438c3cd6
implement remove_old_historical_roots ( #24492 )
2022-04-20 10:43:32 -05:00
cdc3c41001
helpers to skip rewrites in bank ( #24507 )
2022-04-20 10:21:02 -05:00
0d797e2fff
contains_multiple_slots ( #24500 )
2022-04-20 09:53:36 -05:00
cfe2177e16
validator option: accounts-db-skip-rewrites ( #24504 )
2022-04-20 09:41:00 -05:00
957849f4a5
load_to_collect_rent_eagerly returns loaded_slot ( #24506 )
2022-04-20 09:14:03 -05:00
dd15193c69
chore: bump rayon from 1.5.1 to 1.5.2 ( #24470 )
...
* chore: bump rayon from 1.5.1 to 1.5.2
Bumps [rayon](https://github.com/rayon-rs/rayon ) from 1.5.1 to 1.5.2.
- [Release notes](https://github.com/rayon-rs/rayon/releases )
- [Changelog](https://github.com/rayon-rs/rayon/blob/master/RELEASES.md )
- [Commits](https://github.com/rayon-rs/rayon/commits )
---
updated-dependencies:
- dependency-name: rayon
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2022-04-19 23:17:52 -06:00
ef5e7cce6e
type PubkeyAccountSlot ( #24503 )
2022-04-19 21:34:00 -05:00
4be01ec75a
remove temp dead_code markers ( #24508 )
2022-04-19 21:30:57 -05:00
9e4999ef6a
Remove halt_at_slot from RuntimeConfig, it's not a runtime concern
2022-04-19 19:23:58 -07:00
0e2e0c8b7d
Extract most storage-related services from the Tvu abstraction
2022-04-19 19:23:58 -07:00
255a6a729d
add historical_roots_len stat ( #24497 )
2022-04-19 16:53:24 -05:00
d23c04bb68
max_slot_in_storages_exclusive -> INclusive ( #24450 )
2022-04-19 13:59:41 -05:00
94b0186a96
Cost model tracks builtins and bpf programs separately ( #24468 )
...
* Cost model tracks builtins and bpf programs separatele (enables adjusting block cost by actual bpf programs execution costs)
* Address reviews: expand test; add metrics stat
2022-04-19 13:25:47 -05:00
2ad1baa753
Add const fn StakeState::size_of and static assertion ( #24416 )
2022-04-20 01:04:12 +08:00
c4923d29b4
bank_hash_at uses rewrites ( #24439 )
2022-04-19 11:29:29 -05:00
4fdf966393
demote shink counter to debug ( #24458 )
2022-04-19 10:02:23 -05:00
569b15d863
maybe_update_rent_epoch_on_load ( #24055 )
2022-04-19 08:29:36 -05:00
dc98510d6d
accounts hash calls maybe_rehash_skipped_rewrite ( #24316 )
2022-04-19 08:29:09 -05:00
5d1adf1270
Fix signature count ( #24471 )
...
* Fix signature count
* protect the signature count futher
2022-04-19 17:59:06 +08:00
d6ff50bcc1
Remove `KeyedAccount` in builtin program "system instruction processor" ( #24452 )
...
* Makes sure that there is only one KeyedAccount at a time.
Moves keyed_account_at_index() into create_account(), transfer_verified(), transfer() and transfer_with_seed() to do so.
* Uses transaction_context.get_key_of_account_at_index() when only the key is needed.
* Uses "!instruction_context.is_signer()" instead of ".signer_key().is_none()".
* Replaces KeyedAccount by BorrowedAccount.
* Removes unused code.
2022-04-19 09:28:07 +02:00
1e08178aa9
chore: bump flate2 from 1.0.22 to 1.0.23 ( #24460 )
...
* chore: bump flate2 from 1.0.22 to 1.0.23
Bumps [flate2](https://github.com/rust-lang/flate2-rs ) from 1.0.22 to 1.0.23.
- [Release notes](https://github.com/rust-lang/flate2-rs/releases )
- [Commits](https://github.com/rust-lang/flate2-rs/compare/1.0.22...1.0.23 )
---
updated-dependencies:
- dependency-name: flate2
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2022-04-18 23:50:17 -06:00
0e0f306996
mechanism for disabling hash calc caching ( #24437 )
2022-04-18 22:44:30 -05:00
94d3ec37d2
collect_rent_eagerly can just do rewrites ( #24438 )
2022-04-18 22:44:19 -05:00
bf1e0c89e8
rename local: max_slot_in_storages -> inclusive ( #24453 )
2022-04-18 22:42:02 -05:00
8b854d4141
Remove `KeyedAccount` in builtin program "nonce keyed account" ( #24436 )
...
* Makes sure that there is only one KeyedAccount at a time.
Moves keyed_account_at_index() into withdraw_nonce_account() to do so.
* Replaces KeyedAccount by BorrowedAccount.
* Removes unused code.
2022-04-18 22:19:51 +02:00
ee384f7254
remove 'startup' from verify_snapshot_bank ( #24441 )
2022-04-18 14:35:49 -05:00
bcc5b96bda
remove redundant EpochSchedule ( #24444 )
2022-04-18 13:04:48 -05:00
9052e41b32
Ran cargo fmt
2022-04-18 11:57:40 -05:00
4dfeab3b38
Fix build error introduced by my editor setup, part 2
2022-04-18 11:57:40 -05:00
87fb78dc56
Fix build error introduced by my editor setup
2022-04-18 11:57:40 -05:00
c2f7f2fff8
Remove redundant epoch_schedule from AccountsPackage
2022-04-18 11:57:40 -05:00
ed7813e698
Fixes to initial code
2022-04-18 11:57:40 -05:00
5472d2e605
Removing redundant EpochSchedule param from fns
2022-04-18 11:57:40 -05:00
c6a751d658
Detect and report bank drop signal queue full and disconnect events ( #24112 )
...
* nonblocking send when when droping banks
* detect and report drop signal queue full/disconnect events
* comments
* use counter for reporting bank_drop_queue events
* reduce log
* use datapoint to report stats
* logging instead of reporting bank drop signal full
* fix a corner case for reporting
* fix build
2022-04-18 11:19:11 -05:00
578d59c802
Remove the code that handles cost update for separate pr
2022-04-17 19:26:24 -05:00
85281de981
move number utility functions to sdk; log error if cost adjust overflows i64
2022-04-17 19:26:24 -05:00
9dadfb2e2c
Add checked_add_signed() to apply cost adjustment to cost_tracker
2022-04-17 19:26:24 -05:00
094da35b91
Address review comments:
...
1. use was_executed to correctly identify transactions requires cost adjustment;
2. add function to specifically handle executino cost adjustment without have to copy accounts
2022-04-17 19:26:24 -05:00
29ca21ed78
undo transaction cost from cost_tracker if it was not executed successfully
2022-04-17 19:26:24 -05:00
f33ad34531
Add feature_set_override parameter to mock_process_instruction() ( #24386 )
2022-04-15 13:43:04 -05:00
b4fd9124bf
log secondary index contents on startup ( #24348 )
2022-04-15 13:30:03 -05:00
8c9430359e
add early exit in get_corrected_rent_epoch_on_load ( #24331 )
2022-04-15 13:28:16 -05:00
ba7a2efa66
SlotInfoInEpoch ( #24332 )
2022-04-15 13:27:41 -05:00
4ed647d8ec
Test that tick slot hashes update the recent blockhash queue ( #24242 )
2022-04-16 00:30:20 +08:00
0e7b0597db
check for rewrites skipped in closure ( #24330 )
2022-04-14 13:46:18 -05:00
d20b4c9958
rename function param to avoid conflict ( #24342 )
2022-04-14 11:56:32 -05:00
a91b0c8ea3
dashmap -> rwlock<hashmap> for rewrites ( #24327 )
2022-04-14 11:55:58 -05:00
e3ef0741be
simplify bank drop calls ( #24142 )
...
* simplify bank drop calls
* clippy: import
* Update ledger/src/blockstore_processor.rs
Co-authored-by: Brooks Prumo <brooks@prumo.org>
* Update runtime/src/accounts_background_service.rs
Co-authored-by: Brooks Prumo <brooks@prumo.org>
* Update runtime/src/bank.rs
Co-authored-by: Brooks Prumo <brooks@prumo.org>
* cleanup
* format
* use msb of bank_id to indicates that we are dropping
* clippy
* restore bank id
* clippy
* revert is-serialized_with_abs flag
* assert
* clippy
* whitespace
* fix bank drop callback check
* more fix
* remove msb dropping implementation
* fix
Co-authored-by: Brooks Prumo <brooks@prumo.org>
2022-04-14 08:43:54 -05:00
255ef66a27
move feature activation log to correct fn for hash ( #24326 )
2022-04-13 17:43:33 -05:00
6a474f29cd
default to disk index ( #24251 )
2022-04-13 09:24:50 -05:00
b6b8783323
add maybe_update_rent_epoch_on_load ( #24294 )
2022-04-13 08:55:24 -05:00
929753a11f
typo ( #24291 )
2022-04-12 16:46:59 -05:00
69e9ad5571
update comment ( #24288 )
2022-04-12 12:37:46 -05:00
2d4d639635
add expected_rent_collection ( #24028 )
...
* add expected_rent_collection
* update some comments for clarity and resolve a todo
* add test for 'LeaveAloneNoRent'
2022-04-12 11:32:23 -05:00
bdbca3362e
increase test timeout ( #24277 )
2022-04-12 09:54:57 -05:00
1bc49d219d
IndexLimitMb option adds 'Unspecified' state ( #24249 )
2022-04-12 09:38:09 -05:00
605036c117
move test fn into its own mod ( #24212 )
...
* move test fn into its own mod
* pub
2022-04-12 09:36:05 -05:00
f3aa80d3f8
typo ( #24257 )
2022-04-12 09:08:35 -05:00
9b8850f99e
test-validator: Add `--max-compute-units` flag ( #24130 )
...
* test-validator: Add `--max-compute-units` flag
* Add `RuntimeConfig` for tweaking runtime behavior
* Actually add the file
* Move RuntimeConfig to runtime
2022-04-12 02:28:10 +02:00
4fd184c131
Use Release/Acquire instead of SeqCst for is_bank_drop_callback_enabled ( #24134 )
2022-04-11 19:19:17 -05:00
6fbe2b936c
fix comment ( #24254 )
2022-04-11 18:53:45 -05:00
c0019edf00
document WaitableCondvar ( #24252 )
2022-04-11 14:45:23 -05:00
9ac2245970
remove clone ( #24244 )
2022-04-11 13:15:00 -05:00
60b2155bd3
Add accounts-filler-size command line option ( #23896 )
2022-04-11 13:10:09 -05:00
e14933c54d
move bank test fn to its test_utils mod ( #24171 )
2022-04-11 10:42:24 -05:00
8eef3d9713
Add tests to the blockhash queue ( #24238 )
2022-04-11 19:36:24 +08:00
a058f348a2
Address review comments
2022-04-08 14:37:55 -05:00
2ed29771f2
Unittest for cost tracker after process_and_record_transactions
2022-04-08 14:37:55 -05:00
924b8ea1eb
Adjustments to cost_tracker updates
...
- don't store pending tx signatures and costs in CostTracker
- apply tx costs to global state immediately again
- go from commit_or_cancel to update_or_remove, where the cost tracker
is either updated with the true costs for successful tx, or the costs
of a retryable tx is removed
- move the function into qos_service and hold the cost tracker lock for
the whole loop
2022-04-08 14:37:55 -05:00
9e07272af8
- Only commit successfully executed transactions' cost to cost_tracker;
...
- In-fly transactions are pended in cost_tracker until being committed
or cancelled;
2022-04-08 14:37:55 -05:00
03ed334ebb
Double the chunk size for sending the program binary data in tx
2022-04-08 09:06:40 -07:00
210f6a6fab
move hash calculation out of acct bg svc ( #23689 )
...
* move hash calculation out of acct bg svc
* pr feedback
2022-04-08 10:42:03 -05:00
c27150b1a3
reserialize_bank_fields_with_hash ( #23916 )
...
* reserialize_bank_with_new_accounts_hash
* Update runtime/src/serde_snapshot.rs
Co-authored-by: Brooks Prumo <brooks@prumo.org>
* Update runtime/src/serde_snapshot/tests.rs
Co-authored-by: Brooks Prumo <brooks@prumo.org>
* Update runtime/src/serde_snapshot/tests.rs
Co-authored-by: Brooks Prumo <brooks@prumo.org>
* pr feedback
Co-authored-by: Brooks Prumo <brooks@prumo.org>
2022-04-07 14:05:57 -05:00
48d1af01c8
add metrics around rewards ( #24160 )
2022-04-07 11:44:26 -05:00
f7b2951c79
move around some index metrics to reduce locks ( #24161 )
2022-04-07 09:43:19 -05:00
42c094739d
add test cfg attribute ( #24154 )
2022-04-07 09:05:20 -05:00
550ca7bf92
compare contents of serialized banks instead of exact file format ( #24141 )
...
* compare contents of serialized banks instead of exact file format
* Update runtime/src/snapshot_utils.rs
Co-authored-by: Brooks Prumo <brooks@prumo.org>
* Update runtime/src/snapshot_utils.rs
Co-authored-by: Brooks Prumo <brooks@prumo.org>
* pr feedback
* get rid of clone
* pr feedback
Co-authored-by: Brooks Prumo <brooks@prumo.org>
2022-04-06 21:55:44 -05:00
fddd162645
reserialize bank in ahv by first writing to temp file in abs ( #23947 )
2022-04-06 21:39:26 -05:00
c322842257
Replace channel with Mutex<Option> for AccountsPackage ( #24013 )
2022-04-06 05:47:19 -05:00
4ea59d8cb4
Set drop callback on first root bank ( #23999 )
2022-04-05 13:02:33 -05:00
4a11fa072f
hash_account_with_rent_epoch ( #24104 )
2022-04-05 08:10:31 -05:00
6a7f6585ce
persist historical_roots ( #24029 )
2022-04-04 13:13:11 -05:00
132f08486a
remove basically duplicate function ( #24107 )
2022-04-04 12:55:05 -05:00
997db7637c
do simple math without floats
2022-04-04 12:32:00 -05:00
f8f3edac3c
update comment ( #24108 )
2022-04-04 11:06:01 -05:00
2820b64eb3
roots_original -> historical_roots ( #24063 )
2022-04-04 09:12:12 -05:00
ef3e3dce7a
hides implementation details of vote-accounts from public interface ( #24087 )
2022-04-04 13:20:26 +00:00
fa7eb7f30c
improves Stakes::activate_epoch performance ( #24068 )
...
Tested with mainnet stakes obtained from the ledger at 5 recent epoch
boundaries, this code is ~30% faster than current master.
Current code:
epoch: 289, elapsed: 82901us
epoch: 290, elapsed: 80525us
epoch: 291, elapsed: 79122us
epoch: 292, elapsed: 79961us
epoch: 293, elapsed: 78965us
This commit:
epoch: 289, elapsed: 61710us
epoch: 290, elapsed: 55721us
epoch: 291, elapsed: 55886us
epoch: 292, elapsed: 55399us
epoch: 293, elapsed: 56803us
2022-04-02 22:48:51 +00:00
0ca5a0ec68
prior_roots -> historical_roots ( #24064 )
2022-04-02 12:01:43 -05:00
ec97d6d078
rename remove_old_roots ( #24059 )
2022-04-02 12:01:13 -05:00
3ca4fffa78
root -> alive_root ( #24062 )
2022-04-02 12:00:52 -05:00
792bbf75ab
Support sending versioned txs in AsyncClient ( #23982 )
2022-04-02 11:12:02 +08:00
1b45c509c3
Refactor: Use `InstructionContext::get_instruction_data()` ( #24014 )
...
* Adds transaction_context and instruction_context where invoke_context.get_keyed_accounts() is used.
* Use instruction_context.get_instruction_data() instead of an explicit parameter.
* Removes instruction_data parameter from Executor::execute().
* Removes instruction_data parameter from ProcessInstructionWithContext.
2022-04-01 15:48:05 +02:00
31997f8251
hash calc scanning takes config ( #24016 )
2022-03-31 14:26:37 -05:00
9c8dad33c7
add epoch_schedule and rent_collector to hash calc ( #24012 )
2022-03-31 10:51:18 -05:00
da001d54e5
calculate_accounts_hash_helper uses config ( #24003 )
2022-03-31 09:29:45 -05:00
cb5e67d327
Use Rent sysvar directly for stake split instruction ( #24008 )
...
* Use Rent sysvar directly for stake split ix
* Add feature to gate rent sysvar change
* fix tests
* cargo clippy
2022-03-31 16:46:35 +08:00
125f9634fd
add hash calc config.use_write_cache ( #24005 )
2022-03-30 17:19:34 -05:00
82c5230bc2
AccountsPackage::new less brittle ( #23968 )
2022-03-30 14:06:15 -05:00
1fb82d7924
fix typo in comments ( #24004 )
2022-03-30 09:47:51 -05:00
af9344fe22
SortedStorages refactoring ( #23998 )
2022-03-30 09:19:03 -05:00
5636570d6d
add roots_original to roots tracker ( #23849 )
2022-03-30 08:52:45 -05:00
da844d7be5
refactoring of SortedStorages tests to make other changes easier ( #23990 )
2022-03-29 22:06:48 -05:00
5a613e9b6e
use CalcAccountsHashConfig in calculate_accounts_hash ( #23987 )
2022-03-29 22:05:47 -05:00
794645d092
Adds check_number_of_instruction_accounts() to all builtin programs except for the address-lookup-table. ( #23984 )
2022-03-29 19:06:50 +02:00
31b707b625
Specify if archive size datapoint is for full or incremental snapshots ( #23941 )
2022-03-26 12:29:13 -05:00
c24de17278
remove index hash calculation as an option ( #23928 )
2022-03-25 15:32:53 -05:00
ec78702bc8
RollingBitField::get_all_less_than ( #23919 )
2022-03-25 15:20:22 -05:00
acfd22712b
RollingBitFIeld to its own file ( #23917 )
2022-03-25 10:37:00 -05:00
55d61023f7
document 'accounts' hash ( #23907 )
2022-03-24 15:58:52 -05:00
140c8dd01f
Refactor: Replaces KeyedAccount in_get_sysvar_with_account_check ( #23905 )
...
* Replaces all use sites of get_sysvar_with_account_check by get_sysvar_with_account_check2.
* Removes get_sysvar_with_account_check.
* Renames get_sysvar_with_account_check2 to get_sysvar_with_account_check.
2022-03-24 19:30:42 +01:00
37c36ce3fa
pass stats separately from CalcAccountsHashConfig ( #23892 )
2022-03-24 12:48:47 -05:00
82328fd9d8
move max_clean_root deeper in flush cache ( #23869 )
2022-03-24 12:45:49 -05:00
c31db81ac4
Use VoteAccountsHashMap type alias in all applicable spots ( #23904 )
2022-03-24 12:09:48 -05:00
a22a2384bf
fix ci test error ( #23908 )
2022-03-24 11:30:20 -05:00
5b916961b5
HashCalc uses self.accounts_cache ( #23890 )
2022-03-24 10:34:28 -05:00
f2aea3b7c7
flush_slot_cache takes [Slot] ( #23865 )
2022-03-24 10:24:36 -05:00
9d3b17c635
HashCalc uses self.accounts_index ( #23888 )
2022-03-24 10:06:32 -05:00
396b49a7c1
Start saving/loading prior_roots(_with_hash) to snapshot ( #23844 )
...
* Start saving/loading prior_roots(_with_hash) to snapshot
* Update runtime/src/accounts_index.rs
Co-authored-by: Michael Vines <mvines@gmail.com>
* Update runtime/src/accounts_index.rs
Co-authored-by: Michael Vines <mvines@gmail.com>
* update comment
Co-authored-by: Michael Vines <mvines@gmail.com>
2022-03-24 10:06:24 -05:00
b22165ad69
hash calc uses self.filler_account_suffix ( #23887 )
2022-03-24 09:58:06 -05:00
9022931689
calc hash uses self.num_hash_scan_passes ( #23883 )
2022-03-24 09:44:42 -05:00
e3eb002f66
Log storage size stats at hash calc ( #23843 )
2022-03-24 09:40:35 -05:00
f1a411c897
add epoch_schedule and rent_collector to hash calc ( #23857 )
2022-03-24 09:39:22 -05:00
db5d68f01f
HashCalc uses self.accounts_hash_cache_path ( #23882 )
2022-03-24 09:31:55 -05:00
90009f330b
small refactor to shorten the lock on slot_under_contention hashset ( #23891 )
...
* small refactor to shorten the lock on slot_under_contention hashset
* adding comments
* comments
2022-03-24 08:20:56 -05:00
91c2729856
Replaces keyed_account get_signers() by InstructionContext::get_signers(). ( #23863 )
2022-03-24 12:57:51 +01:00
5a892af2fe
disable 'check_hash' on accounts hash calc ( #23873 )
2022-03-23 21:03:31 -05:00
3e22d4b286
calc hash uses self.thread_pool_clean ( #23881 )
2022-03-23 20:52:38 -05:00
486f7b7673
use array access function ( #23895 )
2022-03-23 17:03:01 -07:00
dc3863ef14
flush_slot_cache_with_clean ( #23868 )
2022-03-23 14:09:56 -05:00
260f899eda
write cache: hashmap to set ( #23866 )
2022-03-23 14:05:45 -05:00
9e61fe7583
add AccountsHashConfig to manage parameters ( #23850 )
2022-03-23 13:44:23 -05:00
b1280b670a
calculate_accounts_hash_without_index takes &self ( #23846 )
...
* calculate_accounts_hash_without_index takes &self
* Update runtime/src/snapshot_package.rs
Co-authored-by: Brooks Prumo <brooks@prumo.org>
Co-authored-by: Brooks Prumo <brooks@prumo.org>
2022-03-23 11:57:32 -05:00
7b89222fde
don't start extra threads for shrink/clean/hash ( #23858 )
2022-03-23 11:53:37 -05:00
493a8e2348
remove random flushing of write cache ( #23845 )
2022-03-23 08:45:44 -05:00
bc35e1c5f5
snapshot code needs all storages for hash calc ( #23840 )
2022-03-22 21:27:54 -05:00
9f0ca6d88a
Refactor: Remove `trait` from nonce keyed account ( #23811 )
...
* Removes the trait `NonceKeyedAccount`.
2022-03-23 02:09:30 +01:00
7af48465fa
transaction-status: Add return data to meta ( #23688 )
...
* transaction-status: Add return data to meta
* Add return data to simulation results
* Use pretty-hex for printing return data
* Update arg name, make TransactionRecord struct
* Rename TransactionRecord -> ExecutionRecord
2022-03-22 23:17:05 +01:00
1089a38aaf
AcctIdx: rework scan and write to disk ( #23794 )
2022-03-22 11:54:12 -05:00
89ba3ff139
log fail to evict ( #23815 )
2022-03-22 09:19:38 -05:00
c4ecfa5716
Bump version to v1.11 ( #23807 )
...
* Revert crossbeam_epoch to stable. 0.9.8 only works with nightly
* Remove unneeded unit expression
2022-03-21 17:40:50 -05:00
24f6855f86
AcctIdx: only remove a fixed number of items per write lock ( #23795 )
2022-03-21 16:55:04 -05:00
cb06126388
Set accounts_data_len on feature activation ( #23730 )
2022-03-21 12:28:26 -05:00
965ab9186d
AcctIdx: fix infinite loop ( #23806 )
2022-03-21 10:58:36 -05:00
15357480ec
Refactor instruction compilation and update message account key ordering ( #23729 )
...
* Refactor: Make instruction compilation usable for other message versions
* apply trents feedback
* Fix tests
* Fix bpf compatiblity
2022-03-21 20:53:32 +08:00
258db77100
AcctIdx: factor 'scan' out of flush_internal ( #23777 )
2022-03-20 22:00:38 -05:00
f34434f96b
Drop lock ( #23765 )
2022-03-20 21:27:24 -04:00
dd69f3baf5
throttle index adding to allow disk flushing to keep up and reduce startup ram usage ( #23773 )
2022-03-20 19:56:20 -05:00
335c4b668b
Fix bug in bank/sysvar_cache tests ( #23780 )
2022-03-19 21:38:18 -05:00
df29276eb0
AcctIdx: remove -> evict ( #23775 )
2022-03-18 17:13:21 -05:00
a419374fa4
factor out function ( #23742 )
2022-03-18 14:10:52 -05:00
56428be629
Not exposing inner cost_table to encapsulating implementation details,
...
making future change easier.
2022-03-18 12:58:43 -05:00
998e7d18f9
AcctIdx: never retry a bucket flush ( #23732 )
2022-03-18 12:20:42 -05:00
0ed23899e7
directly use compute_budget MAX_UNITS and DEFAULT_UNITS
2022-03-18 08:53:11 -05:00
a4cacf3389
add deterministic default cost
2022-03-18 08:53:11 -05:00
857576d76f
AcctIdx: move write to disk outside in mem write lock ( #23731 )
2022-03-17 23:09:41 -05:00
7ff8c80e25
Add accounts_data_len to bank snapshot ( #23714 )
2022-03-17 20:14:54 -05:00
664deb2157
AcctIdx: get rid of unused is_dirty ( #23733 )
2022-03-17 16:29:36 -05:00
2f58c9e501
Bump version to 1.10.4 ( #23743 )
2022-03-17 14:02:13 -05:00
66b1f55351
AcctIdx: cheaper check for should evict while flushing ( #23705 )
2022-03-17 08:46:32 -05:00
3a46f45650
AcctIdx: add stats for flushing and estimated memory ( #23709 )
2022-03-17 08:46:00 -05:00
b4350a2522
Make solana-address-lookup-table-program crate bpf compatible ( #23700 )
2022-03-17 08:21:07 +08:00
be0aeea01a
AcctIdx: check for range holds outside lock ( #23706 )
2022-03-16 17:44:59 -05:00
caddb851be
acct idx flush keeps slot_list read lock open ( #23704 )
2022-03-16 17:29:04 -05:00
fa7926580a
minor VoteAccount refactoring ( #23686 )
2022-03-16 09:50:12 -05:00
3252dc7203
uses structural sharing for stake-delegations hash-map ( #23585 )
...
StakeDelegations is using Arc to implement copy-on-write semantics:
https://github.com/solana-labs/solana/blob/58c0db970/runtime/src/stake_delegations.rs#L14-L16
However a single delegation change will still clone the entire hash-map,
resulting in excessive memory use as observed in:
https://github.com/solana-labs/solana/issues/23061#issuecomment-1063444072
This commit instead uses immutable hash-map implementing structural
sharing:
> which means that if two data structures are mostly copies of each
> other, most of the memory they take up will be shared between them.
https://docs.rs/im/latest/im/
2022-03-16 12:58:05 +00:00
584ac80b1e
Revert: `KeyedAccount` refactoings in builtin programs ( #23649 )
...
* Revert "Replaces KeyedAccount by BorrowedAccount in the BPF loader. (#23056 )"
6c56eb9663#23217 )"
ee7e411d68#23214 )"
1a68f81f89#23302 )"
a14c7c37ee#23348 )"
e2fa6a0f7a#23375 )"
ee3fc39f1c
2022-03-16 11:30:01 +01:00
2d3501dff9
make upsert infallible op
2022-03-15 17:05:41 -05:00
b5a99b9b09
avoid data copies in writing to cache ( #23674 )
2022-03-15 16:42:26 -05:00
29af42f428
chore: bump zstd from 0.11.0+zstd.1.5.2 to 0.11.1+zstd.1.5.2 ( #23679 )
...
* chore: bump zstd from 0.11.0+zstd.1.5.2 to 0.11.1+zstd.1.5.2
Bumps [zstd](https://github.com/gyscos/zstd-rs ) from 0.11.0+zstd.1.5.2 to 0.11.1+zstd.1.5.2.
- [Release notes](https://github.com/gyscos/zstd-rs/releases )
- [Commits](https://github.com/gyscos/zstd-rs/commits )
---
updated-dependencies:
- dependency-name: zstd
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <you@example.com>
2022-03-15 15:31:14 -06:00
8c4f010b8d
fix logging: only walk remote dir if it exist ( #23663 )
2022-03-15 08:56:22 -05:00
8c8f9694e0
Refactor: Sanitized transaction creation ( #23558 )
...
* Refactor: SanitizedTransaction::try_create optionally computes hash
* Refactor: Add SimpleAddressLoader
2022-03-15 12:02:22 +08:00
f05ac7a899
move to get_slots_in_epoch ( #23657 )
2022-03-14 22:47:45 -05:00
102dd68a03
Rename AccountsDb plugins to Geyser plugins ( #23604 )
2022-03-14 19:18:46 -06:00
c68c0e881e
consolidate use-jit flag ( #23652 )
2022-03-14 15:00:00 -07:00
behzad nouri
carllin
Jeff Washington (jwash)
Michael Vines
dependabot[bot]
Jon Cinque
Justin Starry
Tao Zhu
HaoranYi
Jack May
Alexander Meißner
Jason Davis
Tao Zhu
Brooks Prumo
Giorgio Gambino
Christian Kamm
Dmitri Makarov
steviez
Will Hickey
Tyera Eulberg