Commit Graph

2867 Commits

Author SHA1 Message Date
apfitzge 7efd05f7b3
Change get_pubkeys to match the new enum (#26177) 2022-06-23 11:56:09 -05:00
Boqin Qin(秦 伯钦) 355e09e1fb
runtime: fix possible deadlock in in_mem_accounts_index (#26046) 2022-06-23 11:35:47 -05:00
Jeff Washington (jwash) 102fb98070
impl StorableAccounts for [StakeReward] (#26165) 2022-06-23 08:34:28 -05:00
Jeff Washington (jwash) 19b79aa764
add metrics for invalid_cached_stake_accounts_rent_epoch (#26143) 2022-06-23 08:33:24 -05:00
Jeff Washington (jwash) 9db44ce352
cleanup some AccountSharedData::from(Account (#26120) 2022-06-23 08:32:48 -05:00
Brooks Prumo 7d29c26315
Remove auto traits from ICE work-around (#26098) 2022-06-23 07:22:30 -05:00
Brooks Prumo a5cf72e446
StatusCache::roots() returns `Slot`s not `u64`s (#26164) 2022-06-23 07:03:09 +00:00
github-actions[bot] 5c2f819f99
Bump Version to 1.11.2 (#26159) 2022-06-22 21:16:18 -05:00
Jeff Washington (jwash) aea75e82e3
Bank::store_accounts takes StorableAccounts (#26113) 2022-06-22 21:11:01 -05:00
Jeff Washington (jwash) a32523c43b
cleanup some tests (#26139) 2022-06-22 21:10:46 -05:00
HaoranYi c4efb9f19e
Stake cache size (#26071)
* reduce threadpool for stake processing

* wip

* Revert "reduce threadpool for stake processing"

This reverts commit 004a4f872ea7a3ef53e38d145b6350c3f57c680c.

* batch update stake_cache

* fix deadlock

* add test

* code review feedbacks

* more review feedbacks

* fix conflicts

* report stake account len and vote account len at epoch boundary

* report num_staked_nodes

* remove batch store, no atomic for redeem timing
2022-06-22 16:22:22 -05:00
Jeff Washington (jwash) c5c2b44ecc
adjust cached stake account compare for rent_epoch (#26135) 2022-06-22 15:11:25 -05:00
ivandzen 1ea6a2cb91
Extended interface for AccountOverrides (#25964)
* Extended interface for AccountOverrides

* fix checker issue

* make accounts private

Co-authored-by: Ivan Loboda <lobodatell@yandex.ru>
2022-06-22 15:22:31 -04:00
apfitzge f4189c0305
ledger-tool minimized snapshots (#25334)
* working on local snapshot

* Parallelization for slot storage minimization

* Additional clean-up and fixes

* make --minimize an option of create-snapshot

* remove now unnecessary function

* Parallelize parts of minimized account set generation

* clippy fixes

* Add rent collection accounts and voting node_pubkeys

* Simplify programdata_accounts generation

* Loop over storages to get slot set

* Parallelize minimized slot set generation

* Parallelize adding owners and programdata_accounts

* Remove some now unncessary checks on the blockstore

* Add a warning for minimized snapshots across epoch boundary

* Simplify ledger-tool minimize

* Clarify names of bank's minimization helper functions

* Remove unnecesary funciton, fix line spacing

* Use DashSets instead of HashSets for minimized account and slot sets

* Filter storages uses all threads instead of thread_pool

* Add some additional comments on functions for minimization

* Moved more into bank and parallelized

* Update programs/bpf/Cargo.lock for dashmap in ledger

* Clippy fix

* ledger-tool: convert minimize_bank_for_snapshot Measure into measure!

* bank.rs: convert minimize_bank_for_snapshot Measure into measure!

* accounts_db.rs: convert minimize_accounts_db Measure into measure!

* accounts_db.rs: add comment about use of minimize_accounts_db

* ledger-tool: CLI argument clarification

* minimization functions: make infos unique

* bank.rs: Add test_get_rent_collection_accounts_between_slots

* bank.rs: Add test_minimization_add_vote_accounts

* bank.rs: Add test_minimization_add_stake_accounts

* bank.rs: Add test_minimization_add_owner_accounts

* bank.rs: Add test_minimization_add_programdata_accounts

* accounts_db.rs: Add test_minimize_accounts_db

* bank.rs: Add negative case and comments in test_get_rent_collection_accounts_between_slots

* bank.rs: Negative test in test_minimization_add_programdata_accounts

* use new static runtime and sdk ids

* bank comments to doc comments

* Only need to insert the maximum slot a key is found in

* rename remove_pubkeys to purge_pubkeys

* add comment on builtins::get_pubkeys

* prevent excessive logging of removed dead slots

* don't need to remove slot from shrink slot candidates

* blockstore.rs: get_accounts_used_in_range shouldn't return Result

* blockstore.rs: get_accounts_used_in_range: parallelize slot loop

* report filtering progress on time instead of count

* parallelize loop over snapshot storages

* WIP: move some bank minimization functionality into a new class

* WIP: move some accounts_db minimization functionality into SnapshotMinimizer

* WIP: Use new SnapshotMinimizer

* SnapshotMinimizer: fix use statements

* remove bank and accounts_db minimization code, where possible

* measure! doesn't take a closure

* fix use statement in blockstore

* log_dead_slots does not need pub(crate)

* get_unique_accounts_from_storages does not need pub(crate)

* different way to get stake accounts/nodes

* fix tests

* move rent collection account functionality to snapshot minimizer

* move accounts_db minimize behavior to snapshot minimizer

* clean up

* Use bank reference instead of Arc. Additional comments

* Add a comment to blockstore function

* Additional clarifying comments

* Moved all non-transaction account accumulation into the SnapshotMinimizer.

* transaction_account_set does not need to be mutable now

* Add comment about load_to_collect_rent_eagerly

* Update log_dead_slots comment

* remove duplicate measure/print of get_minimized_slot_set
2022-06-22 13:17:43 -04:00
Jeff Washington (jwash) cf5ec13f59
Accounts::store_accounts_cached uses StorableAccounts (#26112) 2022-06-22 12:13:56 -05:00
Michael Vines f3639b76ce Remove some clippy lints 2022-06-22 09:23:22 -07:00
HaoranYi 5624ab0d4c
Optimize stake accounts store (#26004)
* reduce threadpool for stake processing

* wip

* Revert "reduce threadpool for stake processing"

This reverts commit 004a4f872ea7a3ef53e38d145b6350c3f57c680c.

* batch update stake_cache

* fix deadlock

* add test

* code review feedbacks

* more review feedbacks

* clippy

* update comments

* fix conflicts

* remove batch store, no atomic for redeem timing

* refactor stake_reward struct
2022-06-22 10:45:58 -05:00
Jeff Washington (jwash) 8e2bae7bf9
batch insert updates stat count correctly (#26107) 2022-06-22 08:45:42 -05:00
Jeff Washington (jwash) d3829b1a76
store_cached takes StorableAccounts to eliminate collect (#26109) 2022-06-22 08:45:03 -05:00
Brooks Prumo 8a18e02f80
Call StatusCache::append() directly (#26125) 2022-06-22 07:42:01 -05:00
Brooks Prumo 31c9b03049
Remove unused SignatureConfirmationStatus (#26124) 2022-06-22 07:41:22 -05:00
Brooks Prumo 97bd81a32b
Add bench for StatusCache::slot_deltas() (#26123) 2022-06-22 07:41:06 -05:00
Brooks Prumo d35c1a7d04
Use BankStatusCache in bank.rs (#26114) 2022-06-21 23:16:16 +00:00
Jeff Washington (jwash) 43e0d29b18
Revert "fix double count of account index del stats (#25797)" (#26103)
This reverts commit 05cb25d8da.
2022-06-21 14:05:26 -05:00
Jeff Washington (jwash) 7a3dc3a57b
remove unused param (#26096) 2022-06-21 13:10:41 -05:00
Will Hickey 51f26dc96e
Bump version to 1.11.1 (#26104) 2022-06-21 12:07:46 -05:00
Boqin Qin(秦 伯钦) 95ea506d75
runtime: fix double-readlock in in_mem_accounts_index.rs (#26047) 2022-06-20 15:44:19 +01:00
Tyera Eulberg 752c851070
Clean up feature preventing new rent-paying accounts (#26000)
* Clean up require_rent_exempt_accounts feature

* Fix tests in bank_client, system_instruction_processor

* Fix tests in snapshot_utils

* Fix bank tests

* Ignore rent tests

* Fix stake-accounts tests
2022-06-16 15:35:25 -06:00
Alexander Meißner bf9ca9827e
Refactor: instruction account index (#25825)
* Adds methods based on instruction_account_index to InstructionContext.
Removes methods which are based on index_in_instruction.

* Adjusts program-runtime.

* Adjusts runtime.

* Adjusts bpf loader.

* Adjusts built-in programs.

* Adjusts program-test and bpf tests.
2022-06-16 18:46:17 +02:00
Brooks Prumo 54675b2dd6
Add bank tests for accounts_data_size and resize transactions / rent collection (#25884) 2022-06-15 19:40:43 -05:00
Jeff Washington (jwash) b02c412d5b
generate_index straight to disk and batched (#25947)
* generate_index straight to disk and batched

* renames and comments

* handle in-mem case correctly

* use mutex
2022-06-15 18:14:39 -05:00
Jeff Washington (jwash) 631ea93259
parallel rent collection avoids overlapping ranges (#25991) 2022-06-15 16:04:38 -05:00
Tyera Eulberg 99cf1e280e
Clean up warp-timestamp features (#25994) 2022-06-15 14:11:41 -06:00
Jeff Washington (jwash) 80f90d6e96
disable logging per thread on rent collection load (#25982) 2022-06-15 13:16:00 -05:00
apfitzge e1efc5bebb
append_vec.rs: remove new_empty_map (unused) (#25885) 2022-06-15 08:59:46 -05:00
Jeff Washington (jwash) ae37359b6b
AcctIdx: only advance age on thread 0 (#25943) 2022-06-14 21:43:42 -05:00
Alexander Meißner 4652377e5b
Removes the feature gate of reject_empty_instruction_without_program. (#25870) 2022-06-14 18:00:12 +02:00
Jeff Washington (jwash) 8c6455022f
remove dead code marker (#25945) 2022-06-14 10:47:26 -05:00
Jeff Washington (jwash) e24cc537a4
avoid adding to 'uncleaned_roots' when generating index and caller passes accounts-db-skip-shrink (#25936) 2022-06-14 10:10:44 -05:00
Jeff Washington (jwash) 01f41b8c76
get rid of dead code marker (#25944) 2022-06-14 08:44:40 -05:00
Steven Luscher 45d11f3d26
Implement `VoteInstruction::AuthorizeWithSeed` & `VoteInstruction::AuthorizeWithSeedChecked` (#25928)
* [vote_authorize_with_seed] Add `VoteInstruction::AuthorizeWithSeed`

* [vote_authorize_with_seed] You can now update a vote account's authority if it's a derived key for which you control the base key

* [vote_authorize_with_seed] Add test helper to create a vote account whose authorities are derived keys

* [vote_authorize_with_seed] Write tests to assert the behavior of `VoteInstruction::AuthorizeWithSeed`

* [vote_authorize_with_seed] Feature gate the `VoteInstruction::AuthorizeWithSeed` processor

* [vote_authorize_with_seed] Add `VoteInstruction::AuthorizeWithSeed` to transaction status parser

* [vote_authorize_with_seed] Add `VoteInstruction::AuthorizeWithSeed` to docs

* [vote_authorize_with_seed] Add `VoteInstruction::AuthorizeCheckedWithSeed`

* [vote_authorize_with_seed] You can now update a vote account's authority (while checking that the new authority has signed) if it's a derived
key for which you control the base key

* [vote_authorize_with_seed] Add `VoteInstruction::AuthorizeCheckedWithSeed` to transaction status parser

* [vote_authorize_with_seed] Write tests to assert the behavior of `VoteInstruction::AuthorizeCheckedWithSeed`
2022-06-13 20:36:44 -07:00
Jeff Washington (jwash) 306faed3a8
update comment (#25949) 2022-06-13 17:43:29 -05:00
apfitzge d56a706b20
Add static vector to get all static ids in runtime (#25909)
* Add function to get all static ids in runtime

* static_ids.rs: use a lazy_static ref instead of a function
2022-06-13 13:21:16 -05:00
apfitzge 3e1fd4dad0
snapshot_utils.rs: remove unnecessary sort (#25893) 2022-06-13 11:12:33 -05:00
apfitzge 04969a2282
serialize snapshots dir first (#25889) 2022-06-13 09:33:32 -05:00
Jeff Washington (jwash) 3eff7648ff
collect rent from multiple partitions in parallel (#25774) 2022-06-13 07:53:49 -05:00
Michael Vines b4237f3f2c Do not exclude failed simple vote transactions from consensus 2022-06-12 22:11:23 -07:00
Brooks Prumo a2c180dc0b
Add EvictionsGuard to InMemAccountsIndex (#25847) 2022-06-12 17:31:58 -05:00
Brooks Prumo 53695ecd21
Remove SlotT type alias (#25908) 2022-06-10 22:04:56 -05:00
Jack May 9fb0e76dc2
cleanup feature; do_support_realloc (#25882) 2022-06-10 15:33:19 -07:00