* AccountSecondaryIndexes.include/exclude
* use normal scan if key is not indexed
* add a test to ask for a scan for an excluded secondary index
* fix up cli args
In a scenario where a bank is unrooted and dropped, any keys that exist
_only_ in that bank are now cleaned up.
This work was originally based on PR #15106.
* stretchy roots tracker
* rename hash to hash_set in tests
* update comment
* try 2 widths in test
* bool iter
* add assert
* helper function for bitfield insert/remove
* introduce RollingBitFieldTester
* another bool iter replacement
* map cleanup
* map to cloned
* Account for possibility of cache flush in load()
* More cleaning
* More cleaning
* Remove unused method and some comment cleaning
* Fix typo
* Make the detected impossible purge race panic()!
* Finally revert to original .expect()
* Fix typos...
* Add assertion for max_root for easier reasoning
* Reframe races with LoadHint as possible opt.
* Fix test
* Make race bug tests run longer for less flaky
* Delay the clone-in-lock slow path even for RPC
* Make get_account panic-free & add its onchain ver.
* Fix rebase conflicts...
* Clean up
* Clean up comment
* Revert fn name change
* Fix flaky test...
* fmt...
Co-authored-by: Ryo Onodera <ryoqun@gmail.com>
* Replace RootsTracker HashSet
* use bitvec
* cleanup, add brenchmark test
* test cleanup
* add lots of tests
* get rid of demo
* change warp test constant
* get rid of unused function
* pr feedback
* reorder use
* rework get_all to remove range checks
* add tests, fix bugs
Clean less keys by tracking the two cases:
* Touched keys per slot in uncleaned_keys derived from
accounts delta hash operation.
* Set of keys with any zero-lamport updates.
* Dont skip eager rent collect across gapped epochs
* Adjust style and comment
* Adjust ascii chart and comment a bit
* Moar assert
* Relax the partition_count assert for completeness
* Tweak comment...
* tweak a bit
* Add gating logic
* Address reviews
* small formatting
* Clarify the code by replacing auto_generated...
* small formatting
* small formatting
* small formatting
* small formatting
* Narrow down conditional compilation scope
* Prioritize shrinking of previously cleaned roots
* measure time of stale slot shrinking
* Disable shrink for test
* shrink: budgeting, store count, force for snapshot
* Polish implementation and fix tests
* Fix ci..
* Clean up a bit
* Further polish implementation and fix/add tests
* Rebase fixes
* Remove unneeded Default for AccountStorageEntry
* Address review comments
* More cleanup
* More cleanup
* Switch AccountsIndex.account_maps from HashMap to BTreeMap
* Introduce eager rent collection
* Start to add tests
* Avoid too short eager rent collection cycles
* Add more tests
* Add more tests...
* Refacotr!!!!!!
* Refactoring follow up
* More tiny cleanups
* Don't rewrite 0-lamport accounts to be deterministic
* Refactor a bit
* Do hard fork, restore tests, and perf. mitigation
* Fix build...
* Refactor and add switch over for testnet (TdS)
* Use to_be_bytes
* cleanup
* More tiny cleanup
* Rebase cleanup
* Set Bank::genesis_hash when resuming from snapshot
* Reorder fns and clean ups
* Better naming and commenting
* Yet more naming clarifications
* Make prefix width strictly uniform for 2-base partition_count
* Fix typo...
* Revert cluster-dependent gate
* kick ci?
* kick ci?
* kick ci?
* Do periodic inbound compaction for rooted slots
* Add comment
* nits
* Consider not_compacted_roots in cleanup_dead_slot
* Renames in AccountsIndex
* Rename to reflect expansion of removed accounts
* Fix a comment
* rename
* Parallelize clean over AccountsIndex
* Some niceties
* Reduce locks and real chunked parallelism
* Measure each step for sampling opportunities
* Just noticed par iter is maybe lazy
* Replace storage scan with optimized index scan
* Various clean-ups
* Clear uncleared_roots even if no updates
* credit_only_credits_forwarding
* whack transfer_now()
* fixup
* bench should retry the airdrop TX
* fixup
* try to make bench-exchange a bit more robust, informative
* accounts_index: RwLock per-account
Lots of lock contention on the accounts_index lock,
only take write-lock on accounts_index if we need to insert/remove an
account.
For updates, take a read-lock and then write-lock on the individual
account.
* Remove unneeded enumerate and add comments.
* Add accounts_index bench
* Don't take the accounts index lock unless needed
* Accounts_index remove insert return vec and add capacity stats
* Use hashbrown hashmap for accounts_index
* Be able to create bank snapshots
* fix clippy
* load snapshot on start
* regenerate account index from the storage
* Remove rc feature dependency
* cleanup
* save snapshot for slot 0
* Add optional depth parameter to pubsub, and store in subscriptions
* Pass bank_forks into rpc_subscription; add method to check depth before notify and impl for account subscriptions
* Impl check-depth for signature subscriptions
* Impl check-depth for program subscriptions
* Plumb fork id through accounts
* Use fork id and root to prevent repeated account notifications; also s/Depth/Confirmations
* Write tests in terms of bank_forks
* Fixup accounts tests
* Add pubsub-confirmations tests
* Update pubsub documentation
* Revert "Revert "account storage is not in sync with the index after gc (#3914)" (#3936)"
This reverts commit 4f47fc00bc.
* fix get_exclusive_storage
* clippy
* account storage is not in sync with the index after gc
* builds
* clippy fmt
* test
* purge dead forks on store
* rm println
* also fixed count_stores
* comments
Jeff Washington (jwash)
carllin
Brooks Prumo
Michael Vines
Greg Fitzgerald
carllin
sakridge
Ryo Onodera
joshua-maros
Kristofer Peterson
Ryo Onodera
Tyera Eulberg
François Garillot
Jack May
sakridge
Rob Walker
Sagar Dhawan
Sathish
Sathish Ambley
Tyera Eulberg
anatoly yakovenko
Pankaj Garg
anatoly yakovenko