Commit Graph

88 Commits

Author SHA1 Message Date
Brooks Prumo 8eeabf9042
Skips taking a bank snapshot for EAH requests (#28757) 2022-11-10 15:14:26 +00:00
Brooks Prumo 9e1cdc7e60
Enables not taking a bank snapshot (#28756) 2022-11-09 12:43:33 -05:00
Brooks Prumo d4c2900590
Removes `snapshot_bank()` wrapper fn (#28753) 2022-11-07 15:09:31 +00:00
Brooks Prumo 0873b90d5f
Treats all bank snapshot errors as fatal (#28752) 2022-11-07 11:03:11 +00:00
Brooks Prumo bfa62da29e
Renames variant to CalcAccountsHashDataSource::IndexForTests (#28623) 2022-10-29 08:55:33 -04:00
Brooks Prumo 2ebf248f47
Renames fn to Bank::update_accounts_hash() (#28619) 2022-10-28 17:31:16 -04:00
Ian Macalinao a2b0d0e6f0
runtime: use correct module doc comments (#27889) 2022-10-27 14:39:44 +02:00
Brooks Prumo 27269d833c
Uses enum for data source with calc_accounts_hash() (#28584) 2022-10-26 13:04:01 -04:00
Brooks Prumo f158bab0ef
Tracks how long background requests wait before processing (#28581) 2022-10-25 12:10:53 -04:00
Brooks Prumo e2438cf3e2
Renames fn to calculate_accounts_hash() (#28572) 2022-10-25 11:53:41 -04:00
Brooks Prumo 70e83e82c6
Waits to handle snapshot requests until after startup verification is complete (#28481) 2022-10-19 13:39:47 -04:00
Jeff Washington (jwash) 28a89a1d99
remove expected rent collection and rehashing completely (#28422) 2022-10-17 07:24:42 -07:00
Brooks Prumo dd7fee8f32
Re-enqueues unhandled ABS requests (#28362) 2022-10-13 16:25:39 -04:00
Brooks Prumo 9cbd00fdbc
Converts PendingAccountsPackage to a channel (#28352) 2022-10-13 12:47:36 -04:00
Brooks Prumo 020cc090d5
Refactors ABS snapshot request handling (#28326) 2022-10-11 13:17:06 -04:00
Brooks Prumo d500391006
Removes redundant accounts_package_type (#28325) 2022-10-10 17:11:38 +00:00
Brooks Prumo 5d044ccb34
abs: Update last_full_snapshot_slot before calling clean_accounts() (#27940) 2022-10-03 09:34:13 -04:00
Brooks Prumo 2f8f6c6a31
Send Epoch Accounts Hash requests from set_root() (#27764) 2022-09-30 14:59:41 -04:00
Jeff Washington (jwash) cfc124c825
acct idx can no longer use write cache (#28150) 2022-09-30 10:55:27 -07:00
Brooks Prumo 5d7130623f
abs: Snapshot requests are handled in priority-order (#27942) 2022-09-21 12:39:09 -04:00
Brooks Prumo ceac6a1094
abs: Refactor determining the accounts package type (#27944) 2022-09-20 20:31:08 -04:00
Jeff Washington (jwash) 3a47dfb8e6
add bank.bank_enable_rehashing_on_accounts_hash (#27768) 2022-09-14 11:29:23 -07:00
Brooks Prumo 6a322de845
Make Accounts Background Services aware of Epoch Accounts Hash (#27626) 2022-09-07 20:41:40 +00:00
Jeff Washington (jwash) 22007a3c96
allow accounts hash calc to specify enable_rehashing (#27615) 2022-09-07 10:16:52 -07:00
Brooks Prumo 93a4f80a2c
Handling snapshot requests is now required (#27537) 2022-09-07 10:08:42 -04:00
Haoran Yi 8f3c1e3ebb report a datapoint 2022-08-29 19:06:30 -05:00
Haoran Yi 7e722f15f2 reduce log frequence to 1m when qlen is above the threshold 2022-08-29 19:06:30 -05:00
Haoran Yi 37bba64613 remove bank drop queue stat
remove try_send
add warn! for excessively long bank drop queue
2022-08-29 19:06:30 -05:00
Haoran Yi 5b64107626 make pruned_bank channel unbonded.
In kin-sim, we found that bounded channel causes halt for account
background services. As the number of accounts grows, the time for
pruning and cleaning increases, which would leads to longer intervals
between the pruning of deaded bank slots. With 1.7B accounts, we will
exceed the 10K bounded channel threshold that causes halt of account
back ground services. Without pruning, the node will eventually run out
of memory.
2022-08-29 19:06:30 -05:00
Brooks Prumo 3c7cd62030
Move pruned_banks_receiver into PrunedBanksRequestHandler (#27445) 2022-08-29 13:30:06 -04:00
Michael Vines 3f4731b37f Standardize thread names
Tenets:
1. Limit thread names to 15 characters
2. Prefix all Solana-controlled threads with "sol"
3. Use Camel case. It's more character dense than Snake or Kebab case
2022-08-20 07:49:39 -07:00
Brooks Prumo 510d195620
Simplify `Bank::clean_accounts()` by removing params (#27254) 2022-08-19 18:15:04 -04:00
Brooks Prumo d2868f439d
Verify snapshot slot deltas (#26666) 2022-08-18 09:48:58 -04:00
steviez 300666dce7
Make `solana-ledger-tool` run AccountsBackgroundService (#26914)
Prior to this change, long running commands like `solana-ledger-tool
verify` would OOM due to AccountsDb cleanup not happening.

Co-authored-by: Michael Vines <mvines@gmail.com>
2022-08-04 15:44:31 -05:00
Jeff Washington (jwash) 817f65bb50
add full_snapshot to hash config (#26811) 2022-07-28 09:46:34 -05:00
Jeff Washington (jwash) 263911e7fd
save off what we find when calculating hash (#26663) 2022-07-19 09:55:52 -05:00
Brooks Prumo 36d13af015
Add stats for Accounts Background Service (#26298)
* Add stats for Accounts Background Service

* pr
2022-06-30 02:21:30 +00:00
Jeff Washington (jwash) 557bf6e656
allow initial hash calc to occur in bg (#26271)
* allow initial hash calc to occur in bg

* validator_initialized -> startup_verification_complete

* add infos for leader and vote

* rework snapshot for startup verification

* change to assert
2022-06-29 16:48:33 -05:00
DimAn 2fa9bc3e70
Add options to store full and/or incremental snapshots in separate locations (#24247) 2022-05-10 16:37:41 -04:00
Jeff Washington (jwash) 96259b4f23
when testing hash calc, also verify cap (#25012) 2022-05-05 15:16:15 -05:00
apfitzge 112a0b475a
Revert "Refactor to use EpochSchedule from within RentCollector struct" (#24893)
* Revert "Ran cargo fmt"

This reverts commit 9052e41b32.

* Revert "Fix build error introduced by my editor setup, part 2"

This reverts commit 4dfeab3b38.

* Revert "Fix build error introduced by my editor setup"

This reverts commit 87fb78dc56.

* Revert "Remove redundant epoch_schedule from AccountsPackage"

This reverts commit c2f7f2fff8.

* Revert "Fix a test"

This reverts commit 36c0bdaa78.

* Revert "Fixes to initial code"

This reverts commit ed7813e698.

* Revert "Removing redundant EpochSchedule param from fns"

This reverts commit 5472d2e605.
2022-05-02 13:46:17 -05:00
Michael Vines 84e3342612 Process blockstore after starting the TVU 2022-04-22 21:17:49 -07:00
Michael Vines 0e2e0c8b7d Extract most storage-related services from the Tvu abstraction 2022-04-19 19:23:58 -07:00
Jason Davis 5472d2e605 Removing redundant EpochSchedule param from fns 2022-04-18 11:57:40 -05:00
HaoranYi c6a751d658
Detect and report bank drop signal queue full and disconnect events (#24112)
* nonblocking send when when droping banks

* detect and report drop signal queue full/disconnect events

* comments

* use counter for reporting bank_drop_queue events

* reduce log

* use datapoint to report stats

* logging instead of reporting bank drop signal full

* fix a corner case for reporting

* fix build
2022-04-18 11:19:11 -05:00
HaoranYi e3ef0741be
simplify bank drop calls (#24142)
* simplify bank drop calls

* clippy: import

* Update ledger/src/blockstore_processor.rs

Co-authored-by: Brooks Prumo <brooks@prumo.org>

* Update runtime/src/accounts_background_service.rs

Co-authored-by: Brooks Prumo <brooks@prumo.org>

* Update runtime/src/bank.rs

Co-authored-by: Brooks Prumo <brooks@prumo.org>

* cleanup

* format

* use msb of bank_id to indicates that we are dropping

* clippy

* restore bank id

* clippy

* revert is-serialized_with_abs flag

* assert

* clippy

* whitespace

* fix bank drop callback check

* more fix

* remove msb dropping implementation

* fix

Co-authored-by: Brooks Prumo <brooks@prumo.org>
2022-04-14 08:43:54 -05:00
Jeff Washington (jwash) 9ac2245970
remove clone (#24244) 2022-04-11 13:15:00 -05:00
Jeff Washington (jwash) 210f6a6fab
move hash calculation out of acct bg svc (#23689)
* move hash calculation out of acct bg svc

* pr feedback
2022-04-08 10:42:03 -05:00
Brooks Prumo c322842257
Replace channel with Mutex<Option> for AccountsPackage (#24013) 2022-04-06 05:47:19 -05:00
Jeff Washington (jwash) c24de17278
remove index hash calculation as an option (#23928) 2022-03-25 15:32:53 -05:00