12f3e8c9cc
Ignores errors when joining background threads in snapshot tests ( #28480 )
2022-10-19 16:54:59 -04:00
d81d2bba59
comments out print in test ( #28475 )
2022-10-19 10:25:11 -05:00
1cc9cf927c
Supports warping with Epoch Accounts Hash ( #28459 )
2022-10-19 10:37:14 -04:00
28a89a1d99
remove expected rent collection and rehashing completely ( #28422 )
2022-10-17 07:24:42 -07:00
31c2b29941
Sends both an EAH and a snapshot request from `set_root()` ( #28363 )
2022-10-14 11:00:04 -04:00
dd7fee8f32
Re-enqueues unhandled ABS requests ( #28362 )
2022-10-13 16:25:39 -04:00
9cbd00fdbc
Converts PendingAccountsPackage to a channel ( #28352 )
2022-10-13 12:47:36 -04:00
5a08eed82d
Cleans up debugging code in EAH tests ( #28324 )
2022-10-10 16:07:55 +00:00
27cd2c324e
Adds tests for EAH and snapshot interactions ( #28304 )
2022-10-10 10:16:13 -04:00
981c9d07a4
Rearranges eah TestEnvironment fields to ensure drop order ( #28270 )
2022-10-06 16:17:32 -04:00
2d936784dd
Ignore errors when joining background threads for EAH tests ( #28263 )
2022-10-06 18:43:56 +00:00
a8c6a9e5fc
Bank::freeze() waits for EAH calculation to complete ( #28170 )
2022-10-05 17:44:35 -04:00
6b17bee5a8
Remove the const default for RocksFifo ( #27965 )
...
#### Summary of Changes
Removes the constant default for ShredStorageType::RocksFifo
as the shred storage size is either user-specified or derived
from --limit-ledger-size in #27459 .
2022-10-01 15:10:54 -07:00
8877ac2aa9
Fix call to calculate_accounts_hash() ( #28169 )
2022-09-30 15:29:18 -04:00
2f8f6c6a31
Send Epoch Accounts Hash requests from set_root() ( #27764 )
2022-09-30 14:59:41 -04:00
97c9af4c6b
plumbs through flag to generate merkle variant of shreds
2022-09-23 16:45:18 +00:00
9a57c64f21
patches clippy errors from new rust nightly release ( #27996 )
2022-09-22 22:23:03 +00:00
ff71df4695
Remove unnecessary call to `set_startup_verification_complete()` ( #27986 )
2022-09-22 16:54:17 -04:00
1ee595ca9c
remove AccountsDb::initial_blockstore_processing_complete ( #27974 )
2022-09-22 13:52:04 -04:00
f2d6a7ecea
bank.initial_blockstore_processing_complete to avoid concurrent hash calculations ( #27776 )
...
* bank.initial_blockstore_processing_complete to avoid concurrent hash calculations
* Update runtime/src/bank.rs
Co-authored-by: Brooks Prumo <brooks@prumo.org>
* Update runtime/src/bank.rs
Co-authored-by: Brooks Prumo <brooks@prumo.org>
* Rename TestValidator::set_startup_verification_complete()
* Initialize with `AtomicBool::new(false)` instead of `default()`
* snapshot tests: move where `initial_blockstore_processing_completed()` is called
* fixup bank_forks.rs calling `is_initial_blockstore_processing_complete()`
* only call initial_blockstore_processing_completed() in blockstore_processor
Co-authored-by: Brooks Prumo <brooks@prumo.org>
Co-authored-by: Brooks Prumo <brooks@solana.com>
2022-09-19 13:00:21 -07:00
765c628546
use exit signal for acct idx bg threads ( #27483 )
2022-09-12 11:51:12 -07:00
abd01553d5
tests: Keypair::new().pubkey() -> pubkey::new_rand ( #27705 )
2022-09-10 13:56:45 -07:00
6a322de845
Make Accounts Background Services aware of Epoch Accounts Hash ( #27626 )
2022-09-07 20:41:40 +00:00
a31d4a597d
serialize epoch_accounts_hash ( #27516 )
2022-09-07 10:07:00 -07:00
93a4f80a2c
Handling snapshot requests is now required ( #27537 )
2022-09-07 10:08:42 -04:00
3c7cd62030
Move pruned_banks_receiver into PrunedBanksRequestHandler ( #27445 )
2022-08-29 13:30:06 -04:00
225cddcffb
serialize incremental_snapshot_hash ( #26839 )
...
* serialize incremental_snapshot_hash
* pr feedback
2022-08-17 15:14:31 -05:00
69598ed4c0
Refactor: Add `RuntimeConfig` field to Bank ( #26946 )
...
* Refactor: Simplify arguments for bank constructor methods
* Refactor: Add RuntimeConfig to Bank fields
* Arc wrap runtime_config
* Arc wrap all runtime config usages
* Remove Copy trait derivation from RuntimeConfig
* Remove some arc wrapping
2022-08-05 20:49:00 +01:00
47716a5e01
async hash verify on load ( #26208 )
...
* verify accounts hash in bg on startup
* fix some tests and loading from genesis
* add extra state for when background thread has completed
2022-07-15 14:29:56 -05:00
602da5e51f
add accounts db config to bank tests ( #26517 )
2022-07-10 19:42:06 -05:00
9ec38a3191
Cleanup snapshot integration tests ( #26390 )
2022-07-05 09:23:23 -05:00
877fedadac
Remove StatusCacheRc type and use StatusCache directly ( #26184 )
2022-06-24 08:38:56 -05:00
23c50a2389
Add StatusCache::root_slot_deltas() and use it ( #26170 )
2022-06-23 15:19:06 -05:00
5b67960c76
(Refactor) Move blocktore options related stuff to blockstore_options.rs ( #25509 )
...
#### Problem
blockstore_db.rs has a mutual dependency between blockstore_metrics.rs.
#### Summary of Changes
This PR removes the mutual dependency by moving the option-related stuff
out from blockstore_db.rs to its new home --- blockstore_options.rs.
By doing this, we address the mutual dependency and also make the code cleaner.
2022-05-26 16:59:26 -07:00
2fa9bc3e70
Add options to store full and/or incremental snapshots in separate locations ( #24247 )
2022-05-10 16:37:41 -04:00
4e58b3870c
Update all BankForks methods to return owned values ( #24801 )
2022-04-28 18:51:00 +00:00
454ef38e43
caches StakeAccount instead of Delegation in Stakes
...
The commit makes values in stake_delegations map in Stakes struct
generic. Stakes<Delegation> is equivalent to the old code and is used
for backward compatibility in BankFieldsTo{Serialize,Deserialize}.
But banks cache Stakes<StakeAccount> which includes the entire stake
account and StakeState deserialized from account. Doing so, will remove
the need to load stake account from accounts-db when working with
stake-delegations.
2022-04-21 15:28:41 +00:00
0e2e0c8b7d
Extract most storage-related services from the Tvu abstraction
2022-04-19 19:23:58 -07:00
210f6a6fab
move hash calculation out of acct bg svc ( #23689 )
...
* move hash calculation out of acct bg svc
* pr feedback
2022-04-08 10:42:03 -05:00
c27150b1a3
reserialize_bank_fields_with_hash ( #23916 )
...
* reserialize_bank_with_new_accounts_hash
* Update runtime/src/serde_snapshot.rs
Co-authored-by: Brooks Prumo <brooks@prumo.org>
* Update runtime/src/serde_snapshot/tests.rs
Co-authored-by: Brooks Prumo <brooks@prumo.org>
* Update runtime/src/serde_snapshot/tests.rs
Co-authored-by: Brooks Prumo <brooks@prumo.org>
* pr feedback
Co-authored-by: Brooks Prumo <brooks@prumo.org>
2022-04-07 14:05:57 -05:00
550ca7bf92
compare contents of serialized banks instead of exact file format ( #24141 )
...
* compare contents of serialized banks instead of exact file format
* Update runtime/src/snapshot_utils.rs
Co-authored-by: Brooks Prumo <brooks@prumo.org>
* Update runtime/src/snapshot_utils.rs
Co-authored-by: Brooks Prumo <brooks@prumo.org>
* pr feedback
* get rid of clone
* pr feedback
Co-authored-by: Brooks Prumo <brooks@prumo.org>
2022-04-06 21:55:44 -05:00
fddd162645
reserialize bank in ahv by first writing to temp file in abs ( #23947 )
2022-04-06 21:39:26 -05:00
c322842257
Replace channel with Mutex<Option> for AccountsPackage ( #24013 )
2022-04-06 05:47:19 -05:00
c24de17278
remove index hash calculation as an option ( #23928 )
2022-03-25 15:32:53 -05:00
db5d68f01f
HashCalc uses self.accounts_hash_cache_path ( #23882 )
2022-03-24 09:31:55 -05:00
ae75b1a25f
(LedgerStore) Add compression type ( #23578 )
...
This PR adds `--rocksdb-ledger-compression` as a hidden argument to the validator
for specifying the compression algorithm for TransactionStatus. Available compression
algorithms include `lz4`, `snappy`, `zlib`. The default value is `none`.
Experimental results show that with lz4 compression, we can achieve ~37% size-reduction
on the TransactionStatus column family, or ~8% size-reduction of the ledger store size.
2022-03-22 02:27:09 -07:00
f999eef452
(LedgerStore) Rename BlockstoreAdvancedOptions to LedgerColumnOptions ( #23764 )
...
This PR renames BlockstoreAdvancedOptions to LedgerColumnOptions, as we will
pass-down this struct to LedgerColumn to allow it to perform metric reporting.
2022-03-18 11:13:35 -07:00
1e20bd8f9a
(LedgerStore) Include storage type as a tag in RocksDB metric reporting ( #23523 )
...
#### Summary of Changes
This PR further enables group by operation on storage type in blockstore_rocksdb_cfs metrics.
Such group-by allows us to further compare the performance metrics between rocks-level and
rocks-fifo.
To make things extensible, this PR introduces BlockstoreAdvancedOptions and move shred_storage_type.
All fields in BlockstoreAdvancedOptions will support group-by operation in blockstore_rocksdb_cfs.
Dependency: #23580
2022-03-11 15:17:34 -08:00
9bbccbe27c
Use `AsRef<Path>` instead of `PathBuf` for parameters ( #23560 )
2022-03-09 16:08:33 -06:00
a1c45d5acb
typo ( #23564 )
2022-03-09 15:11:21 -06:00
Brooks Prumo
HaoranYi
Jeff Washington (jwash)
Yueh-Hsuan Chiang
behzad nouri
Jeff Washington (jwash)
Justin Starry
DimAn
Michael Vines