4ebcacb4a3
Revert "Add run parent directory for accounts files ( #29794 )" ( #29899 )
...
This PR is causing OOM on master. Reverting it for now.
This reverts commit 74f89d1494
2023-01-25 10:03:01 -08:00
74f89d1494
Add run parent directory for accounts files ( #29794 )
...
* Add run parent directory for accounts files
* fix test test_concurrent_snapshot_packaging
* review comments. renamed the path setup function
* Addressed most of the review comments
* remove explict type def for map result
* handle create_accounts_run_and_snapshot_dirs error with expect
* update with more review comments
* minor fixes from review comments
* simplify account_filename option assignment
* handle error from create_accounts_run_and_snapshot_dirs
* use then instead of then_some for lazy evaluation
2023-01-24 16:44:35 -08:00
db10073c36
Promotes accounts delta hash to its own type ( #29768 )
2023-01-20 14:23:56 -05:00
aef8692c8f
remove SnapshotStorageOne ( #29747 )
2023-01-18 07:10:56 -08:00
461dafb887
AccountStorage only holds a single append vec ( #29723 )
...
* AccountStorageReference
* tests build
* pr feedback
2023-01-16 16:22:30 -08:00
14055fc3e2
remove type SnapshotStorage and uses ( #29661 )
2023-01-11 17:57:44 -08:00
544b9745c2
snapshot storage path uses 1 append vec per slot ( #29627 )
2023-01-11 12:05:15 -08:00
4a64f6d421
test_utils::create_test_accounts pre-allocates an append vec first ( #29336 )
...
* test_utils::create_test_accounts pre-allocates an append vec first
* remove comment
2022-12-21 09:02:42 -08:00
49a136a374
move AccountStorage out of accounts_db.rs ( #29261 )
2022-12-14 15:08:39 -08:00
f355066456
remove caching_enabled from serde_snapshot ( #29246 )
...
* fix tests broken by add_root change
* remove caching_enabled from serde_snapshot
2022-12-13 16:32:43 -08:00
c8e7c706df
remove bank_from_streams(caching_enabled) ( #29235 )
2022-12-13 07:49:33 -08:00
db2cc53967
remove skip rewrites from bank ( #29182 )
2022-12-10 09:27:15 -08:00
45ba5ef6fd
remove bank_from_snapshot_archives caching_enabled ( #29171 )
2022-12-09 10:45:21 -08:00
99f0113450
remove Accounts::new_with_config_for_tests.caching_enabled ( #29169 )
2022-12-09 10:44:56 -08:00
ec5098a723
remove bank_test_config_caching_enabled ( #29170 )
2022-12-09 08:28:02 -08:00
c65a8ce6c3
AccountsDb::get_snapshot_storages() takes a Range for slots ( #29054 )
2022-12-05 11:21:21 -05:00
631d93dd2d
tests use write cache ( #29012 )
2022-12-01 12:17:13 -08:00
9327658007
Promotes accounts hash to a strong type ( #28930 )
2022-11-28 10:09:47 -05:00
04016e3bcf
Don't wait for EAH unless feature is enabled ( #28938 )
2022-11-23 09:11:28 -05:00
ff7e82b5ff
Removes Accounts::bank_hash_at() ( #28879 )
2022-11-21 13:25:48 -05:00
7787cb2ab2
Renames BankHashInfo field to accounts_delta_hash ( #28878 )
2022-11-18 15:35:03 -05:00
d798e751a0
Disables EAH with short epochs ( #28803 )
2022-11-15 13:26:19 -05:00
0bfea02056
Snapshots wait for EAH calculations to complete ( #28777 )
2022-11-14 11:34:44 -06:00
d1ba42180d
clippy for rust 1.65.0 ( #28765 )
2022-11-09 19:39:38 +00:00
37507a2de6
Removes EAH parameter from serde_snapshot::reserialize_bank() ( #28669 )
2022-10-31 09:43:17 -04:00
f98e4d9198
Renames BankHashInfo field to accounts_hash ( #28665 )
2022-10-28 15:18:25 -04:00
a8c6a9e5fc
Bank::freeze() waits for EAH calculation to complete ( #28170 )
2022-10-05 17:44:35 -04:00
765c628546
use exit signal for acct idx bg threads ( #27483 )
2022-09-12 11:51:12 -07:00
a31d4a597d
serialize epoch_accounts_hash ( #27516 )
2022-09-07 10:07:00 -07:00
49411aaae0
Feature: Interleaved Snapshot Untar and Indexing - Stage 1 ( #26590 )
...
* Add snapshot_storage_lengths_from_fields
* Add SnapshotUnpacker
* Add SnapshotStorageRebuilder
* Rebuild snapshot storage during unpack
* move snapshot unpacker back into snapshot_utils
* use SerializedAppendVecId in storage rebuilder
* create AccountStorageMap type alias
* Wrap storage and next_append_vec_id in a type
* fixed typo
* move use StorageAndNextAppendVecId
* calculate num_rebuilder_threads more concisely
2022-08-29 13:17:27 -05:00
e4a7d01e10
Rust v1.63 ( #27303 )
...
* Upgrade to Rust v1.63.0
* Add nightly_clippy_allows
* Resolve some new clippy nightly lints
* Increase QUIC packets completion timeout
* Update quinn-udp crate
Co-authored-by: Michael Vines <mvines@gmail.com>
2022-08-22 18:01:03 -07:00
7573000d87
Revert "Rust v1.63.0 ( #27148 )" ( #27245 )
...
This reverts commit a2e7bdf50a
2022-08-19 09:19:44 +01:00
a2e7bdf50a
Rust v1.63.0 ( #27148 )
...
* Upgrade to Rust v1.63.0
* Add nightly_clippy_allows
* Resolve some new clippy nightly lints
* Increase QUIC packets completion timeout
Co-authored-by: Michael Vines <mvines@gmail.com>
2022-08-17 15:48:33 -07:00
225cddcffb
serialize incremental_snapshot_hash ( #26839 )
...
* serialize incremental_snapshot_hash
* pr feedback
2022-08-17 15:14:31 -05:00
69598ed4c0
Refactor: Add `RuntimeConfig` field to Bank ( #26946 )
...
* Refactor: Simplify arguments for bank constructor methods
* Refactor: Add RuntimeConfig to Bank fields
* Arc wrap runtime_config
* Arc wrap all runtime config usages
* Remove Copy trait derivation from RuntimeConfig
* Remove some arc wrapping
2022-08-05 20:49:00 +01:00
ba785cf8ab
removes erroneous uses of std::mem::swap ( #26536 )
...
All instances should be replace by std::mem::{replace,take},
or just plain assignment.
2022-07-11 11:33:15 +00:00
17a99d98dd
Revert "avoid adding to 'uncleaned_roots' when generating index and c… ( #26441 )
...
Revert "avoid adding to 'uncleaned_roots' when generating index and caller passes accounts-db-skip-shrink (#25936 )"
This reverts commit e24cc537a4
2022-07-06 11:32:45 -05:00
877fedadac
Remove StatusCacheRc type and use StatusCache directly ( #26184 )
2022-06-24 08:38:56 -05:00
e24cc537a4
avoid adding to 'uncleaned_roots' when generating index and caller passes accounts-db-skip-shrink ( #25936 )
2022-06-14 10:10:44 -05:00
8caced68ce
Serialize lamports per signature ( #25364 )
...
* Serialize lamports per signature
* Add full snapshot archive test, enable features in previous tests
2022-06-06 13:07:02 -07:00
6c10515ccf
Revert "Serialize lamports per signature in snapshots ( #25181 )" ( #25228 )
...
This reverts commit 35d2a0fd69
2022-05-15 12:04:17 +00:00
35d2a0fd69
Serialize lamports per signature in snapshots ( #25181 )
...
* Serialize lamports per signature
* pr comments
2022-05-14 10:57:51 -07:00
b4491ff4ba
adds StakesEnum type representing Stakes<StakeAccount|Delegation>
...
For backward compatibility, we can only serialize and deserialize
Stakes<Delegation>. However Bank caches Stakes<StakeAccount>. This type
mismatch incurs a conversion cost at epoch boundary when updating
EpochStakes.
This commit adds StakesEnum which allows EpochStakes to include either a
Stakes<StakeAccount> or Stakes<Delegation> and so bypass the conversion
cost between the two at the epoch boundary.
2022-04-21 15:28:41 +00:00
454ef38e43
caches StakeAccount instead of Delegation in Stakes
...
The commit makes values in stake_delegations map in Stakes struct
generic. Stakes<Delegation> is equivalent to the old code and is used
for backward compatibility in BankFieldsTo{Serialize,Deserialize}.
But banks cache Stakes<StakeAccount> which includes the entire stake
account and StakeState deserialized from account. Doing so, will remove
the need to load stake account from accounts-db when working with
stake-delegations.
2022-04-21 15:28:41 +00:00
79923c3b58
Refactor: Rename BlockhashQueue fields and methods for clarity ( #24426 )
2022-04-21 11:57:17 +08:00
c4923d29b4
bank_hash_at uses rewrites ( #24439 )
2022-04-19 11:29:29 -05:00
605036c117
move test fn into its own mod ( #24212 )
...
* move test fn into its own mod
* pub
2022-04-12 09:36:05 -05:00
c27150b1a3
reserialize_bank_fields_with_hash ( #23916 )
...
* reserialize_bank_with_new_accounts_hash
* Update runtime/src/serde_snapshot.rs
Co-authored-by: Brooks Prumo <brooks@prumo.org>
* Update runtime/src/serde_snapshot/tests.rs
Co-authored-by: Brooks Prumo <brooks@prumo.org>
* Update runtime/src/serde_snapshot/tests.rs
Co-authored-by: Brooks Prumo <brooks@prumo.org>
* pr feedback
Co-authored-by: Brooks Prumo <brooks@prumo.org>
2022-04-07 14:05:57 -05:00
6a7f6585ce
persist historical_roots ( #24029 )
2022-04-04 13:13:11 -05:00
396b49a7c1
Start saving/loading prior_roots(_with_hash) to snapshot ( #23844 )
...
* Start saving/loading prior_roots(_with_hash) to snapshot
* Update runtime/src/accounts_index.rs
Co-authored-by: Michael Vines <mvines@gmail.com>
* Update runtime/src/accounts_index.rs
Co-authored-by: Michael Vines <mvines@gmail.com>
* update comment
Co-authored-by: Michael Vines <mvines@gmail.com>
2022-03-24 10:06:24 -05:00
Xiang Zhu
Brooks
Jeff Washington (jwash)
apfitzge
Brennan Watt
Jeff Washington (jwash)
Justin Starry
behzad nouri
Ashwin Sekar
Justin Starry
HaoranYi