Brooks Prumo
0bfea02056
Snapshots wait for EAH calculations to complete ( #28777 )
2022-11-14 11:34:44 -06:00
Brooks Prumo
d1ba42180d
clippy for rust 1.65.0 ( #28765 )
2022-11-09 19:39:38 +00:00
Brooks Prumo
9e1cdc7e60
Enables not taking a bank snapshot ( #28756 )
2022-11-09 12:43:33 -05:00
Brooks Prumo
d4c2900590
Removes `snapshot_bank()` wrapper fn ( #28753 )
2022-11-07 15:09:31 +00:00
Brooks Prumo
59bf1809fe
Uses SnapshotHash type in snapshot archive fields ( #28681 )
2022-10-31 14:28:35 -04:00
Brooks Prumo
37507a2de6
Removes EAH parameter from serde_snapshot::reserialize_bank() ( #28669 )
2022-10-31 09:43:17 -04:00
Brooks Prumo
5a3d252899
Renames fn to Bank::update_accounts_hash_for_tests() ( #28620 )
2022-10-28 14:33:05 -04:00
Brooks Prumo
12f3e8c9cc
Ignores errors when joining background threads in snapshot tests ( #28480 )
2022-10-19 16:54:59 -04:00
Brooks Prumo
dd7fee8f32
Re-enqueues unhandled ABS requests ( #28362 )
2022-10-13 16:25:39 -04:00
Brooks Prumo
9cbd00fdbc
Converts PendingAccountsPackage to a channel ( #28352 )
2022-10-13 12:47:36 -04:00
Brooks Prumo
a8c6a9e5fc
Bank::freeze() waits for EAH calculation to complete ( #28170 )
2022-10-05 17:44:35 -04:00
Brooks Prumo
2f8f6c6a31
Send Epoch Accounts Hash requests from set_root() ( #27764 )
2022-09-30 14:59:41 -04:00
behzad nouri
9a57c64f21
patches clippy errors from new rust nightly release ( #27996 )
2022-09-22 22:23:03 +00:00
Brooks Prumo
ff71df4695
Remove unnecessary call to `set_startup_verification_complete()` ( #27986 )
2022-09-22 16:54:17 -04:00
Brooks Prumo
1ee595ca9c
remove AccountsDb::initial_blockstore_processing_complete ( #27974 )
2022-09-22 13:52:04 -04:00
Jeff Washington (jwash)
f2d6a7ecea
bank.initial_blockstore_processing_complete to avoid concurrent hash calculations ( #27776 )
...
* bank.initial_blockstore_processing_complete to avoid concurrent hash calculations
* Update runtime/src/bank.rs
Co-authored-by: Brooks Prumo <brooks@prumo.org>
* Update runtime/src/bank.rs
Co-authored-by: Brooks Prumo <brooks@prumo.org>
* Rename TestValidator::set_startup_verification_complete()
* Initialize with `AtomicBool::new(false)` instead of `default()`
* snapshot tests: move where `initial_blockstore_processing_completed()` is called
* fixup bank_forks.rs calling `is_initial_blockstore_processing_complete()`
* only call initial_blockstore_processing_completed() in blockstore_processor
Co-authored-by: Brooks Prumo <brooks@prumo.org>
Co-authored-by: Brooks Prumo <brooks@solana.com>
2022-09-19 13:00:21 -07:00
Jeff Washington (jwash)
765c628546
use exit signal for acct idx bg threads ( #27483 )
2022-09-12 11:51:12 -07:00
Jeff Washington (jwash)
abd01553d5
tests: Keypair::new().pubkey() -> pubkey::new_rand ( #27705 )
2022-09-10 13:56:45 -07:00
Brooks Prumo
6a322de845
Make Accounts Background Services aware of Epoch Accounts Hash ( #27626 )
2022-09-07 20:41:40 +00:00
Jeff Washington (jwash)
a31d4a597d
serialize epoch_accounts_hash ( #27516 )
2022-09-07 10:07:00 -07:00
Brooks Prumo
93a4f80a2c
Handling snapshot requests is now required ( #27537 )
2022-09-07 10:08:42 -04:00
Brooks Prumo
3c7cd62030
Move pruned_banks_receiver into PrunedBanksRequestHandler ( #27445 )
2022-08-29 13:30:06 -04:00
Jeff Washington (jwash)
225cddcffb
serialize incremental_snapshot_hash ( #26839 )
...
* serialize incremental_snapshot_hash
* pr feedback
2022-08-17 15:14:31 -05:00
Justin Starry
69598ed4c0
Refactor: Add `RuntimeConfig` field to Bank ( #26946 )
...
* Refactor: Simplify arguments for bank constructor methods
* Refactor: Add RuntimeConfig to Bank fields
* Arc wrap runtime_config
* Arc wrap all runtime config usages
* Remove Copy trait derivation from RuntimeConfig
* Remove some arc wrapping
2022-08-05 20:49:00 +01:00
Jeff Washington (jwash)
47716a5e01
async hash verify on load ( #26208 )
...
* verify accounts hash in bg on startup
* fix some tests and loading from genesis
* add extra state for when background thread has completed
2022-07-15 14:29:56 -05:00
Jeff Washington (jwash)
602da5e51f
add accounts db config to bank tests ( #26517 )
2022-07-10 19:42:06 -05:00
Brooks Prumo
9ec38a3191
Cleanup snapshot integration tests ( #26390 )
2022-07-05 09:23:23 -05:00
Brooks Prumo
877fedadac
Remove StatusCacheRc type and use StatusCache directly ( #26184 )
2022-06-24 08:38:56 -05:00
Brooks Prumo
23c50a2389
Add StatusCache::root_slot_deltas() and use it ( #26170 )
2022-06-23 15:19:06 -05:00
DimAn
2fa9bc3e70
Add options to store full and/or incremental snapshots in separate locations ( #24247 )
2022-05-10 16:37:41 -04:00
Justin Starry
4e58b3870c
Update all BankForks methods to return owned values ( #24801 )
2022-04-28 18:51:00 +00:00
behzad nouri
454ef38e43
caches StakeAccount instead of Delegation in Stakes
...
The commit makes values in stake_delegations map in Stakes struct
generic. Stakes<Delegation> is equivalent to the old code and is used
for backward compatibility in BankFieldsTo{Serialize,Deserialize}.
But banks cache Stakes<StakeAccount> which includes the entire stake
account and StakeState deserialized from account. Doing so, will remove
the need to load stake account from accounts-db when working with
stake-delegations.
2022-04-21 15:28:41 +00:00
Michael Vines
0e2e0c8b7d
Extract most storage-related services from the Tvu abstraction
2022-04-19 19:23:58 -07:00
Jeff Washington (jwash)
210f6a6fab
move hash calculation out of acct bg svc ( #23689 )
...
* move hash calculation out of acct bg svc
* pr feedback
2022-04-08 10:42:03 -05:00
Jeff Washington (jwash)
c27150b1a3
reserialize_bank_fields_with_hash ( #23916 )
...
* reserialize_bank_with_new_accounts_hash
* Update runtime/src/serde_snapshot.rs
Co-authored-by: Brooks Prumo <brooks@prumo.org>
* Update runtime/src/serde_snapshot/tests.rs
Co-authored-by: Brooks Prumo <brooks@prumo.org>
* Update runtime/src/serde_snapshot/tests.rs
Co-authored-by: Brooks Prumo <brooks@prumo.org>
* pr feedback
Co-authored-by: Brooks Prumo <brooks@prumo.org>
2022-04-07 14:05:57 -05:00
Jeff Washington (jwash)
550ca7bf92
compare contents of serialized banks instead of exact file format ( #24141 )
...
* compare contents of serialized banks instead of exact file format
* Update runtime/src/snapshot_utils.rs
Co-authored-by: Brooks Prumo <brooks@prumo.org>
* Update runtime/src/snapshot_utils.rs
Co-authored-by: Brooks Prumo <brooks@prumo.org>
* pr feedback
* get rid of clone
* pr feedback
Co-authored-by: Brooks Prumo <brooks@prumo.org>
2022-04-06 21:55:44 -05:00
Jeff Washington (jwash)
fddd162645
reserialize bank in ahv by first writing to temp file in abs ( #23947 )
2022-04-06 21:39:26 -05:00
Brooks Prumo
c322842257
Replace channel with Mutex<Option> for AccountsPackage ( #24013 )
2022-04-06 05:47:19 -05:00
Jeff Washington (jwash)
c24de17278
remove index hash calculation as an option ( #23928 )
2022-03-25 15:32:53 -05:00
Jeff Washington (jwash)
db5d68f01f
HashCalc uses self.accounts_hash_cache_path ( #23882 )
2022-03-24 09:31:55 -05:00
Brooks Prumo
9bbccbe27c
Use `AsRef<Path>` instead of `PathBuf` for parameters ( #23560 )
2022-03-09 16:08:33 -06:00
HaoranYi
a1c45d5acb
typo ( #23564 )
2022-03-09 15:11:21 -06:00
HaoranYi
181fffb916
rename status filename to be consistent ( #23501 )
2022-03-07 17:34:35 +00:00
Jeff Biseda
8b66625c95
convert std::sync::mpsc to crossbeam_channel ( #22264 )
2022-01-11 02:44:46 -08:00
Michael Vines
b8837c04ec
Reformat imports to a consistent style for imports
...
rustfmt.toml configuration:
imports_granularity = "One"
group_imports = "One"
2021-12-03 09:19:13 -08:00
Michael Vines
ba9dfa0d22
Remove frozen account support
2021-11-29 08:38:11 -08:00
Jeff Washington (jwash)
95e91a4863
disable gossip publish of snapshots when using filler accts ( #20824 )
2021-10-20 18:07:29 +00:00
Lijun Wang
fe97cb2ddf
AccountsDb plugin framework ( #20047 )
...
Summary of Changes
Create a plugin mechanism in the accounts update path so that accounts data can be streamed out to external data stores (be it Kafka or Postgres). The plugin mechanism allows
Data stores of connection strings/credentials to be configured,
Accounts with patterns to be streamed
PostgreSQL implementation of the streaming for different destination stores to be plugged in.
The code comprises 4 major parts:
accountsdb-plugin-intf: defines the plugin interface which concrete plugin should implement.
accountsdb-plugin-manager: manages the load/unload of plugins and provide interfaces which the validator can notify of accounts update to plugins.
accountsdb-plugin-postgres: the concrete plugin implementation for PostgreSQL
The validator integrations: updated streamed right after snapshot restore and after account update from transaction processing or other real updates.
The plugin is optionally loaded on demand by new validator CLI argument -- there is no impact if the plugin is not loaded.
2021-09-30 14:26:17 -07:00
Jeff Washington (jwash)
b57e86abf2
cache account hash info ( #19426 )
...
* cache account hash info
* ledger_path -> accounts_hash_cache_path
2021-09-13 20:39:26 -05:00
Brooks Prumo
62c8bcf565
Add default() to SnapshotConfig ( #19776 )
2021-09-12 13:44:27 -05:00