Commit Graph

3737 Commits

Author SHA1 Message Date
Jeff Washington (jwash) 19f13a0bea
refactor get_store_for_shrink (#29284) 2022-12-15 22:10:39 -08:00
Brooks Prumo 1e0a0e0ced
Cleans up CacheHashData (#29267) 2022-12-15 14:16:27 -05:00
Jeff Washington (jwash) 5a687fa818
make AccountStorage::map private (#29268) 2022-12-14 20:03:25 -08:00
Jeff Washington (jwash) 3a7b67ea55
refactor unwrap_or_default for type that will soon not have a default (#29257) 2022-12-14 15:11:36 -08:00
Jeff Washington (jwash) 49a136a374
move AccountStorage out of accounts_db.rs (#29261) 2022-12-14 15:08:39 -08:00
Jeff Washington (jwash) f9db75768c
Vec<Arc<AccountStorageEntry>> becomes SnapshotStorage (#29253) 2022-12-14 12:18:43 -08:00
Jeff Washington (jwash) fab0ce21a2
remove shrink_candidate_slots_v1 (#29252) 2022-12-14 08:57:16 -08:00
Jeff Washington (jwash) b77bef4ee3
write_version -> write_version_obsolete (#29176) 2022-12-14 07:43:40 -08:00
Jeff Washington (jwash) b8e25282c2
remove rebuild_bank_from_snapshots(caching_enabled) (#29247)
* fix tests broken by add_root change

* remove rebuild_bank_from_snapshots(caching_enabled)
2022-12-13 16:33:06 -08:00
Jeff Washington (jwash) f355066456
remove caching_enabled from serde_snapshot (#29246)
* fix tests broken by add_root change

* remove caching_enabled from serde_snapshot
2022-12-13 16:32:43 -08:00
Jeff Washington (jwash) 62be54a75e
remove process_stale_slot_v1 and other fns (#29244) 2022-12-13 16:32:24 -08:00
Jeff Washington (jwash) 4ae12bf5fa
fix tests broken by add_root change (#29248) 2022-12-13 14:43:52 -08:00
Jeff Washington (jwash) 250ec3f2a4
improve write_version usage in geyser scan from snapshot (#29223) 2022-12-13 14:40:12 -08:00
Alexander Meißner 721496b900
Feature - Program redeployment cooldown (#29136)
* Registers the feature enable_program_redeployment_cooldown.

* Adds redeployment slot constraint.

* Adds test to assert that closed programs can not be reopened.

* Ensure that program close truncates the account data.

* Adds set_sysvar_for_tests() to SBF program tests.
2022-12-13 22:54:24 +01:00
Jeff Washington (jwash) 3e649d2aa0
remove add_root(caching_enabled) (#29245) 2022-12-13 11:26:28 -08:00
Jeff Washington (jwash) 8da82e68fc
remove handle_snapshot_request(caching_enabled) (#29241) 2022-12-13 09:31:47 -08:00
dependabot[bot] dca5d7f9b4
chore: bump test-case from 2.1.0 to 2.2.2 (#28184)
Bumps [test-case](https://github.com/frondeus/test-case) from 2.1.0 to 2.2.2.
- [Release notes](https://github.com/frondeus/test-case/releases)
- [Changelog](https://github.com/frondeus/test-case/blob/master/CHANGELOG.md)
- [Commits](https://github.com/frondeus/test-case/compare/v2.1.0...v2.2.2)

---
updated-dependencies:
- dependency-name: test-case
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-12-13 16:07:50 +00:00
Jeff Washington (jwash) 1a0a465416
handle unclean roots explicitly during index generation (#29230) 2022-12-13 07:51:33 -08:00
Jeff Washington (jwash) b779134020
remove process_stale_slot_with_budget (#29236) 2022-12-13 07:50:14 -08:00
Jeff Washington (jwash) c8e7c706df
remove bank_from_streams(caching_enabled) (#29235) 2022-12-13 07:49:33 -08:00
Brooks Prumo bcbf3374fd
Avoids copying hashes when computing merkle root (#29225) 2022-12-13 10:20:14 -05:00
Jeff Washington (jwash) e1c6de9ef4
remove handle_snapshot_requests(caching_enabled) (#29232) 2022-12-13 07:19:45 -08:00
Jeff Washington (jwash) 7034f4cbbe
remove new_with_config_for_benches(caching_enabled) (#29231) 2022-12-13 07:19:24 -08:00
Jeff Washington (jwash) 5983363b03
remove test_store_scan_consistency(caching_enabled) (#29233) 2022-12-13 07:18:55 -08:00
Jeff Washington (jwash) b95835143e
remove AccountsBackgroundService::new(caching_enabled) (#29234) 2022-12-13 07:18:02 -08:00
Jeff Washington (jwash) 3b4c27b727
remove some if accounts_db_caching_enabled { (#29237) 2022-12-13 07:17:14 -08:00
Jeff Washington (jwash) bb0bfc4214
remove bank_from_latest_snapshot_archives(caching_enabled) (#29238) 2022-12-13 07:16:24 -08:00
Jeff Washington (jwash) a57247a78e
handle write_version correctly in geyser api (#29224) 2022-12-13 06:42:28 -08:00
Jeff Washington (jwash) c3001abef7
don't rely on write_version increasing - just assert that it is for now (#29180) 2022-12-12 19:24:45 -08:00
Jeff Washington (jwash) ceeec9d625
remove enable_caching from Bank::new_with_config_for_tests (#29215) 2022-12-12 15:33:12 -08:00
Jeff Washington (jwash) 2c2324f4ea
remove caching_enabled from Bank::new_with_paths_for_tests (#29214) 2022-12-12 15:30:46 -08:00
Jeff Washington (jwash) dd4393e46d
remove accounts_caching from Bank::new_with_paths (#29213) 2022-12-12 15:18:12 -08:00
Jeff Washington (jwash) 536b879aa7
remove accounts.new_with_config_for_tests.caching_enabled (#29168) 2022-12-12 11:31:51 -08:00
Jeff Washington (jwash) 7b1d00ee80
remove dead just_rewrites (#29188) 2022-12-12 10:04:16 -08:00
Jeff Washington (jwash) 5e799ad563
remove AccountsDb::new_with_config.caching_enabled (#29167) 2022-12-12 08:50:00 -08:00
Jeff Washington (jwash) db2cc53967
remove skip rewrites from bank (#29182) 2022-12-10 09:27:15 -08:00
Brooks Prumo 5f74fb9c87
Removes erroneous duplicate `#[ignore]` (#29193) 2022-12-09 20:58:22 -05:00
Jeff Washington (jwash) f9c4170d34
remove caching_enabled (#29117)
* warp_from_parents works with write_cache enabled

* remove caching_enabled
2022-12-09 14:31:54 -08:00
Jeff Washington (jwash) 631a98a3b6
warp_from_parents works with write_cache enabled (#29185) 2022-12-09 14:28:18 -08:00
Jeff Washington (jwash) 7f1b71a2b0
remove AccountsDb::new_with_config_for_tests.caching_enabled (#29181) 2022-12-09 13:54:37 -08:00
Jeff Washington (jwash) 04c10a9382
panic on process_stale_slot_with_budget and remove test (#29045) 2022-12-09 13:25:34 -08:00
Jeff Washington (jwash) a3c95eaa53
remove BankTestConfig.caching_enabled (#29173) 2022-12-09 11:52:36 -08:00
Jeff Washington (jwash) 6a90abd056
remove handle_snapshot_requests.caching_enabled (#29174) 2022-12-09 10:51:44 -08:00
Jeff Washington (jwash) 45ba5ef6fd
remove bank_from_snapshot_archives caching_enabled (#29171) 2022-12-09 10:45:21 -08:00
Jeff Washington (jwash) 99f0113450
remove Accounts::new_with_config_for_tests.caching_enabled (#29169) 2022-12-09 10:44:56 -08:00
Jeff Washington (jwash) ec5098a723
remove bank_test_config_caching_enabled (#29170) 2022-12-09 08:28:02 -08:00
apfitzge 151727f633
Storage Rebuilder: better error propagation (#29150) 2022-12-09 10:00:12 -06:00
Jeff Washington (jwash) 3d268be800
change RentResult enum values (#29139)
* test_shrink_candidate_slots uses write cache (#29145)

* add tests for collect_from_existing_account
2022-12-08 19:48:42 -08:00
Jeff Washington (jwash) 06a806bb9d
introduce aligned_stored_size to flush write cache (#29147) 2022-12-08 19:44:23 -08:00
Jeff Washington (jwash) 072070ba80
test_shrink_candidate_slots uses write cache (#29145) 2022-12-08 15:13:05 -08:00
Jeff Washington (jwash) ea19fe9219
support for accounts tests loading accounts with excluded feature (#29142) 2022-12-08 11:46:37 -08:00
Brooks Prumo 2d4f17e1c9
Removes AccountsDb::hash_account_with_rent_epoch() (#29138) 2022-12-08 13:08:36 -05:00
Jeff Washington (jwash) 2f4731570a
ledger-tool create-snapshot uses write cache (#29119)
* ledger-tool create-snapshot uses write cache

* ledger tool create snapshot works with write cache
2022-12-08 06:09:52 -08:00
Jeff Washington (jwash) 019f276a91
test_shrink_candidate_slots_cached works with write cache (#29143) 2022-12-08 06:08:31 -08:00
Brooks Prumo 9eedc189fd
Uses `storages` for instance of SortedStorages (#29133) 2022-12-07 19:27:46 -05:00
Jeff Washington (jwash) 1d95720e5e
migrate some tests to work with write cache (#29124)
* make specific tests use write cache

* fix a few more tests
2022-12-07 13:46:06 -08:00
Jeff Washington (jwash) 284c41a6db
test_get_snapshot_storages_only_non_empty uses write cache (#29111) 2022-12-07 09:19:17 -08:00
Jeff Washington (jwash) 59359c3ab4
test_shrink_collect_simple uses write cache (#29115) 2022-12-07 09:18:45 -08:00
Jeff Washington (jwash) 00f5a176a0
test_clean_zero_lamport_and_old_roots uses write cache (#29113) 2022-12-07 07:30:58 -08:00
Jeff Washington (jwash) 18a85edf8c
migrate tests to use write cache (#29108)
* test_store_overhead works with write cache

* test_storage_remove_account_double_remove works with write cache

* test_set_storage_count_and_alive_bytes works with write cache

* test_reuse_append_vec_id works with write cache
2022-12-07 07:03:57 -08:00
Jeff Washington (jwash) 8ce6081744
test_accounts_db_purge1 uses write cache (#29100) 2022-12-06 14:34:40 -08:00
Jeff Washington (jwash) e5e64c8d6e
test_remove_unrooted_slot_snapshot uses write cache (#29110)
* test_remove_unrooted_slot_snapshot uses write cache

* test_purge_alive_unrooted_slots_after_clean uses with write cache

* test_get_snapshot_storages_with_base_slot uses write cache

* test_get_snapshot_storages_only_roots uses write cache

* test_get_snapshot_storages_only_older_than_or_equal_to_snapshot_slot uses write cache

* test_get_snapshot_storages_exclude_empty uses write cache

* test_clean_old_with_zero_lamport_account uses write cache

* test_clean_old_with_normal_account uses write cache

* test_clean_old_with_both_normal_and_zero_lamport_accounts uses write cache

* test_clean_max_slot_zero_lamport_account uses write cache

* test_calculate_storage_count_and_alive_bytes_2_accounts uses write cache

* test_calculate_storage_count_and_alive_bytes uses write cache
2022-12-06 14:34:18 -08:00
Jeff Washington (jwash) 46e13423fd
migrate tests to work with write cache (#29106)
* test_accounts_purge_long_chained_after_snapshot_restore

* test_accountsdb_calculate_accounts_hash_from_storages works with write cache

* test_zero_lamport_new_root_not_cleaned works with write cache

* test_wrapping_append_vec_id works with write cache

* test_verify_bank_hash works with write cache

* change tests to use write cache

* use shared db to pass to tests
2022-12-06 14:33:52 -08:00
Jeff Washington (jwash) c668a58b8e
do_full_clean_refcount uses write cache (#29112) 2022-12-06 14:33:28 -08:00
Jeff Washington (jwash) d3fa93907b
test_clean_zero_lamport_and_dead_slot uses write cache (#29114) 2022-12-06 14:33:07 -08:00
behzad nouri 9524c9dbff patches errors from clippy::uninlined_format_args
https://rust-lang.github.io/rust-clippy/master/index.html#uninlined_format_args
2022-12-06 19:32:15 +00:00
behzad nouri 9433c06745 patches errors from clippy::unchecked_duration_subtraction
https://rust-lang.github.io/rust-clippy/master/index.html#unchecked_duration_subtraction
2022-12-06 19:32:15 +00:00
behzad nouri 401f66c830 patches errors from clippy::seek_to_start_instead_of_rewind
https://rust-lang.github.io/rust-clippy/master/index.html#seek_to_start_instead_of_rewind
2022-12-06 19:32:15 +00:00
Jeff Washington (jwash) ca8372efc1
with_chained_zero_lamport_accounts works with write cache (#29104) 2022-12-06 10:56:37 -08:00
Jeff Washington (jwash) 26472a13ea
migrating tests to use write cache (#29101)
* test_account_update uses write cache

* test_accounts_db_purge_keep_live works with write cache

* enable caching for test_accounts_db_purge_keep_live
2022-12-06 10:25:37 -08:00
Jeff Washington (jwash) 5d0d4a6b95
test_accounts_db_serialize_zero_and_free uses write cache (#29102) 2022-12-06 10:25:07 -08:00
Jeff Washington (jwash) 7180345ee1
test_verify_bank_capitalization works with write cache (#29107) 2022-12-06 10:24:43 -08:00
Jeff Washington (jwash) 5833e2f85e
test_account_grow uses write cache (#29099) 2022-12-06 09:28:24 -08:00
Jeff Washington (jwash) 94a6665995
test_clean_multiple_zero_lamport_decrements_index_ref_count uses write cache (#29088) 2022-12-06 08:56:37 -08:00
Jeff Washington (jwash) fe8b15b43a
test_accountsdb_count_stores uses write cache (#29086) 2022-12-06 07:40:51 -08:00
Jeff Washington (jwash) 93cc3b1d3d
test_shrink_stale_slots_skipped only uses non-write cache (#29081) 2022-12-06 07:03:30 -08:00
Jeff Washington (jwash) 4de13be926
test_accounts_db_serialize1 uses write cache (#29085) 2022-12-06 06:36:28 -08:00
Jeff Washington (jwash) f88398cbcf
test_shrink_stale_slots_processed uses write cache (#29082) 2022-12-06 06:35:49 -08:00
Jeff Washington (jwash) ed0c9e285b
test_partial_clean supports write cache (#29091) 2022-12-06 06:35:02 -08:00
Jeff Washington (jwash) c106c7e349
test_clean_accounts_with_last_full_snapshot_slot uses write cache (#29087) 2022-12-05 20:41:11 -08:00
Jeff Washington (jwash) 6935ff74ce
test_uncleaned_roots_with_account uses write cache (#29084) 2022-12-05 20:37:42 -08:00
Jeff Washington (jwash) 1b49c52d98
store_uncached -> store_for_tests (#29083) 2022-12-05 14:42:16 -08:00
Jeff Washington (jwash) b4075a0e29
bank tests use write cache (#28885) 2022-12-05 14:08:28 -08:00
Jeff Washington (jwash) 9746eb2c8d
test_lazy_gc_slot can use write cache (#29080) 2022-12-05 12:48:15 -08:00
Jeff Washington (jwash) 8c47e2168e
test_accounts_clean_after_snapshot_restore_then_old_revives can use write cache (#29050)
* test_accounts_clean_after_snapshot_restore_then_old_revives can use write cache

* combine flush and set root
2022-12-05 12:39:26 -08:00
Jeff Washington (jwash) 293a5860b8
test_process_stale_slot_with_budget never uses write cache (#29076) 2022-12-05 11:40:14 -08:00
Jeff Washington (jwash) 1036d4409b
test_verify_snapshot_bank works without write cache (#29075) 2022-12-05 11:00:01 -08:00
Brooks Prumo c65a8ce6c3
AccountsDb::get_snapshot_storages() takes a Range for slots (#29054) 2022-12-05 11:21:21 -05:00
Brooks Prumo 544e430a0d
Use AccountsHash instead of bare Hash (#29052) 2022-12-05 11:19:29 -05:00
Jeff Washington (jwash) 7f268e0f6d
rebuild_bank_from_snapshots uses write cache (#29047) 2022-12-02 13:28:21 -08:00
Jeff Washington (jwash) babc241345
tests use write cache (#29027)
* test_add_precompiled_account uses write cache

* test_add_precompiled_account_squatted_while_not_replacing uses write cache
2022-12-02 10:26:51 -08:00
Jeff Washington (jwash) 5104f4ef0d
test_bank_update_vote_stake_rewards works with write cache (#29044) 2022-12-02 10:19:43 -08:00
Jeff Washington (jwash) 00ba78bd2d
test_bank_update_rewards_determinism works with write cache (#29043) 2022-12-02 09:56:58 -08:00
Jeff Washington (jwash) 6852d5768a
test_add_builtin_account* works with write cache (#29030) 2022-12-02 09:48:21 -08:00
Jeff Washington (jwash) 87a3130226
test_bank_hash_internal_state works with write cache (#29042) 2022-12-02 09:37:22 -08:00
Jeff Washington (jwash) d17a740a99
test_bank_hash_internal_state_verify works with write cache (#29041) 2022-12-02 09:22:42 -08:00
Tao Zhu 0809a4df78
remove redundant internal functions (#29035)
remove redundant _internal functions
2022-12-02 10:21:25 -06:00
Jeff Washington (jwash) 63e19c0068
do_test_clean_dropped_unrooted_banks uses write cache (#29029) 2022-12-02 08:04:57 -08:00
Jeff Washington (jwash) 911a8d2a4a
test_add_instruction_processor_for_existing_unrelated_accounts works with write cache (#29031) 2022-12-02 08:04:34 -08:00
Tao Zhu 5850af5316
Refactor to remove requested_cu from cost_trarcker (#29015)
* refactor cost tracker by removing requested_cu from it, call sites to use cost_model forr consistency

* review fix
2022-12-02 00:25:09 +00:00
Jeff Washington (jwash) 20aeff3632
test_purge_empty_accounts uses write cache (#29018) 2022-12-01 15:22:26 -08:00
Jeff Washington (jwash) dbd6ce8f21
test_bank_update_sysvar_account uses write cache (#29022)
* test_add_precompiled_account_inherited_cap_while_replacing uses write cache

* test_bank_update_sysvar_account uses write cache
2022-12-01 15:21:05 -08:00
Jeff Washington (jwash) a9c8f26a66
test_add_precompiled_account_inherited_cap_while_replacing uses write cache (#29019) 2022-12-01 14:07:39 -08:00
Jeff Washington (jwash) 631d93dd2d
tests use write cache (#29012) 2022-12-01 12:17:13 -08:00
Jeff Washington (jwash) e6cc5052c3
pass alive bytes to AccountsToStore::new (#28994) 2022-11-30 15:46:52 -08:00
Jeff Washington (jwash) 2c912c9049
use files for merkle tree inputs on accounts hash (#28965)
* use files for merkle tree inputs on accounts hash

* remove multi-pass scan, simplify

* cleanup and comments

* rework writing into the hash file for clarity

* add test, cleanup

* rename

* move count into option
2022-11-30 12:27:27 -08:00
Jeff Washington (jwash) 58e12cf963
avoid copies when writing to an ancient append vec (#28981)
* avoid copies when writing to an ancient append vec

* update comments
2022-11-30 10:09:23 -08:00
Jeff Washington (jwash) 2427004dba
add 'bytes' to aligned/alive_total for clarity (#28982) 2022-11-30 07:38:57 -08:00
Jeff Washington (jwash) c8cc1270f1
log # append vecs open (#28958) 2022-11-29 16:48:02 -08:00
Jeff Washington (jwash) e42649a8ba
remove cli arg to specify # accounts hash scan passes (#28976) 2022-11-29 16:47:51 -08:00
Jeff Washington (jwash) 19d86bd2b1
add balance to cap overflow message (#28975) 2022-11-29 13:55:07 -08:00
Brooks Prumo 9327658007
Promotes accounts hash to a strong type (#28930) 2022-11-28 10:09:47 -05:00
Brooks Prumo 267bbcadcf
Cleans up accounts hash testing in ABS (#28931) 2022-11-23 10:07:22 -05:00
Jeff Washington (jwash) 9a24743d2a
expect only 1 store per slot (#28844)
expect only 1 store per slot in shrink
2022-11-23 06:20:36 -08:00
Brooks Prumo 04016e3bcf
Don't wait for EAH unless feature is enabled (#28938) 2022-11-23 09:11:28 -05:00
Brooks Prumo 40a7c6d46a
Renames fn to insert_default_bank_hash() (#28929) 2022-11-22 19:31:59 -05:00
Jeff Washington (jwash) 5d88a9b32b
improve perf of storing accounts for shrink/ancient (#28880)
* improve perf of storing accounts for shrink/ancient

* rename

* phantom data

* update comment

* make impl Borrow<Hash> consistent

* remove unused static
2022-11-22 15:36:57 -08:00
Maximilian Schneider c8b0c3ede9
Update cost model to use requested_cu instead of estimated cu #27608 (#28281)
* Update cost model to use requested_cu instead of estimated cu #27608

* remove CostUpdate and CostModel from replay/tvu

* revive cost update service to send cost tracker stats

* CostModel is now static

* remove unused package

Co-authored-by: Tao Zhu <tao@solana.com>
2022-11-22 11:55:56 -06:00
Jeff Washington (jwash) efc681b348
rename test var for clarity (#28914) 2022-11-21 13:22:16 -08:00
Jeff Washington (jwash) d449b9ea26
test_new_from_file_crafted_zero_lamport_account uses static append vec file (#28911) 2022-11-21 12:43:03 -08:00
Jeff Washington (jwash) 8addf93de7
test_store_reuse uses write cache (#28901) 2022-11-21 11:17:33 -08:00
Brooks Prumo ff7e82b5ff
Removes Accounts::bank_hash_at() (#28879) 2022-11-21 13:25:48 -05:00
Jeff Washington (jwash) b29dea7a56
comment function (#28906) 2022-11-21 09:56:07 -08:00
Brooks Prumo d698f982ee
Renames to AccountsHasher (#28883) 2022-11-18 18:25:44 -05:00
Jeff Washington (jwash) 8cbe83c725
cleanup CalcAccountsHashDataSource (#28882) 2022-11-18 14:49:22 -08:00
Jeff Washington (jwash) f22104d46b
use write cache by default in some tests (#28876) 2022-11-18 14:35:52 -08:00
Jeff Washington (jwash) aa1d319d48
add shrink ancient test (#28858)
* add shrink ancient test

* improvements
2022-11-18 13:11:57 -08:00
Brooks Prumo 7787cb2ab2
Renames BankHashInfo field to accounts_delta_hash (#28878) 2022-11-18 15:35:03 -05:00
Jeff Washington (jwash) dcfb73f664
extract test append_account function (#28875) 2022-11-18 10:57:02 -08:00
Jeff Washington (jwash) 9acfa08838
remove per-account slot in ancient append vecs (#28851)
* remove per-account slot in ancient append vecs

* review fixes
2022-11-18 10:15:41 -08:00
Justin Starry 4c85850a73
Refactor: Rename `load_transaction` to `load_transaction_accounts` (#28746)
* Refactor: Rename load_transaction to load_transaction_accounts

* Refactor: map over account keys to load transaction accounts
2022-11-18 23:44:09 +08:00
Justin Starry 7371608722
Support versioned transactions in program test framework (#28739)
* Support versioned transactions in program test framework

* use working bank

* Update to process_transaction_with_metadata

* Migrate client apis from Transaction to Into<VersionedTransaction>

* feedback
2022-11-18 23:43:52 +08:00
Jeff Washington (jwash) 813f1f993a
remove test only append_account (#28850) 2022-11-17 12:26:51 -08:00
Trent Nelson 53a579bed1 ledger-tool: stream output of `accounts` subcommand 2022-11-17 12:45:26 -07:00
Jeff Washington (jwash) e269fe3383
refactor into combine_one_store_into_ancient (#28845) 2022-11-17 08:42:42 -08:00
Brooks Prumo 2bafb0cb12
Requires EAH state cannot be Invalid (#28817) 2022-11-17 11:01:01 -05:00
K-anon e8c8235474
Get Index Key Size RPC Support (#28383)
Co-authored-by: K-anon <IntokuSatori@users.noreply.github.com>
2022-11-16 20:39:11 -07:00
Jeff Washington (jwash) 83a7b2ca6c
only log ancient append vec stats when they are useful (#28843) 2022-11-16 17:15:43 -08:00
Tao Zhu ae48ac97dd
fix account resize test by requesting max tx data size (#28826)
* fix account resize test by requesting max tx data size
* define data size limit in incremental of 1024
2022-11-16 17:52:15 -06:00
Jeff Washington (jwash) c17d594684
collect storage_read_elapsed for shrink and ancient (#28833) 2022-11-16 11:14:07 -08:00
Jeff Washington (jwash) 4837dc071f
cleanup return value from append_accounts (#28836) 2022-11-16 10:59:11 -08:00
Jeff Washington (jwash) f7139532a6
acct hash: don't init accum or create hash file if no storages (#28839) 2022-11-16 10:58:39 -08:00
Jeff Washington (jwash) 2b1db39faa
refactor test ancient population code (#28823) 2022-11-16 08:31:39 -08:00
Brooks Prumo c1e440acb6
Warping calculates and sets new EAH (#28809) 2022-11-15 23:16:38 -05:00
behzad nouri 28956d7653
removes feature gate code preventing crediting accounts that end rent paying (#28818) 2022-11-15 19:55:34 +00:00
Brooks Prumo d798e751a0
Disables EAH with short epochs (#28803) 2022-11-15 13:26:19 -05:00
Jeff Washington (jwash) b572422d24
test ancient append vecs with dead slots (#28800) 2022-11-15 10:21:38 -08:00
Jeff Washington (jwash) a21f536b3a
add scan statistics in hash calc for ancient append vecs (#28816) 2022-11-15 09:31:24 -08:00
Brooks Prumo ac9b11762a
Lowers test's account data size to avoid hitting MaxLoadedAccountsDataSizeExceed (#28814) 2022-11-15 16:52:35 +00:00
Jeff Washington (jwash) 84cfdf23fc
remove LoadedAccount::stored_size (#28802) 2022-11-15 05:47:12 -08:00
Jeff Washington (jwash) ec58cda2e7
improve ancient append vec test (#28799) 2022-11-15 05:46:09 -08:00
Brooks Prumo d4cf18421d
Use 400 slots-per-epoch in EAH tests (#28801) 2022-11-14 17:49:20 -05:00
Ikko Ashimine c882d165e2
Fix typo in cost_tracker.rs (#28781)
Substract -> Subtract
2022-11-14 14:36:59 -07:00
Brooks Prumo 503da50f2c
Verifies EAH in SnapshotHash (#28775) 2022-11-14 16:02:08 -05:00
Brooks Prumo 0bfea02056
Snapshots wait for EAH calculations to complete (#28777) 2022-11-14 11:34:44 -06:00
Tao Zhu 81dc2e56ac
Cap accounts data a transaction can load by its requested limit (#27840)
- Add new compute-budget instruction to set transaction-wide accounts data size limit
- Set default accounts data limit to 10MB, and max to 100MB, per transaction;
- Add getters to make changing default and/or max values easier in the future with feature gates;
- added error counter for transactions exceed data size limit
2022-11-14 10:29:35 -06:00
Jeff Washington (jwash) f69e847137
allow ancient append vec threshold to be set on cli (#28707)
* allow ancient append vec threshold to be set on cli

* pr responses
2022-11-11 11:30:05 -08:00
Jeff Washington (jwash) b2ece6368f
shrink ancient append vecs at 90% (#28736) 2022-11-10 20:05:34 -08:00
Alexander Meißner 4142f42319
Refactor - Move `Executor` in program-runtime crate (#28782)
* Moves CreateMetrics into the program-runtime crate.

* Moves the Executor trait into executor.rs

* Removes the first_instruction_account parameter from Executor::execute().
2022-11-10 21:11:00 +01:00
Brooks Prumo 628ac0d4ea
Refactors SnapshotPackage::new() to use let-else statements (#28779) 2022-11-10 11:29:51 -05:00
Jeff Washington (jwash) 552248906e
report ancient stats correctly in drop_or_recycle_stores (#28747) 2022-11-10 07:27:38 -08:00
Brooks Prumo 8eeabf9042
Skips taking a bank snapshot for EAH requests (#28757) 2022-11-10 15:14:26 +00:00
Brooks Prumo 4d6653598b
Upgrades to Rust 1.65.0 (#28741) 2022-11-09 17:15:03 -05:00
Brooks Prumo d1ba42180d
clippy for rust 1.65.0 (#28765) 2022-11-09 19:39:38 +00:00
Brooks Prumo df81cd11ba
Refactors snapshot_archive_info in bank_from_snapshot_archives() (#28776) 2022-11-09 18:41:27 +00:00
Brooks Prumo 9e1cdc7e60
Enables not taking a bank snapshot (#28756) 2022-11-09 12:43:33 -05:00
Brooks Prumo d4c2900590
Removes `snapshot_bank()` wrapper fn (#28753) 2022-11-07 15:09:31 +00:00
Brooks Prumo 0873b90d5f
Treats all bank snapshot errors as fatal (#28752) 2022-11-07 11:03:11 +00:00
Jeff Washington (jwash) 0bc66c82aa
remove unused stat ancient.skipped_shrink (#28749) 2022-11-05 08:09:19 -07:00
Jeff Washington (jwash) 99565570b8
add ancient append vec stats (#28737) 2022-11-05 08:08:52 -07:00
Jeff Washington (jwash) 5937a10810
ancient append vec stat fixes (#28731) 2022-11-04 04:48:19 -07:00
Brooks Prumo 064cfc70d2
Removes cluster_type from AccountsPackage (#28725) 2022-11-02 18:21:13 -04:00
Jeff Washington (jwash) 88d7f6cc9d
add ability to flush read only accounts cache (#28726) 2022-11-02 12:03:32 -07:00
Jeff Washington (jwash) faf32e2a94
add ancient append vec tests (#28712)
add test test_combine_ancient_slots_simple
2022-11-02 11:24:08 -07:00
Brooks Prumo 0a62f01cab
Removes EAH string from datapoint metric (#28724) 2022-11-02 14:14:08 -04:00
Jeff Washington (jwash) ee0320c9f3
fix ancient append vec throwing away stores bug (#28710) 2022-11-02 08:21:03 -07:00
Jeff Washington (jwash) 7885fdca37
ancient append vecs: pass through can_randomly_shrink (#28706) 2022-11-01 11:55:28 -07:00
Jeff Washington (jwash) 9ff9051d87
move test function to test mod (#28704) 2022-11-01 11:18:02 -07:00
Jeff Washington (jwash) daf71003d3
add tests for handle_dropped_roots_for_ancient (#28703) 2022-11-01 09:03:54 -07:00
Jeff Washington (jwash) 0b51b15af2
remove duplicate pubkey during shrink (#28689) 2022-11-01 07:45:52 -07:00
behzad nouri 17680fe837
removes feature gate code preserving rent_epoch for rent exempt accounts (#28507)
* removes feature gate code preserving rent_epoch for rent exempt accounts

* update test expected hashes

Co-authored-by: jeff washington <jeff.washington@solana.com>
2022-11-01 13:36:59 +00:00
Brooks Prumo f0f108905e
Always verifies deserialized bank's slot and snapshot hash (#28688) 2022-10-31 17:48:18 -04:00
Jeff Washington (jwash) cb4c3668c7
.pubkey() instead of meta.pubkey for stored account (#28682) 2022-10-31 13:58:42 -07:00
Jeff Washington (jwash) 1ba81acf9a
remove public from rent code (#28687) 2022-10-31 13:37:36 -07:00
Jeff Washington (jwash) c47515d055
remove duplicate pubkey when shrinking or combining ancient (#28678) 2022-10-31 12:00:55 -07:00
Brooks Prumo 59bf1809fe
Uses SnapshotHash type in snapshot archive fields (#28681) 2022-10-31 14:28:35 -04:00
Dmitri Makarov 34865d032c chore: update Solana docs and code comments that specify "BPF" to "SBF" 2022-10-31 14:14:25 -04:00
Brooks Prumo 37507a2de6
Removes EAH parameter from serde_snapshot::reserialize_bank() (#28669) 2022-10-31 09:43:17 -04:00
Jeff Washington (jwash) fad6c99e9f
add test variations for shrink_collect (#28640)
add test variations
2022-10-29 14:24:08 -07:00
Jeff Washington (jwash) 5889ee30b0
add test_get_keys_to_unref_ancient (#28664) 2022-10-29 14:23:41 -07:00
Brooks Prumo bfa62da29e
Renames variant to CalcAccountsHashDataSource::IndexForTests (#28623) 2022-10-29 08:55:33 -04:00
Brooks Prumo 2ebf248f47
Renames fn to Bank::update_accounts_hash() (#28619) 2022-10-28 17:31:16 -04:00
Brooks Prumo f98e4d9198
Renames BankHashInfo field to accounts_hash (#28665) 2022-10-28 15:18:25 -04:00
Brooks Prumo 5a3d252899
Renames fn to Bank::update_accounts_hash_for_tests() (#28620) 2022-10-28 14:33:05 -04:00
Jeff Washington (jwash) 8924829e7b
extract 'unref_pubkeys' for shared code (#28653) 2022-10-28 10:12:29 -07:00
Brooks Prumo 0232944c95
Uses Storages to update accounts hash in bank_to_snapshot_archive() (#28618) 2022-10-28 12:22:58 -04:00
Jeff Washington (jwash) 0b2bc987ca
tests for maybe_unref_accounts_already_in_ancient (#28652) 2022-10-28 08:24:10 -07:00
Jeff Washington (jwash) 0c1df82d48
add current_ancient tests (#28651) 2022-10-28 07:32:23 -07:00
Jeff Washington (jwash) 8ae5af2822
ancient append vec tests (#28635) 2022-10-27 15:10:22 -07:00
Jeff Washington (jwash) 3d7cca6296
cleanup drop in ancient append vecs (#28617) 2022-10-27 11:16:46 -07:00
steviez 7b70aef33c
Add help text to genesis creation time assert (#28629)
If a validator starts with a snapshot from the wrong cluster, an assert
will trigger complaining about the genesis_creation_time encoded in the
snapshot differing from the genesis_creation_time embedded in
genesis.bin.

This could be a fairly easy mistake to make so add a help string to the
assertion to point operators in the right direction.
2022-10-27 18:02:36 +02:00
Jeff Washington (jwash) 646514e4d5
give metric around 'remove_old_stores_shrink" better name (#28613) 2022-10-27 07:33:01 -07:00
Jeff Washington (jwash) 4ac52c2a9d
ancient append vec comments and cleanup (#28616) 2022-10-27 07:32:42 -07:00
Jeff Washington (jwash) 251cab4c8b
extract handle_dropped_roots_for_ancient from ancient append vec code (#28611) 2022-10-27 07:32:04 -07:00
Ian Macalinao a2b0d0e6f0
runtime: use correct module doc comments (#27889) 2022-10-27 14:39:44 +02:00
Jeff Washington (jwash) decf3de453
extract ancient maybe_unref_accounts_already_in_ancient (#28610) 2022-10-26 14:20:32 -07:00
Jeff Washington (jwash) d4db649e7c
extract common code remove_old_stores_shrink from shrink and ancient append vecs (#28588) 2022-10-26 11:49:10 -07:00
Brooks Prumo 27269d833c
Uses enum for data source with calc_accounts_hash() (#28584) 2022-10-26 13:04:01 -04:00
Jeff Washington (jwash) 252d7f68da
combine all_are_zero_lamports (#28567) 2022-10-26 08:55:20 -07:00
Justin Starry 2d8665d307
Record inner instruction stack height (#28430)
* Record inner instruction stack height

* fix sbf tests

* feedback
2022-10-26 10:37:44 +08:00
Brooks Prumo 1dbcb78de7
Renames fn to update_accounts_hash() (#28574) 2022-10-25 16:43:19 -04:00
Jeff Washington (jwash) 51e1a7f2af
simplify mark_dirty_dead_stores (#28565) 2022-10-25 11:34:58 -07:00
Jeff Washington (jwash) 470ed4f06e
update stats in shrink_collect (#28563) 2022-10-25 11:27:33 -07:00
Brooks Prumo e79c1cba03
Renames fn to update_accounts_hash_for_tests() (#28571) 2022-10-25 14:03:19 -04:00
Jeff Washington (jwash) 8c57efb828
ancient append vecs: get rid of some locals in shared code (#28559) 2022-10-25 09:32:14 -07:00
Jeff Washington (jwash) 2c837ba4e3
reorder code to prepare for sharing (#28564) 2022-10-25 09:15:26 -07:00
Brooks Prumo f158bab0ef
Tracks how long background requests wait before processing (#28581) 2022-10-25 12:10:53 -04:00
Brooks Prumo bf214d36e3
Renames fn to calculate_accounts_hash_with_verify() (#28573) 2022-10-25 11:55:26 -04:00
Brooks Prumo e2438cf3e2
Renames fn to calculate_accounts_hash() (#28572) 2022-10-25 11:53:41 -04:00
Jeff Washington (jwash) 1ca7c8244f
rename merge() -> accumulate() (#28561) 2022-10-25 08:52:02 -07:00
Brooks Prumo bc02789c43
Renames fn to calculate_accounts_hash_from_storages() (#28566) 2022-10-24 21:07:00 -04:00
Brooks Prumo 2354a0a343
Renames fn to calculate_accounts_hash_from_index() (#28568) 2022-10-24 19:20:08 -04:00
Jeff Washington (jwash) dde8f0dfad
CurrentAncientAppendVec for clarity (#28528) 2022-10-24 15:28:52 -07:00
Jeff Washington (jwash) 8acd6bb34a
share index_read_elapsed code (#28557) 2022-10-24 15:19:40 -07:00
Jeff Washington (jwash) a422387ca8
cleanup ancient append vec merge stats (#28556) 2022-10-24 13:11:12 -07:00
Jeff Washington (jwash) 8a538f8da5
skip_rewrite skips almost all accounts with no rent (#28552) 2022-10-24 11:48:00 -07:00
Trent Nelson 1fbd818647
runtime: remove `Default` req on account scan interfaces (#28533) 2022-10-21 17:53:06 -06:00
Jeff Washington (jwash) 993583d9b3
simplify get_store_for_shrink (#28524) 2022-10-21 14:49:34 -07:00
Ashwin Sekar 9eafad467c
Add convenience methods to VoteInstruction to distinguish vote types (#28526)
* Add convenience methods to VoteInstruction to distinguish vote types

* use matches! macro instead
2022-10-21 14:17:40 -06:00
Jeff Washington (jwash) fb94e50678
remove single use function for clarity (#28523) 2022-10-21 11:42:27 -07:00
Ashwin Sekar f207af765e
Split out voting and banking threads in banking stage (#27931)
* Split out voting and banking threads in banking stage

Additionally this allows us to aggressively prune the buffer for voting threads
as with the new vote state only the latest vote from each validator is
necessary.

* Update local cluster test to use new Vote ix

* Encapsulate transaction storage filtering better

* Address pr comments

* Commit cargo lock change

* clippy

* Remove unsafe impls

* pr comments

* compute_sanitized_transaction -> build_sanitized_transaction

* &Arc -> Arc

* Move test

* Refactor metrics enums

* clippy
2022-10-20 21:10:48 +00:00
Trent Nelson cb7fe87abb bank: factor accumulation out to helper on `TotalAccountsStats` 2022-10-20 10:50:51 -07:00
Jeff Washington (jwash) 09ede8b85c
reuse update_shrink_stats for combine_ancient_slots (#28466) 2022-10-19 14:17:34 -07:00
Alexander Meißner 5aca7df232
Bank::create_executor() (#28474)
* Moves the integration test test_bpf_loader_upgradeable_deploy_with_max_len() from the bpf_loader into the runtime.

* Adds Bank::create_executor().

* Adds a test for Bank::create_executor().
2022-10-19 22:28:57 +02:00
Brooks Prumo 70e83e82c6
Waits to handle snapshot requests until after startup verification is complete (#28481) 2022-10-19 13:39:47 -04:00
Jeff Washington (jwash) 2b11926c9e
simplify combine_ancient_slots (#28464) 2022-10-19 09:35:21 -07:00
Jeff Washington (jwash) 8b115ff489
factor out update_shrink_stats (#28465)
* use local for shrink_stats

* renames and remove .as_us()

* factor out update_shrink_stats

* use integer to report us in metrics
2022-10-19 08:08:04 -07:00
Jeff Washington (jwash) 040035063f
share shrink_collect code (#28454)
* share shrink_collect code

* use correct stats
2022-10-19 08:07:05 -07:00
Brooks Prumo 1cc9cf927c
Supports warping with Epoch Accounts Hash (#28459) 2022-10-19 10:37:14 -04:00
Jeff Washington (jwash) ca115594ff
remove concept of 'full' ancient append vec (#28449) 2022-10-18 15:08:04 -07:00
Jeff Washington (jwash) c89bd0d6a4
make ancient shrink consistent with shrink (#28457) 2022-10-18 15:05:32 -07:00
Jeff Washington (jwash) 825b83c95c
simple minor refactoring (#28456) 2022-10-18 15:05:06 -07:00
Jeff Washington (jwash) 3835ac166e
refactor load_accounts_index_for_shrink for easier sharing (#28455) 2022-10-18 14:55:10 -07:00
Jeff Washington (jwash) d52e28ac22
combine sort into get_unique_accounts_from_storages (#28450) 2022-10-18 11:52:48 -07:00
Jeff Washington (jwash) b583ec79e1
add tests for should_move_to_ancient_append_vec (#28438)
add tests for should_moev_to_ancient_append_vec
2022-10-18 11:41:35 -07:00
Jeff Washington (jwash) fd2e671861
remove slot# from account hash (#28405)
* remove slot# from account hash

* add feature

* fix tests

* constants to help clarify 'irrelevant' changes

* move to enum for enforcing irrelevancy

* ignore unsupported tests
2022-10-18 08:03:37 -07:00
Jeff Washington (jwash) 682999a423
accounts hash calculation uses files instead of memory (#28065) 2022-10-18 07:51:38 -07:00
Jeff Washington (jwash) b669b827fb
refactor and test get_sorted_potential_ancient_slots (#28436) 2022-10-18 07:45:43 -07:00
Justin Starry 70445b7402
Refactor: Rename `max_invoke_depth` to `max_invoke_stack_height` (#28427)
Refactor: Rename max_invoke_depth to max_invoke_stack_height
2022-10-17 23:54:56 +08:00
Jeff Washington (jwash) 28a89a1d99
remove expected rent collection and rehashing completely (#28422) 2022-10-17 07:24:42 -07:00