Commit Graph

3815 Commits

Author SHA1 Message Date
K-anon c5a24e11ba
Add test for executor cache runtime feature upgrades. (#30382)
Co-authored-by: K-anon <IntokuSatori@users.noreply.github.com>
2023-02-21 12:25:29 -07:00
Jeff Washington (jwash) 568a4b1ca0
move AccountInfo bits from size to offset (#30408)
* move AccountInfo bits from size to offset

* feedback
2023-02-21 10:08:37 -08:00
HaoranYi 22f6dfd98e
typo (#30393) 2023-02-21 09:59:45 -06:00
Brooks bcc4bc80c9
Removes unnecessary derives from Accounts{Delta}Hash (#30392) 2023-02-20 16:00:53 -05:00
Jeff Washington (jwash) 46b42e1888
add hash calc scan_chunks metric (#30333) 2023-02-20 12:55:48 -08:00
Brooks ec288ded9b
Serde snapshot uses own types (#30391) 2023-02-20 12:25:15 -05:00
Brooks a8e07c41fe
Adds `is_incremental_accounts_hash_feature_enabled` to `AccountsPackage` (#30368) 2023-02-20 12:21:38 -05:00
Brooks ad6a3abd8b
Uses set_accounts_{delta,}_hash_for_tests() in serde tests (#30390) 2023-02-17 16:00:25 -05:00
Brooks f91e63dfe9
Upgrades Rust to 1.67.1 (#29947) 2023-02-17 14:08:27 -05:00
Pankaj Garg 3a4ba72daf
Remove executors trait and replace with LoadedProgram (#30348)
* Remove executors trait and replace with LoadedProgram

* fill in deployment slot

* address review comments

* fix clippy warnings

* address review comments

* fix failures caught by sbf tests
2023-02-17 12:47:45 +01:00
Pankaj Garg 6c58033df5
API to enumerate executable program accounts in tx batch (#30371) 2023-02-16 13:29:38 -08:00
Brooks febaf36e6d
Apply clippy fixes for future rust upgrade (#30363) 2023-02-16 16:12:51 +00:00
Xiang Zhu fa22389b22
Add version and state_complete flag into bank snapshot (#30099)
* Add version and state_complate flag into bank snapshot

* Add test_get_highest_full_snapshot_slot_and_path

* fix build checks

* fix slot_deltas panic in test_get_highest_full_snapshot_slot_and_path

* make bank rooted for test_get_highest_full_snapshot_slot_and_path

* fix bank scope problem in test_get_highest_full_snapshot_slot_and_path

* minor cleanup

* misc review issues

* remove unneeded error defs and functions

* Fix state_complete macro naming

* fix snapshot_version
2023-02-15 22:38:40 -08:00
Yueh-Hsuan Chiang ce4251be99
Fix the return type of AccountsFile::account_matches_owners() (#30354)
Update AccountsFile::account_matches_owners() to return the matched index
2023-02-16 03:34:36 +00:00
Yueh-Hsuan Chiang aeb6df3507
Abstract out AppendVec into AccountsFile enum (#29815)
Abstracts out AppendVec into AccountsFile enum.
This will allow different implementations of AccountsFile that support the AccountsDB.
2023-02-15 17:24:04 -08:00
Brooks 5680c76c86
Refactors Accounts::new() fns (#30343) 2023-02-15 18:34:52 -05:00
HaoranYi 615d1a8b69
Extract BankHashLamportsVerifyConfig (#30320)
* refactor BankHashLamportsVerifyConfig

* clippy

* comments

* fix bench
2023-02-15 16:03:50 -06:00
Andrew Fitzgerald b86bfbb5c5
measure_us! use Instant and duration_to_us internally (#30339) 2023-02-15 12:43:47 -08:00
Pankaj Garg 3bea5fc326
Return index of matching owner for a program account (#30337) 2023-02-15 19:41:29 +00:00
Xiang Zhu 4909267c88
Add accounts hard-link files into the bank snapshot directory (#29496)
* Add accounts hard-link files into the bank snapshot directory

* Small adjustments and fixes.

* Address some of the review issues

* Fix compilation issues

* Change the latest slot snapshot storage from VecDeque to Option

* IoWithSourceAndFile and expanded comments on accounts

* last_slot_snapshot_storages in return value

* Update comments following the review input

* rename dir_accounts_hard_links to hard_link_path

* Add dir_full_state flag for add_bank_snapshot

* Let appendvec files hardlinking work with multiple accounts paths across multiple partitions

* Fixes for rebasing

* fix tests which generates account_path without adding run/

* rebasing fixes

* fix account path test failures

* fix test test_concurrent_snapshot_packaging

* review comments.  renamed the path setup function

* Addressed most of the review comments

* update with more review comments

* handle error from create_accounts_run_and_snapshot_dirs

* fix rebasing duplicate

* minor accounts_dir path cleanup

* minor cleanup, remove commented code

* misc review comments

* build error fix

* Fix test_incremental_snapshot_download_with_crossing_full_snapshot_interval_at_startup

* fix build error on MAX_BANK_SNAPSHOTS_TO_RETAIN

* rebase fix, update hardlink filename

* minor comment spelling fix

* rebasing fixes

* fix rebase issues; with_extension

* comments changes for review

* misc minor review issues

* bank.fill_bank_with_ticks_for_tests

* error handling on appendvec path

* fix use_jit

* minor comments refining

* Remove type AccountStorages

* get_account_path_from_appendvec_path return changed to Option

* removed appendvec_path.to_path_buf in create_accounts_run_and_snapshot_dirs

* add test_get_snapshot_accounts_hardlink_dir

* update last_snapshot_storages comment

* update last_snapshot_storages comment

* symlink map_err

* simplify test_get_snapshot_accounts_hardlink_dir with fake paths

* log last_snapshot_storages at the end of the loop
2023-02-15 09:52:07 -08:00
Brooks 2e885cf99a
Removes calls to insert_default_bank_hash_stats() in tests (#30311) 2023-02-14 15:00:38 -05:00
Jeff Washington (jwash) b9b8cd9ccc
misc. cleanup (#30295) 2023-02-14 12:00:13 -08:00
HaoranYi 27f18f96e8
typos, clippy and saturating_add_assign (#30272)
* typo

* remove clippy allow

---------

Co-authored-by: haoran <haoran@mbook>
2023-02-14 10:03:16 -06:00
Jeff Washington (jwash) 253517cba3
add combine_ancient_slots_packed (#30276)
* add combine_ancient_slots_new

* pr feedback

* implement ==
2023-02-14 06:53:31 -08:00
Jeff Washington (jwash) 9988978379
add ancient finish_combine_ancient_slots_packed (#30261) 2023-02-13 09:10:13 -08:00
Tao Zhu 60bfc2524b
implement From trait for CostTrackerError to TransactionError (#30267)
implement From trait for CostTrackerError to TransactionError
2023-02-13 11:06:39 -06:00
Jeff Washington (jwash) bcd7cf0821
add ancient write_packed_storages (#30260) 2023-02-13 07:25:47 -08:00
Jeff Washington (jwash) b3778d5044
cleanup (#30251) 2023-02-12 13:42:03 -08:00
Jeff Washington (jwash) dc605b5126
rename parameter and update comments (#30259) 2023-02-11 12:13:03 -08:00
Jeff Washington (jwash) c837af1af4
ancient: pass WriteAncientAccounts by &mut (#30257) 2023-02-11 11:41:33 -08:00
Alexander Meißner 6558c8fdc9
Refactor - Delay visibility of program un-/re-/deployment (#29654)
* Use three separate HashMaps instead of the enum TxBankExecutorCacheDiff.

* Replaces all places which deploy programs by a macro.

* Adds a feature gate.

* Adjust tests.

* Makes undeployment visible immediately.
2023-02-11 11:18:25 +01:00
Jeff Washington (jwash) 51d354defb
simplify AccountsToCombine.accounts_keep_slots (#30246) 2023-02-10 13:53:03 -08:00
Jeff Washington (jwash) a46ba32e9a
remove unused metric (#30248) 2023-02-10 13:36:22 -08:00
Brooks b3c2256799
Refactors a sort's comparator fn (#30243) 2023-02-10 15:41:55 -05:00
Jeff Washington (jwash) 33cc4cc3d5
maintain sorted order on ancient target slots (#30247) 2023-02-10 12:08:13 -08:00
Jeff Washington (jwash) 157a9d725f
add ancient pack_ancient_storages (#30235)
* add ancient pack_ancient_storages

* cleanup
2023-02-10 12:06:36 -08:00
HaoranYi 0b5ae8abe1
Refactor - simplify geyser notify book keeping code and fix total_accounts stat (#30207)
* refactor common code

* fix total_accounts stat

* clippy
2023-02-10 12:18:50 -06:00
Jeff Washington (jwash) 12411e5215
pass shrink_can_be_active through shrink fns (#30244) 2023-02-10 10:06:40 -08:00
Jeff Washington (jwash) dcb2d6c8ae
collect_sort_filter_ancient_slots takes tuning by ref (#30242) 2023-02-10 09:40:13 -08:00
Jeff Washington (jwash) 0113fe340e
add ancient write_ancient_accounts_multiple_refs (#30198) 2023-02-10 09:39:30 -08:00
Brooks d27c860249
Replaces stable sort with unstable sort (#30223) 2023-02-10 09:56:11 -05:00
Jeff Washington (jwash) 128ffbdf9b
restore test to all slot combos (#30231) 2023-02-10 06:25:37 -08:00
Ryo Onodera 3e6162e69e
Add address lookup tables to minimized snapshot (#30158)
* Add address lookup tables to minimized snapshot

* Add comment for future posterity

* Add reference to the issue

* Adjust comment a bit

Co-authored-by: Andrew Fitzgerald <apfitzge@gmail.com>

---------

Co-authored-by: Andrew Fitzgerald <apfitzge@gmail.com>
2023-02-10 14:46:02 +09:00
Jeff Washington (jwash) fc8c590375
improve debugging test fn compare_all_accounts (#30218) 2023-02-09 15:06:09 -08:00
Jeff Washington (jwash) 639f3475f5
test and comment cleanup (#30226)
* test and comment cleanup

* pr feedback
2023-02-09 15:02:10 -08:00
Jeff Washington (jwash) db25ccba52
ancient add write_one_packed_storage (#30220) 2023-02-09 14:47:22 -08:00
ivandzen 2f9146e8c8
Replace txn_signature in ReplicaAccountInfo with transaction (#30189)
#### Current implementation of ReplicaAccountInfo restricts
possibilities for inflight account filtration
Current implementation includes transaction signature in
ReplicaAccountInfo. This approach does not allow to filter accounts by
matching other accounts in transaction in-flight (e. g. accept only
those accounts included in transactions for specific programs). Current
implementation forces to collect ALL accounts and transactions for some
period of time and perform such complex filtration afterwards.

#### Pass reference to transaction object instead of transaction
signature into ReplicaAccountInfo
Advanced in-flight filtration can be implemented in plugins by passing
reference to transaction for every update_account event. This change
doesn't bring any overhead comparing to current implementation (only
data type of reference is changed) and brings only minor changes in
source code.
2023-02-09 12:00:33 -08:00
Jeff Washington (jwash) c6695996b8
add test parameters to fix build (#30227)
#### Problem
build broken due to pr conflicts

#### Summary of Changes


Fixes #
<!-- OPTIONAL: Feature Gate Issue: # -->
<!-- Don't forget to add the "feature-gate" label -->
2023-02-09 11:56:12 -08:00
Jeff Washington (jwash) 62b93df1c7
change ancient tests to correct size (#30221)
#### Problem
fix tests to be more realistic

#### Summary of Changes
allocate correctly sized append vecs

Fixes #
<!-- OPTIONAL: Feature Gate Issue: # -->
<!-- Don't forget to add the "feature-gate" label -->
2023-02-09 11:41:36 -08:00
Jeff Washington (jwash) e0d354f542
fix bug in test fn compare_all_accounts (#30217)
#### Problem
test code compares sets of accounts after clean/shrink operations.
The indexing was off and new tests were giving false positives.
The test code was equivalent as long as everything was in the same
order.

#### Summary of Changes
Fix test code comparison indexes.

Fixes #
<!-- OPTIONAL: Feature Gate Issue: # -->
<!-- Don't forget to add the "feature-gate" label -->
2023-02-09 11:28:53 -08:00