Commit Graph

3827 Commits

Author SHA1 Message Date
Yihau Chen df3ef111f7
chore: workspace inheritance (#29893)
* introduce workspace.package

* introduce workspace.dependencies

* read version from root cargo.toml

* pass check when version = { workspace = true }

* don't bump version when version = { workspace = true }

* including workspace Cargo.toml when bump version

* programs/sbf use workspace inheritance

* fix increasing cargo version ignore program/sbf/Cargo.toml
2023-02-23 22:01:54 +08:00
Pankaj Garg 0e13dcca00
Replenish executor cache eagerly during transaction processing (#30425) 2023-02-23 05:13:39 -08:00
Michael Vines 5136ed3448
Update homepage value for all crates (#30444) 2023-02-23 02:20:18 +00:00
Yueh-Hsuan Chiang ac7e7aa8f0
Move account-meta structs from append_vec.rs to account_meta.rs (#30443)
#### Problem
As we start supporting new storage formats, those account-meta
structs will be shared by different storage formats and thus need
a new home.

#### Summary of Changes
This PR creates meta.rs under account_storage and moves all the account-meta
structs out from append_vec.rs.
2023-02-22 16:10:34 -08:00
Yueh-Hsuan Chiang 833cb75c20
Fix the dependency of accounts_file.rs (#30423)
Replace the use of append_vec::* in accounts_file.rs
2023-02-22 11:13:58 -08:00
Jeff Washington (jwash) a78f763896
simplify remove_if_slot_list_empty_value (#30436) 2023-02-22 10:46:12 -08:00
Jeff Washington (jwash) 6a66d4631c
remove 'stored_size' from account info (#30407) 2023-02-22 07:04:21 -08:00
Jeff Washington (jwash) db8764f98d
hold items with ref count != 1 in memory (#30414)
* hold items with ref count != 1 in memory

* fix tests
2023-02-22 06:57:59 -08:00
Brooks 1689586213
Uses a channel for AHV -> SPS (#30406) 2023-02-22 03:36:29 +00:00
K-anon 2e4b8ea8bb
Revert "Add test for executor cache runtime feature upgrades. (#30382)" (#30424)
This reverts commit c5a24e11ba.

Co-authored-by: K-anon <IntokuSatori@users.noreply.github.com>
2023-02-21 15:47:39 -07:00
Pankaj Garg b1f5b0d790
Find and load missing programs in LoadedPrograms cache (#30275)
* Find and load missing programs in LoadedPrograms cache

- filter program accounts in a transaction batch
- filter the accounts that are missing in LoadedPrograms cache
- load the programs before processing the transactions
- unit tests

* address review comments

* fix clippy

* address review comments

* fix test

* fix more tests
2023-02-21 12:53:35 -08:00
Brooks 35328ca63d
Makes AccountsHash an enum (#30416) 2023-02-21 15:20:51 -05:00
K-anon c5a24e11ba
Add test for executor cache runtime feature upgrades. (#30382)
Co-authored-by: K-anon <IntokuSatori@users.noreply.github.com>
2023-02-21 12:25:29 -07:00
Jeff Washington (jwash) 568a4b1ca0
move AccountInfo bits from size to offset (#30408)
* move AccountInfo bits from size to offset

* feedback
2023-02-21 10:08:37 -08:00
HaoranYi 22f6dfd98e
typo (#30393) 2023-02-21 09:59:45 -06:00
Brooks bcc4bc80c9
Removes unnecessary derives from Accounts{Delta}Hash (#30392) 2023-02-20 16:00:53 -05:00
Jeff Washington (jwash) 46b42e1888
add hash calc scan_chunks metric (#30333) 2023-02-20 12:55:48 -08:00
Brooks ec288ded9b
Serde snapshot uses own types (#30391) 2023-02-20 12:25:15 -05:00
Brooks a8e07c41fe
Adds `is_incremental_accounts_hash_feature_enabled` to `AccountsPackage` (#30368) 2023-02-20 12:21:38 -05:00
Brooks ad6a3abd8b
Uses set_accounts_{delta,}_hash_for_tests() in serde tests (#30390) 2023-02-17 16:00:25 -05:00
Brooks f91e63dfe9
Upgrades Rust to 1.67.1 (#29947) 2023-02-17 14:08:27 -05:00
Pankaj Garg 3a4ba72daf
Remove executors trait and replace with LoadedProgram (#30348)
* Remove executors trait and replace with LoadedProgram

* fill in deployment slot

* address review comments

* fix clippy warnings

* address review comments

* fix failures caught by sbf tests
2023-02-17 12:47:45 +01:00
Pankaj Garg 6c58033df5
API to enumerate executable program accounts in tx batch (#30371) 2023-02-16 13:29:38 -08:00
Brooks febaf36e6d
Apply clippy fixes for future rust upgrade (#30363) 2023-02-16 16:12:51 +00:00
Xiang Zhu fa22389b22
Add version and state_complete flag into bank snapshot (#30099)
* Add version and state_complate flag into bank snapshot

* Add test_get_highest_full_snapshot_slot_and_path

* fix build checks

* fix slot_deltas panic in test_get_highest_full_snapshot_slot_and_path

* make bank rooted for test_get_highest_full_snapshot_slot_and_path

* fix bank scope problem in test_get_highest_full_snapshot_slot_and_path

* minor cleanup

* misc review issues

* remove unneeded error defs and functions

* Fix state_complete macro naming

* fix snapshot_version
2023-02-15 22:38:40 -08:00
Yueh-Hsuan Chiang ce4251be99
Fix the return type of AccountsFile::account_matches_owners() (#30354)
Update AccountsFile::account_matches_owners() to return the matched index
2023-02-16 03:34:36 +00:00
Yueh-Hsuan Chiang aeb6df3507
Abstract out AppendVec into AccountsFile enum (#29815)
Abstracts out AppendVec into AccountsFile enum.
This will allow different implementations of AccountsFile that support the AccountsDB.
2023-02-15 17:24:04 -08:00
Brooks 5680c76c86
Refactors Accounts::new() fns (#30343) 2023-02-15 18:34:52 -05:00
HaoranYi 615d1a8b69
Extract BankHashLamportsVerifyConfig (#30320)
* refactor BankHashLamportsVerifyConfig

* clippy

* comments

* fix bench
2023-02-15 16:03:50 -06:00
Andrew Fitzgerald b86bfbb5c5
measure_us! use Instant and duration_to_us internally (#30339) 2023-02-15 12:43:47 -08:00
Pankaj Garg 3bea5fc326
Return index of matching owner for a program account (#30337) 2023-02-15 19:41:29 +00:00
Xiang Zhu 4909267c88
Add accounts hard-link files into the bank snapshot directory (#29496)
* Add accounts hard-link files into the bank snapshot directory

* Small adjustments and fixes.

* Address some of the review issues

* Fix compilation issues

* Change the latest slot snapshot storage from VecDeque to Option

* IoWithSourceAndFile and expanded comments on accounts

* last_slot_snapshot_storages in return value

* Update comments following the review input

* rename dir_accounts_hard_links to hard_link_path

* Add dir_full_state flag for add_bank_snapshot

* Let appendvec files hardlinking work with multiple accounts paths across multiple partitions

* Fixes for rebasing

* fix tests which generates account_path without adding run/

* rebasing fixes

* fix account path test failures

* fix test test_concurrent_snapshot_packaging

* review comments.  renamed the path setup function

* Addressed most of the review comments

* update with more review comments

* handle error from create_accounts_run_and_snapshot_dirs

* fix rebasing duplicate

* minor accounts_dir path cleanup

* minor cleanup, remove commented code

* misc review comments

* build error fix

* Fix test_incremental_snapshot_download_with_crossing_full_snapshot_interval_at_startup

* fix build error on MAX_BANK_SNAPSHOTS_TO_RETAIN

* rebase fix, update hardlink filename

* minor comment spelling fix

* rebasing fixes

* fix rebase issues; with_extension

* comments changes for review

* misc minor review issues

* bank.fill_bank_with_ticks_for_tests

* error handling on appendvec path

* fix use_jit

* minor comments refining

* Remove type AccountStorages

* get_account_path_from_appendvec_path return changed to Option

* removed appendvec_path.to_path_buf in create_accounts_run_and_snapshot_dirs

* add test_get_snapshot_accounts_hardlink_dir

* update last_snapshot_storages comment

* update last_snapshot_storages comment

* symlink map_err

* simplify test_get_snapshot_accounts_hardlink_dir with fake paths

* log last_snapshot_storages at the end of the loop
2023-02-15 09:52:07 -08:00
Brooks 2e885cf99a
Removes calls to insert_default_bank_hash_stats() in tests (#30311) 2023-02-14 15:00:38 -05:00
Jeff Washington (jwash) b9b8cd9ccc
misc. cleanup (#30295) 2023-02-14 12:00:13 -08:00
HaoranYi 27f18f96e8
typos, clippy and saturating_add_assign (#30272)
* typo

* remove clippy allow

---------

Co-authored-by: haoran <haoran@mbook>
2023-02-14 10:03:16 -06:00
Jeff Washington (jwash) 253517cba3
add combine_ancient_slots_packed (#30276)
* add combine_ancient_slots_new

* pr feedback

* implement ==
2023-02-14 06:53:31 -08:00
Jeff Washington (jwash) 9988978379
add ancient finish_combine_ancient_slots_packed (#30261) 2023-02-13 09:10:13 -08:00
Tao Zhu 60bfc2524b
implement From trait for CostTrackerError to TransactionError (#30267)
implement From trait for CostTrackerError to TransactionError
2023-02-13 11:06:39 -06:00
Jeff Washington (jwash) bcd7cf0821
add ancient write_packed_storages (#30260) 2023-02-13 07:25:47 -08:00
Jeff Washington (jwash) b3778d5044
cleanup (#30251) 2023-02-12 13:42:03 -08:00
Jeff Washington (jwash) dc605b5126
rename parameter and update comments (#30259) 2023-02-11 12:13:03 -08:00
Jeff Washington (jwash) c837af1af4
ancient: pass WriteAncientAccounts by &mut (#30257) 2023-02-11 11:41:33 -08:00
Alexander Meißner 6558c8fdc9
Refactor - Delay visibility of program un-/re-/deployment (#29654)
* Use three separate HashMaps instead of the enum TxBankExecutorCacheDiff.

* Replaces all places which deploy programs by a macro.

* Adds a feature gate.

* Adjust tests.

* Makes undeployment visible immediately.
2023-02-11 11:18:25 +01:00
Jeff Washington (jwash) 51d354defb
simplify AccountsToCombine.accounts_keep_slots (#30246) 2023-02-10 13:53:03 -08:00
Jeff Washington (jwash) a46ba32e9a
remove unused metric (#30248) 2023-02-10 13:36:22 -08:00
Brooks b3c2256799
Refactors a sort's comparator fn (#30243) 2023-02-10 15:41:55 -05:00
Jeff Washington (jwash) 33cc4cc3d5
maintain sorted order on ancient target slots (#30247) 2023-02-10 12:08:13 -08:00
Jeff Washington (jwash) 157a9d725f
add ancient pack_ancient_storages (#30235)
* add ancient pack_ancient_storages

* cleanup
2023-02-10 12:06:36 -08:00
HaoranYi 0b5ae8abe1
Refactor - simplify geyser notify book keeping code and fix total_accounts stat (#30207)
* refactor common code

* fix total_accounts stat

* clippy
2023-02-10 12:18:50 -06:00
Jeff Washington (jwash) 12411e5215
pass shrink_can_be_active through shrink fns (#30244) 2023-02-10 10:06:40 -08:00