Commit Graph

3867 Commits

Author SHA1 Message Date
Jeff Washington (jwash) c8ed54acbd
simplify 2 StorableAccounts impls (#30054) 2023-02-01 12:08:28 -08:00
Brooks d048a1903f
Splits up AccountsDb::bank_hashes (#30024) 2023-02-01 14:32:35 -05:00
Jeff Washington (jwash) 27e8af9621
add pub(crate) to shrink functions (#30052) 2023-02-01 11:04:51 -08:00
Tao Zhu 7bcb3cef02
report less datapoints to block_min_prioritization_fee (#30017) 2023-02-01 11:32:36 -06:00
Jeff Washington (jwash) 5d85a34591
shrink ignores ancient slots (#30026) 2023-02-01 07:23:14 -08:00
Jeff Washington (jwash) 28510f4260
update comments on get_ancient_append_vec_capacity (#30030) 2023-02-01 07:22:34 -08:00
Jeff Washington (jwash) 3e0494051e
tests adding accounts can mark alive in index (#30029) 2023-02-01 07:22:10 -08:00
Jeff Washington (jwash) d5a7b5715c
add cli arg for choosing ancient append vec format (#30025) 2023-02-01 07:21:28 -08:00
Tao Zhu a5af54669a
Limit loaded data per transaction to a fixed cap (#29743) 2023-01-31 22:51:35 -06:00
Richard Patel 179b406acb
runtime: make {Stored,Account}Meta repr(C) (#30027)
Co-authored-by: Richard Patel <ripatel@jumptrading.com>
2023-01-31 20:13:56 -05:00
Will Hickey 04a6a631bc
Bump version to v1.16 (#30028) 2023-01-31 17:48:33 -06:00
K-anon 1e3d6349aa
Add an RPC API that can query the list of Top N secondary index keys and their sizes (#28887)
Co-authored-by: K-anon <IntokuSatori@users.noreply.github.com>
2023-01-31 13:47:03 -07:00
Richard Patel 1ca78845bb
runtime: add test_type_layout (#29987)
adds unit test asserting the type layout of structures in append_vec

---------

Co-authored-by: Richard Patel <ripatel@jumptrading.com>
Co-authored-by: brooks <brooks@solana.com>
2023-01-31 11:08:55 -05:00
Jeff Washington (jwash) 4dbde40e10
measure! -> measure_us! (#29991) 2023-01-31 07:40:50 -08:00
joeaba a12bf8c003
Update maintainers references (#29997)
* update maintainers references

* chore: update maintainers reference
2023-01-31 08:07:13 -05:00
Brooks aa72aa8779
Makes AccountsDb::bank_hashes private (#29990) 2023-01-30 23:25:19 -05:00
Brooks 005dc96bc5
serde-snapshot uses methods to access AccountsDb::bank_hashes (#29989) 2023-01-30 21:32:04 -05:00
Jeff Washington (jwash) ebf3ad5604
remove unnecessary lifetimes (#29988) 2023-01-30 14:11:10 -08:00
Jeff Washington (jwash) 8a56a856d8
refactor shrink_collect (#29982)
* refactor shrink_collect

* Update runtime/src/accounts_db.rs

Co-authored-by: Andrew Fitzgerald <apfitzge@gmail.com>

---------

Co-authored-by: Andrew Fitzgerald <apfitzge@gmail.com>
2023-01-30 13:10:39 -08:00
Jeff Washington (jwash) 3e5ab0437f
remove create_ancient_append_vec (#29984) 2023-01-30 13:07:15 -08:00
Jeff Washington (jwash) ea6dac0556
refactor update_shrink_stats (#29983) 2023-01-30 11:11:13 -08:00
Xiang Zhu 856598969c
Account path add run parent with old path cleanup (#29942)
* Add run parent directory for accounts files

* fix test test_concurrent_snapshot_packaging

* review comments.  renamed the path setup function

* Addressed most of the review comments

* remove explict type def for map result

* handle create_accounts_run_and_snapshot_dirs error with expect

* update with more review comments

* minor fixes from review comments

* simplify account_filename option assignment

* handle error from create_accounts_run_and_snapshot_dirs

* use then instead of then_some for lazy evaluation

* Clean up files in the old account_path before trasitioning to the new run path

* try_exist and accounts_dir removing extra

* sync rmdir, is_dir check

* handle the account_path not deletable case
2023-01-30 10:26:43 -08:00
Tao Zhu 7ef720b76d
refactor: remove obsoleted function (#29964)
refactor: remove obsolated function
2023-01-30 09:16:49 -06:00
Alexander Meißner ac7d0ff633
Feature - Disable builtin loader ownership chains (#29959)
* Adds feature disable_builtin_loader_ownership_chains.

* Removes test_load_accounts_max_call_depth().
2023-01-29 19:27:49 +01:00
Alexander Meißner aa2e3487ba
Refactor - Simplify program accounts in transaction loading (#29728)
* Refactors the "!validated_fee_payer" case from an "else" branch to an early "return".

* Moves the early return upward.

* Removes empty entries.

* Adds account_found_and_dep_index.

* cargo fmt.

* Replaces call site of load_executable_accounts().

* Adjusts number of total loaded accounts in test_load_accounts_multiple_loaders().

* Removes test_accounts_account_not_found().

* Removes load_executable_accounts().

* Refactor back to built-in loader ownership chain loop.
2023-01-27 21:24:21 +01:00
Jeff Washington (jwash) 5e35823b66
add test_stake_account_consistency_with_rent_epoch_max_feature (#29915)
* add test_stake_account_consistency_with_rent_epoch_max_feature

* create_stake_account takes id

* use test_case

* reformat panic message
2023-01-27 11:50:33 -08:00
Tao Zhu 6f4fe37bd2
add cost to additional built-in programs (#29795)
* add cost to additional built-in programs
* update a test with compute-budget cost
2023-01-27 13:47:41 -06:00
Illia Bobyr 8fafbb0a06
doc: Fix documentation warnings and add some links (#29887) 2023-01-25 23:15:58 -08:00
Brooks 867d2581a6
Moves impl of bank_hash_info_at() into hash_internal_state() (#29913) 2023-01-25 22:24:51 -05:00
Brooks dde4b5d1ef
Adds AccountsDb::get_bank_hash_info() (#29908) 2023-01-25 17:01:01 -05:00
Brooks d76c598691
Snapshot serde test uses calculate_accounts_delta_hash() (#29905) 2023-01-25 16:26:15 -05:00
Xiang Zhu 4ebcacb4a3
Revert "Add run parent directory for accounts files (#29794)" (#29899)
This PR is causing OOM on master.  Reverting it for now.

This reverts commit 74f89d1494.
2023-01-25 10:03:01 -08:00
Yihau Chen 9193b4221d
Revert "chore: workspace inheritance (#29509)" (#29892)
This reverts commit a67d239dde.
2023-01-25 15:50:41 +08:00
Yihau Chen a67d239dde
chore: workspace inheritance (#29509)
* introduce workspace.package

* introduce workspace.dependencies

* read version from root cargo.toml

* pass check when version = { workspace = true }

* don't bump version when version = { workspace = true }

* including workspace Cargo.toml when bump version

* programs/sbf use workspace inheritance

* fix increasing cargo version ignore program/sbf/Cargo.toml
2023-01-25 13:59:59 +08:00
Xiang Zhu 74f89d1494
Add run parent directory for accounts files (#29794)
* Add run parent directory for accounts files

* fix test test_concurrent_snapshot_packaging

* review comments.  renamed the path setup function

* Addressed most of the review comments

* remove explict type def for map result

* handle create_accounts_run_and_snapshot_dirs error with expect

* update with more review comments

* minor fixes from review comments

* simplify account_filename option assignment

* handle error from create_accounts_run_and_snapshot_dirs

* use then instead of then_some for lazy evaluation
2023-01-24 16:44:35 -08:00
Brooks bda0c606a1
Renames fn to calculate_accounts_delta_hash() (#29876) 2023-01-24 18:55:56 -05:00
Brooks 655345728b
Use saturating_add_signed() (#29867) 2023-01-24 17:07:50 -05:00
Brooks d67baefd70
Use Option::unzip() (#29866) 2023-01-24 17:07:29 -05:00
behzad nouri 8a146361d1
includes rent_epoch in vote-accounts sanity checks (#29861)
https://github.com/solana-labs/solana/pull/26479
preserves rent epoch for rent-exempt accounts. Since the feature got
activated, vote accounts in stakes-cache have identical rent_epoch field
as the accounts in accounts-db. The commit verifies this in stakes-cache
sanity checks.
2023-01-24 19:47:38 +00:00
Brooks a06f8e68a5
Tests wait for initial accounts hash verification to complete (#29840) 2023-01-24 14:44:39 -05:00
Jeff Washington (jwash) 07afa22671
test fn append_single_account_with_default_hash uses AccountStorageEntry (#29839) 2023-01-24 11:43:03 -08:00
Jeff Washington (jwash) cac52b0819
trait ShrinkCollectRefs (#29830) 2023-01-24 08:43:50 -08:00
Jeff Washington (jwash) 8d4603edfb
update tests storages -> storage (#29837) 2023-01-24 07:54:28 -08:00
Jeff Washington (jwash) 79843fcbe7
update tests for 1 append vec per slot (#29836) 2023-01-24 07:53:54 -08:00
Jeff Washington (jwash) 5d24745464
if 0 alive accounts, don't create ancient append vec (#29842) 2023-01-24 07:52:31 -08:00
Tyera b35d173c04
Remove bank_transaction_count_fix logic (#29695)
* Remove bank_transaction_count_fix logic

* Fix test and add comment
2023-01-22 15:59:35 +08:00
behzad nouri 272e667cb2
deprecates Pubkey::new in favor of Pubkey::{,try_}from (#29805)
The commit deprecates Pubkey::new which lacks type-safety and instead
implements TryFrom<&[u8]> and TryFrom<Vec<u8>> for Pubkey.
2023-01-21 18:06:27 +00:00
Brooks db10073c36
Promotes accounts delta hash to its own type (#29768) 2023-01-20 14:23:56 -05:00
Brooks 6baab92ab5
Refactors calculate_capitalization() (#29790) 2023-01-20 13:38:41 -05:00
Brooks 0a0c532c86
Refactors verify_bank_hash_and_lamports() (#29783) 2023-01-19 16:14:39 -05:00
behzad nouri d463bcc5f8
removes manual implementations of Default for enums (#29779) 2023-01-19 19:25:30 +00:00
Trent Nelson c4e43f1de4
vote: encapsulate `Lockout` (#29753) 2023-01-18 19:28:28 -07:00
carllin 1753a0e3af
Revert "Do not exclude failed simple vote transactions from consensus" (#29745)
This reverts commit b4237f3f2c.
2023-01-18 15:13:13 -05:00
Jeff Washington (jwash) 9d2c71b9a3
remove type SnapshotStoragesOne (#29750) 2023-01-18 11:51:08 -08:00
Jeff Washington (jwash) aef8692c8f
remove SnapshotStorageOne (#29747) 2023-01-18 07:10:56 -08:00
Jeff Washington (jwash) f24e30e4fe
remove test method get_storages_for_slot (#29739) 2023-01-17 12:28:53 -08:00
Brooks 8c62927d59
Adds calculate_incremental_accounts_hash() (#29734)
**Node operators,** please delete your `<ledger>/calculate_accounts_hash_cache` directory (if you have one).  It has been moved/renamed to `<ledger>/accounts_hash_cache/full`.
2023-01-17 15:04:29 -05:00
Jeff Washington (jwash) e1d38c8315
remove test method slot_stores (#29736) 2023-01-17 10:15:25 -08:00
Jeff Washington (jwash) 71a746f10e
remove some write_version checking (#29733) 2023-01-17 10:14:31 -08:00
Jeff Washington (jwash) 4bfe4091b4
make storages not plural (#29732) 2023-01-17 09:50:35 -08:00
Jeff Washington (jwash) 5188da0492
remove get_slot_stores from AccountStorage (#29726) 2023-01-17 09:26:46 -08:00
Tao Zhu 1d88654116
Revert "Cap accounts data a transaction can load by its requested limit" (#29373)
Revert "Cap accounts data a transaction can load by its requested limit (#27840)"

This reverts commit 81dc2e56ac.
2023-01-17 11:34:18 +01:00
Alexander Meißner c5380d4699
Adds symmetric tests for all cases of un-/re-/deployment inside the same transaction (#29725)
Unifies test_program_sbf_upgrade_and_invoke_in_same_tx() and test_program_sbf_upgrade_self_via_cpi() into test_program_sbf_invoke_in_same_tx_as_redeployment().
Adds test_program_sbf_invoke_in_same_tx_as_deployment() and test_program_sbf_invoke_in_same_tx_as_undeployment().
2023-01-17 09:44:27 +01:00
Jeff Washington (jwash) 461dafb887
AccountStorage only holds a single append vec (#29723)
* AccountStorageReference

* tests build

* pr feedback
2023-01-16 16:22:30 -08:00
Brooks 0db14ad39c
Removes full_snapshot from CalcAccountsHashConfig (#29722) 2023-01-16 16:22:46 -05:00
Jeff Washington (jwash) ef30083319
iterate contents of AccountStorage without exposing internals (#29719)
* iterate contents of AccountStorage without exposing internals

* Update runtime/src/accounts_db.rs

Co-authored-by: Brooks <brooks@prumo.org>

* Update runtime/src/accounts_db.rs

Co-authored-by: Brooks <brooks@prumo.org>

* Update runtime/src/accounts_db.rs

Co-authored-by: Brooks <brooks@prumo.org>

* compile errors

Co-authored-by: Brooks <brooks@prumo.org>
2023-01-16 12:01:25 -08:00
Brooks 8a28877910
Use Hash::new_from_array() instead of TryFrom (#29720) 2023-01-16 10:53:38 -05:00
Jeff Washington (jwash) 977d3453a3
storage.remove does not leak SlotStores (#29664) 2023-01-15 11:06:24 -08:00
Alexander Meißner 9d1a0c52f8
Cleanup - `disable_deprecated_loader` (#29653)
* Removes the gating of disable_deprecated_loader.

* Adds create_program().
Renames create_deprecated_program() -> load_program().
Renames load_and_finalize_deprecated_program() -> load_and_finalize_program().

* Fixes test_sol_alloc_free_no_longer_deployable().
2023-01-14 13:02:10 +01:00
Jeff Washington (jwash) 064f163b70
get_account_storage_entry handles missing shrink in progress entry correctly (#29682) 2023-01-13 13:11:25 -08:00
Jeff Washington (jwash) a2933f7fa6
remove some uses of write_version (#29680) 2023-01-13 12:05:15 -08:00
Alexander Meißner ab97d37e3d
Refactor - Simplify loader utils (#29699)
* Moves all the common code of tests and benches into loader utils.

* Removes write_sbf_program().

* Removes ignored test: test_program_sbf_test_use_latest_executor2().
2023-01-13 20:38:36 +01:00
dependabot[bot] 10f8b5faa3
Bump memmap2 from 0.5.3 to 0.5.8 (#29674)
* Bump memmap2 from 0.5.3 to 0.5.8

Bumps [memmap2](https://github.com/RazrFalcon/memmap2-rs) from 0.5.3 to 0.5.8.
- [Release notes](https://github.com/RazrFalcon/memmap2-rs/releases)
- [Changelog](https://github.com/RazrFalcon/memmap2-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/RazrFalcon/memmap2-rs/compare/v0.5.3...v0.5.8)

---
updated-dependencies:
- dependency-name: memmap2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2023-01-12 19:38:38 -07:00
Jeff Washington (jwash) 6359d68f9f
get_slot_stores -> get_slot_storage_entry (#29677) 2023-01-12 09:44:53 -08:00
Brooks d0aa93de21
Refactors accounts hash cache (#29625) 2023-01-12 10:43:50 -05:00
Jeff Washington (jwash) 3412928cad
get_slot_stores -> get_slot_storage_entry (#29678) 2023-01-12 07:17:18 -08:00
Jeff Washington (jwash) b64f58614d
get_slot_stores -> get_slot_storage_entry (#29665) 2023-01-12 07:09:56 -08:00
Jeff Washington (jwash) 9d1c0c5a3c
get_slot_stores -> get_slot_storage_entry (#29663) 2023-01-12 04:08:08 -08:00
Jeff Washington (jwash) 56f8c16497
remove remaining uses of SnapshotStorages (#29660) 2023-01-11 17:58:14 -08:00
Jeff Washington (jwash) 14055fc3e2
remove type SnapshotStorage and uses (#29661) 2023-01-11 17:57:44 -08:00
Brennan Watt 69c4db27c5
Configurable hashes per tick (#29659) 2023-01-11 16:56:11 -08:00
Jeff Washington (jwash) 3e6ba72da1
remove scan_multiple_account_storages_one_slot (#29658) 2023-01-11 16:26:49 -08:00
Jeff Washington (jwash) 0e19b9c849
SortedStorages holds HashMap instead of Vec (#29632)
* SortedStorages holds HashMap instead of Vec

* add comment
2023-01-11 14:20:43 -08:00
Jeff Washington (jwash) 4b93fa85c2
renaming/refactoring for append vecs (#29655) 2023-01-11 14:20:14 -08:00
Jeff Washington (jwash) 544b9745c2
snapshot storage path uses 1 append vec per slot (#29627) 2023-01-11 12:05:15 -08:00
steviez 28071d5bf4
chore: Move new bank metric into metric submodule (#29628)
More cleanup in moving non-essential items out of bank.rs
2023-01-11 13:14:46 -06:00
Jeff Washington (jwash) 0ee9993632
decouple geyser's write_version from append vec on snapshot load (#29623) 2023-01-11 10:59:57 -08:00
Jeff Washington (jwash) 1d6999914a
factor out scan_single_account_storage (#29633) 2023-01-11 10:59:46 -08:00
HaoranYi d693167dfb
Feature: filter stake by min delegation for rewards (#29618)
filter stake by min delegation for rewards
2023-01-11 08:55:18 -06:00
dependabot[bot] 864a7d8d6f
Bump bzip2 from 0.4.3 to 0.4.4 (#29630)
* Bump bzip2 from 0.4.3 to 0.4.4

Bumps [bzip2](https://github.com/alexcrichton/bzip2-rs) from 0.4.3 to 0.4.4.
- [Release notes](https://github.com/alexcrichton/bzip2-rs/releases)
- [Commits](https://github.com/alexcrichton/bzip2-rs/commits/0.4.4)

---
updated-dependencies:
- dependency-name: bzip2
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2023-01-10 23:08:22 -07:00
Jeff Washington (jwash) 0f16120b03
remove FoundStoredAccount (#29591) 2023-01-10 13:57:34 -08:00
Jeff Washington (jwash) ad400419b5
assert write version is increasing per slot as we read from a slot (#29624) 2023-01-10 13:34:30 -08:00
Jeff Washington (jwash) c3353c27c2
verify acct size in index can easily be recalculated (#29601) 2023-01-10 13:33:36 -08:00
Jeff Washington (jwash) 9cd3bb0c4d
never try to shrink a slot that is in the write cache (#29615) 2023-01-10 12:01:37 -08:00
Jeff Washington (jwash) 1faf7d516b
while shrinking, old append vec is held outside of storage.map (#29323)
* while shrinking, old append vec is held outside of storage.map

* update comment

* rework ShrinkInProgress for clarity

* add tests and comment
2023-01-10 11:52:01 -08:00
steviez 1c1c8fb008
chore: Move bank new epoch metrics into separate file (#29599)
chore: Move bank new epoch metric into separate file

bank.rs is very large, so move the metrics reporting code into a
separate file. This reduces line count and also makes the actual logic
in bank.rs more readable by removing a big block of boilerplate code.

There are more metrics that we can do this for, but doing them one at a
time to avoid large diffs.
2023-01-10 13:15:50 -06:00
Jeff Washington (jwash) a5106e110a
just use slot to determine alive while shrinking (#29613) 2023-01-10 09:00:18 -08:00
Jeff Washington (jwash) 1c862f0b66
remove unused ret val from do_shrink_slot_store (#29616) 2023-01-10 08:12:24 -08:00
Jeff Washington (jwash) 6441ed121e
get_unique_accounts_from_storages drops plural (#29600) 2023-01-10 07:49:38 -08:00
HaoranYi d3daea65b9
use atomicu64 for bucket maxsize instead of lock (#29569) 2023-01-10 09:15:27 -06:00
Jeff Washington (jwash) 71467808d2
convert some get_slot_stores -> get_slot_storage_entry (#29549) 2023-01-10 05:34:08 -08:00
Jeff Washington (jwash) c9fe21e3c3
ancient append vecs enabled at -10k by default (#29565) 2023-01-10 05:33:43 -08:00
Jeff Washington (jwash) 1d675c6205
FoundStoredAccount removes store_id (#29590) 2023-01-10 05:28:41 -08:00
Alexander Meißner db277d320b
Adds TX signature to `TransactionContext` in debug mode (#29597)
Adds TX signature to TransactionContext in debug mode.
2023-01-10 00:54:26 +01:00
Jeff Washington (jwash) dda34b208c
get_unique_accounts_from_storages takes 1 append vec (#29588) 2023-01-09 14:53:50 -08:00
Jeff Washington (jwash) 3234af41a7
shrink_collect takes a single append vec (#29523) 2023-01-09 13:33:21 -08:00
Jeff Washington (jwash) a703019caa
feature: don't do rewrites in rent collection (#26491)
* feature: don't do rewrites in rent collection

* modify test to specifically test this feature
2023-01-09 08:14:17 -08:00
Jeff Washington (jwash) d41da2178c
test_bad_bank_hash uses write cache (#29559) 2023-01-09 08:10:28 -08:00
Jeff Washington (jwash) a171402e5c
combine_one_store_into_ancient takes 1 append vec (#29560) 2023-01-09 08:09:38 -08:00
Jeff Washington (jwash) 9692cfef98
accounts tests use store_for_tests (#29555) 2023-01-06 15:30:42 -08:00
Jeff Washington (jwash) 2f0d849c84
set rent epoch to max on new account creation (#29528)
* set rent epoch to max on new account creation

* add comment
2023-01-06 14:21:10 -08:00
Jeff Washington (jwash) 2a0a185846
accounts tests use store_for_tests (#29554) 2023-01-06 12:07:36 -08:00
Jeff Washington (jwash) 33119f8a87
convert some get_slot_stores -> get_slot_storage_entry (#29557) 2023-01-06 12:06:59 -08:00
Jeff Washington (jwash) 453b61676e
filler accounts only work gradually (#29553) 2023-01-06 12:05:32 -08:00
Jeff Washington (jwash) 3fc4015a4d
do_shrink_slot_stores -> do_shrink_slot_store (#29552) 2023-01-06 09:23:32 -08:00
Jeff Washington (jwash) 39dc034c05
ShrinkCandidates only hold 1 append vec per slot (#29525)
* ShrinkCandidates only hold 1 append vec per slot

* Fixed select_candidates_by_total_usage unit tests, use distinct slots for stores
as each slot has one store now.

* rename

Co-authored-by: Lijun Wang <83639177+lijunwangs@users.noreply.github.com>
2023-01-06 06:54:34 -08:00
Jeff Washington (jwash) 272e5a05f2
remove AccountStorage::is_empty() (#29548) 2023-01-06 06:51:25 -08:00
Jeff Washington (jwash) 8e19909c7d
stop returning remaining # stores from mark_dirty_dead_stores (#29545) 2023-01-05 19:21:01 -08:00
Xiang Zhu a0f045d2c3
storage rebuilder regex cleanup (#29408)
* storage rebuilder regex cleanup

* Update runtime/src/snapshot_utils/snapshot_storage_rebuilder.rs

Co-authored-by: apfitzge <apfitzge@gmail.com>

Co-authored-by: apfitzge <apfitzge@gmail.com>
2023-01-05 16:16:24 -08:00
Lijun Wang a6b492b2c1
Removed assert on write_version ordering (#29530)
Removed assert on write_version ordering as snapshot created by earlier version
is not honoring that.
2023-01-05 15:14:55 -08:00
Xiang Zhu a6bf68cec8
Refine appendvec sanitize error message to include path (#29541) 2023-01-05 14:43:10 -08:00
Jeff Washington (jwash) 72a7cbd6a2
shrink fns take a single append vec (#29522) 2023-01-05 13:17:04 -08:00
behzad nouri 12da2da389
fixes errors from clippy::redundant_clone (#29536)
https://rust-lang.github.io/rust-clippy/master/index.html#redundant_clone
2023-01-05 18:42:19 +00:00
Jeff Washington (jwash) 3b7ebfe50e
migrate tests to use get_storage_for_slot (#29518) 2023-01-05 10:38:58 -08:00
Jeff Washington (jwash) d3f45c6901
shrink_slot_forced uses a single append vec (#29521) 2023-01-05 10:38:05 -08:00
behzad nouri d87128e02c
fixes errors from clippy::needless_borrow (#29535)
https://rust-lang.github.io/rust-clippy/master/index.html#needless_borrow
2023-01-05 18:21:56 +00:00
Brooks d611cc2e75
Fixes format string (#29533) 2023-01-05 13:06:00 -05:00
behzad nouri 5c9beef498
fixes errors from clippy::useless_conversion (#29534)
https://rust-lang.github.io/rust-clippy/master/index.html#useless_conversion
2023-01-05 18:05:32 +00:00
Lijun Wang 1e8a8e07b6
Stream the executed transaction count in the block notification (#29272)
Problem

The plugins need to know when all transactions for a block have been all notified to serve getBlock request correctly. As block and transaction notifications are sent asynchronously to each other it will be difficult.

Summary of Changes

Include the executed transaction count in block notification which can be used to check if all transactions have been notified.
2023-01-05 09:36:19 -08:00
Jeff Washington (jwash) dbac798ef6
process_storage_slot takes a single append vec (#29519) 2023-01-05 07:33:47 -08:00
Jeff Washington (jwash) a140f06a67
use get_storage_for_slot() in tests (#29517) 2023-01-05 07:32:00 -08:00
Jeff Washington (jwash) 8cdf5cdb50
get_storage_to_move_to_ancient_append_vec returns a single append vec (#29482)
get_storages_to_move_to_ancient_append_vec returns a single append vec
2023-01-05 07:31:26 -08:00
Jeff Washington (jwash) 221f547ba9
remove store_ids from a few shrink data structures (#29360) 2023-01-05 07:30:43 -08:00
Brooks 05d5aed4d8
Adds SnapshotError::IoWithSourceAndFile (#29527) 2023-01-05 09:08:15 -05:00
Jeff Washington (jwash) fc8cd82309
migrate tests to get_slot_storage_entry (#29515) 2023-01-04 13:12:48 -08:00
Jeff Washington (jwash) 4e636ab153
cleanup ancient append vec tests (#29514) 2023-01-04 12:39:42 -08:00
Jeff Washington (jwash) 150ac80cfa
filter_storages ignores reclaims (#29512) 2023-01-04 12:38:30 -08:00
Jeff Washington (jwash) ef72140173
remove unnecessary to_vec() (#29516) 2023-01-04 12:38:16 -08:00
Jeff Washington (jwash) 093b1a5e10
get_storages_for_slot uses get_slot_storage_entry (#29498) 2023-01-04 10:15:56 -08:00
Jeff Washington (jwash) 10135dbd10
should_move_to_ancient_append_vec works with a single storage (#29484) 2023-01-04 07:03:46 -08:00
Illia Bobyr d7bd1bf970
bank: Record non-vote transaction count (#29383)
A subsequent change to `SamplePerformanceService` introduces non-vote transaction counts, which `bank`s need to store.

Part of work on https://github.com/solana-labs/solana/issues/29159
2023-01-03 14:46:20 -08:00
Jeff Washington (jwash) 89deecb979
add AccountStorage.is_empty_entry for tests (#29489) 2023-01-03 12:36:31 -08:00
Jeff Washington (jwash) 0f2bfa2b02
cleanup get_snapshot_storages (#29488)
* cleanup get_snapshot_storages

* pr feedback
2023-01-03 12:22:28 -08:00
Jeff Washington (jwash) 95d2f002bd
add AccountStorage.get_slot_storage_entry (#29480) 2023-01-03 12:10:34 -08:00
Jeff Washington (jwash) 9ee4555f53
remove should_retain from mark_dirty_dead_stores (#29358) 2023-01-03 09:53:43 -08:00
Jeff Washington (jwash) 102dc08378
get_snapshot_storages removes call to AccountStorage.get (#29466) 2023-01-03 09:15:07 -08:00
Jeff Washington (jwash) 9d15ead873
remove single use AccountStorage.slot_store_count (#29479) 2023-01-03 09:12:27 -08:00
Jeff Washington (jwash) 10eadb6dd9
add test method assert_no_storages_at_slot() (#29483) 2023-01-03 08:59:21 -08:00
Jeff Washington (jwash) ca9166201c
add test method get_and_assert_single_storage (#29481) 2023-01-03 08:38:15 -08:00
Jeff Washington (jwash) 04508ccfa7
add AccountStorage.is_empty (#29478) 2023-01-03 08:35:10 -08:00
Jeff Washington (jwash) 8c878daada
refactor RecycleStores::add_entries (#29475)
* refactor RecycleStores::add_entries

* pr feedback
2023-01-03 07:56:30 -08:00
Jeff Washington (jwash) 89e733c2b3
remove unnecessary type (#29473) 2023-01-03 06:34:05 -08:00
Xiang Zhu 3363c08ac0
Move async remove to snapshot_utils.rs (#29406) 2023-01-03 06:15:32 -08:00
Jeff Washington (jwash) 1fd1e57f50
add test (#29471) 2023-01-02 20:53:43 -08:00
Jeff Washington (jwash) e86628a823
filter get_snapshot_storages for requested_slots earlier (#29465)
* filter get_snapshot_storages for requested_slots earlier

* Update runtime/src/accounts_db.rs

Co-authored-by: apfitzge <apfitzge@gmail.com>

Co-authored-by: apfitzge <apfitzge@gmail.com>
2023-01-02 20:53:30 -08:00
Jeff Washington (jwash) 4cc1890f00
skip_rewrites will only be feature driven (#29468) 2023-01-02 20:15:42 -08:00
Jeff Washington (jwash) 64466725d4
remove skip rewrite code from collect_rent_from_accounts (#29472) 2023-01-02 20:15:05 -08:00
Jeff Washington (jwash) 6294d1eff1
cleanup in account_storage.rs (#29467) 2023-01-02 14:55:02 -08:00
Jeff Washington (jwash) a31927419b
test_rent_exempt_temporal_escape works in passes (#29460) 2023-01-02 14:32:09 -08:00
Jeff Washington (jwash) d5fdb7bd0a
remove metric time_hashing_skipped_rewrites_us (#29470) 2023-01-02 14:03:04 -08:00
Jeff Washington (jwash) 9615965219
decrease frequency of random shrink of ancient append vec (#29462) 2023-01-02 13:29:04 -08:00
Jeff Washington (jwash) bf93a1e13f
migrate tests to not use AccountStorage::get (#29464) 2023-01-02 13:02:09 -08:00
Jeff Washington (jwash) e99afb4e85
handle ancient overflow case correctly (#29458) 2023-01-02 12:39:59 -08:00
Jeff Washington (jwash) 8397f295c0
simplify get_storages_for_slot (#29463) 2023-01-02 12:38:48 -08:00
Jeff Washington (jwash) 38d771a823
add test and comments (#29459) 2023-01-02 12:38:21 -08:00
Jeff Washington (jwash) 67c7e343a2
test permutations of set_exempt_rent_epoch_max (#29461) 2023-01-02 11:59:50 -08:00
Jeff Washington (jwash) cf1aa4beea
feature: set rent_epoch to Epoch::MAX (#28690)
* check android builds

* feature: set rent_epoch to Epoch::MAX

* tweaks

* Update runtime/src/rent_collector.rs

Co-authored-by: Brooks Prumo <brooks@prumo.org>

* simplify changes to tests

* back out some test changes

* calculate_rent_result passes through Exempt

* move calc outside loop

* if rent epoch is already max, use 'NoRentCollectionNow'

Co-authored-by: Brooks Prumo <brooks@prumo.org>
2023-01-02 07:51:08 -08:00
Jeff Washington (jwash) 3d72012a31
cleanup 'shrinking_in_progress' (#29359) 2023-01-02 07:41:36 -08:00
Jeff Washington (jwash) 6edbb61fa0
add test_shrink_ancient_overflow (#29363) 2023-01-02 07:09:53 -08:00
Tyera 1e0d3931fd
Fix "tranaction" typo in code base (#29347)
Fix typos
2022-12-21 18:02:36 -07:00
Jeff Washington (jwash) 0f7ef085a0
introduce ShrinkInProgress (#29329)
* introduce ShrinkInProgress

* remove redundant check

* add comments
2022-12-21 09:03:29 -08:00
Jeff Washington (jwash) 4a64f6d421
test_utils::create_test_accounts pre-allocates an append vec first (#29336)
* test_utils::create_test_accounts pre-allocates an append vec first

* remove comment
2022-12-21 09:02:42 -08:00
Tao Zhu 0244b0144a
Add metrics for min/max priority fee per slot, and counters for fee/non-fee transactions (#29330)
* Add metrics for min/max priority fee per slot, and counters for fee/non-fee txs

* get fee range of prioritized transactions only

* Update runtime/src/prioritization_fee.rs

Co-authored-by: Trent Nelson <trent.a.b.nelson@gmail.com>

* Update runtime/src/prioritization_fee.rs

Co-authored-by: Trent Nelson <trent.a.b.nelson@gmail.com>

* fix format

Co-authored-by: Trent Nelson <trent.a.b.nelson@gmail.com>
2022-12-21 16:42:26 +00:00
Tao Zhu 926debd18c
exclude Vote transactions from updating min-fee-cache (#29341) 2022-12-20 20:05:46 +00:00
Jeff Washington (jwash) c3ca5b78b2
remove acctdb.min_num_stores (#29335) 2022-12-20 10:39:20 -08:00
Jeff Washington (jwash) 098b86b883
test_minimize_accounts_db uses write cache correctly (#29333) 2022-12-20 09:50:56 -08:00
Brooks 053775ad77
Elides unnecessary lifetimes (#29299) 2022-12-20 12:44:17 -05:00
Jeff Washington (jwash) c816157979
shrink avoids reshaping data by using StorableAccounts trait (#29307) 2022-12-19 12:29:21 -08:00
Jeff Washington (jwash) 3eca364190
remove metrics from prior to rent_epoch feature activation (#29322) 2022-12-19 08:18:02 -08:00
Brennan Watt 86b2e545e1
Prune redundant const SLOT_MS (#29278)
* Alias redundant const SLOT_MS to DEFAULT_MS_PER_SLOT

* Slate SLOT_MS for deprecation

* Add doc comments

Co-authored-by: Brooks Prumo <brooks@prumo.org>
2022-12-16 08:05:09 -08:00
Jeff Washington (jwash) 19f13a0bea
refactor get_store_for_shrink (#29284) 2022-12-15 22:10:39 -08:00
Brooks Prumo 1e0a0e0ced
Cleans up CacheHashData (#29267) 2022-12-15 14:16:27 -05:00
Jeff Washington (jwash) 5a687fa818
make AccountStorage::map private (#29268) 2022-12-14 20:03:25 -08:00
Jeff Washington (jwash) 3a7b67ea55
refactor unwrap_or_default for type that will soon not have a default (#29257) 2022-12-14 15:11:36 -08:00
Jeff Washington (jwash) 49a136a374
move AccountStorage out of accounts_db.rs (#29261) 2022-12-14 15:08:39 -08:00
Jeff Washington (jwash) f9db75768c
Vec<Arc<AccountStorageEntry>> becomes SnapshotStorage (#29253) 2022-12-14 12:18:43 -08:00
Jeff Washington (jwash) fab0ce21a2
remove shrink_candidate_slots_v1 (#29252) 2022-12-14 08:57:16 -08:00
Jeff Washington (jwash) b77bef4ee3
write_version -> write_version_obsolete (#29176) 2022-12-14 07:43:40 -08:00
Jeff Washington (jwash) b8e25282c2
remove rebuild_bank_from_snapshots(caching_enabled) (#29247)
* fix tests broken by add_root change

* remove rebuild_bank_from_snapshots(caching_enabled)
2022-12-13 16:33:06 -08:00
Jeff Washington (jwash) f355066456
remove caching_enabled from serde_snapshot (#29246)
* fix tests broken by add_root change

* remove caching_enabled from serde_snapshot
2022-12-13 16:32:43 -08:00
Jeff Washington (jwash) 62be54a75e
remove process_stale_slot_v1 and other fns (#29244) 2022-12-13 16:32:24 -08:00
Jeff Washington (jwash) 4ae12bf5fa
fix tests broken by add_root change (#29248) 2022-12-13 14:43:52 -08:00
Jeff Washington (jwash) 250ec3f2a4
improve write_version usage in geyser scan from snapshot (#29223) 2022-12-13 14:40:12 -08:00
Alexander Meißner 721496b900
Feature - Program redeployment cooldown (#29136)
* Registers the feature enable_program_redeployment_cooldown.

* Adds redeployment slot constraint.

* Adds test to assert that closed programs can not be reopened.

* Ensure that program close truncates the account data.

* Adds set_sysvar_for_tests() to SBF program tests.
2022-12-13 22:54:24 +01:00
Jeff Washington (jwash) 3e649d2aa0
remove add_root(caching_enabled) (#29245) 2022-12-13 11:26:28 -08:00
Jeff Washington (jwash) 8da82e68fc
remove handle_snapshot_request(caching_enabled) (#29241) 2022-12-13 09:31:47 -08:00
dependabot[bot] dca5d7f9b4
chore: bump test-case from 2.1.0 to 2.2.2 (#28184)
Bumps [test-case](https://github.com/frondeus/test-case) from 2.1.0 to 2.2.2.
- [Release notes](https://github.com/frondeus/test-case/releases)
- [Changelog](https://github.com/frondeus/test-case/blob/master/CHANGELOG.md)
- [Commits](https://github.com/frondeus/test-case/compare/v2.1.0...v2.2.2)

---
updated-dependencies:
- dependency-name: test-case
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-12-13 16:07:50 +00:00
Jeff Washington (jwash) 1a0a465416
handle unclean roots explicitly during index generation (#29230) 2022-12-13 07:51:33 -08:00
Jeff Washington (jwash) b779134020
remove process_stale_slot_with_budget (#29236) 2022-12-13 07:50:14 -08:00
Jeff Washington (jwash) c8e7c706df
remove bank_from_streams(caching_enabled) (#29235) 2022-12-13 07:49:33 -08:00