Commit Graph

3791 Commits

Author SHA1 Message Date
Brooks 06e801a8a5
Moves BankHashInfo into serde_snapshot (#30120) 2023-02-03 13:17:50 -05:00
Tao Zhu 4293f11cf1
feature gate to enable compute_budget::request_heap_frame on mainnetBeta (#30077) 2023-02-03 09:21:57 -06:00
Brooks 9b297ac89c
Removes {get,set}_bank_hash_info() test helper functions (#30101) 2023-02-03 10:17:38 -05:00
Jeff Washington (jwash) eeb622c4ed
test create storages can ignore TempFile (#30094) 2023-02-03 07:12:33 -08:00
Brennan 8312141f87
Move bank tests to separate file (#29912) 2023-02-02 16:32:22 -08:00
Brooks 3249e4a123
Removes {get,set}_bank_hash_info_from_snapshot() (#30087) 2023-02-02 16:50:44 -05:00
Jeff Washington (jwash) ae7803a55e
add test accounts with custom data size (#30089) 2023-02-02 10:47:12 -08:00
Jeff Washington (jwash) 890f29be0c
fix alive_bytes test fn (#30088) 2023-02-02 10:46:46 -08:00
Jeff Washington (jwash) d563d37860
combine lookups in StorableAccountsWithHashesAndWriteVersions (#30061) 2023-02-02 10:46:17 -08:00
Jeff Washington (jwash) 385a6e01f7
combine some stats into ShrinkStatsSub (#30034) 2023-02-02 06:21:36 -08:00
Brooks a6f43c2e3d
Slot 0 is not as special in set_bank_hash_info_from_snapshot() (#30071) 2023-02-02 08:57:11 -05:00
Jeff Washington (jwash) 7ea0a72cc4
total_bytes -> capacity (#30065) 2023-02-02 05:46:13 -08:00
Brooks 79e9c9f8b7
Do not populate slot 0 with default accounts delta hash/accounts hash (#30069) 2023-02-01 20:43:26 -05:00
Jeff Washington (jwash) 22afdb21ec
rename original_bytes -> capacity (#30064) 2023-02-01 16:58:41 -08:00
Xiang Zhu f107b8b607
Add slot deltas into the bank snapshot directory (#29409) 2023-02-01 16:51:32 -08:00
dependabot[bot] 232e252014
Bump serde from 1.0.144 to 1.0.152 (#29696)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
Co-authored-by: Tyera <tyera@solana.com>
2023-02-01 16:27:17 -07:00
Brooks 33165040bf
Only insert default BankHashStats in Accounts::new_from_parent() (#30063) 2023-02-01 17:47:55 -05:00
Tao Zhu 090b990e15
remove congestion multiplier from transaction fee calculation (#29828) 2023-02-01 16:26:50 -06:00
Jeff Washington (jwash) 67d3beeb35
remove unnecessary lifetime (#30019) 2023-02-01 13:18:46 -08:00
Jeff Washington (jwash) e11c22a13c
clean up remove_old_stores_shrink stats (#30018) 2023-02-01 13:18:07 -08:00
Jeff Washington (jwash) 1c4e84b8ca
impl default version of contains_multiple_slots (#30055) 2023-02-01 13:01:01 -08:00
Jeff Washington (jwash) 5769dbf36b
is_cached_store, bool -> Storage enum (#30057) 2023-02-01 12:33:27 -08:00
Tao Zhu 0cf93da4b4
remove obsoleted else-if clause (#30020) 2023-02-01 14:28:35 -06:00
Jeff Washington (jwash) 3b2c2ebf9e
add StorableAccountsBySlot to move accounts from multiple slots to 1 (#30023) 2023-02-01 12:10:12 -08:00
Jeff Washington (jwash) c8ed54acbd
simplify 2 StorableAccounts impls (#30054) 2023-02-01 12:08:28 -08:00
Brooks d048a1903f
Splits up AccountsDb::bank_hashes (#30024) 2023-02-01 14:32:35 -05:00
Jeff Washington (jwash) 27e8af9621
add pub(crate) to shrink functions (#30052) 2023-02-01 11:04:51 -08:00
Tao Zhu 7bcb3cef02
report less datapoints to block_min_prioritization_fee (#30017) 2023-02-01 11:32:36 -06:00
Jeff Washington (jwash) 5d85a34591
shrink ignores ancient slots (#30026) 2023-02-01 07:23:14 -08:00
Jeff Washington (jwash) 28510f4260
update comments on get_ancient_append_vec_capacity (#30030) 2023-02-01 07:22:34 -08:00
Jeff Washington (jwash) 3e0494051e
tests adding accounts can mark alive in index (#30029) 2023-02-01 07:22:10 -08:00
Jeff Washington (jwash) d5a7b5715c
add cli arg for choosing ancient append vec format (#30025) 2023-02-01 07:21:28 -08:00
Tao Zhu a5af54669a
Limit loaded data per transaction to a fixed cap (#29743) 2023-01-31 22:51:35 -06:00
Richard Patel 179b406acb
runtime: make {Stored,Account}Meta repr(C) (#30027)
Co-authored-by: Richard Patel <ripatel@jumptrading.com>
2023-01-31 20:13:56 -05:00
Will Hickey 04a6a631bc
Bump version to v1.16 (#30028) 2023-01-31 17:48:33 -06:00
K-anon 1e3d6349aa
Add an RPC API that can query the list of Top N secondary index keys and their sizes (#28887)
Co-authored-by: K-anon <IntokuSatori@users.noreply.github.com>
2023-01-31 13:47:03 -07:00
Richard Patel 1ca78845bb
runtime: add test_type_layout (#29987)
adds unit test asserting the type layout of structures in append_vec

---------

Co-authored-by: Richard Patel <ripatel@jumptrading.com>
Co-authored-by: brooks <brooks@solana.com>
2023-01-31 11:08:55 -05:00
Jeff Washington (jwash) 4dbde40e10
measure! -> measure_us! (#29991) 2023-01-31 07:40:50 -08:00
joeaba a12bf8c003
Update maintainers references (#29997)
* update maintainers references

* chore: update maintainers reference
2023-01-31 08:07:13 -05:00
Brooks aa72aa8779
Makes AccountsDb::bank_hashes private (#29990) 2023-01-30 23:25:19 -05:00
Brooks 005dc96bc5
serde-snapshot uses methods to access AccountsDb::bank_hashes (#29989) 2023-01-30 21:32:04 -05:00
Jeff Washington (jwash) ebf3ad5604
remove unnecessary lifetimes (#29988) 2023-01-30 14:11:10 -08:00
Jeff Washington (jwash) 8a56a856d8
refactor shrink_collect (#29982)
* refactor shrink_collect

* Update runtime/src/accounts_db.rs

Co-authored-by: Andrew Fitzgerald <apfitzge@gmail.com>

---------

Co-authored-by: Andrew Fitzgerald <apfitzge@gmail.com>
2023-01-30 13:10:39 -08:00
Jeff Washington (jwash) 3e5ab0437f
remove create_ancient_append_vec (#29984) 2023-01-30 13:07:15 -08:00
Jeff Washington (jwash) ea6dac0556
refactor update_shrink_stats (#29983) 2023-01-30 11:11:13 -08:00
Xiang Zhu 856598969c
Account path add run parent with old path cleanup (#29942)
* Add run parent directory for accounts files

* fix test test_concurrent_snapshot_packaging

* review comments.  renamed the path setup function

* Addressed most of the review comments

* remove explict type def for map result

* handle create_accounts_run_and_snapshot_dirs error with expect

* update with more review comments

* minor fixes from review comments

* simplify account_filename option assignment

* handle error from create_accounts_run_and_snapshot_dirs

* use then instead of then_some for lazy evaluation

* Clean up files in the old account_path before trasitioning to the new run path

* try_exist and accounts_dir removing extra

* sync rmdir, is_dir check

* handle the account_path not deletable case
2023-01-30 10:26:43 -08:00
Tao Zhu 7ef720b76d
refactor: remove obsoleted function (#29964)
refactor: remove obsolated function
2023-01-30 09:16:49 -06:00
Alexander Meißner ac7d0ff633
Feature - Disable builtin loader ownership chains (#29959)
* Adds feature disable_builtin_loader_ownership_chains.

* Removes test_load_accounts_max_call_depth().
2023-01-29 19:27:49 +01:00
Alexander Meißner aa2e3487ba
Refactor - Simplify program accounts in transaction loading (#29728)
* Refactors the "!validated_fee_payer" case from an "else" branch to an early "return".

* Moves the early return upward.

* Removes empty entries.

* Adds account_found_and_dep_index.

* cargo fmt.

* Replaces call site of load_executable_accounts().

* Adjusts number of total loaded accounts in test_load_accounts_multiple_loaders().

* Removes test_accounts_account_not_found().

* Removes load_executable_accounts().

* Refactor back to built-in loader ownership chain loop.
2023-01-27 21:24:21 +01:00
Jeff Washington (jwash) 5e35823b66
add test_stake_account_consistency_with_rent_epoch_max_feature (#29915)
* add test_stake_account_consistency_with_rent_epoch_max_feature

* create_stake_account takes id

* use test_case

* reformat panic message
2023-01-27 11:50:33 -08:00
Tao Zhu 6f4fe37bd2
add cost to additional built-in programs (#29795)
* add cost to additional built-in programs
* update a test with compute-budget cost
2023-01-27 13:47:41 -06:00
Illia Bobyr 8fafbb0a06
doc: Fix documentation warnings and add some links (#29887) 2023-01-25 23:15:58 -08:00
Brooks 867d2581a6
Moves impl of bank_hash_info_at() into hash_internal_state() (#29913) 2023-01-25 22:24:51 -05:00
Brooks dde4b5d1ef
Adds AccountsDb::get_bank_hash_info() (#29908) 2023-01-25 17:01:01 -05:00
Brooks d76c598691
Snapshot serde test uses calculate_accounts_delta_hash() (#29905) 2023-01-25 16:26:15 -05:00
Xiang Zhu 4ebcacb4a3
Revert "Add run parent directory for accounts files (#29794)" (#29899)
This PR is causing OOM on master.  Reverting it for now.

This reverts commit 74f89d1494.
2023-01-25 10:03:01 -08:00
Yihau Chen 9193b4221d
Revert "chore: workspace inheritance (#29509)" (#29892)
This reverts commit a67d239dde.
2023-01-25 15:50:41 +08:00
Yihau Chen a67d239dde
chore: workspace inheritance (#29509)
* introduce workspace.package

* introduce workspace.dependencies

* read version from root cargo.toml

* pass check when version = { workspace = true }

* don't bump version when version = { workspace = true }

* including workspace Cargo.toml when bump version

* programs/sbf use workspace inheritance

* fix increasing cargo version ignore program/sbf/Cargo.toml
2023-01-25 13:59:59 +08:00
Xiang Zhu 74f89d1494
Add run parent directory for accounts files (#29794)
* Add run parent directory for accounts files

* fix test test_concurrent_snapshot_packaging

* review comments.  renamed the path setup function

* Addressed most of the review comments

* remove explict type def for map result

* handle create_accounts_run_and_snapshot_dirs error with expect

* update with more review comments

* minor fixes from review comments

* simplify account_filename option assignment

* handle error from create_accounts_run_and_snapshot_dirs

* use then instead of then_some for lazy evaluation
2023-01-24 16:44:35 -08:00
Brooks bda0c606a1
Renames fn to calculate_accounts_delta_hash() (#29876) 2023-01-24 18:55:56 -05:00
Brooks 655345728b
Use saturating_add_signed() (#29867) 2023-01-24 17:07:50 -05:00
Brooks d67baefd70
Use Option::unzip() (#29866) 2023-01-24 17:07:29 -05:00
behzad nouri 8a146361d1
includes rent_epoch in vote-accounts sanity checks (#29861)
https://github.com/solana-labs/solana/pull/26479
preserves rent epoch for rent-exempt accounts. Since the feature got
activated, vote accounts in stakes-cache have identical rent_epoch field
as the accounts in accounts-db. The commit verifies this in stakes-cache
sanity checks.
2023-01-24 19:47:38 +00:00
Brooks a06f8e68a5
Tests wait for initial accounts hash verification to complete (#29840) 2023-01-24 14:44:39 -05:00
Jeff Washington (jwash) 07afa22671
test fn append_single_account_with_default_hash uses AccountStorageEntry (#29839) 2023-01-24 11:43:03 -08:00
Jeff Washington (jwash) cac52b0819
trait ShrinkCollectRefs (#29830) 2023-01-24 08:43:50 -08:00
Jeff Washington (jwash) 8d4603edfb
update tests storages -> storage (#29837) 2023-01-24 07:54:28 -08:00
Jeff Washington (jwash) 79843fcbe7
update tests for 1 append vec per slot (#29836) 2023-01-24 07:53:54 -08:00
Jeff Washington (jwash) 5d24745464
if 0 alive accounts, don't create ancient append vec (#29842) 2023-01-24 07:52:31 -08:00
Tyera b35d173c04
Remove bank_transaction_count_fix logic (#29695)
* Remove bank_transaction_count_fix logic

* Fix test and add comment
2023-01-22 15:59:35 +08:00
behzad nouri 272e667cb2
deprecates Pubkey::new in favor of Pubkey::{,try_}from (#29805)
The commit deprecates Pubkey::new which lacks type-safety and instead
implements TryFrom<&[u8]> and TryFrom<Vec<u8>> for Pubkey.
2023-01-21 18:06:27 +00:00
Brooks db10073c36
Promotes accounts delta hash to its own type (#29768) 2023-01-20 14:23:56 -05:00
Brooks 6baab92ab5
Refactors calculate_capitalization() (#29790) 2023-01-20 13:38:41 -05:00
Brooks 0a0c532c86
Refactors verify_bank_hash_and_lamports() (#29783) 2023-01-19 16:14:39 -05:00
behzad nouri d463bcc5f8
removes manual implementations of Default for enums (#29779) 2023-01-19 19:25:30 +00:00
Trent Nelson c4e43f1de4
vote: encapsulate `Lockout` (#29753) 2023-01-18 19:28:28 -07:00
carllin 1753a0e3af
Revert "Do not exclude failed simple vote transactions from consensus" (#29745)
This reverts commit b4237f3f2c.
2023-01-18 15:13:13 -05:00
Jeff Washington (jwash) 9d2c71b9a3
remove type SnapshotStoragesOne (#29750) 2023-01-18 11:51:08 -08:00
Jeff Washington (jwash) aef8692c8f
remove SnapshotStorageOne (#29747) 2023-01-18 07:10:56 -08:00
Jeff Washington (jwash) f24e30e4fe
remove test method get_storages_for_slot (#29739) 2023-01-17 12:28:53 -08:00
Brooks 8c62927d59
Adds calculate_incremental_accounts_hash() (#29734)
**Node operators,** please delete your `<ledger>/calculate_accounts_hash_cache` directory (if you have one).  It has been moved/renamed to `<ledger>/accounts_hash_cache/full`.
2023-01-17 15:04:29 -05:00
Jeff Washington (jwash) e1d38c8315
remove test method slot_stores (#29736) 2023-01-17 10:15:25 -08:00
Jeff Washington (jwash) 71a746f10e
remove some write_version checking (#29733) 2023-01-17 10:14:31 -08:00
Jeff Washington (jwash) 4bfe4091b4
make storages not plural (#29732) 2023-01-17 09:50:35 -08:00
Jeff Washington (jwash) 5188da0492
remove get_slot_stores from AccountStorage (#29726) 2023-01-17 09:26:46 -08:00
Tao Zhu 1d88654116
Revert "Cap accounts data a transaction can load by its requested limit" (#29373)
Revert "Cap accounts data a transaction can load by its requested limit (#27840)"

This reverts commit 81dc2e56ac.
2023-01-17 11:34:18 +01:00
Alexander Meißner c5380d4699
Adds symmetric tests for all cases of un-/re-/deployment inside the same transaction (#29725)
Unifies test_program_sbf_upgrade_and_invoke_in_same_tx() and test_program_sbf_upgrade_self_via_cpi() into test_program_sbf_invoke_in_same_tx_as_redeployment().
Adds test_program_sbf_invoke_in_same_tx_as_deployment() and test_program_sbf_invoke_in_same_tx_as_undeployment().
2023-01-17 09:44:27 +01:00
Jeff Washington (jwash) 461dafb887
AccountStorage only holds a single append vec (#29723)
* AccountStorageReference

* tests build

* pr feedback
2023-01-16 16:22:30 -08:00
Brooks 0db14ad39c
Removes full_snapshot from CalcAccountsHashConfig (#29722) 2023-01-16 16:22:46 -05:00
Jeff Washington (jwash) ef30083319
iterate contents of AccountStorage without exposing internals (#29719)
* iterate contents of AccountStorage without exposing internals

* Update runtime/src/accounts_db.rs

Co-authored-by: Brooks <brooks@prumo.org>

* Update runtime/src/accounts_db.rs

Co-authored-by: Brooks <brooks@prumo.org>

* Update runtime/src/accounts_db.rs

Co-authored-by: Brooks <brooks@prumo.org>

* compile errors

Co-authored-by: Brooks <brooks@prumo.org>
2023-01-16 12:01:25 -08:00
Brooks 8a28877910
Use Hash::new_from_array() instead of TryFrom (#29720) 2023-01-16 10:53:38 -05:00
Jeff Washington (jwash) 977d3453a3
storage.remove does not leak SlotStores (#29664) 2023-01-15 11:06:24 -08:00
Alexander Meißner 9d1a0c52f8
Cleanup - `disable_deprecated_loader` (#29653)
* Removes the gating of disable_deprecated_loader.

* Adds create_program().
Renames create_deprecated_program() -> load_program().
Renames load_and_finalize_deprecated_program() -> load_and_finalize_program().

* Fixes test_sol_alloc_free_no_longer_deployable().
2023-01-14 13:02:10 +01:00
Jeff Washington (jwash) 064f163b70
get_account_storage_entry handles missing shrink in progress entry correctly (#29682) 2023-01-13 13:11:25 -08:00
Jeff Washington (jwash) a2933f7fa6
remove some uses of write_version (#29680) 2023-01-13 12:05:15 -08:00
Alexander Meißner ab97d37e3d
Refactor - Simplify loader utils (#29699)
* Moves all the common code of tests and benches into loader utils.

* Removes write_sbf_program().

* Removes ignored test: test_program_sbf_test_use_latest_executor2().
2023-01-13 20:38:36 +01:00
dependabot[bot] 10f8b5faa3
Bump memmap2 from 0.5.3 to 0.5.8 (#29674)
* Bump memmap2 from 0.5.3 to 0.5.8

Bumps [memmap2](https://github.com/RazrFalcon/memmap2-rs) from 0.5.3 to 0.5.8.
- [Release notes](https://github.com/RazrFalcon/memmap2-rs/releases)
- [Changelog](https://github.com/RazrFalcon/memmap2-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/RazrFalcon/memmap2-rs/compare/v0.5.3...v0.5.8)

---
updated-dependencies:
- dependency-name: memmap2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2023-01-12 19:38:38 -07:00
Jeff Washington (jwash) 6359d68f9f
get_slot_stores -> get_slot_storage_entry (#29677) 2023-01-12 09:44:53 -08:00
Brooks d0aa93de21
Refactors accounts hash cache (#29625) 2023-01-12 10:43:50 -05:00
Jeff Washington (jwash) 3412928cad
get_slot_stores -> get_slot_storage_entry (#29678) 2023-01-12 07:17:18 -08:00