Commit Graph

4492 Commits

Author SHA1 Message Date
Alexander Meißner a5c7c999e2
Bump solana_rbpf to v0.8.0 (#33679)
* Bumps solana_rbpf to v0.8.0

* Adjustments:
Replaces declare_syscall!() with declare_builtin_function!().
Removes Config::encrypt_runtime_environment.
Simplifies error propagation.
2023-10-20 21:39:50 +02:00
Pankaj Garg 59cb3b57ee
Set a global fork graph in program cache (#33776)
* Set a global fork graph in program cache

* fix deadlock

* review feedback
2023-10-20 08:47:03 -07:00
steviez c98c24bd6d
Revert "Split compute budget instructions process from struct itself … (#33784)
Revert "Split compute budget instructions process from struct itself (#33513)"

This reverts commit c73bebe984. This
was found to be a consensus breaking change.
2023-10-20 15:53:45 +02:00
Tao Zhu c73bebe984
Split compute budget instructions process from struct itself (#33513)
* Split compute budget instruction processing from ComputeBudget struct itself, allow compute_budget_instructions be processed elsewhere without having to instantiate ComputeBudget

* updated tests
2023-10-19 11:10:42 -05:00
Brooks c1353e172c
Removes cap_accounts_data_allocations_per_transaction featurization (#33754) 2023-10-19 07:01:15 -04:00
Brooks 93d882f158
Moves solana-store-tool into the accounts-db crate (#33755) 2023-10-18 16:16:38 -04:00
Jeff Washington (jwash) 69495f4c13
Fix non-determinism in account_hash_ignore_slot on genesis (#33692) 2023-10-16 12:47:07 -07:00
Jeff Washington (jwash) d948e5bf69
ancient shrink on its own cadence (#33712) 2023-10-16 10:06:20 -07:00
Alexander Meißner a3f85aba21
Refactor - LoadedPrograms part 2 (#33694) 2023-10-13 12:59:48 -07:00
Pankaj Garg fd9297775a
Prevent delay_visibility_of_program_deployment feature from cache pruning (#33689) 2023-10-13 14:21:18 +00:00
Sean Young 47511999bb
bank: do not remove trailing 0 bytes from return data (#33639)
This is creating havoc for Solang, as the return data is borsh encoded
and therefore `u64` values like 0x100 get truncated.
2023-10-13 08:00:41 +01:00
Ryo Onodera 53925b6182
Make goto_end_of_slot() take Arc<Bank> (#33650) 2023-10-13 13:08:38 +09:00
Brooks 452fd5d384
Adds `--no-skip-initial-accounts-db-clean` *hidden* CLI flag (#33664) 2023-10-12 13:32:40 -04:00
Brennan b36d051b51
Update hashes per tick with feature gates (#33600)
* Update hashes per tick with feature gates
2023-10-11 09:58:06 -07:00
Alexander Meißner cb695c7b32
Cleanup - Feature Gate of `enable_early_verification_of_account_modifications` (#33579)
Cleans up the feature gate of enable_early_verification_of_account_modifications:
- Removes PreAccount
- Removes InvokeContext::pre_accounts and InvokeContext::rent
- Removes InvokeContext::verify() and InvokeContext::verify_and_update()
- Removes TransactionContext::is_early_verification_of_account_modifications_enabled()
- Removes TransactionAccounts::is_early_verification_of_account_modifications_enabled
- No longer optional: TransactionContext::rent
2023-10-10 17:26:17 +02:00
Ryo Onodera 1704789247
Define tick related helper test methods (#33537)
* Define tick related helper methods

* dcou VoteSimulator

* blacklist ledger-tool for dcou

* fix dcou ci...

* github
2023-10-10 09:23:18 +09:00
Trent Nelson 55f3f203c6
bank: remove ambiguating "helper" method (#33606) 2023-10-09 21:47:21 +00:00
Brooks fc73813db2
Adds AccountHash newtype (#33597) 2023-10-09 16:00:52 -04:00
Brooks e0091d6995
Removes unnecessary borrow (#33539) 2023-10-05 12:41:33 -04:00
Joe C 93d8bcca4b
update feature gate no-op program id (#33535) 2023-10-05 18:28:44 +02:00
Joe C 25460f76e7
Bank: Add function to replace empty account with upgradeable program on feature activation (#32783)
* replace program account

* modify for all cases

* remove non-data swap

* address tests & conditional feedback

* get the rent involved

* mix in owner & executable

* feature-related cases

* stripped back to feature-specific case only

* added feature

* address initial feedback

* added more lamport checks

* condense tests

* using test_case

* add fail cases to tests

* more cleanup

* add verifiably built program

* update program account state

* cleaned up serializing logic

* use full word capitalization

* rename old & new to dst & src

* swap src and dst in parameters

* add warnings and errors

* rename feature to programify

* test suite description clarity

* remove strings from datapoints

* spell out source and destination

* more verbose comments in account replace functions

* move lamport calculation

* swap lamport check for state check

* move replace functions to helper module

* make replace_account methods fallible

* refactor error handling

* add test for source program state
2023-10-04 19:54:59 +02:00
steviez 3eae980293
Minor cleanup on some snapshot related tests (#33485) 2023-10-02 20:25:30 +02:00
steviez 73e9e6dd70
Deprecate bz2/gzip/none/tar snapshot compression types (#33484)
These options are now disallowed on the command line for
solana-validator and solana-ledger-tool, which effectively means no more
snapshots will be created with this types in normal usecases. However,
support for reading the deprecated types is still in place.
2023-10-02 19:40:31 +02:00
Alexander Meißner 8033be333e
Refactor - `LoadedPrograms` (#33482)
* Adds type ProgramRuntimeEnvironment.

* Moves LoadedPrograms::remove_expired_entries() into LoadedPrograms::prune().

* Adds Stats::prunes_environment and renames Stats::prunes_orphan and Stats::prunes_expired.

* Adds LoadedPrograms::latest_root_epoch.

* Typo fix, authored-by: Dmitri Makarov <dmakarov@users.noreply.github.com>
2023-10-02 19:01:23 +02:00
Tao Zhu cc4e9283db
reduce WARN logging to only necessary scenario (#33408) 2023-09-28 09:53:17 -05:00
steviez 5b9a167c51
Add uniform start/stop log lines for background hash verification (#33441)
Add uniform start/stop log lines for background hash verification
2023-09-28 15:13:51 +02:00
Alexander Meißner 7ff797bcef
Refactor - Remove parameter `feature_set` from `load_program_from_bytes()` (#33395)
Replaces parameter feature_set with delay_visibility_of_program_deployment in load_program_from_bytes().
2023-09-25 17:16:31 +02:00
ananas-block 997aa0a3f8
Feat(syscall): add altbn128 g1 & g2 compression (#32870)
* solana-program - altbn128: add g1 & g2 compression

still fixing tests for point of infinity

feat: proof compression syscall working

add rust test to ci

remove prints

added c test

added sycall pricing

* fixed ci checks

* refactored altbn128 and compression
2023-09-25 05:43:34 -07:00
Trent Nelson 1840fd7ab3
Feature - better error codes for tx lamport check (#33343)
Replaces `TransactionError::InstructionError(0, InstructionError::UnbalancedInstruction)` with `TransactionError::UnbalancedTransaction`.

Co-authored-by: Alexander Meißner <AlexanderMeissner@gmx.net>
2023-09-22 23:58:05 +00:00
Tao Zhu a2ad820309
compute_budget heap_size does not have to be optional (#33313) 2023-09-21 09:24:47 -05:00
Tyera bca41edf20
Make active stake consistent in split (#33295)
* Add feature gate

* Add helper fn

* Require split destination to be rent-exempt if it is active

* Update cli to prefund split accounts

* cli: require rent param with sign-only

* Update tokens to prefund split accounts

* Update split tests with sysvar accounts

* Fix test_split_to_account_with_rent_exempt_reserve

* Fix test_staked_split_destination_minimum_balance

* Fix test_split_more_than_staked

* Fix test_split_minimum_stake_delegation and remove misleading StakeState::Initialized case

* Fix test_split_from_larger_sized_account

* Add test for pre-/post-activation behavior splitting some or all of stake account

* Assert active stake

* Fix runtime test

* Ignore stake-pool downstream

* Review comments

* Feature gate sysvar reads
2023-09-20 06:00:51 +00:00
Pankaj Garg f50342a790
Split vote related code from runtime to its own crate (#32882)
* Move vote related code to its own crate

* Update imports in code and tests

* update programs/sbf/Cargo.lock

* fix check errors

* update abi_digest

* rebase fixes

* fixes after rebase
2023-09-19 10:46:37 -07:00
Brooks 889d6c655e
Moves accounts-db benches to accounts-db crate (#33306) 2023-09-19 13:09:42 -04:00
Alexander Meißner bc38ef27d8
Feature - Enable Program-Runtime-v2 and Loader-v4 (#33294)
* Adds a new feature `enable_program_runtime_v2_and_loader_v4`.

* Adds a feature gated builtin entry for the solana_loader_v4_program.
2023-09-19 16:29:52 +02:00
Brooks bc2b372762
Adds DCOU to verify_snapshot_archive() (#33298) 2023-09-18 19:11:28 -04:00
Tao Zhu 8b8a21a52f
cleanup feature: enable request heap frame instruction #30076 (#33243)
* cleanup feature: enable request heap frame instruction #30076

* update sbf tests

* removed out dated comments and test
2023-09-18 16:06:24 -05:00
steviez 9e11ae6275
Make program owners a const array instead of Vec<_> (#33275)
The program owners pubkeys are constant, no need to reconstruct the
Vec<Pubkey> and Vec<&Pubkey> each time this function runs (every time we
execute transactions).
2023-09-18 10:59:03 -05:00
behzad nouri c1090d3959
moves new_warmup_cooldown_rate_epoch outside iterators and for loops (#33259)
Recalculating new_warmup_cooldown_rate_epoch for each item is redundant
and wasteful and instead can be done only once outside the iterators and
for loops.
Also NewWarmupCooldownRateEpoch is unnecessary and verbose and is
removed in this commit.
2023-09-15 00:06:34 +00:00
Christian Kamm 525e59f01a
sdk, programs/bpf_loader: add sol_remaining_compute_units syscall (#31640)
bpf_loader: add sol_remaining_compute_units syscall

Co-authored-by: jonch <9093549+jon-chuang@users.noreply.github.com>
2023-09-13 08:57:08 -06:00
Pankaj Garg 5562f79cc5
Do not unnecessarily re-verify unloaded program (#32722)
* Do not unnecessarily re-verify unloaded program

* clippy fixes

* new unit test for extract

* fixes after rebase

* fixes after rebase

* address review comments

* fix clippy
2023-09-13 06:25:56 -07:00
Alexander Meißner bbb57be0a5
Refactor - Move interfaces of address-lookup-table into the program SDK (#33165)
* Adds a module `address_lookup_table` to the SDK.

* Adds a module `address_lookup_table::instruction` to the SDK.

* Adds a module `address_lookup_table::error` to the SDK.

* Adds a module `address_lookup_table::state` to the SDK.

* Moves AddressLookupTable into SDK as well.

* Moves AddressLookupTableAccount into address_lookup_table.

* Adds deprecation messages.

* Disentangles dependencies across cargo files.
2023-09-11 21:10:40 +02:00
Tao Zhu 4f4ce69f5f
purge duplicated bank prioritization fee from cache (#33062)
* purge duplicated bank prioritization fee from cache

* add test for purge dup bank

* Added metrics counts to monitor anomalies

* fix a flaky test
2023-09-11 10:08:55 -05:00
Pankaj Garg 377b6003a8
Check loader-v4::id() as a valid program owner (#33167) 2023-09-06 11:54:18 -07:00
Brooks a80819b029
PrunedBanksRequestHandler::handle_request() is only pub with dcou (#33155) 2023-09-06 17:02:52 +00:00
behzad nouri 88ee8f5820
replaces once_cell::sync::OnceCell with std::sync::OnceLock (#33140)
std::sync::OnceLock has become stable since rust 1.70.0 and there is no
longer a need for an external crate dependency.
2023-09-06 16:46:51 +00:00
Brooks 9e156f88f4
Removes invariant `is_serialized_with_abs` param (#33154) 2023-09-06 12:00:18 -04:00
Brooks d921b9a44e
Adds metrics when purging banks with the same slot (#33153) 2023-09-06 11:38:30 -04:00
steviez 224eea42d5
Add unit test for Bank::register_hard_fork() (#32902) 2023-09-06 09:07:09 -05:00
Brooks 2cf53b6293
Purge banks, with the same slot, sequentially (#33149) 2023-09-05 17:06:35 -04:00
Alexander Meißner e9542200e8
Fix - Superfluous `if` condition for overwriting `environments` in feature activation code (#33138)
Removes the `if` block around the overwriting of the `environments`.
2023-09-05 15:01:20 +02:00
Alexander Meißner 9e703f85de
Upgrades Rust to 1.72.0 & nightly-2023-08-25 (#32961)
* allow pedantic invalid cast lint

* allow lint with false-positive triggered by `test-case` crate

* nightly `fmt` correction

* adapt to rust layout changes

* remove dubious test

* Use transmute instead of pointer cast and de/ref when check_aligned is false.

* Renames clippy::integer_arithmetic to clippy::arithmetic_side_effects.

* bump rust nightly to 2023-08-25

* Upgrades Rust to 1.72.0

---------

Co-authored-by: Trent Nelson <trent@solana.com>
2023-09-01 07:26:13 +00:00
Ryo Onodera 4fe0812f3c
Make should_panic tests compatible with 1.73+ panic format (#33099)
* Adjust should_panic tests for 1.73+ panic format

* Update more panic format change affected tests...
2023-09-01 02:04:22 +00:00
behzad nouri 39615bd075
removes manual let...else (#33089)
https://rust-lang.github.io/rust-clippy/master/index.html#/manual_let_else
2023-08-31 22:35:47 +00:00
behzad nouri 4ec5ea6f7b
replaces assert!(matches!(...)) with assert_matches!(...) (#33068)
assert_matches!(...) provides more informative error message when it
fails and it is part of nightly rust:
https://doc.rust-lang.org/std/assert_matches/macro.assert_matches.html
2023-08-30 13:48:27 -04:00
Trent Nelson b8dc5daedb
preliminaries for bumping nightly to 2023-08-25 (#33047)
* remove unnecessary hashes around raw string literals

* remove unncessary literal `unwrap()`s

* remove panicking `unwrap()`

* remove unnecessary `unwrap()`

* use `[]` instead of `vec![]` where applicable

* remove (more) unnecessary explicit `into_iter()` calls

* remove redundant pattern matching

* don't cast to same type and constness

* do not `cfg(any(...` a single item

* remove needless pass by `&mut`

* prefer `or_default()` to `or_insert_with(T::default())`

* `filter_map()` better written as `filter()`

* incorrect `PartialOrd` impl on `Ord` type

* replace "slow zero-filled `Vec` initializations"

* remove redundant local bindings

* add required lifetime to associated constant
2023-08-29 23:05:35 +00:00
Alexander Meißner 1238e53c9f
Refactor - Loader v4 deployment status (#33024)
Refactors "is_deployed: bool" and "authority_address: Option<Pubkey>".
Replaces them with "status: LoaderV4Status" and "authority_address: Pubkey".
2023-08-28 20:14:01 +02:00
Ryo Onodera d3fb54e0b2
Remove From<LogCollector> for Vec<String> (#33012) 2023-08-27 13:10:02 +09:00
Alexander Meißner 29bbda0c11
General cleanup (Part 2) (#32998)
* bucket-map: remove unused `mut`

* remove unnecessary explicit `into_iter()` calls

* don't default-construct unit-structs

* prefer `or_default()` to `or_insert_with(T::default())`

* replace "slow zero-filled vec initialization"

---------

Co-authored-by: Trent Nelson <trent@solana.com>
2023-08-25 19:07:38 +02:00
Alexander Meißner a8be70fa7a
General cleanup (#32980)
* Fixes `cargo::` in build.rs

* Fixes overshadowing reexports.

* Removes unnecessary `as *const u8`.

* Removes unnecessary `.into_iter()`.

* cargo clippy
2023-08-24 21:44:19 +02:00
Alexander Meißner 025b952598
Fix - missing delay visibility feature triggering cache flush (#32974)
Adds delay_visibility_of_program_deployment to FEATURES_AFFECTING_RBPF.
2023-08-24 16:41:23 +02:00
steviez cd96d5b6c3
Only update last restart slot sysvar if value has changed (#32925)
Read account first to check for change in value
2023-08-22 22:33:03 +02:00
Jon Cinque 0fe902ced7
Bump rand to 0.8, rand_chacha to 0.3, getrandom to 0.2 (#32871)
* sdk: Add concurrent support for rand 0.7 and 0.8

* Update rand, rand_chacha, and getrandom versions

* Run command to replace `gen_range`

Run `git grep -l gen_range | xargs sed -i'' -e 's/gen_range(\(\S*\), /gen_range(\1../'

* sdk: Fix users of older `gen_range`

* Replace `hash::new_rand` with `hash::new_with_thread_rng`

Run:
```
git grep -l hash::new_rand | xargs sed -i'' -e 's/hash::new_rand([^)]*/hash::new_with_thread_rng(/'
```

* perf: Use `Keypair::new()` instead of `generate`

* Use older rand version in zk-token-sdk

* program-runtime: Inline random key generation

* bloom: Fix clippy warnings in tests

* streamer: Scope rng usage correctly

* perf: Fix clippy warning

* accounts-db: Map to char to generate a random string

* Remove `from_secret_key_bytes`, it's just `keypair_from_seed`

* ledger: Generate keypairs by hand

* ed25519-tests: Use new rand

* runtime: Use new rand in all tests

* gossip: Clean up clippy and inline keypair generators

* core: Inline keypair generation for tests

* Push sbf lockfile change

* sdk: Sort dependencies correctly

* Remove `hash::new_with_thread_rng`, use `Hash::new_unique()`

* Use Keypair::new where chacha isn't used

* sdk: Fix build by marking rand 0.7 optional

* Hardcode secret key length, add static assertion

* Unify `getrandom` crate usage to fix linking errors

* bloom: Fix tests that require a random hash

* Remove some dependencies, try to unify others

* Remove unnecessary uses of rand and rand_core

* Update lockfiles

* Add back some dependencies to reduce rebuilds

* Increase max rebuilds from 14 to 15

* frozen-abi: Remove `getrandom`

* Bump rebuilds to 17

* Remove getrandom from zk-token-proof
2023-08-21 19:11:21 +02:00
Brooks a8c1dabf6a
Renames AccountsPackageType to AccountsPackageKind (#32908) 2023-08-21 13:00:00 -04:00
HaoranYi 910b0f5d12
fix sbf sysvar test (#32803)
* fix sbf sysvar test

* typo

---------

Co-authored-by: HaoranYi <haoran.yi@solana.com>
2023-08-21 09:49:48 -06:00
Brooks a563b4cccb
Renames AccountsHashEnum to AccountsHashKind (#32904) 2023-08-21 10:34:56 -04:00
Brooks 9a6cf3f528
Renames SnapshotRequestType to SnapshotRequestKind (#32907) 2023-08-21 07:12:20 -04:00
Brooks f9bc3cec24
Renames SnapshotType to SnapshotKind (#32906) 2023-08-19 19:55:29 -04:00
steviez a4c8cc3ce0
Remove improper uses of &Arc<Bank> (#32802)
In most cases, either a &Bank or an Arc<Bank> is more proper.
- &Bank is used if the function only needs a momentary reference
- Arc<Bank> is used if the function needs its' own copy

This PR leaves several instances of &Arc<Bank> around; these instances
are situations where a clone may only happen conditionally.
2023-08-18 16:46:34 -05:00
Brooks 3cda810358
Adds docs to get_all_accounts_modified_since_parent() (#32895) 2023-08-18 16:01:50 -04:00
steviez 480851b865
Reduce repeated logic in BankForks unit tests (#32852)
* Move repeated BankForks logic into a loop
* Do away with redundant Vec<Arc<Bank>>
* Condense repeated logic into helper function
2023-08-18 11:58:04 -05:00
Brooks 280bb53802
Adds docs to get_all_accounts() (#32887) 2023-08-18 16:09:50 +00:00
Brooks 004577d94a
Adds docs to scan_all_accounts() (#32888) 2023-08-18 16:04:07 +00:00
Brooks 1e8f7b5d1e
Adds docs to non_vote_transaction_count_since_restart() (#32886) 2023-08-18 15:37:44 +00:00
Alexander Meißner c5a251e8c3
Refactor - `Bank::compute_active_feature_set()` (#32872)
Returns the feature set instead of overwriting it inside Bank::compute_active_feature_set().
2023-08-17 19:14:10 +02:00
Pankaj Garg c17b938204
Integrate program loader-v4 with bank (#32832)
* Integrate program loader-v4 with bank

* fix tests

* new struct for ProgramRuntimeEnvironments

* remove environment from program_runtime_environment_v

* move find_program_in_cache() to invoke_context

* cleanup
2023-08-16 10:50:23 -07:00
Brooks e316db2ab4
dcou: set_accounts_hash() (#32829) 2023-08-15 07:18:59 -04:00
steviez 6bbf514e78
Add ability to output components that go into Bank hash (#32632)
When a consensus divergance occurs, the current workflow involves a
handful of manual steps to hone in on the offending slot and
transaction. This process isn't overly difficult to execute; however, it
is tedious and currently involves creating and parsing logs.

This change introduces functionality to output a debug file that
contains the components go into the bank hash. The file can be generated
in two ways:
- Via solana-validator when the node realizes it has diverged
- Via solana-ledger-tool verify by passing a flag

When a divergance occurs now, the steps to debug would be:
- Grab the file from the node that diverged
- Generate a file for the same slot with ledger-tool with a known good
  version
- Diff the files, they are pretty-printed json
2023-08-15 00:12:05 -05:00
Brooks ce57cac370
Use update_accounts_hash() in AccountsHashVerifier (#32830) 2023-08-14 13:11:23 -04:00
Ryo Onodera ce63640d6d
dcou: make internal function `pub` conditionally (#32822)
dcou: make internal function pub conditionally
2023-08-13 14:12:20 +09:00
Pankaj Garg a1a0829a8b
Initialize and store PRv2 environment in cache (#32812)
* Initialize and store PRv2 environment in cache

* address review comments
2023-08-11 18:58:22 +00:00
Jon Cinque c73a56faf8
stake: Rename `StakeStateWithFlags` -> `StakeStateV2` (#32795) 2023-08-11 01:07:21 +02:00
Pankaj Garg f4287d70bb
Move accounts-db code to its own crate (#32766) 2023-08-09 13:03:36 -07:00
Jon Cinque 8e4a9a94ed
sdk: Add new version of `StakeState` to avoid breaking downstream users (#32736)
* sdk: Rename `StakeState` -> `StakeStateWithFlags`

* Add back `StakeFlags` with a deprecation warning
2023-08-09 00:05:40 +02:00
Jeff Washington (jwash) 7c1cf298aa
add active stats for pieces of hash calc (#32750) 2023-08-08 07:16:10 -07:00
Jeff Washington (jwash) 32cb381f69
remove some stats that are no longer kept (#32751)
* remove some stats that are no longer kept

* Update runtime/src/accounts_hash.rs

Co-authored-by: Brooks <brooks@prumo.org>

* reorder to remove drop

---------

Co-authored-by: Brooks <brooks@prumo.org>
2023-08-08 07:15:37 -07:00
Jeff Washington (jwash) 21b8b70bee
add metrics around purge_exact (#32752) 2023-08-08 07:15:21 -07:00
Yueh-Hsuan Chiang bf6d0aba4c
TieredStorage struct (4/N) -- remove_on_drop (#32603)
#### Summary of Changes
This PR makes TieredStorage implement Drop which does the remove-on-drop
behavior that follows the AccountsFile convention.
2023-08-07 15:39:46 -07:00
Brooks df31bc13db
Removes `set_no_remove_on_drop()` from AppendVec/AccountsFile (#32742) 2023-08-07 17:48:31 -04:00
Brooks 4894eb0333
Moves some accounts-db test-only code into a dev-context-only-utils feature (#32748) 2023-08-07 17:28:15 -04:00
Ashwin Sekar fa3506631a
stake: deprecate on chain warmup/cooldown rate and config (#32723)
* stake: deprecate on chain warmup/cooldown rate and config

* Pr feedback: Deprecate since 1.16.7

Co-authored-by: Jon Cinque <me@jonc.dev>

---------

Co-authored-by: Jon Cinque <me@jonc.dev>
2023-08-07 13:23:24 -07:00
behzad nouri b9a2030537
uses atomics for read-only accounts cache entry index (#32518)
Using atomics for entry indices allows load function to use self.cache.get
instead of get_mut which reduces lock contention on the respective
dash-map shard.
2023-08-07 17:47:09 +00:00
Brooks c2dec254c8
Removes `remove_on_drop` field from AppendVec (#32741) 2023-08-07 13:25:07 -04:00
Pankaj Garg 511cf28be8
Move `serde-snapshot dependent accounts-db tests (#32671)
Move serde-snapshot dependent accounts-db tests
2023-08-07 09:35:41 -07:00
Brooks 6ce647a3e0
Do not drop AppendVec in store-tool (#32739) 2023-08-07 12:15:01 -04:00
Yueh-Hsuan Chiang a5bde0a79c
TieredStorage struct (3/N) -- new_readonly and reader structs (#32579)
#### Summary of Changes
This PR implements TieredStorage::new_readonly() and introduces
TieredStorageReader and HotStorageReader.


#### Test Plan
Updated the existing unit test.
2023-08-04 12:18:38 -07:00
HaoranYi 533f42dae6
remove type cast in scan metric report (#32704)
Co-authored-by: HaoranYi <haoran.yi@solana.com>
2023-08-04 13:03:32 -05:00
Brooks 04e25ff3ce
Adds doc comments for Bank fns that get accounts/snapshot hashes (#32716) 2023-08-04 13:56:15 -04:00
Jeff Washington (jwash) ca7a7ad2f3
add some doc comments identified in audit (#32709)
* add some doc comments identified in audit

* Update runtime/src/bank.rs

Co-authored-by: Brooks <brooks@prumo.org>

* Update runtime/src/accounts_db.rs

Co-authored-by: Brooks <brooks@prumo.org>

* Update runtime/src/accounts_db.rs

Co-authored-by: Brooks <brooks@prumo.org>

---------

Co-authored-by: Brooks <brooks@prumo.org>
2023-08-04 10:52:01 -07:00
Alexander Meißner a310dd776c
Fix - Uses `std::mem::transmute` and `std::ptr::write` in unsafe code in append_vec.rs (#32711)
Uses std::mem::transmute and std::ptr::write in unsafe code in append_vec.rs
2023-08-04 17:18:13 +02:00
steviez 5edd032c79
Remove unnecessary clone (#32707) 2023-08-03 13:54:43 -06:00
Jeff Washington (jwash) f5c8bb789a
remove unused bank creation/freezing progress (#32699) 2023-08-03 10:12:44 -07:00
Alexander Meißner 33c33f5f79
Fix - Adds missing feature pubkeys to the `FEATURES_AFFECTING_RBPF` list (#32703)
Adds missing feature pubkeys to the FEATURES_AFFECTING_RBPF list.
2023-08-03 17:45:30 +02:00
Jeff Washington (jwash) 3dcb382731
avoid giant memory allocation in hash calc (#32646)
* avoid giant memory allocation in hash calc

* update comment

* reorder to avoid clone

* simplify references

* update comment on get_item
2023-07-31 13:13:19 -07:00
Brooks f4504d055a
Uses Mutex::into_inner() at end of de_dup_accounts() (#32667) 2023-07-31 17:02:45 +00:00
Jeff Washington (jwash) 91fd2a3a93
abs purge slots in parallel (#32536)
* abs purge slots in parallel

* remove arc clone
2023-07-31 09:22:53 -07:00
Jeff Washington (jwash) bb407d2b62
add metric read_only_cache_load_us (#32559) 2023-07-31 07:07:13 -07:00
Brooks 58e796dc64
Removes unnecessary clippy-allow for needless-collect when generating index (#32656) 2023-07-29 22:30:11 -04:00
Brooks fc8cfb63ae
Unstable sort slots when generating index (#32653) 2023-07-29 22:28:33 -04:00
behzad nouri ad4ddd3cb0
stores stake-accounts in parallel after epoch rewards calculations (#32633) 2023-07-29 20:47:28 +00:00
Brooks 16ec9a68c9
Unzips Option when deserializing snapshot data files (#32652) 2023-07-28 15:21:22 -04:00
Brooks 965ed1ab1f
Simplifies if-let when rebuilding bank (#32654)
Simplifies let-Some when rebuilding bank
2023-07-28 17:55:58 +00:00
Brooks e1d0785f4f
Unzips Option when rebuilding bank (#32651) 2023-07-28 12:28:49 -04:00
Brooks de02601d73
Cleanups metrics in add_bank_snapshot() (#32645) 2023-07-27 20:24:25 +00:00
Brooks 4417f49728
Adds more timing metrics to add_bank_snapshot() (#32639) 2023-07-27 14:52:34 -04:00
Yueh-Hsuan Chiang 2448486106
TieredStorage struct (2/N) -- write_accounts part 1 (#32541)
#### Summary of Changes
This PR initiates the implementation of TieredAccountsFile::write_accounts,
which will later support AccountsFile::append_accounts.

This PR also introduces TieredAccountsFileWriter -- the main writer struct for a
TieredAccountsFile.

#### Test Plan
Extended existing unit tests.
2023-07-27 14:55:48 +08:00
Brooks 89c1f94cc4
Do not add bank snapshots for AccountsHashVerifier requests (#32589) 2023-07-25 18:30:06 -04:00
Pankaj Garg ef8d3206d7
Move bank specific code out of epoch_accounts_hash/utils.rs (#32623)
* Move bank specific code out of epoch_accounts_hash/utils.rs

* cleanup
2023-07-25 14:12:20 -07:00
Yueh-Hsuan Chiang 28d7e59cac
TieredStorage struct (1/N) -- new_writable() (#32538)
#### Summary of Changes
This PR initiates the implementation of the main struct for the
tiered accounts storage --- TieredStorage.  Specifically,
it defines the TieredStorage struct, TieredStorageFormat,
and skeleton implementation of new_writable().

#### Test Plan
Unit tests are included in this PR.
2023-07-26 01:51:56 +08:00
Pankaj Garg aba637d5d9
Split snapshot_utils.rs into snapshot_bank_utils.rs (#32612) 2023-07-24 16:31:03 -07:00
Brooks de4ff348b4
Removes manual call to `.0.display()` with thiserror (#32605) 2023-07-24 12:02:55 -04:00
HaoranYi 17af3ab10a
Add stake_flags to stake state (#32524)
add stake_flags to stake state

Co-authored-by: HaoranYi <haoran.yi@solana.com>
2023-07-24 09:09:40 -05:00
Tao Zhu 62b9fcfa76
cleanup feature: Use default units per instruction in fee calculation (#32570)
* cleanup feature: Use default units per instruction in fee calculation
2023-07-21 18:37:58 -05:00
Pankaj Garg fc35b13365
Cleanup load_program() in bank.rs (#32146) 2023-07-21 13:43:46 -07:00
behzad nouri 952d8861c6
removes feature-gate code for updating rewards from cached accounts (#32514) 2023-07-21 19:52:44 +00:00
HaoranYi 9e8639f7ae
Add distributed rewards for the block in partitioned reward metrics (#32580)
report distributed rewards for the block in metrics

Co-authored-by: HaoranYi <haoran.yi@solana.com>
2023-07-21 14:42:52 -05:00
Pankaj Garg faff34403c
Move bank dependent tests out of serde_snapshot/tests.rs (#32571) 2023-07-21 12:08:32 -07:00
Tao Zhu 19f5b49abb
Priority queue supports deprecated compute budget ix (#32567)
Supports deprecated compute budget instruction when getting transaction's priority details
2023-07-20 19:17:56 -05:00
Brooks faa002c2b6
Adds error types for all of `add_bank_snapshot()` (#32564) 2023-07-20 18:30:19 -04:00
Brooks 979913e1d5
Returns Self instead of Result<Self> for AccountsPackage::new_for_snapshot() (#32545) 2023-07-20 12:52:31 -04:00
Brooks 3fbfac41da
AccountsBackgroundService gracefully exits when snapshot request handling errors (#32543) 2023-07-20 11:53:06 -04:00
Jeff Washington (jwash) 0f70098410
allow shrinking to be in progress during `scan_account_storage` (#32553) 2023-07-20 08:01:10 -07:00
Brooks 6d30429dd1
Replaces `unwrap` with `?` when writing snapshot version file (#32542) 2023-07-19 16:42:46 -04:00
behzad nouri a8b0c92873
simplifies update_stakes_cache implementation (#32534) 2023-07-19 18:13:30 +00:00
HaoranYi 5408872476
test force reward interval end fn (#32513)
Co-authored-by: HaoranYi <haoran.yi@solana.com>
2023-07-17 15:37:10 -05:00
Yueh-Hsuan Chiang a2d9870678
[TieredStorage] Implementation of AccountIndexFormat for hot accounts (#32497)
#### Summary of Changes
This PR implements AccountIndexFormat::AddressAndOffset, the index format
that will be used by the hot account storage.

#### Test Plan
Unit tests are included in this PR.
Tested via the prototype implementation of tiered-storage.
2023-07-16 07:23:08 +08:00
Pankaj Garg 2a86420777
Move calculate_fee() out of bank.rs (#32409)
* Move calculate_fee() to fee.rs

* fixup code for compilation
2023-07-14 20:17:38 -07:00
behzad nouri cfb028819a
deprecates Signature::new in favor of Signature::{try_,}from (#32481) 2023-07-14 22:51:12 +00:00
Brooks 78230050fb
Fixes measure in log when calculating accounts hash (#32502) 2023-07-14 17:02:14 -04:00
Pankaj Garg 7177dd9809
Remove calculate_fee()'s dependency on ComputeBudget (#32487) 2023-07-14 12:25:04 -07:00
Pankaj Garg eb74562124
Move methods to get num signatures and locks to SanitizedMessage (#32489) 2023-07-14 10:59:55 -07:00
Yueh-Hsuan Chiang e6d1217dc3
[TieredStorage] Replace 'a by 'accounts_file in TieredReadableAccount (#32470)
#### Summary of Changes
This PR replaces the struct life-time parameter `'a` to `'accounts_file` in
TieredReadableAccount to improve readability.
2023-07-14 14:35:05 +08:00
Brooks 3b0e4c8452
Removes unnecessary default() (#32477) 2023-07-13 13:59:57 -04:00
Brooks 3c825f28a5
Uses next_back() (#32478) 2023-07-13 13:38:18 -04:00
Jeff Washington (jwash) 09ddbd731c
correctly log stats at remove_unrooted_slots (#32467) 2023-07-13 06:55:35 -07:00
Ryo Onodera d80745cfaf
dcou: BorrowedAccount::set_data() (#32424)
* dcou: BorrowedAccount::set_data()

* Properly depend on dev-context-only-utils
2023-07-13 22:44:08 +09:00
Brooks b99ff04b7b
Removes default impls on TieredAccountMeta (#32464)
#### Problem

`TieredAccountMeta` has default impls on many of its methods. But I don't think it should, because these defaults are not actually useful defaults (i.e. they are `unimplemented!()`).

We're only going to have a small number of structs that'll implement this trait, and each struct should be responsible for picking the correct impl for each method. 

Currently, `HotAccountMeta` is the only one, and it implements every method.

#### Summary of Changes

Remove default impls on TieredAccountMeta
2023-07-13 13:45:19 +08:00
HaoranYi ae6abda786
Remove unnecessary dead_code attributes (#32416)
* remove unnecessary dead_code attr

* move test fn to test files

---------

Co-authored-by: HaoranYi <haoran.yi@solana.com>
2023-07-12 18:41:17 -05:00
Justin Starry d344ae5297
feature flag cleanup: fix_recent_blockhashes (#32443)
* feature flag cleanup: fix_recent_blockhashes

* flip test logic
2023-07-11 20:43:50 -05:00
Jeff Washington (jwash) 245cd9631a
hook up ancient.slots_considered metric (#32458) 2023-07-11 11:51:16 -07:00
HaoranYi 466564686b
update epoch rewards sysvar (#32415)
* update epoch rewards sysvar

* add back the one reward distribution reward test

---------

Co-authored-by: HaoranYi <haoran.yi@solana.com>
2023-07-11 11:24:20 -05:00