blockworks-foundation
/
solana
mirror of https://github.com/blockworks-foundation/solana.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
23,785 Commits 168 Branches 497 Tags 539 MiB
Commit Graph

4374 Commits

Author SHA1 Message Date
Brooks 3cda810358
Adds docs to get_all_accounts_modified_since_parent() (#32895) 2023-08-18 16:01:50 -04:00
steviez 480851b865
Reduce repeated logic in BankForks unit tests (#32852) 
* Move repeated BankForks logic into a loop
* Do away with redundant Vec<Arc<Bank>>
* Condense repeated logic into helper function
 2023-08-18 11:58:04 -05:00
Brooks 280bb53802
Adds docs to get_all_accounts() (#32887) 2023-08-18 16:09:50 +00:00
Brooks 004577d94a
Adds docs to scan_all_accounts() (#32888) 2023-08-18 16:04:07 +00:00
Brooks 1e8f7b5d1e
Adds docs to non_vote_transaction_count_since_restart() (#32886) 2023-08-18 15:37:44 +00:00
Alexander Meißner c5a251e8c3
Refactor - `Bank::compute_active_feature_set()` (#32872) 
Returns the feature set instead of overwriting it inside Bank::compute_active_feature_set().
 2023-08-17 19:14:10 +02:00
Pankaj Garg c17b938204
Integrate program loader-v4 with bank (#32832) 
* Integrate program loader-v4 with bank

* fix tests

* new struct for ProgramRuntimeEnvironments

* remove environment from program_runtime_environment_v

* move find_program_in_cache() to invoke_context

* cleanup
 2023-08-16 10:50:23 -07:00
Brooks e316db2ab4
dcou: set_accounts_hash() (#32829) 2023-08-15 07:18:59 -04:00
steviez 6bbf514e78
Add ability to output components that go into Bank hash (#32632) 
When a consensus divergance occurs, the current workflow involves a
handful of manual steps to hone in on the offending slot and
transaction. This process isn't overly difficult to execute; however, it
is tedious and currently involves creating and parsing logs.

This change introduces functionality to output a debug file that
contains the components go into the bank hash. The file can be generated
in two ways:
- Via solana-validator when the node realizes it has diverged
- Via solana-ledger-tool verify by passing a flag

When a divergance occurs now, the steps to debug would be:
- Grab the file from the node that diverged
- Generate a file for the same slot with ledger-tool with a known good
  version
- Diff the files, they are pretty-printed json
 2023-08-15 00:12:05 -05:00
Brooks ce57cac370
Use update_accounts_hash() in AccountsHashVerifier (#32830) 2023-08-14 13:11:23 -04:00
Ryo Onodera ce63640d6d
dcou: make internal function `pub` conditionally (#32822) 
dcou: make internal function pub conditionally
 2023-08-13 14:12:20 +09:00
Pankaj Garg a1a0829a8b
Initialize and store PRv2 environment in cache (#32812) 
* Initialize and store PRv2 environment in cache

* address review comments
 2023-08-11 18:58:22 +00:00
Jon Cinque c73a56faf8
stake: Rename `StakeStateWithFlags` -> `StakeStateV2` (#32795) 2023-08-11 01:07:21 +02:00
Pankaj Garg f4287d70bb
Move accounts-db code to its own crate (#32766) 2023-08-09 13:03:36 -07:00
Jon Cinque 8e4a9a94ed
sdk: Add new version of `StakeState` to avoid breaking downstream users (#32736) 
* sdk: Rename `StakeState` -> `StakeStateWithFlags`

* Add back `StakeFlags` with a deprecation warning
 2023-08-09 00:05:40 +02:00
Jeff Washington (jwash) 7c1cf298aa
add active stats for pieces of hash calc (#32750) 2023-08-08 07:16:10 -07:00
Jeff Washington (jwash) 32cb381f69
remove some stats that are no longer kept (#32751) 
* remove some stats that are no longer kept

* Update runtime/src/accounts_hash.rs

Co-authored-by: Brooks <brooks@prumo.org>

* reorder to remove drop

---------

Co-authored-by: Brooks <brooks@prumo.org>
 2023-08-08 07:15:37 -07:00
Jeff Washington (jwash) 21b8b70bee
add metrics around purge_exact (#32752) 2023-08-08 07:15:21 -07:00
Yueh-Hsuan Chiang bf6d0aba4c
TieredStorage struct (4/N) -- remove_on_drop (#32603) 
#### Summary of Changes
This PR makes TieredStorage implement Drop which does the remove-on-drop
behavior that follows the AccountsFile convention.
 2023-08-07 15:39:46 -07:00
Brooks df31bc13db
Removes `set_no_remove_on_drop()` from AppendVec/AccountsFile (#32742) 2023-08-07 17:48:31 -04:00
Brooks 4894eb0333
Moves some accounts-db test-only code into a dev-context-only-utils feature (#32748) 2023-08-07 17:28:15 -04:00
Ashwin Sekar fa3506631a
stake: deprecate on chain warmup/cooldown rate and config (#32723) 
* stake: deprecate on chain warmup/cooldown rate and config

* Pr feedback: Deprecate since 1.16.7

Co-authored-by: Jon Cinque <me@jonc.dev>

---------

Co-authored-by: Jon Cinque <me@jonc.dev>
 2023-08-07 13:23:24 -07:00
behzad nouri b9a2030537
uses atomics for read-only accounts cache entry index (#32518) 
Using atomics for entry indices allows load function to use self.cache.get
instead of get_mut which reduces lock contention on the respective
dash-map shard.
 2023-08-07 17:47:09 +00:00
Brooks c2dec254c8
Removes `remove_on_drop` field from AppendVec (#32741) 2023-08-07 13:25:07 -04:00
Pankaj Garg 511cf28be8
Move `serde-snapshot dependent accounts-db tests (#32671) 
Move serde-snapshot dependent accounts-db tests
 2023-08-07 09:35:41 -07:00
Brooks 6ce647a3e0
Do not drop AppendVec in store-tool (#32739) 2023-08-07 12:15:01 -04:00
Yueh-Hsuan Chiang a5bde0a79c
TieredStorage struct (3/N) -- new_readonly and reader structs (#32579) 
#### Summary of Changes
This PR implements TieredStorage::new_readonly() and introduces
TieredStorageReader and HotStorageReader.


#### Test Plan
Updated the existing unit test.
 2023-08-04 12:18:38 -07:00
HaoranYi 533f42dae6
remove type cast in scan metric report (#32704) 
Co-authored-by: HaoranYi <haoran.yi@solana.com>
 2023-08-04 13:03:32 -05:00
Brooks 04e25ff3ce
Adds doc comments for Bank fns that get accounts/snapshot hashes (#32716) 2023-08-04 13:56:15 -04:00
Jeff Washington (jwash) ca7a7ad2f3
add some doc comments identified in audit (#32709) 
* add some doc comments identified in audit

* Update runtime/src/bank.rs

Co-authored-by: Brooks <brooks@prumo.org>

* Update runtime/src/accounts_db.rs

Co-authored-by: Brooks <brooks@prumo.org>

* Update runtime/src/accounts_db.rs

Co-authored-by: Brooks <brooks@prumo.org>

---------

Co-authored-by: Brooks <brooks@prumo.org>
 2023-08-04 10:52:01 -07:00
Alexander Meißner a310dd776c
Fix - Uses `std::mem::transmute` and `std::ptr::write` in unsafe code in append_vec.rs (#32711) 
Uses std::mem::transmute and std::ptr::write in unsafe code in append_vec.rs
 2023-08-04 17:18:13 +02:00
steviez 5edd032c79
Remove unnecessary clone (#32707) 2023-08-03 13:54:43 -06:00
Jeff Washington (jwash) f5c8bb789a
remove unused bank creation/freezing progress (#32699) 2023-08-03 10:12:44 -07:00
Alexander Meißner 33c33f5f79
Fix - Adds missing feature pubkeys to the `FEATURES_AFFECTING_RBPF` list (#32703) 
Adds missing feature pubkeys to the FEATURES_AFFECTING_RBPF list.
 2023-08-03 17:45:30 +02:00
Jeff Washington (jwash) 3dcb382731
avoid giant memory allocation in hash calc (#32646) 
* avoid giant memory allocation in hash calc

* update comment

* reorder to avoid clone

* simplify references

* update comment on get_item
 2023-07-31 13:13:19 -07:00
Brooks f4504d055a
Uses Mutex::into_inner() at end of de_dup_accounts() (#32667) 2023-07-31 17:02:45 +00:00
Jeff Washington (jwash) 91fd2a3a93
abs purge slots in parallel (#32536) 
* abs purge slots in parallel

* remove arc clone
 2023-07-31 09:22:53 -07:00
Jeff Washington (jwash) bb407d2b62
add metric read_only_cache_load_us (#32559) 2023-07-31 07:07:13 -07:00
Brooks 58e796dc64
Removes unnecessary clippy-allow for needless-collect when generating index (#32656) 2023-07-29 22:30:11 -04:00
Brooks fc8cfb63ae
Unstable sort slots when generating index (#32653) 2023-07-29 22:28:33 -04:00
behzad nouri ad4ddd3cb0
stores stake-accounts in parallel after epoch rewards calculations (#32633) 2023-07-29 20:47:28 +00:00
Brooks 16ec9a68c9
Unzips Option when deserializing snapshot data files (#32652) 2023-07-28 15:21:22 -04:00
Brooks 965ed1ab1f
Simplifies if-let when rebuilding bank (#32654) 
Simplifies let-Some when rebuilding bank
 2023-07-28 17:55:58 +00:00
Brooks e1d0785f4f
Unzips Option when rebuilding bank (#32651) 2023-07-28 12:28:49 -04:00
Brooks de02601d73
Cleanups metrics in add_bank_snapshot() (#32645) 2023-07-27 20:24:25 +00:00
Brooks 4417f49728
Adds more timing metrics to add_bank_snapshot() (#32639) 2023-07-27 14:52:34 -04:00
Yueh-Hsuan Chiang 2448486106
TieredStorage struct (2/N) -- write_accounts part 1 (#32541) 
#### Summary of Changes
This PR initiates the implementation of TieredAccountsFile::write_accounts,
which will later support AccountsFile::append_accounts.

This PR also introduces TieredAccountsFileWriter -- the main writer struct for a
TieredAccountsFile.

#### Test Plan
Extended existing unit tests.
 2023-07-27 14:55:48 +08:00
Brooks 89c1f94cc4
Do not add bank snapshots for AccountsHashVerifier requests (#32589) 2023-07-25 18:30:06 -04:00
Pankaj Garg ef8d3206d7
Move bank specific code out of epoch_accounts_hash/utils.rs (#32623) 
* Move bank specific code out of epoch_accounts_hash/utils.rs

* cleanup
 2023-07-25 14:12:20 -07:00
Yueh-Hsuan Chiang 28d7e59cac
TieredStorage struct (1/N) -- new_writable() (#32538) 
#### Summary of Changes
This PR initiates the implementation of the main struct for the
tiered accounts storage --- TieredStorage.  Specifically,
it defines the TieredStorage struct, TieredStorageFormat,
and skeleton implementation of new_writable().

#### Test Plan
Unit tests are included in this PR.
 2023-07-26 01:51:56 +08:00
Pankaj Garg aba637d5d9
Split snapshot_utils.rs into snapshot_bank_utils.rs (#32612) 2023-07-24 16:31:03 -07:00
Brooks de4ff348b4
Removes manual call to `.0.display()` with thiserror (#32605) 2023-07-24 12:02:55 -04:00
HaoranYi 17af3ab10a
Add stake_flags to stake state (#32524) 
add stake_flags to stake state

Co-authored-by: HaoranYi <haoran.yi@solana.com>
 2023-07-24 09:09:40 -05:00
Tao Zhu 62b9fcfa76
cleanup feature: Use default units per instruction in fee calculation (#32570) 
* cleanup feature: Use default units per instruction in fee calculation
 2023-07-21 18:37:58 -05:00
Pankaj Garg fc35b13365
Cleanup load_program() in bank.rs (#32146) 2023-07-21 13:43:46 -07:00
behzad nouri 952d8861c6
removes feature-gate code for updating rewards from cached accounts (#32514) 2023-07-21 19:52:44 +00:00
HaoranYi 9e8639f7ae
Add distributed rewards for the block in partitioned reward metrics (#32580) 
report distributed rewards for the block in metrics

Co-authored-by: HaoranYi <haoran.yi@solana.com>
 2023-07-21 14:42:52 -05:00
Pankaj Garg faff34403c
Move bank dependent tests out of serde_snapshot/tests.rs (#32571) 2023-07-21 12:08:32 -07:00
Tao Zhu 19f5b49abb
Priority queue supports deprecated compute budget ix (#32567) 
Supports deprecated compute budget instruction when getting transaction's priority details
 2023-07-20 19:17:56 -05:00
Brooks faa002c2b6
Adds error types for all of `add_bank_snapshot()` (#32564) 2023-07-20 18:30:19 -04:00
Brooks 979913e1d5
Returns Self instead of Result<Self> for AccountsPackage::new_for_snapshot() (#32545) 2023-07-20 12:52:31 -04:00
Brooks 3fbfac41da
AccountsBackgroundService gracefully exits when snapshot request handling errors (#32543) 2023-07-20 11:53:06 -04:00
Jeff Washington (jwash) 0f70098410
allow shrinking to be in progress during `scan_account_storage` (#32553) 2023-07-20 08:01:10 -07:00
Brooks 6d30429dd1
Replaces `unwrap` with `?` when writing snapshot version file (#32542) 2023-07-19 16:42:46 -04:00
behzad nouri a8b0c92873
simplifies update_stakes_cache implementation (#32534) 2023-07-19 18:13:30 +00:00
HaoranYi 5408872476
test force reward interval end fn (#32513) 
Co-authored-by: HaoranYi <haoran.yi@solana.com>
 2023-07-17 15:37:10 -05:00
Yueh-Hsuan Chiang a2d9870678
[TieredStorage] Implementation of AccountIndexFormat for hot accounts (#32497) 
#### Summary of Changes
This PR implements AccountIndexFormat::AddressAndOffset, the index format
that will be used by the hot account storage.

#### Test Plan
Unit tests are included in this PR.
Tested via the prototype implementation of tiered-storage.
 2023-07-16 07:23:08 +08:00
Pankaj Garg 2a86420777
Move calculate_fee() out of bank.rs (#32409) 
* Move calculate_fee() to fee.rs

* fixup code for compilation
 2023-07-14 20:17:38 -07:00
behzad nouri cfb028819a
deprecates Signature::new in favor of Signature::{try_,}from (#32481) 2023-07-14 22:51:12 +00:00
Brooks 78230050fb
Fixes measure in log when calculating accounts hash (#32502) 2023-07-14 17:02:14 -04:00
Pankaj Garg 7177dd9809
Remove calculate_fee()'s dependency on ComputeBudget (#32487) 2023-07-14 12:25:04 -07:00
Pankaj Garg eb74562124
Move methods to get num signatures and locks to SanitizedMessage (#32489) 2023-07-14 10:59:55 -07:00
Yueh-Hsuan Chiang e6d1217dc3
[TieredStorage] Replace 'a by 'accounts_file in TieredReadableAccount (#32470) 
#### Summary of Changes
This PR replaces the struct life-time parameter `'a` to `'accounts_file` in
TieredReadableAccount to improve readability.
 2023-07-14 14:35:05 +08:00
Brooks 3b0e4c8452
Removes unnecessary default() (#32477) 2023-07-13 13:59:57 -04:00
Brooks 3c825f28a5
Uses next_back() (#32478) 2023-07-13 13:38:18 -04:00
Jeff Washington (jwash) 09ddbd731c
correctly log stats at remove_unrooted_slots (#32467) 2023-07-13 06:55:35 -07:00
Ryo Onodera d80745cfaf
dcou: BorrowedAccount::set_data() (#32424) 
* dcou: BorrowedAccount::set_data()

* Properly depend on dev-context-only-utils
 2023-07-13 22:44:08 +09:00
Brooks b99ff04b7b
Removes default impls on TieredAccountMeta (#32464) 
#### Problem

`TieredAccountMeta` has default impls on many of its methods. But I don't think it should, because these defaults are not actually useful defaults (i.e. they are `unimplemented!()`).

We're only going to have a small number of structs that'll implement this trait, and each struct should be responsible for picking the correct impl for each method. 

Currently, `HotAccountMeta` is the only one, and it implements every method.

#### Summary of Changes

Remove default impls on TieredAccountMeta
 2023-07-13 13:45:19 +08:00
HaoranYi ae6abda786
Remove unnecessary dead_code attributes (#32416) 
* remove unnecessary dead_code attr

* move test fn to test files

---------

Co-authored-by: HaoranYi <haoran.yi@solana.com>
 2023-07-12 18:41:17 -05:00
Justin Starry d344ae5297
feature flag cleanup: fix_recent_blockhashes (#32443) 
* feature flag cleanup: fix_recent_blockhashes

* flip test logic
 2023-07-11 20:43:50 -05:00
Jeff Washington (jwash) 245cd9631a
hook up ancient.slots_considered metric (#32458) 2023-07-11 11:51:16 -07:00
HaoranYi 466564686b
update epoch rewards sysvar (#32415) 
* update epoch rewards sysvar

* add back the one reward distribution reward test

---------

Co-authored-by: HaoranYi <haoran.yi@solana.com>
 2023-07-11 11:24:20 -05:00
HaoranYi 943fba5afd
update max reward block factor (#32445) 
* update max reward block factor

* review comments

* more tests

* reduce the number of stakes per block to speed up tests

---------

Co-authored-by: HaoranYi <haoran.yi@solana.com>
 2023-07-11 09:36:11 -05:00
HaoranYi 2ed30793cc
add PartitionedEpochRewardsConfigRewardBlock (#32444) 
Co-authored-by: HaoranYi <haoran.yi@solana.com>
 2023-07-10 17:04:40 -05:00
Yueh-Hsuan Chiang a58b84ce95
[TieredStorage] TieredReadableAccount::data() (#32433) 
#### Problem
While TieredReadableAccount implements ReadableAccount that
already includes `fn data(&self) -> &'a [u8]`, a `data()` function that
directly under TieredReadableAccount is still needed in order to
correctly link the lifetime of the returned value and its member
account_block.  Otherwise, cargo will complain lifetime may not
live long enough.

#### Summary of Changes
This PR adds TieredReadableAccount::data() that directly links the lifetime
of its account_block to the returned value.
 2023-07-11 00:33:36 +08:00
Brooks 65e90ab0bc
Uses `'storage` for lifetime in StoredAccountMeta (#32440) 2023-07-10 10:57:20 -04:00
behzad nouri d54b6204be
removes instances of clippy::manual_let_else (#32417) 2023-07-09 21:41:36 +00:00
HaoranYi a3171d3de2
Fix typos (#32420) 
* typo

* more typos

---------

Co-authored-by: HaoranYi <haoran.yi@solana.com>
 2023-07-07 21:05:32 +00:00
HaoranYi 37a36384f3
Test stake program execution restricted in reward period (#32404) 
test stake program execution restricted in reward period

Co-authored-by: HaoranYi <haoran.yi@solana.com>
 2023-07-07 10:24:49 -05:00
Illia Bobyr 282e043177
`cargo fmt` using 1.6.0-nightly (#32390) 
Seems like rustfmt 1.6.0 can now format `let/else` statements.
1.5.2 we use in our `solana-nightly` does not mind the reformatting.

```
$ cargo +nightly fmt --version
rustfmt 1.6.0-nightly (f20afcc 2023-07-04)

$ cargo +nightly fmt
$ git add -u
$ git commit

$ ./cargo nightly fmt --version
+ exec cargo +nightly-2023-04-19 fmt --version
rustfmt 1.5.2-nightly (c609da5 2023-04-18)

$ ./cargo nightly fmt
$ git diff
[empty output]
```
 2023-07-06 20:45:29 -07:00
HaoranYi 8273b103d2
Test rewards compuation and partitioned rewards distribution at the epoch boundary (#32349) 
* Test rewards compuation and partitioned rewards distribution at the epoch boundary

* reviews

* match to if

* clippy

* peeling of bank1 from the loop

* collapse if_same

---------

Co-authored-by: HaoranYi <haoran.yi@solana.com>
 2023-07-06 14:49:09 -05:00
Pankaj Garg 4674b0099f
Move CostModel and CostTracker to its own crate (#32354) 
* Move CostModel and CostTracker to its own crate

* compile new crate and update imports

* update sbf Cargo.lock

* fix AbiExample

* fix cargo sort

* Fix AbiExample
 2023-07-06 10:08:18 -07:00
Pankaj Garg b060f62004
Move transaction results data structures out of bank.rs (#32386) 
* Move transaction results data structures out of bank.rs

* Fix CI check failures
 2023-07-06 09:45:01 -07:00
Yueh-Hsuan Chiang fbb1dd5fe5
Replace AccountsFile::cloned_account by ReadableAccount::to_account_shared_data() (#32399) 
#### Problem
Both AppendVec::cloned_account() and ReadableAccount::to_account_shard_data()
returns an AccountSharedData instance by cloning itself as pointed out under
a comment in PR #32380.

#### Summary of Changes
This PR removes AccountsFile::cloned_account() and replaces its call-sites by
ReadableAccount::to_account_shared_data().
 2023-07-07 00:08:50 +08:00
Yueh-Hsuan Chiang f1836bb10f
[TieredAccountStorage] TieredReadableAccount (#32380) 
#### Summary of Changes
This PR introduces TieredReadableAccount, a struct that takes a generic
TieredAccountMeta that implements ReadableAccount and support
StoredAccountMeta.

#### Test Plan
More tests will be added once more reader and writer functions for hot
accounts have been added.
 2023-07-07 00:08:09 +08:00
Brooks d08ce116b3
Measures accounts hash calculation time (#32385) 2023-07-05 15:53:36 -04:00
Brooks f1f0f451b7
Adds cache_hash_data_us to HashStats (#32371) 2023-07-05 13:57:07 -04:00
Pankaj Garg 356a827087
Move NonceInfo, NonceFull and NoncePartial out of bank.rs (#32375) 
* Move NonceInfo, NonceFull and NoncePartial out of bank.rs

* fix imports

* move test_nonce_info to nonce_info.rs
 2023-07-05 10:44:54 -07:00
Brooks c80981070c
Adds mark_time and sort_time to HashStats total_time (#32370) 2023-07-05 12:26:31 -04:00
Brooks 52fe7eb1cf
Renames HashStats "sort" to "sort_us" (#32369) 2023-07-05 09:35:44 -04:00