Commit Graph

3162 Commits

Author SHA1 Message Date
HaoranYi bf14440895
clean up and optimize account hash verify (#26560)
* remove unused code

* extract test related fault hash inject fn

* use rotate to optimize hashes removal

* use rotate to optimize snapshot hashes removal

* address code reveiw feedbacks

* revise comments

* inline
2022-07-12 19:27:28 +00:00
Jeff Washington (jwash) f54f528148
make bank test creation standard (#26519) 2022-07-12 09:07:55 -05:00
github-actions[bot] fd5df1cf25
Bump Version to 1.11.4 (#26578)
Co-authored-by: willhickey <willhickey@users.noreply.github.com>
2022-07-11 23:30:38 -05:00
Jeff Washington (jwash) 1db136a8be
do not count 0 lamport accounts as rent-paying (#26514) 2022-07-11 22:16:46 -05:00
Jeff Washington (jwash) 36bd46bece
ancient append vec: reuse AccountsToStore (#26572) 2022-07-11 22:13:09 -05:00
Nicholas Clarke ee0a40937e
Add validator argument log_messages_bytes_limit to change log truncation limit.
Add new cli argument log_messages_bytes_limit to solana-validator to control how long program logs can be before truncation
2022-07-11 10:53:18 -05:00
behzad nouri ba785cf8ab
removes erroneous uses of std::mem::swap (#26536)
All instances should be replace by std::mem::{replace,take},
or just plain assignment.
2022-07-11 11:33:15 +00:00
Jeff Washington (jwash) 602da5e51f
add accounts db config to bank tests (#26517) 2022-07-10 19:42:06 -05:00
Jeff Washington (jwash) 2b6c9852f1
add more slop to test_age_time (#26537) 2022-07-10 17:52:07 -05:00
Jeff Washington (jwash) 1babf460ab
collapse some bank test creation fns (#26532) 2022-07-10 17:51:38 -05:00
behzad nouri 105c7e19e1
moves check for feature activation out of accounts loop (#26535) 2022-07-10 17:39:19 +00:00
Jeff Washington (jwash) 8576832f2a
rename Bank::new_with_config_for_tests (#26533) 2022-07-10 09:33:13 -05:00
Brooks Prumo 785a7a5936
Track more accounts data size changes (#26467) 2022-07-08 19:17:15 -05:00
behzad nouri c99d9f00a9
preserves rent_epoch for rent exempt accounts (#26479)
https://github.com/solana-labs/solana/pull/22292
prevents rent paying account creation going forward. As a result
rent_epoch field for rent exempt accounts is redundant, and advancing
this field will incur expensive account rewrites and cause discrepancy
between accounts-db and cached vote/stake accounts.

This commit adds a feature which upon activation preserves rent_epoch
field for rent exempt accounts so that the field is frozen and is no
longer advanced.
2022-07-08 20:04:08 +00:00
HaoranYi 995756a550
Use RefCount type alias instead of u64 (#26472)
use RefCount type alias instead of u64
2022-07-08 13:51:16 -05:00
Brooks Prumo 78b8a8a6db
Remove `get_total_resize_remaining()` and `total_resize_limit` from `TransactionContext` (#26504) 2022-07-08 20:26:54 +02:00
dependabot[bot] d1370f2c7d
chore: bump bytemuck from 1.9.1 to 1.10.0 (#26495)
* chore: bump bytemuck from 1.9.1 to 1.10.0

Bumps [bytemuck](https://github.com/Lokathor/bytemuck) from 1.9.1 to 1.10.0.
- [Release notes](https://github.com/Lokathor/bytemuck/releases)
- [Changelog](https://github.com/Lokathor/bytemuck/blob/main/changelog.md)
- [Commits](https://github.com/Lokathor/bytemuck/compare/v1.9.1...v1.10.0)

---
updated-dependencies:
- dependency-name: bytemuck
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2022-07-08 10:04:43 -06:00
Jeff Washington (jwash) d9eee72edb
remove FoundStoredAccount.account_size (#26460) 2022-07-07 18:37:14 -05:00
Brooks Prumo 3b759bcffb
Add fn to get max accounts data size from Bank (#26478) 2022-07-07 18:08:42 -05:00
Jeff Washington (jwash) 38d53ef8a2
add metrics for # scans active and scan distance (#26395) 2022-07-07 16:20:32 -05:00
Jeff Washington (jwash) ee0a54ce80
improve clean acct idx calls (#26320) 2022-07-07 15:50:12 -05:00
Jeff Washington (jwash) b582e4ce0f
introduce UpsertReclaim (#26462) 2022-07-07 15:40:17 -05:00
github-actions[bot] 9d937fb8a0
Bump Version to 1.11.3 (#26481)
Co-authored-by: willhickey <willhickey@users.noreply.github.com>
2022-07-07 14:39:46 -05:00
Jeff Washington (jwash) c9f1d01e79
refactor, add GetUniqueAccountsResult (#26457) 2022-07-07 13:49:33 -05:00
Jeff Washington (jwash) 8e64b5883e
restore population of 'accounts' metric in rent collection (#26454) 2022-07-07 13:19:23 -05:00
Jeff Washington (jwash) 72256ac54d
use iter for append vecs (#26459) 2022-07-07 13:13:47 -05:00
Justin Starry 8416090732
Clean up `enable_durable_nonce` feature (#26470)
Clean up enable_durable_nonce feature
2022-07-07 17:14:46 +01:00
Justin Starry eb6a722eba
Clean up `nonce_must_be_writable` feature (#26444)
Clean up nonce_must_be_writable feature
2022-07-07 15:45:19 +01:00
Jeff Washington (jwash) b3a47de1ce
improve startup stats for rent payers (#26432) 2022-07-07 08:39:57 -05:00
Brooks Prumo f8bdedb596
Make accounts data size tests more robust (#26466) 2022-07-07 00:01:13 -05:00
Xiang Zhu 134303714b
Fix the order assumption of the pubkeys created by Pubkey::new_unique() (#26451)
new_unique() does not gurantee the increment order due to the bytes
array storage and its eq-partial trait interpreting the bytes in the
big-endian way.
2022-07-06 15:35:32 -07:00
carllin 90ef2cd02a
Parse snapshot for bank fields (#26016) 2022-07-06 17:30:30 -05:00
Jeff Washington (jwash) 16219e300e
generating index sets uncleaned_roots correctly (#26431)
* generating index sets uncleaned_roots correctly

* fix test failures

* rename

* update comments
2022-07-06 16:12:47 -05:00
behzad nouri d7201a8d1a
names fields in RentResullt::CollectRent enum variant (#26449)
Avoiding ambiguous raw tuple:
    CollectRent((Epoch, u64))

Using named fields instead:
    CollectRent {
        new_rent_epoch: Epoch,
        rent_due: u64,
    },
2022-07-06 20:01:16 +00:00
Justin Starry f4718be22f
Clean up `leave_nonce_on_success` feature (#26447)
Clean up leave_nonce_on_success feature
2022-07-06 17:59:34 +00:00
Justin Starry ff1e6fcbed
Clean up `nonce_must_be_advanceable` feature (#26446)
Clean up nonce_must_be_advanceable feature
2022-07-06 17:36:33 +00:00
Alexander Meißner 06ebfa1eb2
Replicates `AccountsDataMeter` in `TransactionContext` (#26438)
Replicates AccountsDataMeter in TransactionContext.
2022-07-06 19:27:42 +02:00
Justin Starry 611ac33718
Clean up `nonce_must_be_authorized` feature (#26445)
Clean up nonce_must_be_authorized feature
2022-07-06 17:17:31 +00:00
Xiang Zhu 44f499cff3
Refactor account index scan function parameters (#26428)
* Refactor account index scan function parameters

F: FnMut(bool, &SlotList<T>, &'a Pubkey, RefCount) -> AccountsIndexScanResult,
Refactor to take  (&'a Pubkey, Option<(&SlotList<T>, RefCount))

* Update comments based on review feedback

* cargo fmt --all fixes
2022-07-06 09:49:11 -07:00
Jeff Washington (jwash) 17a99d98dd
Revert "avoid adding to 'uncleaned_roots' when generating index and c… (#26441)
Revert "avoid adding to 'uncleaned_roots' when generating index and caller passes accounts-db-skip-shrink (#25936)"

This reverts commit e24cc537a4.
2022-07-06 11:32:45 -05:00
Jeff Washington (jwash) ab164fc975
simplify code (#26443) 2022-07-06 11:22:26 -05:00
Justin Starry f8dccd4602
Clean up `max_tx_account_locks` feature (#26440)
Clean up max_tx_account_locks feature
2022-07-06 15:06:03 +00:00
Jeff Washington (jwash) 5afe4d938d
combine lookup in calc_delete_dependencies (#26429) 2022-07-06 08:50:14 -05:00
Jeff Washington (jwash) 71c7b4f9cb
use append vec iter in construct_candidate_clean_keys (#26426) 2022-07-06 08:49:35 -05:00
behzad nouri 6f5857a5db
removes feature gate code separating durable nonce from blockhash domain (#26055) 2022-07-06 12:03:13 +00:00
dependabot[bot] 37f4621c06
chore: bump serde from 1.0.137 to 1.0.138 (#26421)
* chore: bump serde from 1.0.137 to 1.0.138

Bumps [serde](https://github.com/serde-rs/serde) from 1.0.137 to 1.0.138.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.137...v1.0.138)

---
updated-dependencies:
- dependency-name: serde
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2022-07-05 23:18:08 -06:00
Tao Zhu c1d89ad749
forward packets by prioritization in desc order (#25406)
- Forward packets by prioritization in desc order
- Add support of cost-tracking by transaction requested compute units
- Hook up account buckets to forwarder
- Add metrics for forwardable batches count
- Remove redundant invalid packets filtering at end of slot since forwarder will do the same when batch forwardable packets
- Add bench test for forwarding
2022-07-05 23:24:58 -05:00
Jeff Washington (jwash) 38216aa781
update comment for the next weary traveller (#26413) 2022-07-05 23:12:35 -05:00
Jeff Washington (jwash) 75149fd624
move mark_old_slots_as_dirty to be called from ahv (#26411) 2022-07-05 23:11:32 -05:00
Jeff Washington (jwash) 2aafef38f9
is_shrinking_productive calls avoid arc clone (#26422) 2022-07-05 17:53:29 -05:00
Jeff Washington (jwash) b7e34aea15
refactor calculate_rent_result (#26416)
reactor calculate_rent_result
2022-07-05 16:45:59 -05:00
Brooks Prumo 53b9420562
Remove `INTO!` macros in serde_snapshot (#26409) 2022-07-05 16:37:20 -05:00
carllin ce39c14025
Add end-to-end replay slot metrics (#25752) 2022-07-05 13:58:51 -05:00
Jeff Washington (jwash) 0ab521d5f1
inc counter when append vec drop fails (#26396) 2022-07-05 11:27:01 -05:00
Jeff Washington (jwash) 5f3b7bdd16
prevent ledger tool from calculating hash on non-rooted slots (#26355) 2022-07-02 11:50:01 -05:00
Jeff Washington (jwash) 88b5b7e30a
improve log (#26360) 2022-07-01 13:12:52 -05:00
HaoranYi 728986ddf4
Spell correction and add remove slot in AccountDB purge assert message (#26358)
nit spell. add remove slots in accoutdb purge assert message
2022-07-01 17:39:35 +00:00
Michael Vines fdb186ba3b pacify nightly clippy 2022-06-30 21:31:49 -07:00
Alexander Meißner 72a968fbe8
Fix/system instruction processor tests (#26338)
* Make mock_process_instruction() stricter by invoking the verify() method.

* Fixes broken test cases.
2022-07-01 00:06:24 +02:00
Jack May 4563bf40f6
cleanup feature: tx-wide-compute-cap (#26326) 2022-06-29 23:54:45 -07:00
Brooks Prumo 36d13af015
Add stats for Accounts Background Service (#26298)
* Add stats for Accounts Background Service

* pr
2022-06-30 02:21:30 +00:00
behzad nouri d053ce79d4
removes RwLock+Once in favor of OnceCell in caching staked-nodes (#26313)
VoteAccounts uses std::sync::{RwLock, Once} to lazily compute and cache
staked_nodes:
https://github.com/solana-labs/solana/blob/032bee13a/runtime/src/vote_account.rs#L89-L104

This commit instead switches to using once_cell::sync::OnceCell which
provides this exact intended functionality by design.
2022-06-29 22:22:22 +00:00
Jeff Washington (jwash) 557bf6e656
allow initial hash calc to occur in bg (#26271)
* allow initial hash calc to occur in bg

* validator_initialized -> startup_verification_complete

* add infos for leader and vote

* rework snapshot for startup verification

* change to assert
2022-06-29 16:48:33 -05:00
Jeff Washington (jwash) 608191352f
track # roots added/removed since last clean (#26319) 2022-06-29 16:47:57 -05:00
Jeff Washington (jwash) 2deb396490
remove noisy and useless 'error!' and add test (#26138) 2022-06-29 16:26:23 -05:00
Alexander Meißner c01a8f271e
Prepare built-in program tests to deal with stricter account modification checks (#26314)
* Fixes test_vote_state_withdraw().

* Set is_writable to true where needed.
2022-06-29 19:29:12 +02:00
Jeff Washington (jwash) e241906db2
remove outdated, confusing comment (#26315) 2022-06-29 10:42:34 -05:00
Jeff Washington (jwash) ca83f143aa
load_accounts_index_for_shrink uses scan (#26260) 2022-06-29 09:02:27 -05:00
behzad nouri af7f08eba4
uses OnceCell instead of RwLock+Once to cache vote-state in vote-account (#26257)
RwLock seems excessive since only the very 1st call to VoteAccount::vote_state
will write-lock the inner field. Future calls would also incur overhead of an
RwLockReadGuard.
once_cell::sync::OnceCell provides a matching api to the desired functionality.
2022-06-29 11:45:53 +00:00
Justin Starry 50631e274d
Ensure that staked nodes is populated before returning length (#26293) 2022-06-28 19:15:42 +00:00
Jeff Washington (jwash) 89e5640c63
AccountsIndex::Scan takes iter (#26268) 2022-06-28 12:47:17 -05:00
Jeff Washington (jwash) d07b079850
make test_age_time less intermittent (#26280) 2022-06-28 09:13:01 -05:00
Jeff Washington (jwash) bfc390f9ed
refactor acct idx scan func for reuse (#26252) 2022-06-28 09:01:04 -05:00
Jeff Washington (jwash) 4b591d7b59
AccountsDb::load_accounts_index_for_shrink takes slice (#26267) 2022-06-28 08:59:57 -05:00
Alexander Meißner e8fed88669
Both kinds of LamportsError are turned into InstructionError::ArithmeticOverflow anyway. (#26273) 2022-06-28 10:04:54 +02:00
Jeff Washington (jwash) e164879f20
load_accounts_index_for_shrink uses &item instead of (&item.0, &item.1) (#26266) 2022-06-27 22:40:45 -05:00
Jeff Washington (jwash) d6a28cb7a8
load_accounts_index_for_shrink unrefed_pubkeys is optional (#26265) 2022-06-27 17:23:46 -05:00
apfitzge 50eb9965ab
fix test_new_from_file_crafted_executable for m1 (#26009)
* fix test_new_from_file_crafted_executable for m1

* re-add old assert/comment, but disable for aarch64
2022-06-27 12:03:52 -05:00
Jeff Washington (jwash) d8516b5c88
comment and expand on test (#26231) 2022-06-27 11:35:46 -05:00
Brooks Prumo 662818ef0d
Use `VoteAccount::node_pubkey()` (#26207) 2022-06-27 09:09:06 -05:00
Jeff Washington (jwash) cbefc581c0
test_rent_collect_rent_from_accounts (#26204) 2022-06-25 19:04:28 -05:00
Justin Starry 0ca963a869
Refactor: Add separate methods for retrieving stake info from VoteAccounts (#26224) 2022-06-25 20:03:15 +00:00
Justin Starry 7cd7173b71
Refactor: Add get_delegated_stake method to VoteAccounts (#26221) 2022-06-25 16:41:35 +00:00
Justin Starry 44d1e62007
Refactor: No need to return stake in Bank::get_vote_account (#26220) 2022-06-25 16:27:43 +00:00
behzad nouri 2efdb965dd
encapsulates AuthorizeNonceAccount implementation in the nonce module (#26202)
Follow up commit removes feature gate code separating durable nonce from
blockhash domain. This commit allows to encapsulate any logic
distinguishing legacy vs current nonce versions in the nonce module
after removing the feature gate.
2022-06-25 12:17:39 +00:00
Jeff Washington (jwash) a3395a786a
vote_account uses AccountSharedData to avoid copies (#23687)
* vote_account uses AccountSharedData to avoid copies

* simpler deserialize
2022-06-24 15:08:01 -05:00
Nicholas Rempel 2b21c1da29 Sort Cargo.toml files 2022-06-24 12:41:38 -07:00
Jeff Washington (jwash) 18397527ad
retain prior metric behavior for invalid_cached_stake_accounts (#26181) 2022-06-24 14:12:16 -05:00
Jeff Washington (jwash) 054b25729d
eliminate vec creation in stake rewards (#26191) 2022-06-24 13:40:53 -05:00
HaoranYi fbcb4d8216
extract get_stake_reward fn from StakeReward (#26183) 2022-06-24 08:44:48 -05:00
Brooks Prumo 877fedadac
Remove StatusCacheRc type and use StatusCache directly (#26184) 2022-06-24 08:38:56 -05:00
apfitzge 4a729effff
Bugfix/test get rent collection accounts remove random account (#26178)
* Use default GenisisConfig so we don't have a random account

* Formatting
2022-06-24 09:22:50 -04:00
Jeff Washington (jwash) a36abfda28
don't skip rewrite when account was written IN the current slot already (#26197) 2022-06-24 07:28:37 -05:00
Brooks Prumo 5b842669e7
Remove InMemAccountsIndex::map() and use map_internal directly (#26189) 2022-06-23 15:55:35 -05:00
Brooks Prumo 23c50a2389
Add StatusCache::root_slot_deltas() and use it (#26170) 2022-06-23 15:19:06 -05:00
apfitzge 7efd05f7b3
Change get_pubkeys to match the new enum (#26177) 2022-06-23 11:56:09 -05:00
Boqin Qin(秦 伯钦) 355e09e1fb
runtime: fix possible deadlock in in_mem_accounts_index (#26046) 2022-06-23 11:35:47 -05:00
Jeff Washington (jwash) 102fb98070
impl StorableAccounts for [StakeReward] (#26165) 2022-06-23 08:34:28 -05:00
Jeff Washington (jwash) 19b79aa764
add metrics for invalid_cached_stake_accounts_rent_epoch (#26143) 2022-06-23 08:33:24 -05:00
Jeff Washington (jwash) 9db44ce352
cleanup some AccountSharedData::from(Account (#26120) 2022-06-23 08:32:48 -05:00
Brooks Prumo 7d29c26315
Remove auto traits from ICE work-around (#26098) 2022-06-23 07:22:30 -05:00
Brooks Prumo a5cf72e446
StatusCache::roots() returns `Slot`s not `u64`s (#26164) 2022-06-23 07:03:09 +00:00
github-actions[bot] 5c2f819f99
Bump Version to 1.11.2 (#26159) 2022-06-22 21:16:18 -05:00
Jeff Washington (jwash) aea75e82e3
Bank::store_accounts takes StorableAccounts (#26113) 2022-06-22 21:11:01 -05:00
Jeff Washington (jwash) a32523c43b
cleanup some tests (#26139) 2022-06-22 21:10:46 -05:00
HaoranYi c4efb9f19e
Stake cache size (#26071)
* reduce threadpool for stake processing

* wip

* Revert "reduce threadpool for stake processing"

This reverts commit 004a4f872ea7a3ef53e38d145b6350c3f57c680c.

* batch update stake_cache

* fix deadlock

* add test

* code review feedbacks

* more review feedbacks

* fix conflicts

* report stake account len and vote account len at epoch boundary

* report num_staked_nodes

* remove batch store, no atomic for redeem timing
2022-06-22 16:22:22 -05:00
Jeff Washington (jwash) c5c2b44ecc
adjust cached stake account compare for rent_epoch (#26135) 2022-06-22 15:11:25 -05:00
ivandzen 1ea6a2cb91
Extended interface for AccountOverrides (#25964)
* Extended interface for AccountOverrides

* fix checker issue

* make accounts private

Co-authored-by: Ivan Loboda <lobodatell@yandex.ru>
2022-06-22 15:22:31 -04:00
apfitzge f4189c0305
ledger-tool minimized snapshots (#25334)
* working on local snapshot

* Parallelization for slot storage minimization

* Additional clean-up and fixes

* make --minimize an option of create-snapshot

* remove now unnecessary function

* Parallelize parts of minimized account set generation

* clippy fixes

* Add rent collection accounts and voting node_pubkeys

* Simplify programdata_accounts generation

* Loop over storages to get slot set

* Parallelize minimized slot set generation

* Parallelize adding owners and programdata_accounts

* Remove some now unncessary checks on the blockstore

* Add a warning for minimized snapshots across epoch boundary

* Simplify ledger-tool minimize

* Clarify names of bank's minimization helper functions

* Remove unnecesary funciton, fix line spacing

* Use DashSets instead of HashSets for minimized account and slot sets

* Filter storages uses all threads instead of thread_pool

* Add some additional comments on functions for minimization

* Moved more into bank and parallelized

* Update programs/bpf/Cargo.lock for dashmap in ledger

* Clippy fix

* ledger-tool: convert minimize_bank_for_snapshot Measure into measure!

* bank.rs: convert minimize_bank_for_snapshot Measure into measure!

* accounts_db.rs: convert minimize_accounts_db Measure into measure!

* accounts_db.rs: add comment about use of minimize_accounts_db

* ledger-tool: CLI argument clarification

* minimization functions: make infos unique

* bank.rs: Add test_get_rent_collection_accounts_between_slots

* bank.rs: Add test_minimization_add_vote_accounts

* bank.rs: Add test_minimization_add_stake_accounts

* bank.rs: Add test_minimization_add_owner_accounts

* bank.rs: Add test_minimization_add_programdata_accounts

* accounts_db.rs: Add test_minimize_accounts_db

* bank.rs: Add negative case and comments in test_get_rent_collection_accounts_between_slots

* bank.rs: Negative test in test_minimization_add_programdata_accounts

* use new static runtime and sdk ids

* bank comments to doc comments

* Only need to insert the maximum slot a key is found in

* rename remove_pubkeys to purge_pubkeys

* add comment on builtins::get_pubkeys

* prevent excessive logging of removed dead slots

* don't need to remove slot from shrink slot candidates

* blockstore.rs: get_accounts_used_in_range shouldn't return Result

* blockstore.rs: get_accounts_used_in_range: parallelize slot loop

* report filtering progress on time instead of count

* parallelize loop over snapshot storages

* WIP: move some bank minimization functionality into a new class

* WIP: move some accounts_db minimization functionality into SnapshotMinimizer

* WIP: Use new SnapshotMinimizer

* SnapshotMinimizer: fix use statements

* remove bank and accounts_db minimization code, where possible

* measure! doesn't take a closure

* fix use statement in blockstore

* log_dead_slots does not need pub(crate)

* get_unique_accounts_from_storages does not need pub(crate)

* different way to get stake accounts/nodes

* fix tests

* move rent collection account functionality to snapshot minimizer

* move accounts_db minimize behavior to snapshot minimizer

* clean up

* Use bank reference instead of Arc. Additional comments

* Add a comment to blockstore function

* Additional clarifying comments

* Moved all non-transaction account accumulation into the SnapshotMinimizer.

* transaction_account_set does not need to be mutable now

* Add comment about load_to_collect_rent_eagerly

* Update log_dead_slots comment

* remove duplicate measure/print of get_minimized_slot_set
2022-06-22 13:17:43 -04:00
Jeff Washington (jwash) cf5ec13f59
Accounts::store_accounts_cached uses StorableAccounts (#26112) 2022-06-22 12:13:56 -05:00
Michael Vines f3639b76ce Remove some clippy lints 2022-06-22 09:23:22 -07:00
HaoranYi 5624ab0d4c
Optimize stake accounts store (#26004)
* reduce threadpool for stake processing

* wip

* Revert "reduce threadpool for stake processing"

This reverts commit 004a4f872ea7a3ef53e38d145b6350c3f57c680c.

* batch update stake_cache

* fix deadlock

* add test

* code review feedbacks

* more review feedbacks

* clippy

* update comments

* fix conflicts

* remove batch store, no atomic for redeem timing

* refactor stake_reward struct
2022-06-22 10:45:58 -05:00
Jeff Washington (jwash) 8e2bae7bf9
batch insert updates stat count correctly (#26107) 2022-06-22 08:45:42 -05:00
Jeff Washington (jwash) d3829b1a76
store_cached takes StorableAccounts to eliminate collect (#26109) 2022-06-22 08:45:03 -05:00
Brooks Prumo 8a18e02f80
Call StatusCache::append() directly (#26125) 2022-06-22 07:42:01 -05:00
Brooks Prumo 31c9b03049
Remove unused SignatureConfirmationStatus (#26124) 2022-06-22 07:41:22 -05:00
Brooks Prumo 97bd81a32b
Add bench for StatusCache::slot_deltas() (#26123) 2022-06-22 07:41:06 -05:00
Brooks Prumo d35c1a7d04
Use BankStatusCache in bank.rs (#26114) 2022-06-21 23:16:16 +00:00
Jeff Washington (jwash) 43e0d29b18
Revert "fix double count of account index del stats (#25797)" (#26103)
This reverts commit 05cb25d8da.
2022-06-21 14:05:26 -05:00
Jeff Washington (jwash) 7a3dc3a57b
remove unused param (#26096) 2022-06-21 13:10:41 -05:00
Will Hickey 51f26dc96e
Bump version to 1.11.1 (#26104) 2022-06-21 12:07:46 -05:00
Boqin Qin(秦 伯钦) 95ea506d75
runtime: fix double-readlock in in_mem_accounts_index.rs (#26047) 2022-06-20 15:44:19 +01:00
Tyera Eulberg 752c851070
Clean up feature preventing new rent-paying accounts (#26000)
* Clean up require_rent_exempt_accounts feature

* Fix tests in bank_client, system_instruction_processor

* Fix tests in snapshot_utils

* Fix bank tests

* Ignore rent tests

* Fix stake-accounts tests
2022-06-16 15:35:25 -06:00
Alexander Meißner bf9ca9827e
Refactor: instruction account index (#25825)
* Adds methods based on instruction_account_index to InstructionContext.
Removes methods which are based on index_in_instruction.

* Adjusts program-runtime.

* Adjusts runtime.

* Adjusts bpf loader.

* Adjusts built-in programs.

* Adjusts program-test and bpf tests.
2022-06-16 18:46:17 +02:00
Brooks Prumo 54675b2dd6
Add bank tests for accounts_data_size and resize transactions / rent collection (#25884) 2022-06-15 19:40:43 -05:00
Jeff Washington (jwash) b02c412d5b
generate_index straight to disk and batched (#25947)
* generate_index straight to disk and batched

* renames and comments

* handle in-mem case correctly

* use mutex
2022-06-15 18:14:39 -05:00
Jeff Washington (jwash) 631ea93259
parallel rent collection avoids overlapping ranges (#25991) 2022-06-15 16:04:38 -05:00
Tyera Eulberg 99cf1e280e
Clean up warp-timestamp features (#25994) 2022-06-15 14:11:41 -06:00
Jeff Washington (jwash) 80f90d6e96
disable logging per thread on rent collection load (#25982) 2022-06-15 13:16:00 -05:00
apfitzge e1efc5bebb
append_vec.rs: remove new_empty_map (unused) (#25885) 2022-06-15 08:59:46 -05:00
Jeff Washington (jwash) ae37359b6b
AcctIdx: only advance age on thread 0 (#25943) 2022-06-14 21:43:42 -05:00
Alexander Meißner 4652377e5b
Removes the feature gate of reject_empty_instruction_without_program. (#25870) 2022-06-14 18:00:12 +02:00
Jeff Washington (jwash) 8c6455022f
remove dead code marker (#25945) 2022-06-14 10:47:26 -05:00
Jeff Washington (jwash) e24cc537a4
avoid adding to 'uncleaned_roots' when generating index and caller passes accounts-db-skip-shrink (#25936) 2022-06-14 10:10:44 -05:00
Jeff Washington (jwash) 01f41b8c76
get rid of dead code marker (#25944) 2022-06-14 08:44:40 -05:00
Steven Luscher 45d11f3d26
Implement `VoteInstruction::AuthorizeWithSeed` & `VoteInstruction::AuthorizeWithSeedChecked` (#25928)
* [vote_authorize_with_seed] Add `VoteInstruction::AuthorizeWithSeed`

* [vote_authorize_with_seed] You can now update a vote account's authority if it's a derived key for which you control the base key

* [vote_authorize_with_seed] Add test helper to create a vote account whose authorities are derived keys

* [vote_authorize_with_seed] Write tests to assert the behavior of `VoteInstruction::AuthorizeWithSeed`

* [vote_authorize_with_seed] Feature gate the `VoteInstruction::AuthorizeWithSeed` processor

* [vote_authorize_with_seed] Add `VoteInstruction::AuthorizeWithSeed` to transaction status parser

* [vote_authorize_with_seed] Add `VoteInstruction::AuthorizeWithSeed` to docs

* [vote_authorize_with_seed] Add `VoteInstruction::AuthorizeCheckedWithSeed`

* [vote_authorize_with_seed] You can now update a vote account's authority (while checking that the new authority has signed) if it's a derived
key for which you control the base key

* [vote_authorize_with_seed] Add `VoteInstruction::AuthorizeCheckedWithSeed` to transaction status parser

* [vote_authorize_with_seed] Write tests to assert the behavior of `VoteInstruction::AuthorizeCheckedWithSeed`
2022-06-13 20:36:44 -07:00
Jeff Washington (jwash) 306faed3a8
update comment (#25949) 2022-06-13 17:43:29 -05:00
apfitzge d56a706b20
Add static vector to get all static ids in runtime (#25909)
* Add function to get all static ids in runtime

* static_ids.rs: use a lazy_static ref instead of a function
2022-06-13 13:21:16 -05:00
apfitzge 3e1fd4dad0
snapshot_utils.rs: remove unnecessary sort (#25893) 2022-06-13 11:12:33 -05:00
apfitzge 04969a2282
serialize snapshots dir first (#25889) 2022-06-13 09:33:32 -05:00
Jeff Washington (jwash) 3eff7648ff
collect rent from multiple partitions in parallel (#25774) 2022-06-13 07:53:49 -05:00
Michael Vines b4237f3f2c Do not exclude failed simple vote transactions from consensus 2022-06-12 22:11:23 -07:00
Brooks Prumo a2c180dc0b
Add EvictionsGuard to InMemAccountsIndex (#25847) 2022-06-12 17:31:58 -05:00
Brooks Prumo 53695ecd21
Remove SlotT type alias (#25908) 2022-06-10 22:04:56 -05:00
Jack May 9fb0e76dc2
cleanup feature; do_support_realloc (#25882) 2022-06-10 15:33:19 -07:00
apfitzge 0fffccb4a3
account_info.rs: docs offset is reduced_offset * ALIGN_BOUNDARY_OFFSET (#25848) 2022-06-10 16:19:53 -05:00
ivandzen 2251aa2809
Report transaction signature on update account (#25726)
There's no direct linkage between account-update events and transaction-events

Issue is linked with geyser plugin interface
Currently, there's no straightforward way to determine what notify_transaction call is related to particular update_account call. So, there's no simple way to determine what transaction caused what changes in accounts. This issue is especially valuable in case when several transactions modifies single account in the same slot.

Add txn_signature field to ReplicaTransactionInfo structure

This additional field will contain the signature of transaction which caused this account update event. This modification is not bringing significant overhead because all necessary information for such linkage already available inside Accounts::collect_accounts_to_store method
2022-06-10 10:24:35 -07:00
Brooks Prumo bea35d5fbe
Add ApplyFeatureActivationsCaller enum (#25858) 2022-06-10 10:15:13 -05:00
Brooks Prumo fedd3f26d0
Fix new cost tracker with accounts data size limit (#25843) 2022-06-10 07:39:43 -05:00
behzad nouri b4190319a1
feat(nonce): adds system instruction to upgrade legacy nonce versions (#25789)
https://github.com/solana-labs/solana/pull/25788
permanently disables durable transactions with legacy nonce versions
which are within chain blockhash domain.

This commit adds a new system instruction for a one-time idempotent
upgrade of legacy nonce accounts in order to bump them out of chain
blockhash domain.
2022-06-10 00:04:29 +00:00
behzad nouri 3c1ce3cc93
permanently disables durable nonces with chain blockhash domain (#25788)
https://github.com/solana-labs/solana/pull/25744
separated durable nonce and blockhash domains, which will stop double
execution going forward. However it is possible that a durable
transaction has *already* been executed once as a normal transaction and
it is now a valid durable transaction. #25744 cannot stop such
transactions to be re-executed until the nonce accounts are advanced.

This commit adds a new nonce version indicating that the nonce is moved
out of the blockhash domain, and permanently disables durable
transactions for legacy nonces which are in the blockhash domain.
2022-06-09 15:28:37 +00:00
Brooks Prumo e5f36aa371
Change `WaitableCondvar`'s mutex to hold `()` (#25857) 2022-06-09 09:03:22 -05:00
Justin Starry 6a7edc02c6
Reject durable nonce txs that don't use an advanceable nonce (#25832)
* Reject durable nonce txs that use the latest durable nonce hash

* feedback
2022-06-08 21:34:57 +00:00
Justin Starry b2b426d4bf
Reject durable nonce transactions not signed by authority (#25831) 2022-06-08 14:43:09 -05:00
HaoranYi 40b1655eb5
fix intermittent bucket age test failure (#25841)
* fix double count of account index del stats

* refactor to seperate delete and insert account index stats

* fix test

* increase test timing tolerance for bucket age

* Revert "increase test timing tolerance for bucket age"

This reverts commit 2b91cf902208e4585d25dff1d4298d412c9da516.

* Revert "Revert "increase test timing tolerance for bucket age""

This reverts commit 8f07367b64ddc45a9a693729ea2725895627d14c.
2022-06-08 16:15:49 +00:00
HaoranYi 28142f8a6d
Refactor: separate delete and insert account index stats (#25800)
* fix double count of account index del stats

* refactor to seperate delete and insert account index stats

* fix test

* increase test timing tolerance for bucket age

* Revert "increase test timing tolerance for bucket age"

This reverts commit 2b91cf902208e4585d25dff1d4298d412c9da516.
2022-06-08 09:05:37 -05:00
Brooks Prumo d4e4871c47
Add tests to Bank for accounts data size (#25816) 2022-06-07 18:28:53 +00:00
HaoranYi 05cb25d8da
fix double count of account index del stats (#25797) 2022-06-07 11:50:57 -05:00
apfitzge e6c21a3036
Convert Measure::this to measure! and remove Measure::this (#25776)
* Remove the args param from Measure::this since we don't ever use it

* banking_stage.rs: convert to measure!

* poh_recorder.rs: convert to measure!

* cost_update_service.rs: convert to measure!

* poh_service.rs: convert to measure!

* bank.rs: convert to measure!

* measure.rs: Remove Measure::this now that all have been converted to measure!
2022-06-06 20:21:05 -05:00
Ashwin Sekar 8caced68ce
Serialize lamports per signature (#25364)
* Serialize lamports per signature

* Add full snapshot archive test, enable features in previous tests
2022-06-06 13:07:02 -07:00
Brooks Prumo ec64d5261f
Avoid RMWs on shared data inside parallel loops: collect_rent_from_accounts() (#25790) 2022-06-06 13:32:22 -05:00
Justin Starry 61ad8fcc55
Fix bank executor stats and remove copy-on-write semantics (#25621)
* Fix bank executor stats and remove copy-on-write semantics

* Remove clone implementation for CachedExecutors

* feedback
2022-06-04 13:54:26 +00:00
behzad nouri 9851774133 adds feature gate enabling durable nonce
Previous commit separates durable nonce and blockhash domains with a
feature gate. A 2nd feature added in this commit enables durable nonce
at least one epoch after the 1st feature.
By the time 2nd feature is activated, some nonce accounts will have an
old blockhash, but no nonce account can have a recent blockhash.
As a result no transaction (durable or normal) can be executed twice.
2022-06-04 12:39:31 +00:00
behzad nouri 5ee157f43d separates durable nonce and blockhash domains
AdvanceNonceAccount instruction updates nonce to blockhash. This makes it
possible that a durable transaction is executed twice both as a normal
transaction and a nonce transaction if it uses blockhash (as opposed to nonce)
for its recent_blockhash field.

The commit prevents this double execution by separating nonce and blockhash
domains; when advancing nonce account, blockhash is hashed with a fixed string.
As a result a blockhash cannot be a valid nonce value; and if transaction was
once executed as a normal transaction it cannot be re-executed as a durable
transaction again and vice-versa.
2022-06-04 12:39:31 +00:00
Jeff Washington (jwash) 8b0d5b2485
parallelize rent collection (#25767) 2022-06-03 18:00:02 -05:00
Brooks Prumo edebd7e495
Grab read-lock instead of write-lock in From<PreAllocatedAccountMapEntry<T>> for (Slot, T) (#25770) 2022-06-03 20:48:32 +00:00
Christian Kamm 0da0e0da63
Bank: Batch account stores in collect_rent_eagerly (#25707)
* Bank: Batch account stores in collect_rent_eagerly

Previously store() calls were done one-by-one, which leads to suboptimal
performance.

* Accounts: Remove store_slow_cached()

* clippy

Co-authored-by: Jeff Washington (jwash) <wash678@gmail.com>
2022-06-03 15:00:39 -05:00
Jeff Washington (jwash) 646bd2e488
extract prefix_from_pubkey (#25766) 2022-06-03 14:03:15 -05:00
Jeff Washington (jwash) a9a94166ef
load_account_into_read_cache (#25760) 2022-06-03 13:34:17 -05:00
sakridge e6e2058758
Increase accounts stack size to 8mb (#25592) 2022-06-03 19:46:01 +02:00
Jeff Washington (jwash) 779ec564e4
add data to assert (#25749) 2022-06-03 09:32:43 -05:00
Jeff Washington (jwash) 7c95ae341d
disk idx: reduce lock calls (#25734)
* disk idx: reduce lock calls

* adjust ordering for count_buckets_flushed
2022-06-02 18:26:08 -05:00
Justin Starry 6fc057c1d9
Remove activated evict_invalid_stakes_cache_entries feature (#25739) 2022-06-02 21:04:54 +00:00
apfitzge ed68c0b889
epoch_stakes_reation -> epoch_stakes_creation in Bank::_new_from_parent (#25660) 2022-06-02 12:26:01 -05:00
carllin eeec3fb9c9
Remove unwrap (#25538) 2022-06-02 12:18:46 -05:00
Jeff Washington (jwash) e8df90c34b
rename count_buckets_flushed (#25733) 2022-06-02 12:16:01 -05:00
Jeff Washington (jwash) ddd0ed0af1
fix ordering in disk idx (#25735) 2022-06-02 11:50:44 -05:00
Jeff Washington (jwash) 96f71c7d4d
add metrics on rent collection pieces (#25713) 2022-06-02 09:08:53 -05:00
Jeff Washington (jwash) d1a6cebaa2
u64 -> duration (#25708) 2022-06-02 07:56:47 -05:00
apfitzge 369ff0858d
Helper functions for getting rent partitions between slots (#25669) 2022-06-01 20:20:44 -05:00
Jeff Washington (jwash) 149a54b786
remove par_iter on update index below threshold (#25699) 2022-06-01 17:41:56 -05:00
Jeff Washington (jwash) 891dcb2084
move rent collection time into datapoint (#25714) 2022-06-01 17:28:47 -05:00
Jeff Washington (jwash) dd0bf4ff4b
add filler accounts gradually with cache flush (#25678) 2022-06-01 15:47:21 -05:00
Trent Nelson b80a82a9e5 runtime: gate nonce disable at restart slot 2022-06-01 12:00:10 -06:00
Trent Nelson 635bd40f09 runtime: disable durable nonces 2022-06-01 11:15:16 -06:00
Jeff Washington (jwash) 905fef29cb
add filter to AppendVecScan (#25664) 2022-06-01 11:55:18 -05:00
Jeff Washington (jwash) 94b0ce5d43
remove_old_historical_roots uses accounts_hash_complete_one_epoch_old (#25673) 2022-06-01 10:45:26 -05:00
Jeff Washington (jwash) c91fbd289f
set active state to shrink_ancient when doing intersting work (#25677) 2022-06-01 10:36:58 -05:00
Jeff Washington (jwash) 8bb76fcd87
add ancient metrics (#25656)
* add ancient metrics

* review feedback
2022-06-01 10:36:23 -05:00
Jeff Washington (jwash) 07958fd8ed
use self.init_accum (#25663) 2022-06-01 08:38:37 -05:00
dependabot[bot] 27978c4d0c
chore: bump flate2 from 1.0.23 to 1.0.24 (#25636)
* chore: bump flate2 from 1.0.23 to 1.0.24

Bumps [flate2](https://github.com/rust-lang/flate2-rs) from 1.0.23 to 1.0.24.
- [Release notes](https://github.com/rust-lang/flate2-rs/releases)
- [Commits](https://github.com/rust-lang/flate2-rs/commits)

---
updated-dependencies:
- dependency-name: flate2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2022-05-31 23:15:46 -06:00
Tyera Eulberg 8584a3a331
Add feature to increase timestamp bounding on the slow side to 150% of poh estimate (1s/slot) (#25666) 2022-05-31 16:53:06 -06:00
Jeff Washington (jwash) 5994b4f53f
use object for hash scanning (#25418) 2022-05-31 11:51:56 -05:00
apfitzge 56d38e4954
Add lz4 to snapshot filename regex for matching files on load (#25633)
* Add lz4 to snapshot filename regex for matching files on load

* Add tests for lz4 archive format filenames
2022-05-31 11:06:41 -05:00
steviez 6b2f58152b
Ignore rocksdb directories when unpacking genesis archive (#25575)
Unpacking the rocksdb directory from genesis archive can have unintended
consequences, such as clobbering files in pre-existing rocksdb
directory.
2022-05-31 00:38:45 -05:00
Wei Hu 4b1dd0f921
Fix a possible test hang in runtime::test_shared_buffer_sweep (#25545)
* Fix a possible test hang in runtime::test_shared_buffer_sweep

The test may hang if the total number of threads it wants to start
is greater than the number of threads configured in the rayon
threadpool. Check to avoid this situation from happening.

* Incorporate comment from Xiang Zhu, using std::cmp::min() to simplify the code
2022-05-30 23:39:32 +08:00
Christian Kamm 387d9d9c77 Store missing unupdated executors earlier 2022-05-28 13:42:35 +10:00
carllin 740ee0c502
Fix datapoint name (#25614) 2022-05-28 01:01:29 +00:00
Trent Nelson 4e6b276f6c
bump executor cache max entries (#25600)
* executor-cache: unify `remove` calls

* executor-cache: metrics - add one-hit-wonder counter

* executor-cache: bump max entries to 256
2022-05-28 01:54:10 +10:00
Justin Starry ce14c63bf0
fix: stop double counting execution timings (#25605) 2022-05-28 00:06:29 +10:00
buffalu e58de2c233
Check fee payer before loading all accounts (#25371)
* fix vote account loading

* fix clippy and rename some stuff

* fix bug

Co-authored-by: Justin Starry <justin@solana.com>
2022-05-27 13:44:29 +10:00
Brooks Prumo 7384c41ea7
Speed up bank::tests::test_max_accounts_data_size_exceeded() (#25523) 2022-05-25 22:22:30 -04:00
Justin Starry 26a02f6f59
Add invoke logging to builtin programs that don't have it yet (#25230)
* Refactor process_executor_chain

* Enable builtin program invoke logs by default

* feedback
2022-05-26 07:27:00 +08:00
Alexander Meißner f10c80b49f
Refactor: Rebase offset of `index_in_caller` (#25531)
* Removes the offset InstructionContext::get_number_of_program_accounts() from InstructionAccount::index_in_caller.

* Removes unreachable SyscallError::InvalidLength in orig_data_lens.get().
2022-05-25 13:43:20 +02:00
buffalu 7b98ff1929
Refactor: pull out fee payer balance check (#25519) 2022-05-25 20:44:03 +10:00
Brooks Prumo b7149f7f54
Add helper fn to set accounts_data_size_initial for tests (#25522) 2022-05-25 02:43:54 +00:00
Brooks Prumo 0dd3c6bf1f
Split up accounts data size field (#25495) 2022-05-24 20:26:32 -04:00
Alexander Meißner 2fb096c486
Refactor: Adds `index_in_callee` to `InstructionAccount` (#25490)
* Adds InstructionAccount::index_in_callee

* Adjusts tests and benches.

* Adds documentation for InstructionAccount.

* Adds InstructionContext::is_duplicate()
2022-05-25 00:04:46 +02:00
Justin Starry e66ea7cb6a Clean up Bank::commit_transactions parameters 2022-05-24 20:24:42 +08:00
dependabot[bot] 3566fb5660
chore: bump regex from 1.5.5 to 1.5.6 (#25488)
* chore: bump regex from 1.5.5 to 1.5.6

Bumps [regex](https://github.com/rust-lang/regex) from 1.5.5 to 1.5.6.
- [Release notes](https://github.com/rust-lang/regex/releases)
- [Changelog](https://github.com/rust-lang/regex/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/regex/compare/1.5.5...1.5.6)

---
updated-dependencies:
- dependency-name: regex
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2022-05-23 13:45:43 -06:00
Jeff Washington (jwash) bac05dc55a
calc slot info on max slot once per hash calc (#25422) 2022-05-23 10:11:00 -05:00
Jeff Washington (jwash) 41f30a2383
stop logging misleading bank hash mismatch (#25427) 2022-05-23 08:43:25 -05:00
Michael Vines b05c7d91ed Fix derive_partial_eq_without_eq clippy lint 2022-05-22 22:22:21 -07:00
Brooks Prumo f8842032c6
clippy: fix "this let-binding has unit value" warnings (#25429) 2022-05-22 12:17:59 -04:00
Alexander Meißner 7eae98c6d9
Introduces `Result` return types to mutable accessors of `BorrowedAccount` (#25380)
* Introduces result return types to get_data_mut(), set_data() and set_data_length() of BorrowedAccount.

* Introduces result return types to set_owner(), set_lamports() and set_executable() of BorrowedAccount.
2022-05-21 17:47:09 +02:00
Brooks Prumo e9a6d1ecef
clippy: is_digit(10) -> is_ascii_digit() (#25428) 2022-05-21 07:03:11 -05:00
Jeff Washington (jwash) f584b249dd
special case for common case of 1 storage on append vec scan (#25414) 2022-05-20 17:36:30 -05:00
Jack May 851958f77a
Disallow deployment of deprecated _sol_alloc_free syscall (#25179) 2022-05-20 13:19:41 -07:00
Jeff Washington (jwash) cfcc18f7f4
apply AccountsInAppendVec to more places (#25408) 2022-05-20 14:25:46 -05:00
Brooks Prumo a25212b087
Move append_vec::test_utils into its own file (#25407) 2022-05-20 13:37:45 -05:00
Jeff Washington (jwash) 9ef0acdf5d
allow hash calc to use all threads at startup (#25369) 2022-05-20 11:45:01 -05:00
Jeff Washington (jwash) 467431de89
add ledger-tool accounts-db-skip-initial-hash-calc (#25354) 2022-05-20 10:27:00 -05:00
Jeff Washington (jwash) 950245c29b
simulate filler account hash calc time at delta hash (#25387) 2022-05-20 10:20:44 -05:00
Jeff Washington (jwash) 1b4e4ef548
impl squash to ancient append vecs (#24538)
* squash to ancient append vecs

* pr feedback

* clippy
2022-05-20 10:20:17 -05:00
Jeff Washington (jwash) de03a3d3aa
filler accounts get a non-zero rent_epoch (#25386)
* fill accounts get a non-zero rent_epoch

* use snapshot_slot

* get rid of perf simulation code
2022-05-20 09:03:55 -05:00
Jeff Washington (jwash) 3fe7655378
use iterator instead of vec to iterate accounts (#25370)
* user iterator instead of vec to iterate accounts

* rename
2022-05-20 08:50:24 -05:00
Jeff Washington (jwash) 36ed36edf0
use previous epoch's # slots per epoch (#25382) 2022-05-20 08:47:55 -05:00
Michael Vines c54e06355f
voteSubscribe pubsub notification now includes the vote transaction signature (#25291) 2022-05-19 18:28:46 -07:00
Brooks Prumo 021276c955
Move updating accounts data len from execute to commit (#25256) 2022-05-19 12:24:13 -05:00
Jeff Washington (jwash) 0f703b71cc
minor test renaming (#25297) 2022-05-18 12:53:55 -05:00
Jeff Washington (jwash) fe91f40f8a
use previous epoch rent collector in expected rent collection (#25289) 2022-05-18 12:53:45 -05:00
dependabot[bot] 542bd0ec3c
chore: bump rayon from 1.5.2 to 1.5.3 (#25242)
* chore: bump rayon from 1.5.2 to 1.5.3

Bumps [rayon](https://github.com/rayon-rs/rayon) from 1.5.2 to 1.5.3.
- [Release notes](https://github.com/rayon-rs/rayon/releases)
- [Changelog](https://github.com/rayon-rs/rayon/blob/master/RELEASES.md)
- [Commits](https://github.com/rayon-rs/rayon/compare/v1.5.2...v1.5.3)

---
updated-dependencies:
- dependency-name: rayon
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2022-05-18 09:39:57 -06:00
Justin Starry a1522d0024
Use consistent naming for compute unit limit (#25229)
* Use consistent naming for compute unit limit

* feedback
2022-05-18 13:14:31 +08:00
Jon Cinque 0820065c98
rpc: Support token-2022 in token-specific calls (#25150)
* rpc: Support token-2022 in token-specific calls

* Address feedback
2022-05-17 21:02:43 +02:00
HaoranYi 85a2e599bb
revert memory map for reading snapshot (#25259) 2022-05-17 09:32:18 -05:00
Jeff Washington (jwash) 2cf7c73ced
cleanup types in tests (#25254) 2022-05-16 15:54:40 -05:00
Jack May 21e066ef26
Merge pull request from GHSA-wxrh-ff9f-fc6w 2022-05-16 12:30:37 -07:00
HaoranYi 3e44046a73
Support lz4 for snapshot archives (#25089)
* add lz4

* add cargo package file

* fix tests

* use fast lz4 compression

* report snapshot archive format

* add test

* code review feedback

* add cargo.lock

* fix var name

* refactor archive format parsing and add default compress cli arg

* clippy

* add from_cli_arg test

* update cargo.lock

* add lz4 support for mmap

* cargo.lock

* clippy
2022-05-16 12:44:15 -05:00
Justin Starry f81c5df1f0
Include account index in rent paying account tx error (#25189) 2022-05-16 23:35:34 +08:00
HaoranYi 389d78e424
Don't use mmarinus package (#25177)
* Revert "Revert "Use memory map to speed up snapshot untar (#24889)" (#25174)"

This reverts commit fc793de2dd.

* not use mmarinus

* enable secondary build

* Revert "enable secondary build"

This reverts commit 5aa43a94bbad59497506f02235597e7adc0dc47b.

* macbuild

* Revert "macbuild"

This reverts commit 0da92945a40dc5431eff4bd91054b407e22418fb.
2022-05-16 08:49:17 -05:00
Tao Zhu b1b3702e6d
Prioritize transactions in banking stage by their compute unit price (#25178)
* - get prioritization fee from compute_budget instruction;
- update compute_budget::process_instruction function to take instruction iter to support sanitized versioned message;
- updated runtime.md

* update transaction fee calculation for prioritization fee rate as lamports per 10K CUs

* review changes

* fix test

* fix a bpf test

* fix bpf test

* patch feedback

* fix clippy

* fix bpf test

* feedback

* rename prioritization fee rate to compute unit price

* feedback

Co-authored-by: Justin Starry <justin@solana.com>
2022-05-16 12:06:33 +08:00
Justin Starry 6c10515ccf
Revert "Serialize lamports per signature in snapshots (#25181)" (#25228)
This reverts commit 35d2a0fd69.
2022-05-15 12:04:17 +00:00
Ashwin Sekar 35d2a0fd69
Serialize lamports per signature in snapshots (#25181)
* Serialize lamports per signature

* pr comments
2022-05-14 10:57:51 -07:00
Ryo Onodera 592aa4c21b
Improve hard fork message with more detail (#25214) 2022-05-15 01:14:23 +09:00
sakridge 3d96a1ab76
Block packets in vote-only mode (#24906) 2022-05-14 17:53:37 +02:00
Jeff Washington (jwash) 7826fef120
clippy (#25206) 2022-05-13 21:03:54 -05:00
Jeff Washington (jwash) 9ccdc8a050
populate ancient append vec metrics (#25204)
* populate ancient append vec metrics

* count ancient storages while enumerating storages anyway.
2022-05-13 18:20:15 -05:00
Jeff Washington (jwash) d7dcb62fe1
ancient_append_vec refactoring (#25203) 2022-05-13 16:40:26 -05:00
Jeff Washington (jwash) cb037e49ea
is_candidate_for_shrink respects ancient append vecs (#25205) 2022-05-13 16:26:46 -05:00
apfitzge 5eeb07fead
coalesce purge stats and reclaim result (#25183) 2022-05-13 12:34:11 -05:00