738c4a79b7
metric for 'ancient_account_cleans' ( #26664 )
2022-07-19 09:16:23 -05:00
6efe7e095b
temporarily make ancient append vecs 10x smaller to find intermittent bugs ( #26574 )
2022-07-19 08:51:36 -05:00
c4d2c95540
move hash stat to correct level for ahv to work ( #26649 )
2022-07-18 12:44:33 -05:00
1c08f83c7e
eliminate flatten().collect() of reclaims in clean ( #26647 )
2022-07-18 12:37:17 -05:00
fd6038884b
Clean up forgotten error log
2022-07-17 12:21:52 +02:00
be8813e588
shrink stops using HandleReclaims ( #26594 )
2022-07-15 17:54:48 -05:00
83c614da3c
change type to be more clear ( #26646 )
2022-07-15 17:17:17 -05:00
47716a5e01
async hash verify on load ( #26208 )
...
* verify accounts hash in bg on startup
* fix some tests and loading from genesis
* add extra state for when background thread has completed
2022-07-15 14:29:56 -05:00
bf225bae73
restricts rent-paying accounts lifetime extension ( #26606 )
...
https://github.com/solana-labs/solana/pull/22292
prevents rent paying accounts creation going forward. However a rent
paying account can linger on for ever if it is continually topped up but
stays below the rent-exempt minimum.
This can prevent eliminating accounts-rewrites and the problematic
rent_epoch field in accounts.
Link to discord discussion:
https://discord.com/channels/428295358100013066/943609352068145162/995202300001927219
This commit restricts rent-paying accounts lifetime extension by
preventing increasing lamports on the account if the account stays below
the rent-exempt minimum.
2022-07-15 13:23:37 +00:00
038da82b6f
Feature: Early verification of account modifications in `BorrowedAccount` ( #25899 )
...
* Adjusts test cases for stricter requirements.
* Removes account reset in deserialization test.
* Removes verify related test cases.
* Replicates account modification verification logic of PreAccount in BorrowedAccount.
* Adds TransactionContext::account_touched_flags.
* Adds account modification verification to the BPF ABIv0 and ABIv1 deserialization, CPI syscall and program-test.
* Replicates the total sum of all lamports verification of PreAccounts in InstructionContext
* Check that the callers instruction balance is maintained during a call / push.
* Replicates PreAccount statistics in TransactionContext.
* Disable verify() and verify_and_update() if the feature enable_early_verification_of_account_modifications is enabled.
* Moves Option<Rent> of enable_early_verification_of_account_modifications into TransactionContext::new().
* Relaxes AccountDataMeter related test cases.
* Don't touch the account if nothing changes.
* Adds two tests to trigger InstructionError::UnbalancedInstruction.
Co-authored-by: Justin Starry <justin@solana.com>
2022-07-15 09:31:34 +02:00
f13b5c832d
Remove obsoleted metrics reporting to reduce lock contention on cost_model ( #26608 )
...
remove obsoleted metrics reporting to reduce lock contention on cost_model
2022-07-14 23:02:49 -05:00
819d6512ab
add PartialEq, Eq and spell correction ( #26614 )
2022-07-14 14:55:03 +00:00
3e9bd6170c
use OnceCell instead of RwLock for rent_paying_accounts_by_partition ( #26601 )
2022-07-14 08:52:41 -05:00
4dea32e8e5
fix active_scans metric to not swap ( #26602 )
2022-07-13 20:11:23 -05:00
2b7cc107d8
typo ( #26618 )
2022-07-13 17:55:02 -05:00
1aa9215411
Refactor: add account_iter api for appendvec ( #26591 )
...
* add account iterator for appendvec
* semicolon
2022-07-12 23:56:05 +00:00
6b0eb5a42b
handle rent paying accounts differently ( #26487 )
...
* handle rent paying accounts differently
* restore collection
* downgrade assert to metric and warn until we get more runtime
2022-07-12 17:51:33 -05:00
1dfbb225e3
Fix test_accounts_data_size_and_rent_collection ( #26576 )
2022-07-12 17:05:21 -05:00
c62c55a16b
shrink stores by id ( #26490 )
2022-07-12 15:43:41 -05:00
9d31216d23
add StoreReclaims::Ignore ( #26489 )
2022-07-12 15:05:47 -05:00
bf14440895
clean up and optimize account hash verify ( #26560 )
...
* remove unused code
* extract test related fault hash inject fn
* use rotate to optimize hashes removal
* use rotate to optimize snapshot hashes removal
* address code reveiw feedbacks
* revise comments
* inline
2022-07-12 19:27:28 +00:00
f54f528148
make bank test creation standard ( #26519 )
2022-07-12 09:07:55 -05:00
fd5df1cf25
Bump Version to 1.11.4 ( #26578 )
...
Co-authored-by: willhickey <willhickey@users.noreply.github.com>
2022-07-11 23:30:38 -05:00
1db136a8be
do not count 0 lamport accounts as rent-paying ( #26514 )
2022-07-11 22:16:46 -05:00
36bd46bece
ancient append vec: reuse AccountsToStore ( #26572 )
2022-07-11 22:13:09 -05:00
ee0a40937e
Add validator argument log_messages_bytes_limit to change log truncation limit.
...
Add new cli argument log_messages_bytes_limit to solana-validator to control how long program logs can be before truncation
2022-07-11 10:53:18 -05:00
ba785cf8ab
removes erroneous uses of std::mem::swap ( #26536 )
...
All instances should be replace by std::mem::{replace,take},
or just plain assignment.
2022-07-11 11:33:15 +00:00
602da5e51f
add accounts db config to bank tests ( #26517 )
2022-07-10 19:42:06 -05:00
2b6c9852f1
add more slop to test_age_time ( #26537 )
2022-07-10 17:52:07 -05:00
1babf460ab
collapse some bank test creation fns ( #26532 )
2022-07-10 17:51:38 -05:00
105c7e19e1
moves check for feature activation out of accounts loop ( #26535 )
2022-07-10 17:39:19 +00:00
8576832f2a
rename Bank::new_with_config_for_tests ( #26533 )
2022-07-10 09:33:13 -05:00
785a7a5936
Track more accounts data size changes ( #26467 )
2022-07-08 19:17:15 -05:00
c99d9f00a9
preserves rent_epoch for rent exempt accounts ( #26479 )
...
https://github.com/solana-labs/solana/pull/22292
prevents rent paying account creation going forward. As a result
rent_epoch field for rent exempt accounts is redundant, and advancing
this field will incur expensive account rewrites and cause discrepancy
between accounts-db and cached vote/stake accounts.
This commit adds a feature which upon activation preserves rent_epoch
field for rent exempt accounts so that the field is frozen and is no
longer advanced.
2022-07-08 20:04:08 +00:00
995756a550
Use RefCount type alias instead of u64 ( #26472 )
...
use RefCount type alias instead of u64
2022-07-08 13:51:16 -05:00
78b8a8a6db
Remove `get_total_resize_remaining()` and `total_resize_limit` from `TransactionContext` ( #26504 )
2022-07-08 20:26:54 +02:00
d1370f2c7d
chore: bump bytemuck from 1.9.1 to 1.10.0 ( #26495 )
...
* chore: bump bytemuck from 1.9.1 to 1.10.0
Bumps [bytemuck](https://github.com/Lokathor/bytemuck ) from 1.9.1 to 1.10.0.
- [Release notes](https://github.com/Lokathor/bytemuck/releases )
- [Changelog](https://github.com/Lokathor/bytemuck/blob/main/changelog.md )
- [Commits](https://github.com/Lokathor/bytemuck/compare/v1.9.1...v1.10.0 )
---
updated-dependencies:
- dependency-name: bytemuck
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2022-07-08 10:04:43 -06:00
d9eee72edb
remove FoundStoredAccount.account_size ( #26460 )
2022-07-07 18:37:14 -05:00
3b759bcffb
Add fn to get max accounts data size from Bank ( #26478 )
2022-07-07 18:08:42 -05:00
38d53ef8a2
add metrics for # scans active and scan distance ( #26395 )
2022-07-07 16:20:32 -05:00
ee0a54ce80
improve clean acct idx calls ( #26320 )
2022-07-07 15:50:12 -05:00
b582e4ce0f
introduce UpsertReclaim ( #26462 )
2022-07-07 15:40:17 -05:00
9d937fb8a0
Bump Version to 1.11.3 ( #26481 )
...
Co-authored-by: willhickey <willhickey@users.noreply.github.com>
2022-07-07 14:39:46 -05:00
c9f1d01e79
refactor, add GetUniqueAccountsResult ( #26457 )
2022-07-07 13:49:33 -05:00
8e64b5883e
restore population of 'accounts' metric in rent collection ( #26454 )
2022-07-07 13:19:23 -05:00
72256ac54d
use iter for append vecs ( #26459 )
2022-07-07 13:13:47 -05:00
8416090732
Clean up `enable_durable_nonce` feature ( #26470 )
...
Clean up enable_durable_nonce feature
2022-07-07 17:14:46 +01:00
eb6a722eba
Clean up `nonce_must_be_writable` feature ( #26444 )
...
Clean up nonce_must_be_writable feature
2022-07-07 15:45:19 +01:00
b3a47de1ce
improve startup stats for rent payers ( #26432 )
2022-07-07 08:39:57 -05:00
f8bdedb596
Make accounts data size tests more robust ( #26466 )
2022-07-07 00:01:13 -05:00
134303714b
Fix the order assumption of the pubkeys created by Pubkey::new_unique() ( #26451 )
...
new_unique() does not gurantee the increment order due to the bytes
array storage and its eq-partial trait interpreting the bytes in the
big-endian way.
2022-07-06 15:35:32 -07:00
90ef2cd02a
Parse snapshot for bank fields ( #26016 )
2022-07-06 17:30:30 -05:00
16219e300e
generating index sets uncleaned_roots correctly ( #26431 )
...
* generating index sets uncleaned_roots correctly
* fix test failures
* rename
* update comments
2022-07-06 16:12:47 -05:00
d7201a8d1a
names fields in RentResullt::CollectRent enum variant ( #26449 )
...
Avoiding ambiguous raw tuple:
CollectRent((Epoch, u64))
Using named fields instead:
CollectRent {
new_rent_epoch: Epoch,
rent_due: u64,
},
2022-07-06 20:01:16 +00:00
f4718be22f
Clean up `leave_nonce_on_success` feature ( #26447 )
...
Clean up leave_nonce_on_success feature
2022-07-06 17:59:34 +00:00
ff1e6fcbed
Clean up `nonce_must_be_advanceable` feature ( #26446 )
...
Clean up nonce_must_be_advanceable feature
2022-07-06 17:36:33 +00:00
06ebfa1eb2
Replicates `AccountsDataMeter` in `TransactionContext` ( #26438 )
...
Replicates AccountsDataMeter in TransactionContext.
2022-07-06 19:27:42 +02:00
611ac33718
Clean up `nonce_must_be_authorized` feature ( #26445 )
...
Clean up nonce_must_be_authorized feature
2022-07-06 17:17:31 +00:00
44f499cff3
Refactor account index scan function parameters ( #26428 )
...
* Refactor account index scan function parameters
F: FnMut(bool, &SlotList<T>, &'a Pubkey, RefCount) -> AccountsIndexScanResult,
Refactor to take (&'a Pubkey, Option<(&SlotList<T>, RefCount))
* Update comments based on review feedback
* cargo fmt --all fixes
2022-07-06 09:49:11 -07:00
17a99d98dd
Revert "avoid adding to 'uncleaned_roots' when generating index and c… ( #26441 )
...
Revert "avoid adding to 'uncleaned_roots' when generating index and caller passes accounts-db-skip-shrink (#25936 )"
This reverts commit e24cc537a4
2022-07-06 11:32:45 -05:00
ab164fc975
simplify code ( #26443 )
2022-07-06 11:22:26 -05:00
f8dccd4602
Clean up `max_tx_account_locks` feature ( #26440 )
...
Clean up max_tx_account_locks feature
2022-07-06 15:06:03 +00:00
5afe4d938d
combine lookup in calc_delete_dependencies ( #26429 )
2022-07-06 08:50:14 -05:00
71c7b4f9cb
use append vec iter in construct_candidate_clean_keys ( #26426 )
2022-07-06 08:49:35 -05:00
6f5857a5db
removes feature gate code separating durable nonce from blockhash domain ( #26055 )
2022-07-06 12:03:13 +00:00
37f4621c06
chore: bump serde from 1.0.137 to 1.0.138 ( #26421 )
...
* chore: bump serde from 1.0.137 to 1.0.138
Bumps [serde](https://github.com/serde-rs/serde ) from 1.0.137 to 1.0.138.
- [Release notes](https://github.com/serde-rs/serde/releases )
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.137...v1.0.138 )
---
updated-dependencies:
- dependency-name: serde
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2022-07-05 23:18:08 -06:00
c1d89ad749
forward packets by prioritization in desc order ( #25406 )
...
- Forward packets by prioritization in desc order
- Add support of cost-tracking by transaction requested compute units
- Hook up account buckets to forwarder
- Add metrics for forwardable batches count
- Remove redundant invalid packets filtering at end of slot since forwarder will do the same when batch forwardable packets
- Add bench test for forwarding
2022-07-05 23:24:58 -05:00
38216aa781
update comment for the next weary traveller ( #26413 )
2022-07-05 23:12:35 -05:00
75149fd624
move mark_old_slots_as_dirty to be called from ahv ( #26411 )
2022-07-05 23:11:32 -05:00
2aafef38f9
is_shrinking_productive calls avoid arc clone ( #26422 )
2022-07-05 17:53:29 -05:00
b7e34aea15
refactor calculate_rent_result ( #26416 )
...
reactor calculate_rent_result
2022-07-05 16:45:59 -05:00
53b9420562
Remove `INTO!` macros in serde_snapshot ( #26409 )
2022-07-05 16:37:20 -05:00
ce39c14025
Add end-to-end replay slot metrics ( #25752 )
2022-07-05 13:58:51 -05:00
0ab521d5f1
inc counter when append vec drop fails ( #26396 )
2022-07-05 11:27:01 -05:00
5f3b7bdd16
prevent ledger tool from calculating hash on non-rooted slots ( #26355 )
2022-07-02 11:50:01 -05:00
88b5b7e30a
improve log ( #26360 )
2022-07-01 13:12:52 -05:00
728986ddf4
Spell correction and add remove slot in AccountDB purge assert message ( #26358 )
...
nit spell. add remove slots in accoutdb purge assert message
2022-07-01 17:39:35 +00:00
fdb186ba3b
pacify nightly clippy
2022-06-30 21:31:49 -07:00
72a968fbe8
Fix/system instruction processor tests ( #26338 )
...
* Make mock_process_instruction() stricter by invoking the verify() method.
* Fixes broken test cases.
2022-07-01 00:06:24 +02:00
4563bf40f6
cleanup feature: tx-wide-compute-cap ( #26326 )
2022-06-29 23:54:45 -07:00
36d13af015
Add stats for Accounts Background Service ( #26298 )
...
* Add stats for Accounts Background Service
* pr
2022-06-30 02:21:30 +00:00
d053ce79d4
removes RwLock+Once in favor of OnceCell in caching staked-nodes ( #26313 )
...
VoteAccounts uses std::sync::{RwLock, Once} to lazily compute and cache
staked_nodes:
https://github.com/solana-labs/solana/blob/032bee13a/runtime/src/vote_account.rs#L89-L104
This commit instead switches to using once_cell::sync::OnceCell which
provides this exact intended functionality by design.
2022-06-29 22:22:22 +00:00
557bf6e656
allow initial hash calc to occur in bg ( #26271 )
...
* allow initial hash calc to occur in bg
* validator_initialized -> startup_verification_complete
* add infos for leader and vote
* rework snapshot for startup verification
* change to assert
2022-06-29 16:48:33 -05:00
608191352f
track # roots added/removed since last clean ( #26319 )
2022-06-29 16:47:57 -05:00
2deb396490
remove noisy and useless 'error!' and add test ( #26138 )
2022-06-29 16:26:23 -05:00
c01a8f271e
Prepare built-in program tests to deal with stricter account modification checks ( #26314 )
...
* Fixes test_vote_state_withdraw().
* Set is_writable to true where needed.
2022-06-29 19:29:12 +02:00
e241906db2
remove outdated, confusing comment ( #26315 )
2022-06-29 10:42:34 -05:00
ca83f143aa
load_accounts_index_for_shrink uses scan ( #26260 )
2022-06-29 09:02:27 -05:00
af7f08eba4
uses OnceCell instead of RwLock+Once to cache vote-state in vote-account ( #26257 )
...
RwLock seems excessive since only the very 1st call to VoteAccount::vote_state
will write-lock the inner field. Future calls would also incur overhead of an
RwLockReadGuard.
once_cell::sync::OnceCell provides a matching api to the desired functionality.
2022-06-29 11:45:53 +00:00
50631e274d
Ensure that staked nodes is populated before returning length ( #26293 )
2022-06-28 19:15:42 +00:00
89e5640c63
AccountsIndex::Scan takes iter ( #26268 )
2022-06-28 12:47:17 -05:00
d07b079850
make test_age_time less intermittent ( #26280 )
2022-06-28 09:13:01 -05:00
bfc390f9ed
refactor acct idx scan func for reuse ( #26252 )
2022-06-28 09:01:04 -05:00
4b591d7b59
AccountsDb::load_accounts_index_for_shrink takes slice ( #26267 )
2022-06-28 08:59:57 -05:00
e8fed88669
Both kinds of LamportsError are turned into InstructionError::ArithmeticOverflow anyway. ( #26273 )
2022-06-28 10:04:54 +02:00
e164879f20
load_accounts_index_for_shrink uses &item instead of (&item.0, &item.1) ( #26266 )
2022-06-27 22:40:45 -05:00
d6a28cb7a8
load_accounts_index_for_shrink unrefed_pubkeys is optional ( #26265 )
2022-06-27 17:23:46 -05:00
50eb9965ab
fix test_new_from_file_crafted_executable for m1 ( #26009 )
...
* fix test_new_from_file_crafted_executable for m1
* re-add old assert/comment, but disable for aarch64
2022-06-27 12:03:52 -05:00
d8516b5c88
comment and expand on test ( #26231 )
2022-06-27 11:35:46 -05:00
662818ef0d
Use `VoteAccount::node_pubkey()` ( #26207 )
2022-06-27 09:09:06 -05:00
Jeff Washington (jwash)
Justin Starry
behzad nouri
Alexander Meißner
Tao Zhu
HaoranYi
Brooks Prumo
github-actions[bot]
Nicholas Clarke
dependabot[bot]
Xiang Zhu
carllin
Michael Vines
Jack May
apfitzge