38cd29810f
Fix ordering for AccountEntry ref_count ( #26842 )
...
fix ordering for ref_count
2022-07-29 15:47:03 -05:00
69b5dd9b07
cleanup ignore_eof_error ( #26838 )
2022-07-29 12:10:22 -05:00
c1f7d1a367
Add AppendVec::new_from_file_unchecked ( #26795 )
...
new_from_file_unchecked
2022-07-28 14:57:26 -05:00
467cb5def5
Concurrent slot replay ( #26465 )
...
* Concurrent replay slots
* Split out concurrent and single bank replay paths
* Sub function processing of replay results for readability
* Add feature switch for concurrent replay
2022-07-28 11:33:19 -07:00
817f65bb50
add full_snapshot to hash config ( #26811 )
2022-07-28 09:46:34 -05:00
8d69e8d447
Compact vote state updates to reduce block size ( #26616 )
...
* Compact vote state updates to reduce block size
* Add rpc transaction tests
2022-07-27 13:23:44 -06:00
1a5b830294
CalcAccountsHashConfig default for tests ( #26810 )
2022-07-27 13:21:38 -05:00
1421833c5e
log when shrink still contains extra stores ( #26794 )
2022-07-27 09:14:16 -05:00
30b408ed0c
allow caching on ancient append vecs for accounts hash calc ( #26770 )
2022-07-27 09:13:32 -05:00
cd4340ed06
chore: bump bytemuck from 1.10.0 to 1.11.0 ( #26792 )
...
* chore: bump bytemuck from 1.10.0 to 1.11.0
Bumps [bytemuck](https://github.com/Lokathor/bytemuck ) from 1.10.0 to 1.11.0.
- [Release notes](https://github.com/Lokathor/bytemuck/releases )
- [Changelog](https://github.com/Lokathor/bytemuck/blob/main/changelog.md )
- [Commits](https://github.com/Lokathor/bytemuck/compare/v1.10.0...v1.11.0 )
---
updated-dependencies:
- dependency-name: bytemuck
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2022-07-26 20:50:30 -06:00
293d651ca8
store-tool use accounts_iter ( #26793 )
2022-07-26 17:29:53 -05:00
5e622f82ac
Add feature gate to use default units per instruction for fee calculation ( #26786 )
...
* Add feature gate to use default units per instruction for fee calculation
* fix bpf tests
2022-07-26 19:39:24 +00:00
5de1a7accf
Add streaming_unpack_snapshot to send entry files during unpack ( #26781 )
...
* Add streaming_unpack_snapshot to send entry files during unpack
* Update unpack tests to call internal function
2022-07-26 14:09:28 -05:00
a7e4df5d6e
Separate remapping append_vecs into functions ( #26696 )
...
Separate remapping append_vecs into remap_and_reconstruct_storages and helpers
2022-07-25 14:55:30 -05:00
6f5995c841
add stats for disk_index_find_entry_mut_us ( #26753 )
2022-07-25 14:47:17 -05:00
5d038b9d2a
Bump Version to 1.11.5 ( #26758 )
...
Co-authored-by: willhickey <willhickey@users.noreply.github.com>
2022-07-25 13:05:14 -06:00
01e367f19b
Add Bank::update_accounts_data_size_delta_off_chain_for_tests() ( #26771 )
2022-07-25 18:58:04 +00:00
d9c7bc7e78
Revert "cleanup feature: default units per instruction ( #26684 )" ( #26750 )
...
This reverts commit 39a34db52a
2022-07-23 11:03:46 -05:00
f2b81fe665
Separate field_from_stream to get bank/account fields from individual snapshot stream ( #26679 )
2022-07-21 12:27:31 -05:00
075a5ac44e
Removes `AccountSharedData` from `SysvarCache`. ( #26712 )
...
* Removes AccountSharedData from SysvarCache.
* Fixes incorrect transaction account index in stake_instruction::test_set_lockup().
2022-07-21 18:32:28 +02:00
cbb74a190f
Cleanup: `record_instruction_in_transaction_context_push` ( #26658 )
...
Cleanup feature gate of record_instruction_in_transaction_context_push.
2022-07-21 12:49:34 +02:00
6ec89ee861
log error for cache_hash_data save failure ( #26676 )
...
log error for cach_hash_data save
2022-07-20 16:19:35 -05:00
39a34db52a
cleanup feature: default units per instruction ( #26684 )
2022-07-20 19:13:34 +00:00
b20edd232b
Remove useless enumerate from test_hold_range_in_memory2 ( #26683 )
2022-07-20 10:06:08 -05:00
9f2f2a45cd
AcctIdx: ref_count_from_storage remove Arc::clone ( #26678 )
2022-07-20 10:05:13 -05:00
f8e9af5f1e
Cap the number of accounts passed to a bpf program ix ( #26630 )
...
* Cap the number of accounts passed to a bpf program ix
* update bank abi hash
* fix ci failures
2022-07-20 14:12:43 +02:00
07f41628db
untar_snapshot_create_shared_buffer doesn't need to return a result ( #26686 )
2022-07-19 23:15:41 +00:00
2d689ac53a
get rid of rwlock per bin in accounts index ( #26675 )
2022-07-19 17:48:40 -05:00
e7cd6daebe
Add helper fn to create shared buffer - simplify archive format matching ( #26674 )
2022-07-19 20:30:30 +00:00
84c8cfe901
Refactor/hardened unpack more generic unpack archive ( #26677 )
...
* Add entry_processor fn to unpack_archive - not used yet
* Add generic unpack_snapshot_with_processors to separate account paths handling from entry checking
2022-07-19 14:48:57 -05:00
7efe72a74f
eliminate write lock usage in acct idx ( #26673 )
2022-07-19 13:24:03 -05:00
77843306c7
rename parallel_archivers -> parallel_divisions for consistency ( #26672 )
...
parallel_archivers -> parallel_divisions for consistency
2022-07-19 13:15:33 -05:00
263911e7fd
save off what we find when calculating hash ( #26663 )
2022-07-19 09:55:52 -05:00
738c4a79b7
metric for 'ancient_account_cleans' ( #26664 )
2022-07-19 09:16:23 -05:00
6efe7e095b
temporarily make ancient append vecs 10x smaller to find intermittent bugs ( #26574 )
2022-07-19 08:51:36 -05:00
c4d2c95540
move hash stat to correct level for ahv to work ( #26649 )
2022-07-18 12:44:33 -05:00
1c08f83c7e
eliminate flatten().collect() of reclaims in clean ( #26647 )
2022-07-18 12:37:17 -05:00
fd6038884b
Clean up forgotten error log
2022-07-17 12:21:52 +02:00
be8813e588
shrink stops using HandleReclaims ( #26594 )
2022-07-15 17:54:48 -05:00
83c614da3c
change type to be more clear ( #26646 )
2022-07-15 17:17:17 -05:00
47716a5e01
async hash verify on load ( #26208 )
...
* verify accounts hash in bg on startup
* fix some tests and loading from genesis
* add extra state for when background thread has completed
2022-07-15 14:29:56 -05:00
bf225bae73
restricts rent-paying accounts lifetime extension ( #26606 )
...
https://github.com/solana-labs/solana/pull/22292
prevents rent paying accounts creation going forward. However a rent
paying account can linger on for ever if it is continually topped up but
stays below the rent-exempt minimum.
This can prevent eliminating accounts-rewrites and the problematic
rent_epoch field in accounts.
Link to discord discussion:
https://discord.com/channels/428295358100013066/943609352068145162/995202300001927219
This commit restricts rent-paying accounts lifetime extension by
preventing increasing lamports on the account if the account stays below
the rent-exempt minimum.
2022-07-15 13:23:37 +00:00
038da82b6f
Feature: Early verification of account modifications in `BorrowedAccount` ( #25899 )
...
* Adjusts test cases for stricter requirements.
* Removes account reset in deserialization test.
* Removes verify related test cases.
* Replicates account modification verification logic of PreAccount in BorrowedAccount.
* Adds TransactionContext::account_touched_flags.
* Adds account modification verification to the BPF ABIv0 and ABIv1 deserialization, CPI syscall and program-test.
* Replicates the total sum of all lamports verification of PreAccounts in InstructionContext
* Check that the callers instruction balance is maintained during a call / push.
* Replicates PreAccount statistics in TransactionContext.
* Disable verify() and verify_and_update() if the feature enable_early_verification_of_account_modifications is enabled.
* Moves Option<Rent> of enable_early_verification_of_account_modifications into TransactionContext::new().
* Relaxes AccountDataMeter related test cases.
* Don't touch the account if nothing changes.
* Adds two tests to trigger InstructionError::UnbalancedInstruction.
Co-authored-by: Justin Starry <justin@solana.com>
2022-07-15 09:31:34 +02:00
f13b5c832d
Remove obsoleted metrics reporting to reduce lock contention on cost_model ( #26608 )
...
remove obsoleted metrics reporting to reduce lock contention on cost_model
2022-07-14 23:02:49 -05:00
819d6512ab
add PartialEq, Eq and spell correction ( #26614 )
2022-07-14 14:55:03 +00:00
3e9bd6170c
use OnceCell instead of RwLock for rent_paying_accounts_by_partition ( #26601 )
2022-07-14 08:52:41 -05:00
4dea32e8e5
fix active_scans metric to not swap ( #26602 )
2022-07-13 20:11:23 -05:00
2b7cc107d8
typo ( #26618 )
2022-07-13 17:55:02 -05:00
1aa9215411
Refactor: add account_iter api for appendvec ( #26591 )
...
* add account iterator for appendvec
* semicolon
2022-07-12 23:56:05 +00:00
6b0eb5a42b
handle rent paying accounts differently ( #26487 )
...
* handle rent paying accounts differently
* restore collection
* downgrade assert to metric and warn until we get more runtime
2022-07-12 17:51:33 -05:00
1dfbb225e3
Fix test_accounts_data_size_and_rent_collection ( #26576 )
2022-07-12 17:05:21 -05:00
c62c55a16b
shrink stores by id ( #26490 )
2022-07-12 15:43:41 -05:00
9d31216d23
add StoreReclaims::Ignore ( #26489 )
2022-07-12 15:05:47 -05:00
bf14440895
clean up and optimize account hash verify ( #26560 )
...
* remove unused code
* extract test related fault hash inject fn
* use rotate to optimize hashes removal
* use rotate to optimize snapshot hashes removal
* address code reveiw feedbacks
* revise comments
* inline
2022-07-12 19:27:28 +00:00
f54f528148
make bank test creation standard ( #26519 )
2022-07-12 09:07:55 -05:00
fd5df1cf25
Bump Version to 1.11.4 ( #26578 )
...
Co-authored-by: willhickey <willhickey@users.noreply.github.com>
2022-07-11 23:30:38 -05:00
1db136a8be
do not count 0 lamport accounts as rent-paying ( #26514 )
2022-07-11 22:16:46 -05:00
36bd46bece
ancient append vec: reuse AccountsToStore ( #26572 )
2022-07-11 22:13:09 -05:00
ee0a40937e
Add validator argument log_messages_bytes_limit to change log truncation limit.
...
Add new cli argument log_messages_bytes_limit to solana-validator to control how long program logs can be before truncation
2022-07-11 10:53:18 -05:00
ba785cf8ab
removes erroneous uses of std::mem::swap ( #26536 )
...
All instances should be replace by std::mem::{replace,take},
or just plain assignment.
2022-07-11 11:33:15 +00:00
602da5e51f
add accounts db config to bank tests ( #26517 )
2022-07-10 19:42:06 -05:00
2b6c9852f1
add more slop to test_age_time ( #26537 )
2022-07-10 17:52:07 -05:00
1babf460ab
collapse some bank test creation fns ( #26532 )
2022-07-10 17:51:38 -05:00
105c7e19e1
moves check for feature activation out of accounts loop ( #26535 )
2022-07-10 17:39:19 +00:00
8576832f2a
rename Bank::new_with_config_for_tests ( #26533 )
2022-07-10 09:33:13 -05:00
785a7a5936
Track more accounts data size changes ( #26467 )
2022-07-08 19:17:15 -05:00
c99d9f00a9
preserves rent_epoch for rent exempt accounts ( #26479 )
...
https://github.com/solana-labs/solana/pull/22292
prevents rent paying account creation going forward. As a result
rent_epoch field for rent exempt accounts is redundant, and advancing
this field will incur expensive account rewrites and cause discrepancy
between accounts-db and cached vote/stake accounts.
This commit adds a feature which upon activation preserves rent_epoch
field for rent exempt accounts so that the field is frozen and is no
longer advanced.
2022-07-08 20:04:08 +00:00
995756a550
Use RefCount type alias instead of u64 ( #26472 )
...
use RefCount type alias instead of u64
2022-07-08 13:51:16 -05:00
78b8a8a6db
Remove `get_total_resize_remaining()` and `total_resize_limit` from `TransactionContext` ( #26504 )
2022-07-08 20:26:54 +02:00
d1370f2c7d
chore: bump bytemuck from 1.9.1 to 1.10.0 ( #26495 )
...
* chore: bump bytemuck from 1.9.1 to 1.10.0
Bumps [bytemuck](https://github.com/Lokathor/bytemuck ) from 1.9.1 to 1.10.0.
- [Release notes](https://github.com/Lokathor/bytemuck/releases )
- [Changelog](https://github.com/Lokathor/bytemuck/blob/main/changelog.md )
- [Commits](https://github.com/Lokathor/bytemuck/compare/v1.9.1...v1.10.0 )
---
updated-dependencies:
- dependency-name: bytemuck
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2022-07-08 10:04:43 -06:00
d9eee72edb
remove FoundStoredAccount.account_size ( #26460 )
2022-07-07 18:37:14 -05:00
3b759bcffb
Add fn to get max accounts data size from Bank ( #26478 )
2022-07-07 18:08:42 -05:00
38d53ef8a2
add metrics for # scans active and scan distance ( #26395 )
2022-07-07 16:20:32 -05:00
ee0a54ce80
improve clean acct idx calls ( #26320 )
2022-07-07 15:50:12 -05:00
b582e4ce0f
introduce UpsertReclaim ( #26462 )
2022-07-07 15:40:17 -05:00
9d937fb8a0
Bump Version to 1.11.3 ( #26481 )
...
Co-authored-by: willhickey <willhickey@users.noreply.github.com>
2022-07-07 14:39:46 -05:00
c9f1d01e79
refactor, add GetUniqueAccountsResult ( #26457 )
2022-07-07 13:49:33 -05:00
8e64b5883e
restore population of 'accounts' metric in rent collection ( #26454 )
2022-07-07 13:19:23 -05:00
72256ac54d
use iter for append vecs ( #26459 )
2022-07-07 13:13:47 -05:00
8416090732
Clean up `enable_durable_nonce` feature ( #26470 )
...
Clean up enable_durable_nonce feature
2022-07-07 17:14:46 +01:00
eb6a722eba
Clean up `nonce_must_be_writable` feature ( #26444 )
...
Clean up nonce_must_be_writable feature
2022-07-07 15:45:19 +01:00
b3a47de1ce
improve startup stats for rent payers ( #26432 )
2022-07-07 08:39:57 -05:00
f8bdedb596
Make accounts data size tests more robust ( #26466 )
2022-07-07 00:01:13 -05:00
134303714b
Fix the order assumption of the pubkeys created by Pubkey::new_unique() ( #26451 )
...
new_unique() does not gurantee the increment order due to the bytes
array storage and its eq-partial trait interpreting the bytes in the
big-endian way.
2022-07-06 15:35:32 -07:00
90ef2cd02a
Parse snapshot for bank fields ( #26016 )
2022-07-06 17:30:30 -05:00
16219e300e
generating index sets uncleaned_roots correctly ( #26431 )
...
* generating index sets uncleaned_roots correctly
* fix test failures
* rename
* update comments
2022-07-06 16:12:47 -05:00
d7201a8d1a
names fields in RentResullt::CollectRent enum variant ( #26449 )
...
Avoiding ambiguous raw tuple:
CollectRent((Epoch, u64))
Using named fields instead:
CollectRent {
new_rent_epoch: Epoch,
rent_due: u64,
},
2022-07-06 20:01:16 +00:00
f4718be22f
Clean up `leave_nonce_on_success` feature ( #26447 )
...
Clean up leave_nonce_on_success feature
2022-07-06 17:59:34 +00:00
ff1e6fcbed
Clean up `nonce_must_be_advanceable` feature ( #26446 )
...
Clean up nonce_must_be_advanceable feature
2022-07-06 17:36:33 +00:00
06ebfa1eb2
Replicates `AccountsDataMeter` in `TransactionContext` ( #26438 )
...
Replicates AccountsDataMeter in TransactionContext.
2022-07-06 19:27:42 +02:00
611ac33718
Clean up `nonce_must_be_authorized` feature ( #26445 )
...
Clean up nonce_must_be_authorized feature
2022-07-06 17:17:31 +00:00
44f499cff3
Refactor account index scan function parameters ( #26428 )
...
* Refactor account index scan function parameters
F: FnMut(bool, &SlotList<T>, &'a Pubkey, RefCount) -> AccountsIndexScanResult,
Refactor to take (&'a Pubkey, Option<(&SlotList<T>, RefCount))
* Update comments based on review feedback
* cargo fmt --all fixes
2022-07-06 09:49:11 -07:00
17a99d98dd
Revert "avoid adding to 'uncleaned_roots' when generating index and c… ( #26441 )
...
Revert "avoid adding to 'uncleaned_roots' when generating index and caller passes accounts-db-skip-shrink (#25936 )"
This reverts commit e24cc537a4
2022-07-06 11:32:45 -05:00
ab164fc975
simplify code ( #26443 )
2022-07-06 11:22:26 -05:00
f8dccd4602
Clean up `max_tx_account_locks` feature ( #26440 )
...
Clean up max_tx_account_locks feature
2022-07-06 15:06:03 +00:00
5afe4d938d
combine lookup in calc_delete_dependencies ( #26429 )
2022-07-06 08:50:14 -05:00
71c7b4f9cb
use append vec iter in construct_candidate_clean_keys ( #26426 )
2022-07-06 08:49:35 -05:00
6f5857a5db
removes feature gate code separating durable nonce from blockhash domain ( #26055 )
2022-07-06 12:03:13 +00:00
37f4621c06
chore: bump serde from 1.0.137 to 1.0.138 ( #26421 )
...
* chore: bump serde from 1.0.137 to 1.0.138
Bumps [serde](https://github.com/serde-rs/serde ) from 1.0.137 to 1.0.138.
- [Release notes](https://github.com/serde-rs/serde/releases )
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.137...v1.0.138 )
---
updated-dependencies:
- dependency-name: serde
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2022-07-05 23:18:08 -06:00
c1d89ad749
forward packets by prioritization in desc order ( #25406 )
...
- Forward packets by prioritization in desc order
- Add support of cost-tracking by transaction requested compute units
- Hook up account buckets to forwarder
- Add metrics for forwardable batches count
- Remove redundant invalid packets filtering at end of slot since forwarder will do the same when batch forwardable packets
- Add bench test for forwarding
2022-07-05 23:24:58 -05:00
HaoranYi
Jeff Washington (jwash)
apfitzge
Brennan Watt
Ashwin Sekar
dependabot[bot]
Tao Zhu
github-actions[bot]
Brooks Prumo
Alexander Meißner
Jack May
Justin Starry
behzad nouri
Nicholas Clarke
Xiang Zhu
carllin