891dcb2084
move rent collection time into datapoint ( #25714 )
2022-06-01 17:28:47 -05:00
dd0bf4ff4b
add filler accounts gradually with cache flush ( #25678 )
2022-06-01 15:47:21 -05:00
b80a82a9e5
runtime: gate nonce disable at restart slot
2022-06-01 12:00:10 -06:00
635bd40f09
runtime: disable durable nonces
2022-06-01 11:15:16 -06:00
905fef29cb
add filter to AppendVecScan ( #25664 )
2022-06-01 11:55:18 -05:00
94b0ce5d43
remove_old_historical_roots uses accounts_hash_complete_one_epoch_old ( #25673 )
2022-06-01 10:45:26 -05:00
c91fbd289f
set active state to shrink_ancient when doing intersting work ( #25677 )
2022-06-01 10:36:58 -05:00
8bb76fcd87
add ancient metrics ( #25656 )
...
* add ancient metrics
* review feedback
2022-06-01 10:36:23 -05:00
07958fd8ed
use self.init_accum ( #25663 )
2022-06-01 08:38:37 -05:00
27978c4d0c
chore: bump flate2 from 1.0.23 to 1.0.24 ( #25636 )
...
* chore: bump flate2 from 1.0.23 to 1.0.24
Bumps [flate2](https://github.com/rust-lang/flate2-rs ) from 1.0.23 to 1.0.24.
- [Release notes](https://github.com/rust-lang/flate2-rs/releases )
- [Commits](https://github.com/rust-lang/flate2-rs/commits )
---
updated-dependencies:
- dependency-name: flate2
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2022-05-31 23:15:46 -06:00
8584a3a331
Add feature to increase timestamp bounding on the slow side to 150% of poh estimate (1s/slot) ( #25666 )
2022-05-31 16:53:06 -06:00
5994b4f53f
use object for hash scanning ( #25418 )
2022-05-31 11:51:56 -05:00
56d38e4954
Add lz4 to snapshot filename regex for matching files on load ( #25633 )
...
* Add lz4 to snapshot filename regex for matching files on load
* Add tests for lz4 archive format filenames
2022-05-31 11:06:41 -05:00
6b2f58152b
Ignore rocksdb directories when unpacking genesis archive ( #25575 )
...
Unpacking the rocksdb directory from genesis archive can have unintended
consequences, such as clobbering files in pre-existing rocksdb
directory.
2022-05-31 00:38:45 -05:00
4b1dd0f921
Fix a possible test hang in runtime::test_shared_buffer_sweep ( #25545 )
...
* Fix a possible test hang in runtime::test_shared_buffer_sweep
The test may hang if the total number of threads it wants to start
is greater than the number of threads configured in the rayon
threadpool. Check to avoid this situation from happening.
* Incorporate comment from Xiang Zhu, using std::cmp::min() to simplify the code
2022-05-30 23:39:32 +08:00
387d9d9c77
Store missing unupdated executors earlier
2022-05-28 13:42:35 +10:00
740ee0c502
Fix datapoint name ( #25614 )
2022-05-28 01:01:29 +00:00
4e6b276f6c
bump executor cache max entries ( #25600 )
...
* executor-cache: unify `remove` calls
* executor-cache: metrics - add one-hit-wonder counter
* executor-cache: bump max entries to 256
2022-05-28 01:54:10 +10:00
ce14c63bf0
fix: stop double counting execution timings ( #25605 )
2022-05-28 00:06:29 +10:00
e58de2c233
Check fee payer before loading all accounts ( #25371 )
...
* fix vote account loading
* fix clippy and rename some stuff
* fix bug
Co-authored-by: Justin Starry <justin@solana.com>
2022-05-27 13:44:29 +10:00
7384c41ea7
Speed up bank::tests::test_max_accounts_data_size_exceeded() ( #25523 )
2022-05-25 22:22:30 -04:00
26a02f6f59
Add invoke logging to builtin programs that don't have it yet ( #25230 )
...
* Refactor process_executor_chain
* Enable builtin program invoke logs by default
* feedback
2022-05-26 07:27:00 +08:00
f10c80b49f
Refactor: Rebase offset of `index_in_caller` ( #25531 )
...
* Removes the offset InstructionContext::get_number_of_program_accounts() from InstructionAccount::index_in_caller.
* Removes unreachable SyscallError::InvalidLength in orig_data_lens.get().
2022-05-25 13:43:20 +02:00
7b98ff1929
Refactor: pull out fee payer balance check ( #25519 )
2022-05-25 20:44:03 +10:00
b7149f7f54
Add helper fn to set accounts_data_size_initial for tests ( #25522 )
2022-05-25 02:43:54 +00:00
0dd3c6bf1f
Split up accounts data size field ( #25495 )
2022-05-24 20:26:32 -04:00
2fb096c486
Refactor: Adds `index_in_callee` to `InstructionAccount` ( #25490 )
...
* Adds InstructionAccount::index_in_callee
* Adjusts tests and benches.
* Adds documentation for InstructionAccount.
* Adds InstructionContext::is_duplicate()
2022-05-25 00:04:46 +02:00
e66ea7cb6a
Clean up Bank::commit_transactions parameters
2022-05-24 20:24:42 +08:00
3566fb5660
chore: bump regex from 1.5.5 to 1.5.6 ( #25488 )
...
* chore: bump regex from 1.5.5 to 1.5.6
Bumps [regex](https://github.com/rust-lang/regex ) from 1.5.5 to 1.5.6.
- [Release notes](https://github.com/rust-lang/regex/releases )
- [Changelog](https://github.com/rust-lang/regex/blob/master/CHANGELOG.md )
- [Commits](https://github.com/rust-lang/regex/compare/1.5.5...1.5.6 )
---
updated-dependencies:
- dependency-name: regex
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2022-05-23 13:45:43 -06:00
bac05dc55a
calc slot info on max slot once per hash calc ( #25422 )
2022-05-23 10:11:00 -05:00
41f30a2383
stop logging misleading bank hash mismatch ( #25427 )
2022-05-23 08:43:25 -05:00
b05c7d91ed
Fix derive_partial_eq_without_eq clippy lint
2022-05-22 22:22:21 -07:00
f8842032c6
clippy: fix "this let-binding has unit value" warnings ( #25429 )
2022-05-22 12:17:59 -04:00
7eae98c6d9
Introduces `Result` return types to mutable accessors of `BorrowedAccount` ( #25380 )
...
* Introduces result return types to get_data_mut(), set_data() and set_data_length() of BorrowedAccount.
* Introduces result return types to set_owner(), set_lamports() and set_executable() of BorrowedAccount.
2022-05-21 17:47:09 +02:00
e9a6d1ecef
clippy: is_digit(10) -> is_ascii_digit() ( #25428 )
2022-05-21 07:03:11 -05:00
f584b249dd
special case for common case of 1 storage on append vec scan ( #25414 )
2022-05-20 17:36:30 -05:00
851958f77a
Disallow deployment of deprecated _sol_alloc_free syscall ( #25179 )
2022-05-20 13:19:41 -07:00
cfcc18f7f4
apply AccountsInAppendVec to more places ( #25408 )
2022-05-20 14:25:46 -05:00
a25212b087
Move append_vec::test_utils into its own file ( #25407 )
2022-05-20 13:37:45 -05:00
9ef0acdf5d
allow hash calc to use all threads at startup ( #25369 )
2022-05-20 11:45:01 -05:00
467431de89
add ledger-tool accounts-db-skip-initial-hash-calc ( #25354 )
2022-05-20 10:27:00 -05:00
950245c29b
simulate filler account hash calc time at delta hash ( #25387 )
2022-05-20 10:20:44 -05:00
1b4e4ef548
impl squash to ancient append vecs ( #24538 )
...
* squash to ancient append vecs
* pr feedback
* clippy
2022-05-20 10:20:17 -05:00
de03a3d3aa
filler accounts get a non-zero rent_epoch ( #25386 )
...
* fill accounts get a non-zero rent_epoch
* use snapshot_slot
* get rid of perf simulation code
2022-05-20 09:03:55 -05:00
3fe7655378
use iterator instead of vec to iterate accounts ( #25370 )
...
* user iterator instead of vec to iterate accounts
* rename
2022-05-20 08:50:24 -05:00
36ed36edf0
use previous epoch's # slots per epoch ( #25382 )
2022-05-20 08:47:55 -05:00
c54e06355f
voteSubscribe pubsub notification now includes the vote transaction signature ( #25291 )
2022-05-19 18:28:46 -07:00
021276c955
Move updating accounts data len from execute to commit ( #25256 )
2022-05-19 12:24:13 -05:00
0f703b71cc
minor test renaming ( #25297 )
2022-05-18 12:53:55 -05:00
fe91f40f8a
use previous epoch rent collector in expected rent collection ( #25289 )
2022-05-18 12:53:45 -05:00
542bd0ec3c
chore: bump rayon from 1.5.2 to 1.5.3 ( #25242 )
...
* chore: bump rayon from 1.5.2 to 1.5.3
Bumps [rayon](https://github.com/rayon-rs/rayon ) from 1.5.2 to 1.5.3.
- [Release notes](https://github.com/rayon-rs/rayon/releases )
- [Changelog](https://github.com/rayon-rs/rayon/blob/master/RELEASES.md )
- [Commits](https://github.com/rayon-rs/rayon/compare/v1.5.2...v1.5.3 )
---
updated-dependencies:
- dependency-name: rayon
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2022-05-18 09:39:57 -06:00
a1522d0024
Use consistent naming for compute unit limit ( #25229 )
...
* Use consistent naming for compute unit limit
* feedback
2022-05-18 13:14:31 +08:00
0820065c98
rpc: Support token-2022 in token-specific calls ( #25150 )
...
* rpc: Support token-2022 in token-specific calls
* Address feedback
2022-05-17 21:02:43 +02:00
85a2e599bb
revert memory map for reading snapshot ( #25259 )
2022-05-17 09:32:18 -05:00
2cf7c73ced
cleanup types in tests ( #25254 )
2022-05-16 15:54:40 -05:00
21e066ef26
Merge pull request from GHSA-wxrh-ff9f-fc6w
2022-05-16 12:30:37 -07:00
3e44046a73
Support lz4 for snapshot archives ( #25089 )
...
* add lz4
* add cargo package file
* fix tests
* use fast lz4 compression
* report snapshot archive format
* add test
* code review feedback
* add cargo.lock
* fix var name
* refactor archive format parsing and add default compress cli arg
* clippy
* add from_cli_arg test
* update cargo.lock
* add lz4 support for mmap
* cargo.lock
* clippy
2022-05-16 12:44:15 -05:00
f81c5df1f0
Include account index in rent paying account tx error ( #25189 )
2022-05-16 23:35:34 +08:00
389d78e424
Don't use mmarinus package ( #25177 )
...
* Revert "Revert "Use memory map to speed up snapshot untar (#24889 )" (#25174 )"
This reverts commit fc793de2dd
2022-05-16 08:49:17 -05:00
b1b3702e6d
Prioritize transactions in banking stage by their compute unit price ( #25178 )
...
* - get prioritization fee from compute_budget instruction;
- update compute_budget::process_instruction function to take instruction iter to support sanitized versioned message;
- updated runtime.md
* update transaction fee calculation for prioritization fee rate as lamports per 10K CUs
* review changes
* fix test
* fix a bpf test
* fix bpf test
* patch feedback
* fix clippy
* fix bpf test
* feedback
* rename prioritization fee rate to compute unit price
* feedback
Co-authored-by: Justin Starry <justin@solana.com>
2022-05-16 12:06:33 +08:00
6c10515ccf
Revert "Serialize lamports per signature in snapshots ( #25181 )" ( #25228 )
...
This reverts commit 35d2a0fd69
2022-05-15 12:04:17 +00:00
35d2a0fd69
Serialize lamports per signature in snapshots ( #25181 )
...
* Serialize lamports per signature
* pr comments
2022-05-14 10:57:51 -07:00
592aa4c21b
Improve hard fork message with more detail ( #25214 )
2022-05-15 01:14:23 +09:00
3d96a1ab76
Block packets in vote-only mode ( #24906 )
2022-05-14 17:53:37 +02:00
7826fef120
clippy ( #25206 )
2022-05-13 21:03:54 -05:00
9ccdc8a050
populate ancient append vec metrics ( #25204 )
...
* populate ancient append vec metrics
* count ancient storages while enumerating storages anyway.
2022-05-13 18:20:15 -05:00
d7dcb62fe1
ancient_append_vec refactoring ( #25203 )
2022-05-13 16:40:26 -05:00
cb037e49ea
is_candidate_for_shrink respects ancient append vecs ( #25205 )
2022-05-13 16:26:46 -05:00
5eeb07fead
coalesce purge stats and reclaim result ( #25183 )
2022-05-13 12:34:11 -05:00
295df294de
chore: bump zstd from 0.11.1+zstd.1.5.2 to 0.11.2+zstd.1.5.2 ( #25156 )
...
* chore: bump zstd from 0.11.1+zstd.1.5.2 to 0.11.2+zstd.1.5.2
Bumps [zstd](https://github.com/gyscos/zstd-rs ) from 0.11.1+zstd.1.5.2 to 0.11.2+zstd.1.5.2.
- [Release notes](https://github.com/gyscos/zstd-rs/releases )
- [Commits](https://github.com/gyscos/zstd-rs/commits/v0.11.2 )
---
updated-dependencies:
- dependency-name: zstd
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2022-05-12 20:28:50 -06:00
6f1cc5a2a7
assert we haven't squashed to ancient after hash calc ( #25168 )
2022-05-12 18:26:01 -05:00
fc793de2dd
Revert "Use memory map to speed up snapshot untar ( #24889 )" ( #25174 )
...
This reverts commit 3367e44671
2022-05-12 14:48:29 -05:00
08da486c05
additional costs in block capacity calc ( #25059 )
...
* Added additional costs to block capacity computation, and pushed alloc of CostModel all the way to the top of the call chain, instead of reallocing
* Fix two compiler errors
* Update block processing to propagate computed costs, rather than re-computing deeper in the call stack
* Clippy fix
* Reformatting fix after merge
* Add CostModel::sum_without_bpf
2022-05-12 13:52:20 -05:00
3367e44671
Use memory map to speed up snapshot untar ( #24889 )
...
* mmap
* add no_os_memory_reporting_args to ledger-tool
* use safe memory map package for unix. use regular file for windows
* fallback to untaring with filebuf when mmap fails
* update comments
2022-05-12 13:35:19 -05:00
cde15ff687
support 64-bit prioritization fee ( #25027 )
2022-05-12 11:07:36 -07:00
896729f25e
keep track of oldest slot used by last hash calculation ( #25152 )
2022-05-12 11:18:08 -05:00
abb4993a3b
SortedStorages::max_slot_inclusive() ( #25153 )
2022-05-12 10:24:03 -05:00
849cbe93aa
add AccountsToStore::has_overflow ( #25143 )
2022-05-12 10:14:27 -05:00
c56d978cb1
minor cleanup ( #25144 )
2022-05-11 14:58:43 -05:00
68800982ea
Add size_of const fns for upgradeable loader states ( #25131 )
2022-05-11 22:22:59 +08:00
a8930ee14b
add --accounts-db-ancient-append-vecs ( #25125 )
2022-05-11 08:47:07 -05:00
542a14d9b7
test-validator: Always use overridden compute budget in bank ( #25128 )
2022-05-11 15:46:51 +02:00
11fa0db850
more ancient append vec fns ( #25127 )
2022-05-11 08:02:05 -05:00
3ce877d1ab
add metric for ancient append vec count ( #25126 )
2022-05-11 08:01:04 -05:00
c4506269a6
add ancient_append_vecs ( #25119 )
2022-05-10 16:41:04 -05:00
c838e15234
Unset needs_unlock for rebatched transactions batches ( #25095 )
...
* Unset needs_unlock for rebatched transactions batches
* address review comments
2022-05-10 13:39:08 -07:00
2fa9bc3e70
Add options to store full and/or incremental snapshots in separate locations ( #24247 )
2022-05-10 16:37:41 -04:00
dc6e28099b
mark_dirty_dead_stores ( #25114 )
2022-05-10 15:08:29 -05:00
e9e267947e
simplify storage_finder -> storage ( #25109 )
2022-05-10 14:18:51 -05:00
e9a9604ca7
avoid zeroing out root stats with empty data ( #25111 )
2022-05-10 13:56:30 -05:00
b91cc87680
assert when rewrites skipped ( #25110 )
2022-05-10 13:55:48 -05:00
8e5bb0a1e2
check file size of accounts cache mmap before indexing ( #25077 )
2022-05-10 13:33:22 -05:00
2d7ce2a6c2
Revert "uses designated thread-pools for account db parallel ops ( #24954 )" ( #25053 )
...
This reverts commit e8bdc27080
2022-05-06 15:20:50 -05:00
082502d4f3
Fail tx sanitization when ix program id uses lookup table ( #25035 )
...
* Fail tx sanitization when ix program id uses lookup table
* feedback
2022-05-07 03:19:50 +08:00
df9a4a0c9d
Bank::new_for_benches uses bench config ( #25017 )
2022-05-06 13:35:18 -05:00
7401bc0fa5
remove_old_historical_roots uses correct epoch schedule ( #25020 )
...
* remove_old_historical_roots uses correct epoch schedule
* Update runtime/src/accounts_db.rs
Co-authored-by: apfitzge <apfitzge@users.noreply.github.com>
Co-authored-by: apfitzge <apfitzge@users.noreply.github.com>
2022-05-06 12:38:09 -05:00
47cd0d9061
Speed up get-data-cost function ( #25013 )
...
* Combine data_bytes_cost into the get_transaction_cost function, for speedup
* Remove a dead function, fix a value in a test
* Respond to comments by changing call signature to take a &mut struct rather than return a tuple
2022-05-06 11:39:00 -05:00
492f89a170
checks account owner when initializing a vote-account ( #25018 )
...
A VoteAccount may only wrap an account if the account owner is
solana_vote_program:id or equivalently this check returns true:
solana_vote_program::check_id(account.owner())
2022-05-06 16:22:49 +00:00
3fff34a65f
explicitly removes accounts with zero lamports from stakes cache ( #25015 )
...
Zero lamport accounts are not stored in accounts-db:
https://github.com/solana-labs/solana/blob/7b5835ddf/runtime/src/accounts_db.rs#L5007-L5014
https://github.com/solana-labs/solana/blob/7b5835ddf/runtime/src/accounts_db.rs#L4481-L4483
However, in order to purge the account from cache, stakes cache update
partially relies on accounts data to be also zeroed out :
https://github.com/solana-labs/solana/blob/7b5835ddf/runtime/src/bank.rs#L3471-L3478
This can be error-prone and introduce inconsistency between accounts-db
and vote/stake cache. This commit instead explicitly removes accounts
from cache if lamports == 0.
2022-05-06 14:04:49 +00:00
0cc97689f9
add stats for stakes_cache_check_and_store_us ( #24349 )
...
* add stats for stakes_cache_check_and_store_us
* remove redundant metrics
2022-05-05 17:22:48 -05:00
Jeff Washington (jwash)
Trent Nelson
dependabot[bot]
Tyera Eulberg
apfitzge
steviez
Wei Hu
Christian Kamm
carllin
Justin Starry
buffalu
Brooks Prumo
Alexander Meißner
Michael Vines
Jack May
Jon Cinque
HaoranYi
Tao Zhu
Justin Starry
Ashwin Sekar
Ryo Onodera
sakridge
Jeff Washington (jwash)
Jason
Pankaj Garg
DimAn
behzad nouri