0486df02ba
don't limit to thread pool when cleaning on startup ( #17317 )
2021-05-20 14:36:35 -05:00
a9aa533684
replace Ancestors HashMap for performance ( #16477 )
...
* replace Ancestors HashMap for performance
* add ancestors module
2021-05-20 10:11:56 -05:00
ed9cbd50f0
move Ancestors to its own module ( #17316 )
2021-05-19 11:50:34 -05:00
fd88db7339
ancestors::new -> default ( #17195 )
2021-05-14 15:20:59 -05:00
f39dda00e0
type AccountSecondaryIndexes = HashSet ( #17108 )
2021-05-10 14:22:48 +00:00
1a9954f85b
bank deposit checked_add error ( #16917 )
...
* bank deposit checked_add error
* add id
* rename variables
* rename error and metric
2021-04-30 16:22:17 -05:00
5591cd6ef3
.owner -> .owner() ( #16757 )
...
* .owner -> .get_owner()
* use
2021-04-23 09:35:09 -05:00
03f7b251b8
ReadableAccount.data returns slice ( #16686 )
...
* ReadAbleAccount.data returns slice
* more slice fixup
* more slice
* slice
2021-04-20 16:41:16 -05:00
9dfcb921cf
Refactoring: Move KeyedAccounts to InvokeContext ( #15410 )
...
Collects all parametric occurrences and the construction of keyed_accounts and puts them into InvokeContext.
2021-04-19 18:48:48 +02:00
d747614b27
Account for possibility of cache flush in load() ( #15454 )
...
* Account for possibility of cache flush in load()
* More cleaning
* More cleaning
* Remove unused method and some comment cleaning
* Fix typo
* Make the detected impossible purge race panic()!
* Finally revert to original .expect()
* Fix typos...
* Add assertion for max_root for easier reasoning
* Reframe races with LoadHint as possible opt.
* Fix test
* Make race bug tests run longer for less flaky
* Delay the clone-in-lock slow path even for RPC
* Make get_account panic-free & add its onchain ver.
* Fix rebase conflicts...
* Clean up
* Clean up comment
* Revert fn name change
* Fix flaky test...
* fmt...
Co-authored-by: Ryo Onodera <ryoqun@gmail.com>
2021-04-17 00:23:32 +09:00
85eb37fab0
Merge pull request from GHSA-8v47-8c53-wwrc
...
* Track transaction check time separately from account loads
* banking packet process metrics
* Remove signature clone in status cache lookup
* Reduce allocations when converting packets to transactions
* Add blake3 hash of transaction messages in status cache
* Bug fixes
* fix tests and run fmt
* Address feedback
* fix simd tx entry verification
* Fix rebase
* Feedback
* clean up
* Add tests
* Remove feature switch and fall back to signature check
* Bump programs/bpf Cargo.lock
* clippy
* nudge benches
* Bump `BankSlotDelta` frozen ABI hash`
* Add blake3 to sdk/programs/Cargo.lock
* nudge bpf tests
* short circuit status cache checks
Co-authored-by: Trent Nelson <trent@solana.com>
2021-04-13 00:28:08 -06:00
6930a77a0f
prepare replace Ancestors HashMap for performance ( #16476 )
2021-04-12 10:51:57 -05:00
3f63ed9a72
removes OrderedIterator and transaction batch iteration order ( #16153 )
...
In TransactionBatch,
https://github.com/solana-labs/solana/blob/e50f59844/runtime/src/transaction_batch.rs#L4-L11
lock_results[i] is aligned with transactions[iteration_order[i]]:
https://github.com/solana-labs/solana/blob/e50f59844/runtime/src/bank.rs#L2414-L2424
https://github.com/solana-labs/solana/blob/e50f59844/runtime/src/accounts.rs#L788-L817
However load_and_execute_transactions is iterating over
lock_results[iteration_order[i]]
https://github.com/solana-labs/solana/blob/e50f59844/runtime/src/bank.rs#L2878-L2889
and then returning i as for the index of the retryable transaction.
If iteratorion_order is [1, 2, 0], and i is 0, then:
lock_results[iteration_order[i]] = lock_results[1]
which corresponds to
transactions[iteration_order[1]] = transactions[2]
so neither i = 0, nor iteration_order[i] = 1 gives the correct index for the
corresponding transaction (which is 2).
This commit removes OrderedIterator and transaction batch iteration order
entirely. There is only one place in blockstore processor which the
iteration order is not ordinal:
https://github.com/solana-labs/solana/blob/e50f59844/ledger/src/blockstore_processor.rs#L269-L271
It seems like, instead of using an iteration order, that can shuffle entry
transactions in-place.
2021-03-31 23:59:19 +00:00
416312b30b
update timings only on outer call ( #16275 )
...
* update timings only on outer call
* test & format
2021-03-31 18:02:59 -05:00
5e5b63712b
Status cache improvements ( #16174 )
2021-03-28 19:10:14 -07:00
4bbeb9c033
Remove old feature: simple_capitalization ( #15763 )
...
* Remove old feature: simple_capitalization
* Fix another failing test in core
* Finish up test cleanup
* Further clean up a bit
2021-03-12 11:12:40 +09:00
9c1198c0c7
Improve load_largest_accounts more ( #15785 )
...
* Add load_largest_accounts bench
* Check lamports before address filter
* Use BinaryHeap, add Accounts test
* Use pubkey reference in the min-heap
Also, flatten code with early returns
Co-authored-by: Greg Fitzgerald <greg@solana.com>
2021-03-10 11:22:02 -07:00
369e13b111
cleanup old runtime features ( #15787 )
2021-03-10 09:48:41 -08:00
52e54e1100
account.data -> data() ( #15778 )
2021-03-09 22:31:33 +00:00
8a3135d17b
Account->AccountSharedData ( #15691 )
2021-03-09 15:06:07 -06:00
be35c1c1b7
add execute detail timings ( #15638 )
2021-03-03 17:07:45 -06:00
7f7370c306
Re-allow clippy::integer_arithmetic at crate-level
2021-02-17 13:55:08 -07:00
77572a7c53
Track account writable deescalation ( #14626 )
2021-01-22 15:28:01 -08:00
2745b79b74
Parallel cache scan ( #14544 )
...
* Parallel cache scan
* PR comments
* PR comments
Co-authored-by: Carl Lin <carl@solana.com>
2021-01-20 08:50:17 +00:00
6dfad0652f
Cache account stores, flush from AccountsBackgroundService ( #13140 )
2021-01-11 17:00:23 -08:00
5affd8aa72
Add secondary indexes ( #14212 )
2020-12-31 18:06:03 -08:00
7143aaa89b
Clippy
2020-12-14 08:03:29 -08:00
de9ac43ebf
Simple cap by including sysvars and native programs ( #13884 )
...
* Simpler cap by including sysvars and native programs
* Fix tests
* Add comment
* revert some unrelated code
* Update test_bank_update_sysvar_account for cap.
* Test cap. for add_native_program using new helper
* Improve the cap adjustment with new tests
* Fix typo...
* Adjust test for improved code coverage
* Rename simpler_capitalization => simple_capitalization
* More rename and bonus commenting
2020-12-14 21:03:24 +09:00
8f0796436a
shares the lock on gossip when processing prune messages ( #13339 )
...
Processing prune messages acquires an exclusive lock on gossip:
https://github.com/solana-labs/solana/blob/55b0428ff/core/src/cluster_info.rs#L1824-L1825
This can be reduced to a shared lock if active-sets are changed to use
atomic bloom filters:
https://github.com/solana-labs/solana/blob/55b0428ff/core/src/crds_gossip_push.rs#L50
2020-11-05 15:42:00 +00:00
bc62313c66
Allow feature builtins to overwrite existing builtins ( #13403 )
...
* Allow feature builtins to overwrite existing builtins
* Add feature_builtin ActivationType
* Correctly retain idempotent for replacing case
* Fix test
Co-authored-by: Ryo Onodera <ryoqun@gmail.com>
2020-11-05 15:37:07 +00:00
225bed11c7
Remove Entrypoint type
2020-10-29 21:45:24 -07:00
df8dab9d2b
Native/builtin programs now receive an InvokeContext
2020-10-29 21:45:24 -07:00
1b343665a1
Move KeyedAccount out of solana-program. Native programs are not supported by solana-program
2020-10-26 18:54:54 -07:00
959880db60
Remove unused pubkey::Pubkey imports
2020-10-21 19:08:13 -07:00
17c391121a
Run `codemod --extensions rs Hash::new_rand solana_sdk: #️⃣ :new_rand`
2020-10-21 19:08:13 -07:00
7bc073defe
Run `codemod --extensions rs Pubkey::new_rand solana_sdk::pubkey::new_rand`
2020-10-21 19:08:13 -07:00
e6b821c392
Finer grained AccountsIndex locking ( #12787 )
...
Co-authored-by: Carl Lin <carl@solana.com>
2020-10-21 17:05:27 -07:00
c8fc0a6ba1
Don't hold dashmap write lock in store create ( #13007 )
...
Co-authored-by: Carl Lin <carl@solana.com>
2020-10-21 12:54:09 -07:00
f8d338c9cb
Switch accounts storage lock to DashMap ( #12126 )
...
Co-authored-by: Carl Lin <carl@solana.com>
2020-10-13 18:29:50 -07:00
403790760c
Default to 50% rent burn
2020-10-09 21:52:31 -07:00
06f84c65f1
Fix rooted accounts cleanup, simplify locking ( #12194 )
...
Co-authored-by: Carl Lin <carl@solana.com>
2020-09-28 16:04:46 -07:00
31696a1d72
Port BPFLoader2 activation to FeatureSet and rework built-in program activation
2020-09-28 12:50:19 -07:00
bb183938d9
adds an atomic variant of the bloom filter ( #12422 )
...
For crds_gossip_pull, we want to parallelize build_crds_filters, which
requires concurrent writes to bloom filters.
This commit implements a variant of the bloom filter which uses atomics
for its bits vector and so is thread-safe.
2020-09-24 18:37:19 +00:00
68e5a2ef56
Add RPC notify and banking keys debug ( #12396 )
2020-09-23 18:46:42 -07:00
de4a613610
Check bank capitalization ( #11927 )
...
* Check bank capitalization
* Simplify and unify capitalization calculation
* Improve and add tests
* Avoid overflow and inhibit automatic restart
* Fix test
* Tweak checked sum for cap. and add tests
* Fix broken build after merge conflicts..
* Rename to ClusterType
* Rename confusing method
* Clarify comment
* Verify cap. in rent and inflation tests
Co-authored-by: Stephen Akridge <sakridge@gmail.com>
2020-09-12 01:48:06 +09:00
53b8ea4464
Rename to ClusterType and restore devnet compat. ( #12068 )
...
* Rename to ClusterType and restore devnet compat.
* De-duplicate parse code and add comments
* Adjust default Devnet genesis & reduce it in tests
2020-09-08 23:55:09 +09:00
af08221aec
Switch account hashing to blake3 ( #11969 )
...
* Switch account hashing to blake3
Co-authored-by: Carl <carl@solana.com>
2020-09-02 00:37:36 -07:00
7e25130529
Send votes from banking stage to vote listener ( #11434 )
...
* Send votes from banking stage to vote listener
Co-authored-by: Carl <carl@solana.com>
2020-08-07 11:21:35 -07:00
8bf3a0aa05
accounts_clean: Convert stack dependency calculation with iterative ( #11067 )
...
* accounts_clean: Convert stack dependency calculation with iterative
* optimize clean with by creating a reverse-lookup hashset of the affected
keys
* Add dependency bench
reduce bench
* Huge clean
2020-07-15 06:49:22 -07:00
7ade330b23
Rename Client methods to match proposed BanksClient ( #10793 )
2020-06-25 03:35:38 +00:00
1c498369b5
Remove fee-payer guesswork from Message and Transaction ( #10776 )
...
* Make Message::new_with_payer the default constructor
* Remove Transaction::new_[un]signed_instructions
These guess the fee-payer instead of stating it explicitly
2020-06-24 14:52:38 -06:00
e23340d89e
Clippy cleanup for all targets and nighly rust (also support 1.44.0) ( #10445 )
...
* address warnings from 'rustup run beta cargo clippy --workspace'
minor refactoring in:
- cli/src/cli.rs
- cli/src/offline/blockhash_query.rs
- logger/src/lib.rs
- runtime/src/accounts_db.rs
expect some performance improvement AccountsDB::clean_accounts()
* address warnings from 'rustup run beta cargo clippy --workspace --tests'
* address warnings from 'rustup run nightly cargo clippy --workspace --all-targets'
* rustfmt
* fix warning stragglers
* properly fix clippy warnings test_vote_subscribe()
replace ref-to-arc with ref parameters where arc not cloned
* Remove lock around JsonRpcRequestProcessor (#10417 )
automerge
* make ancestors parameter optional to avoid forcing construction of empty hash maps
Co-authored-by: Greg Fitzgerald <greg@solana.com>
2020-06-09 09:38:14 +09:00
36627fb8b3
move builtin programs out of bank ( #10132 )
...
automerge
2020-05-19 19:45:30 -07:00
16af67d5e1
Focus bench on squash and fix log errors ( #9759 )
...
* Focus bench on squash
Squash performance does not depend on adding a small number accounts. It mainly depends on total number of accounts that need to be hashed during freeze operation. New bank means a clone of accounts db, so we don't get previous errors in log.
* Fix fmt and add slot counter
2020-05-05 08:33:41 -07:00
068f12fd6f
Add Cross-program invocations ( #9582 )
2020-04-28 14:33:56 -07:00
efad193180
Make default programs static ( #9717 )
2020-04-27 21:05:12 -07:00
76b1c2baf0
One less alloc per transaction ( #9705 )
...
* One less alloc per transaction
* Fix benches
* Fix compiler warnings in bench build
* Fix move build
* Fix bench
2020-04-24 13:03:46 -06:00
58887c591b
Add and update tests ( #9566 )
2020-04-18 17:04:13 -07:00
ad0482be73
Revert "Add native loader entry points ( #9275 )" Breaks genesis_config abi ( #9377 )
...
This reverts commit ed86d8d1fc
2020-04-08 14:36:18 -07:00
ed86d8d1fc
Add native loader entry points ( #9275 )
2020-04-03 17:40:59 -07:00
130c0b484d
Enforce an executable's rent exemption in the runtime ( #9134 )
2020-03-31 10:07:38 -07:00
88ba8439fc
Add frozen account support ( #8989 )
...
automerge
2020-03-22 11:10:04 -07:00
947a339714
Add snapshot hash of full accounts state ( #8295 )
...
* Add snapshot hash of full accounts state
* Use normal hashing for the accounts delta state
* Add merkle
2020-02-22 13:46:40 -08:00
0e6aca5a7e
Reorganize message processor in prep for cross-program-invocation ( #8338 )
2020-02-21 11:30:00 -08:00
ab361a8073
Rename KeypairUtil to Signer ( #8360 )
...
automerge
2020-02-20 13:28:55 -08:00
83718a3b3e
Cleanup runtime use syntax ( #8002 )
2020-01-28 17:03:20 -08:00
c95e5346a4
Boot the mut ( #7926 )
2020-01-22 17:54:06 -08:00
023074650f
Allow the same account to be passed multiple times to a single instruction ( #7795 )
2020-01-22 09:11:56 -08:00
87598c7612
Consolidate tx error counters and update metrics dashboard ( #7724 )
...
automerge
2020-01-16 23:26:50 -08:00
3c361eb759
Check account hashes in snapshot ( #7559 )
...
* Check for incorrect hash value
* Finish up checking for incorrect hash value
* Fix comment a bit
Co-authored-by: sakridge <sakridge@gmail.com>
2019-12-20 09:39:30 +09:00
b7d4330dd4
Fail fast if account paths cannot be canonicalized ( #7300 )
...
* Canonicalize account paths to avoid symlink issues
* fixes
2019-12-05 21:41:29 -05:00
c09469fa3a
Rename verify_instruction() to verify_account_changes() ( #7020 )
2019-11-18 15:01:14 -07:00
9807f47d4e
Rename genesis block to genesis config ( #6816 )
2019-11-08 23:56:57 -05:00
b9d8e3e55a
Only copy whats needed to verify an instruction after processing ( #6669 )
2019-11-05 10:57:32 -08:00
5416c114cf
SDK: Add sysvar to expose recent block hashes to programs ( #6663 )
...
* SDK: Add sysvar to expose recent block hashes to programs
* Blockhashes is one word
* Missed one
* Avoid allocs on update
* unwrap_or_else
* Use iterators
* Add microbench
* Revert "unwrap_or_else"
This reverts commit a8f8c3bfbe0d88e2229e67a29e9d2b489b93ff62.
* Revert "Avoid allocs on update"
This reverts commit 486f01790c271cf09ac3dbffba0e40e526e81324.
2019-11-04 10:51:15 -07:00
c9cea2152b
optimize verify_instruction ( #6539 )
2019-10-25 21:47:16 -07:00
d68e2c4d06
Revert "Make instruction data opaque to runtime ( #6470 )" ( #6564 )
...
This reverts commit 6eeca9c6f1
2019-10-25 16:22:41 -07:00
6eeca9c6f1
Make instruction data opaque to runtime ( #6470 )
2019-10-24 22:38:57 -07:00
f46a2cec3c
owner and executable checks ( #6526 )
...
* owner_checks
* only system program may assign owner, and only if pre.owner is system
* moar coverage!
* moar coverage, allow re-assignment IFF data is zeroed
2019-10-24 11:06:00 -07:00
35ed432d1a
Make benchmark useful ( #6440 )
...
function is verify_hash_internal_state
2019-10-18 12:59:47 -07:00
fb39bd45d7
Revert "Rename solana-runtime to sealevel ( #6239 )" ( #6247 )
...
This reverts commit 2e921437cd
2019-10-04 19:33:29 -06:00
2e921437cd
Rename solana-runtime to sealevel ( #6239 )
...
automerge
2019-10-04 15:02:44 -07:00
19ae556857
hash account state on store ( #5573 )
2019-09-20 13:21:12 -07:00
6657312f44
dyn for runtime benches ( #5983 )
2019-09-19 14:21:09 -07:00
7dfb735db9
randomize tx ordering ( #4978 )
...
Summary of Changes:
This change adds functionality to randomize tx execution for every entry. It does this by implementing OrderedIterator that iterates tx slice as per the order specified. The order is generated randomly for every entry.
2019-08-28 21:08:32 +05:30
58d4e32c97
Remove serialization of future AppendVecs and serialize AccountStorage correctly ( #5510 )
2019-08-13 16:05:37 -07:00
a43922ccbf
Boot hashbrown ( #5505 )
...
As of Rust 1.36.0, hashbrown now implements the HashMap in std (which
implements HashSet).
https://blog.rust-lang.org/2019/07/04/Rust-1.36.0.html#a-new-hashmapk,-v%3E-implementation
2019-08-12 16:46:49 -06:00
9dcf3347f5
Refactor status cache and remove complex serialize/deserialize ( #5335 )
...
automerge
2019-08-06 18:47:30 -07:00
8d243221f0
Ignore flaky local cluster tests ( #5347 )
...
* Add logging to local_cluster tests
* Ignore flaky test_leader_failure_4, test_repairman_catchup
And crashing banking benchmarks.
2019-07-30 13:48:46 -07:00
264a3d7dde
Increase ticks_per_slot for banking benchmark ( #5221 )
2019-07-22 13:57:28 -07:00
111d0eb89b
runtime: Add bench for accounts::hash_internal_state ( #5157 )
...
* runtime: Add bench for accounts::hash_internal_state
* fixup! cargo fmt
* fixup! cargo clippy
* fixup! Use a more representitive number of accounts
* fixup! More descriptive name for accounts creation helper
2019-07-19 10:32:29 -06:00
04649de6a6
Boot remote native loads, take 2 ( #5106 )
...
* Drop dependencies on remote native loads
* Remove remote native loads
2019-07-15 13:16:09 -06:00
966b6999d1
Accounts index opt ( #4621 )
...
* Add accounts_index bench
* Don't take the accounts index lock unless needed
* Accounts_index remove insert return vec and add capacity stats
* Use hashbrown hashmap for accounts_index
2019-06-10 18:15:39 -07:00
64e8a21d73
Add tick height syscall ( #4497 )
...
* Remove tick_height from entrypoint signature
* Impl tick_height syscall and use in storage program
* Properly remove tick height from bpf handling
2019-05-31 16:29:21 -06:00
182096dc1a
Create bank snapshots ( #4244 )
...
* Revert "Revert "Create bank snapshots (#3671 )" (#4243 )"
This reverts commit 81fa69d347
2019-05-30 21:31:35 -07:00
a2e3a92b01
Extend GetBlockHash RPC API to include the fee scehdule for using the returned blockhash ( #4222 )
2019-05-13 12:49:37 -07:00
69eeb7cf08
Fix parent record locks usage in child banks ( #4159 )
...
* Introduce record locks on txs that will be recorded
* Add tests for LockedAccountsResults
* Fix broken bench
* Exit process_entries on detecting conflicting locks within same entry
2019-05-07 15:51:35 -07:00
8e400fc4bd
rework genesis (passive staking groundwork) ( #4187 )
...
* rework genesis
* fixup
2019-05-07 11:16:22 -07:00
c545e812d0
Expand bank benches to include async/sync and native_loader ( #4026 )
2019-04-29 13:09:11 -07:00
5fb8baed04
Process async BankClient transactions in batches ( #3738 )
...
* Process async transactions in batches
This aims to process transactions at least as fast as LocalCluster
* Add benchmark
2019-04-19 07:29:07 -06:00
Jeff Washington (jwash)
Alexander Meißner
carllin
Justin Starry
behzad nouri
sakridge
Ryo Onodera
Tyera Eulberg
Jack May
Trent Nelson
carllin
Michael Vines
Greg Fitzgerald
Kristofer Peterson
jon-chuang
Ryo Onodera
Rob Walker
sakridge
Parth
Sagar Dhawan
Tyera Eulberg
Sathish