Commit Graph

77 Commits

Author SHA1 Message Date
Tyera Eulberg 6e9deaf1bd
Move block-time caching earlier (#17109)
* Require that blockstore block-time only be recognized slot, instead of root

* Move cache_block_time to after Bank freeze

* Single use statement

* Pass transaction_status_sender by reference

* Remove unnecessary slot-existence check before caching block time altogether

* Move block-time existence check into Blockstore::cache_block_time, Blockstore no longer needed in blockstore_processor helper
2021-05-10 13:14:56 -06:00
Jeff Washington (jwash) f39dda00e0
type AccountSecondaryIndexes = HashSet (#17108) 2021-05-10 14:22:48 +00:00
Justin Starry 85eb37fab0
Merge pull request from GHSA-8v47-8c53-wwrc
* Track transaction check time separately from account loads

* banking packet process metrics

* Remove signature clone in status cache lookup

* Reduce allocations when converting packets to transactions

* Add blake3 hash of transaction messages in status cache

* Bug fixes

* fix tests and run fmt

* Address feedback

* fix simd tx entry verification

* Fix rebase

* Feedback

* clean up

* Add tests

* Remove feature switch and fall back to signature check

* Bump programs/bpf Cargo.lock

* clippy

* nudge benches

* Bump `BankSlotDelta` frozen ABI hash`

* Add blake3 to sdk/programs/Cargo.lock

* nudge bpf tests

* short circuit status cache checks

Co-authored-by: Trent Nelson <trent@solana.com>
2021-04-13 00:28:08 -06:00
Christian Drappi 54a04bac3d
Apple M1 compatibility (#16346)
Co-authored-by: Christian Drappi <christiandrappi@Christians-MacBook-Pro.local>
2021-04-09 17:21:01 -07:00
behzad nouri 3f63ed9a72
removes OrderedIterator and transaction batch iteration order (#16153)
In TransactionBatch,
https://github.com/solana-labs/solana/blob/e50f59844/runtime/src/transaction_batch.rs#L4-L11
lock_results[i] is aligned with transactions[iteration_order[i]]:
https://github.com/solana-labs/solana/blob/e50f59844/runtime/src/bank.rs#L2414-L2424
https://github.com/solana-labs/solana/blob/e50f59844/runtime/src/accounts.rs#L788-L817

However load_and_execute_transactions is iterating over
  lock_results[iteration_order[i]]
https://github.com/solana-labs/solana/blob/e50f59844/runtime/src/bank.rs#L2878-L2889
and then returning i as for the index of the retryable transaction.

If iteratorion_order is [1, 2, 0], and i is 0, then:
  lock_results[iteration_order[i]] = lock_results[1]
which corresponds to
  transactions[iteration_order[1]] = transactions[2]
so neither i = 0, nor iteration_order[i] = 1 gives the correct index for the
corresponding transaction (which is 2).

This commit removes OrderedIterator and transaction batch iteration order
entirely. There is only one place in blockstore processor which the
iteration order is not ordinal:
https://github.com/solana-labs/solana/blob/e50f59844/ledger/src/blockstore_processor.rs#L269-L271
It seems like, instead of using an iteration order, that can shuffle entry
transactions in-place.
2021-03-31 23:59:19 +00:00
Tyera Eulberg 433f1ead1c
Rpc: enable getConfirmedBlock and getConfirmedTransaction to return confirmed (not yet finalized) data (#16142)
* Add Blockstore block and tx apis that allow unrooted responses

* Add TransactionStatusMessage, and send on bank freeze; also refactor TransactionStatusSender

* Track highest slot with tx-status writes complete

* Rename and unpub fn

* Add commitment to GetConfirmed input configs

* Support confirmed blocks in getConfirmedBlock

* Support confirmed txs in getConfirmedTransaction

* Update sigs-for-addr2 comment

* Enable confirmed block in cli

* Enable confirmed transaction in cli

* Review comments

* Rename blockstore method
2021-03-26 16:47:35 -06:00
Jeff Washington (jwash) 1135ffd595
mut data refs as slice (#15782) 2021-03-10 15:28:03 -06:00
Jeff Washington (jwash) 8a3135d17b
Account->AccountSharedData (#15691) 2021-03-09 15:06:07 -06:00
Jeff Washington (jwash) 9448540ea6
log timing after ledger process (#15716) 2021-03-05 11:01:52 -06:00
Jeff Washington (jwash) be35c1c1b7
add execute detail timings (#15638) 2021-03-03 17:07:45 -06:00
sakridge f1223fb783
Lower blockstore processor error severity (#15578) 2021-03-01 14:57:37 -08:00
Michael Vines 33eaa2b238 Sort forks in "ledger processed..." log message 2021-02-28 17:32:50 -08:00
sakridge bbae23358c
ledger-tool cleanup and additions (#15179)
* Plumb allow-dead-slots to ledger-tool verify

* ledger-tool cleanup and add some useful missing args

Print root slots and how many unrooted past last root.
2021-02-06 17:26:42 -08:00
Tyera Eulberg cbb8b79a60
Add validator flag to opt in to cpi and logs storage (#14922)
* Add validator flag to opt in to cpi and logs storage

* Default TestValidator to opt-in; allow using in multinode-demo

* No clone

Co-authored-by: Carl Lin <carl@solana.com>
2021-02-01 14:00:51 -07:00
sakridge 907f518f6d
Add load/execute/store timings (#14561) 2021-01-14 14:14:16 -08:00
Ryo Onodera e95ebcf864
Don't stop to find newer cluster-confirmed roots (#14557)
* Don't stop to find newer cluster-confirmed roots

* Fix and add new tests

* nits
2021-01-13 15:47:06 +00:00
carllin 6dfad0652f
Cache account stores, flush from AccountsBackgroundService (#13140) 2021-01-11 17:00:23 -08:00
Michael Vines 86c81a0ba2 Humanize the 'ledger processed...' time 2021-01-08 22:56:21 -08:00
carllin 5affd8aa72
Add secondary indexes (#14212) 2020-12-31 18:06:03 -08:00
Trent Nelson 5b903318b2 vote: Add helper for creating current-versioned states 2020-12-22 19:37:26 -07:00
sakridge baa9602411
Add shrink paths (#14238) 2020-12-21 21:33:37 -08:00
Michael Vines 501fd83afd Fix up upgradeable bpf loader activation 2020-12-16 06:36:14 +00:00
Michael Vines 7143aaa89b Clippy 2020-12-14 08:03:29 -08:00
Josh 13db3eca9f
SPL token balance in transaction metadata (#13673)
* feat: store pre / post token balances

* move helper functions into separate include

* move token balance functionality to transaction-status crate

* fix blockstore processor test

* fix bigtable legacy test

* add caching to decimals
2020-12-10 19:25:07 -08:00
Alexander Meißner a706706572
Validator CLI option to enable just-in-time compilation of BPF (#13789)
* Adds a CLI option to the validator to enable just-in-time compilation of BPF.

* Refactoring to use bpf_loader_program instead of feature_set to pass JIT flag from the validator CLI to the executor.
2020-12-07 09:49:55 +01:00
carllin 51d8f36dae
Ledger-tool recreate snapshot of starting snapshots
Co-authored-by: Carl Lin <carl@solana.com>
2020-12-04 21:14:59 -08:00
Trent Nelson 404fc1570d runtime: Replace `HashAgeKind` with `NonceRollbackInfo` 2020-12-02 20:10:08 +00:00
behzad nouri e1793e5a13
caches vote-state de-serialized from vote accounts (#13795)
Gossip and other places repeatedly de-serialize vote-state stored in
vote accounts. Ideally the first de-serialization should cache the
result.

This commit adds new VoteAccount type which lazily de-serializes
VoteState from Account data and caches the result internally.

Serialize and Deserialize traits are manually implemented to match
existing code. So, despite changes to frozen_abi, this commit should be
backward compatible.
2020-11-30 17:18:33 +00:00
Michael Vines 959880db60 Remove unused pubkey::Pubkey imports 2020-10-21 19:08:13 -07:00
Michael Vines 7bc073defe Run `codemod --extensions rs Pubkey::new_rand solana_sdk::pubkey::new_rand` 2020-10-21 19:08:13 -07:00
Ryo Onodera 1f4bcf70b0
Fix various ledger-tool error due to no builtins (#12759)
* Fix various ledger-tool error due to no builtins

* Add missing file...
2020-10-09 12:19:36 -06:00
Josh 8f5431551e
Store program logs in blockstore / bigtable (TransactionWithStatusMeta) (#12678)
* introduce store program logs in blockstore / bigtable

* fix test, transaction logs created for successful transactions

* fix test for legacy bincode implementation around log_messages

* only api nodes should record logs

* truncate transaction logs to 100KB

* refactor log truncate for improved coverage
2020-10-08 12:06:15 -07:00
Michael Vines 31696a1d72 Port BPFLoader2 activation to FeatureSet and rework built-in program activation 2020-09-28 12:50:19 -07:00
Michael Vines c10da16d7b Port instructions sysvar and secp256k1 program activation to FeatureSet 2020-09-25 11:40:36 -07:00
Justin Starry 6601ec8f26
Record and store invoked instructions in transaction meta (#12311)
* Record invoked instructions and store in transaction meta

* Enable cpi recording if transaction sender is some

* Rename invoked to innerInstructions
2020-09-24 22:36:22 +08:00
sakridge 68e5a2ef56
Add RPC notify and banking keys debug (#12396) 2020-09-23 18:46:42 -07:00
carllin 3533e11786
Fix blockstore processor squash (#12319) 2020-09-17 21:33:08 -07:00
sakridge 3930cb865a
Add keccak-secp256k1 instruction (#11839)
* Implement keccak-secp256k1 instruction

Verifies eth addreses with ecrecover function

* Move secp256k1 test
2020-09-15 18:23:21 -07:00
Ryo Onodera de4a613610
Check bank capitalization (#11927)
* Check bank capitalization

* Simplify and unify capitalization calculation

* Improve and add tests

* Avoid overflow and inhibit automatic restart

* Fix test

* Tweak checked sum for cap. and add tests

* Fix broken build after merge conflicts..

* Rename to ClusterType

* Rename confusing method

* Clarify comment

* Verify cap. in rent and inflation tests

Co-authored-by: Stephen Akridge <sakridge@gmail.com>
2020-09-12 01:48:06 +09:00
Ryo Onodera c274e26eb8
Prevent unbound memory growth by blockstore_processor (#12110)
* Prevent unbound memory growth by blockstore_processor

* Promote log to info! considering infrequency

* Exclude the time of freeing from interval...

* Skip not-shrinkable slots even if forced

* Add comment
2020-09-09 12:05:10 +09:00
Ryo Onodera 53b8ea4464
Rename to ClusterType and restore devnet compat. (#12068)
* Rename to ClusterType and restore devnet compat.

* De-duplicate parse code and add comments

* Adjust default Devnet genesis & reduce it in tests
2020-09-08 23:55:09 +09:00
Jack May db4bbb3569
Switch programs activation to whole-set based gating (#11750)
* Implement Debug for MessageProcessor

* Switch from delta-based gating to whole-set gating

* Remove dbg!

* Fix clippy

* Clippy

* Add test

* add loader to stable operating mode at proper epoch

* refresh_programs_and_inflation after ancestor setup

* Callback via snapshot; avoid account re-add; Debug

* Fix test

* Fix test and fix the past history

* Make callback management stricter and cleaner

* Fix test

* Test overwrite and frozen for native programs

* Test epoch callback with genesis-programs

* Add assertions for parent bank

* Add tests and some minor cleaning

* Remove unsteady assertion...

* Fix test...

* Fix DOS

* Skip ensuring account by dual (whole/delta) gating

* Fix frozen abi implementation...

* Move compute budget constatnt init back into bank

Co-authored-by: Ryo Onodera <ryoqun@gmail.com>
2020-08-26 01:49:15 +09:00
carllin f7adb68599
Squash supermajority root on blockstore replay at startup (#11727) 2020-08-20 21:56:25 -07:00
Jack May 7c736f71fe
Make BPF Loader static (#11516) 2020-08-14 12:32:45 -07:00
carllin 7e25130529
Send votes from banking stage to vote listener (#11434)
*  Send votes from banking stage to vote listener

Co-authored-by: Carl <carl@solana.com>
2020-08-07 11:21:35 -07:00
Greg Fitzgerald 3fdd8ffdf0
Remove circular dep between InstructionError and SystemError (#11427) 2020-08-06 22:04:43 +00:00
carllin bf18524368
Add hook for getting vote transactions on replay (#11264)
* Add hook for getting vote transactions on replay

Co-authored-by: Carl <carl@solana.com>
2020-07-29 23:17:40 -07:00
Tyera Eulberg 6c38369042
Use OrderedIterator in TransactionStatusService (#11149)
* Split out get-first-err for unit testing

* Add failing test

* Add missing ordering
2020-07-21 20:06:49 +00:00
Tyera Eulberg 9a80e31bae
Expose tss to the other blockstore_processor path (#11070) 2020-07-14 21:14:48 -06:00
sakridge 658de5b347
Skip and warn for hard-forks which are less than the start slot (#10918)
* Skip and warn for hard-forks which are less than the start slot

Option is used during a restart, but then after the restart is
complete, then the option is not needed if the starting slot
is past the hard-fork since the hard-fork should already be
in the snapshot it started from.

* Update ledger/src/blockstore_processor.rs

Co-authored-by: Michael Vines <mvines@gmail.com>

Co-authored-by: Michael Vines <mvines@gmail.com>
2020-07-05 13:51:43 -07:00