Commit Graph

226 Commits

Author SHA1 Message Date
Jeff Washington (jwash) cfc124c825
acct idx can no longer use write cache (#28150) 2022-09-30 10:55:27 -07:00
steviez e4affb9fea
Add Blockstore::highest_slot() method (#27981) 2022-09-23 04:53:43 -05:00
Brooks Prumo 1ee595ca9c
remove AccountsDb::initial_blockstore_processing_complete (#27974) 2022-09-22 13:52:04 -04:00
Jeff Washington (jwash) 7ddacc4bb4
remove extra accounts cap call at startup (#27949) 2022-09-21 09:13:00 -07:00
Jeff Washington (jwash) f2d6a7ecea
bank.initial_blockstore_processing_complete to avoid concurrent hash calculations (#27776)
* bank.initial_blockstore_processing_complete to avoid concurrent hash calculations

* Update runtime/src/bank.rs

Co-authored-by: Brooks Prumo <brooks@prumo.org>

* Update runtime/src/bank.rs

Co-authored-by: Brooks Prumo <brooks@prumo.org>

* Rename TestValidator::set_startup_verification_complete()

* Initialize with `AtomicBool::new(false)` instead of `default()`

* snapshot tests: move where `initial_blockstore_processing_completed()` is called

* fixup bank_forks.rs calling `is_initial_blockstore_processing_complete()`

* only call initial_blockstore_processing_completed() in blockstore_processor

Co-authored-by: Brooks Prumo <brooks@prumo.org>
Co-authored-by: Brooks Prumo <brooks@solana.com>
2022-09-19 13:00:21 -07:00
Jeff Washington (jwash) 765c628546
use exit signal for acct idx bg threads (#27483) 2022-09-12 11:51:12 -07:00
Jeff Washington (jwash) f0770c199e
fix ledger tool final hash calc (#27725)
fix ledger tool final hash calc
2022-09-12 11:09:34 -07:00
Tao Zhu 8bb039d08d
collect min prioritization fees when replaying sanitized transactions (#26709)
* Collect blocks' minimum prioritization fees when replaying sanitized transactions

* Limits block min-fee metrics reporting to top 10 writable accounts

* Add service thread to asynchronously update and finalize prioritization fee cache

* Add bench test for prioritization_fee_cache

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
2022-08-31 08:00:55 -05:00
Brennan Watt 5150877fb3
Attempt to skip redundant startup account verification (#26999) 2022-08-25 09:29:57 -07:00
Michael Vines 3f4731b37f Standardize thread names
Tenets:
1. Limit thread names to 15 characters
2. Prefix all Solana-controlled threads with "sol"
3. Use Camel case. It's more character dense than Snake or Kebab case
2022-08-20 07:49:39 -07:00
steviez 1d66ae0f66
Bail out of execute_batches() early for empty batches slice (#26932)
The caller of execute_batches() that assembles batches may call this
function with empty batches; we know we can bail early in this scenario.
2022-08-08 12:14:49 -05:00
Justin Starry 69598ed4c0
Refactor: Add `RuntimeConfig` field to Bank (#26946)
* Refactor: Simplify arguments for bank constructor methods

* Refactor: Add RuntimeConfig to Bank fields

* Arc wrap runtime_config

* Arc wrap all runtime config usages

* Remove Copy trait derivation from RuntimeConfig

* Remove some arc wrapping
2022-08-05 20:49:00 +01:00
Brooks Prumo 4e43aa6c18
Remove accounts data size checks in blockstore_processor (#26776) 2022-08-05 09:55:41 -04:00
Tyera Eulberg 2dca239480
Remove runtime dependency from solana-transaction-status (#26930)
* Move RewardType out of runtime

* Move collect_token_balances to solana-ledger

* Remove solana-runtime dependency
2022-08-05 00:20:27 -06:00
Jeff Washington (jwash) f6e4f76dac
--halt-at-slot-store-hash-raw-data works for root bank (#26865) 2022-08-01 14:25:47 -05:00
Jeff Washington (jwash) c7462b7a52
ledger tool verify can store debug info on hash calc (#26837) 2022-07-29 15:54:56 -05:00
Brennan Watt 467cb5def5
Concurrent slot replay (#26465)
* Concurrent replay slots

* Split out concurrent and single bank replay paths

* Sub function processing of replay results for readability

* Add feature switch for concurrent replay
2022-07-28 11:33:19 -07:00
Jeff Washington (jwash) 47716a5e01
async hash verify on load (#26208)
* verify accounts hash in bg on startup

* fix some tests and loading from genesis

* add extra state for when background thread has completed
2022-07-15 14:29:56 -05:00
Alexander Meißner 038da82b6f
Feature: Early verification of account modifications in `BorrowedAccount` (#25899)
* Adjusts test cases for stricter requirements.

* Removes account reset in deserialization test.

* Removes verify related test cases.

* Replicates account modification verification logic of PreAccount in BorrowedAccount.

* Adds TransactionContext::account_touched_flags.

* Adds account modification verification to the BPF ABIv0 and ABIv1 deserialization, CPI syscall and program-test.

* Replicates the total sum of all lamports verification of PreAccounts in InstructionContext

* Check that the callers instruction balance is maintained during a call / push.

* Replicates PreAccount statistics in TransactionContext.

* Disable verify() and verify_and_update() if the feature enable_early_verification_of_account_modifications is enabled.

* Moves Option<Rent> of enable_early_verification_of_account_modifications into TransactionContext::new().

* Relaxes AccountDataMeter related test cases.

* Don't touch the account if nothing changes.

* Adds two tests to trigger InstructionError::UnbalancedInstruction.

Co-authored-by: Justin Starry <justin@solana.com>
2022-07-15 09:31:34 +02:00
Tao Zhu f13b5c832d
Remove obsoleted metrics reporting to reduce lock contention on cost_model (#26608)
remove obsoleted metrics reporting to reduce lock contention on cost_model
2022-07-14 23:02:49 -05:00
steviez 26ad45e723
Add log for missing start_slot in blockstore_processor (#26566)
blockstore_processor can do nothing in this case; the log might help
user realize that their snapshot and ledger are disjoint.
2022-07-11 18:11:16 -05:00
Nicholas Clarke ee0a40937e
Add validator argument log_messages_bytes_limit to change log truncation limit.
Add new cli argument log_messages_bytes_limit to solana-validator to control how long program logs can be before truncation
2022-07-11 10:53:18 -05:00
Jeff Washington (jwash) 602da5e51f
add accounts db config to bank tests (#26517) 2022-07-10 19:42:06 -05:00
carllin ce39c14025
Add end-to-end replay slot metrics (#25752) 2022-07-05 13:58:51 -05:00
Jeff Washington (jwash) 5f3b7bdd16
prevent ledger tool from calculating hash on non-rooted slots (#26355) 2022-07-02 11:50:01 -05:00
Ryo Onodera cd2878acf9
Avoid to miss to root for local slots before the hard fork (#19912)
* Make sure to root local slots even with hard fork

* Address review comments

* Cleanup a bit

* Further clean up

* Further clean up a bit

* Add comment

* Tweak hard fork reconciliation code placement
2022-06-26 15:14:17 +09:00
Tyera Eulberg a6ba5a9a05
Add transaction index in slot to geyser plugin TransactionInfo (#25688)
* Define shuffle to prep using same shuffle for multiple slices

* Determine transaction indexes and plumb to execute_batch

* Pair transaction_index with transaction in TransactionStatusService

* Add new ReplicaTransactionInfoVersion

* Plumb transaction_indexes through BankingStage

* Prepare BankingStage to receive transaction indexes from PohRecorder

* Determine transaction indexes in PohRecorder; add field to WorkingBank

* Add PohRecorder::record unit test

* Only pass starting_transaction_index around PohRecorder

* Add helper structs to simplify test DashMap

* Pass entry and starting-index into process_entries_with_callback together

* Add tx-index checks to test_rebatch_transactions

* Revert shuffle definition and use zip/unzip

* Only zip/unzip if randomize

* Add confirm_slot_entries test

* Review nits

* Add type alias to make sender docs more clear
2022-06-23 13:37:38 -06:00
Will Hickey 3096b64f9d
Update error that results when snapshot is missing (#24839) 2022-06-21 13:06:37 -05:00
Brooks Prumo b4b191e446
Enforce accounts data size limit per block in ReplayStage (#25524) 2022-06-15 20:35:33 -05:00
Tyera Eulberg 8a3d48b0ee
Reduce 2 iterators to one (#25973) 2022-06-14 22:49:58 +00:00
Michael Vines b4237f3f2c Do not exclude failed simple vote transactions from consensus 2022-06-12 22:11:23 -07:00
Yueh-Hsuan Chiang 5b67960c76
(Refactor) Move blocktore options related stuff to blockstore_options.rs (#25509)
#### Problem
blockstore_db.rs has a mutual dependency between blockstore_metrics.rs.

#### Summary of Changes
This PR removes the mutual dependency by moving the option-related stuff
out from blockstore_db.rs to its new home --- blockstore_options.rs.

By doing this, we address the mutual dependency and also make the code cleaner.
2022-05-26 16:59:26 -07:00
Jeff Washington (jwash) 41f30a2383
stop logging misleading bank hash mismatch (#25427) 2022-05-23 08:43:25 -05:00
Michael Vines c54e06355f
voteSubscribe pubsub notification now includes the vote transaction signature (#25291) 2022-05-19 18:28:46 -07:00
Jason 08da486c05
additional costs in block capacity calc (#25059)
* Added additional costs to block capacity computation, and pushed alloc of CostModel all the way to the top of the call chain, instead of reallocing

* Fix two compiler errors

* Update block processing to propagate computed costs, rather than re-computing deeper in the call stack

* Clippy fix

* Reformatting fix after merge

* Add CostModel::sum_without_bpf
2022-05-12 13:52:20 -05:00
Pankaj Garg c838e15234
Unset needs_unlock for rebatched transactions batches (#25095)
* Unset needs_unlock for rebatched transactions batches

* address review comments
2022-05-10 13:39:08 -07:00
behzad nouri 492f89a170
checks account owner when initializing a vote-account (#25018)
A VoteAccount may only wrap an account if the account owner is
solana_vote_program:id or equivalently this check returns true:
solana_vote_program::check_id(account.owner())
2022-05-06 16:22:49 +00:00
behzad nouri a01291069a
initializes thread-pools with lazy_static instead of thread_local (#24853)
In addition to thread_local -> lazy_static change, a number of thread-pools are
initialized with get_max_thread_count to achieve parity with the older code in
terms of number of validator threads.
2022-05-05 20:00:50 +00:00
Jeff Washington (jwash) e8bf812a8a
allow ledger-tool halt at slot to calc hash using write cache (#24974) 2022-05-04 16:27:49 -05:00
steviez cb96edcaa7
Pass ProcessOptions struct around by reference (#24900) 2022-05-02 16:58:00 -05:00
steviez 428cf54c91
Change BlockStore TryPrimaryThenSecondary to just Secondary (#23391) 2022-04-29 20:05:39 -05:00
Justin Starry 4e58b3870c
Update all BankForks methods to return owned values (#24801) 2022-04-28 18:51:00 +00:00
Justin Starry b6bc5d2fa1 Aggregate executor update timing metrics 2022-04-26 23:15:22 +08:00
Michael Vines 83e041299a Run real snapshot packager while processing blockstore at validator startup 2022-04-22 21:17:49 -07:00
Justin Starry d5127abf46
Only add hashes for completed blocks to recent blockhashes (#24389)
* Only add hashes for completed blocks to recent blockhashes

* feedback
2022-04-21 21:05:29 +08:00
Michael Vines 9e4999ef6a Remove halt_at_slot from RuntimeConfig, it's not a runtime concern 2022-04-19 19:23:58 -07:00
Michael Vines c6f3da4879 blockstore_processor now accepts an Arc<Rwlock<BankForks>> 2022-04-19 19:23:58 -07:00
Justin Starry 4ed647d8ec
Test that tick slot hashes update the recent blockhash queue (#24242) 2022-04-16 00:30:20 +08:00
HaoranYi e3ef0741be
simplify bank drop calls (#24142)
* simplify bank drop calls

* clippy: import

* Update ledger/src/blockstore_processor.rs

Co-authored-by: Brooks Prumo <brooks@prumo.org>

* Update runtime/src/accounts_background_service.rs

Co-authored-by: Brooks Prumo <brooks@prumo.org>

* Update runtime/src/bank.rs

Co-authored-by: Brooks Prumo <brooks@prumo.org>

* cleanup

* format

* use msb of bank_id to indicates that we are dropping

* clippy

* restore bank id

* clippy

* revert is-serialized_with_abs flag

* assert

* clippy

* whitespace

* fix bank drop callback check

* more fix

* remove msb dropping implementation

* fix

Co-authored-by: Brooks Prumo <brooks@prumo.org>
2022-04-14 08:43:54 -05:00
Jon Cinque 9b8850f99e
test-validator: Add `--max-compute-units` flag (#24130)
* test-validator: Add `--max-compute-units` flag

* Add `RuntimeConfig` for tweaking runtime behavior

* Actually add the file

* Move RuntimeConfig to runtime
2022-04-12 02:28:10 +02:00