Bubble up the constants to the CLI that control the sizes of the
following two thread pools:
- The thread pool used to replay multiple forks in parallel
- The thread pool used to execute transactions in parallel
The default value was previously being determined down where the thread
pool is being created. Providing a default value at the CLI level is
consistent with other args, and gives an operator better visibility into
what the default will actually be
In cluster restart scenarios, an important step is scanning the
Blockstore for blocks that occur after the chosen restart slot with an
incorrect shred version. This check ensures that any blocks that
occurred pre-cluster restart and after the chosen restart slot get
deleted. If a node skips this step, the node can encounter problems when
that block is created again, after the cluster has restarted.
This check only occurs if --wait-for-supermajority AND
--expected-shred-version are set; however, --expected-... is currently
optional when using --wait-...
Our restart instructions typically mention that one should specify
--expected-... as well, but we should just enforce it at the CLI level
to prevent mistakes / wasted time debuggging.
RuntimeConfig doesn't use anything SVM specific and logically belongs
in program runtime rather than SVM. This change moves the definition
of RuntimeConfig struct from the SVM crate to program-runtime and
adjusts `use` statements accordingly.
There are lots of string literals in the /validator directory,
including many for CLI help and error messages. Any string literals
that exceed 100 characters prevent rustfmt from running properly.
This change temporarily set format_string = true in rustfmt.toml, and
then ran the linter on the validator directory. This was followed up
by manually tweaking several strings that were already well crafted
for readability (and within the 100 character limit)
* Check feature_set for enable_partitioned_epoch_reward
* Keep common variable outside if case
* Keep common early return out of if case, since the first_slot_in_epoch must exist for partiion PDA to exist
* Get and parse epoch partition data PDA
* Find partition index for all addresses
* Pull relevant blocks and get rewards
* Reuse ordering and reformatting
* Remove feature deactivation from TestValidator
* Restore rewards iteration in first block in epoch for feature case to catch Voting rewards
* Add fn get_reward_map helper to dedupe code
* No need to start 2nd get_block_with_limit call with first block again
* Replace filter_map to parameterize RewardType filter expression
* Weird thing to make clippy and compiler agree (https://github.com/rust-lang/rust-clippy/issues/8098)
* Use activated_slot to ensure the right approach for past rewards epochs
* Update proto files with tonic-build v0.9.2
* Manually ignore invalid doc-tests
* Add new ReadRowsRequest fields
* Add LedgerStorageConfig::max_message_size and default value
* Add BigtableConnection::max_message_size and use on client creation
* Add max_message_size to RpcBigtableConfig and make const pub
* Add solana-validator cli arg
```
warning: in a `match` scrutinee, avoid complex blocks or closures with blocks; instead, move the block or closure higher and bind it with a `let`
--> validator/src/dashboard.rs:223:38
|
223 | match async move {
| ______________________________________^
224 | | let rpc_addr = admin_client.rpc_addr().await?;
225 | | let start_time = admin_client.start_time().await?;
226 | | Ok::<_, jsonrpc_core_client::RpcError>((rpc_addr, start_time))
227 | | }
| |_____________________^
|
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#blocks_in_conditions
= note: `#[warn(clippy::blocks_in_conditions)]` on by default
warning: `solana-validator` (lib) generated 1 warning
```
```
error: unnecessary use of `.write(true)` because there is `.append(true)`
--> logger/src/lib.rs:61:9
|
61 | .write(true)
| ^^^^^^^^^^^^ help: remove `.write(true)`
|
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#ineffective_open_options
= note: `-D clippy::ineffective-open-options` implied by `-D warnings`
= help: to override `-D warnings` add `#[allow(clippy::ineffective_open_options)]`
```
These services currently live in core/; however, they operate on the
ledger. Mores so, these two services operate on the blockstore only,
and not necessarily the entire ledger. So, it makes sense to move these
services out of core and into ledger. We've recently been doing similar
changes with breaking things out into individual crates in order to
reduce the scope of core.
So, this change moves the services from core/ to ledger/, and replaces
ledger with blockstore.
* allow test feature to skip rewrites
* hook up cli arg for test skip rewrites, update tests
* fix sanity checker
* add account hash to abi to fix a test
* reviews
* use hashmap to collect skip_rewrites. exclude skip_rewrites from dirty
pubkey set
* accumulate skipped_rewrite in reduce
* mutex
* fmt
* skip hash verify for this test flag
* add skipped rewrites num stat
* skip bank hash verify not account hash verify
* reviews
---------
Co-authored-by: HaoranYi <haoran.yi@solana.com>
* Initialize fork graph in program cache during bank_forks creation
* rename BankForks::new to BankForks::new_rw_arc
* fix compilation
* no need to set fork_graph on insert()
* fix partition tests
The current getHealth mechanism checks a local accounts hash slot vs.
those of other nodes as specified by --known-validator. This is a
very coarse comparison given that the default for this value is 100
slots. More so, any nodes using a value larger than the default
(ie --incremental-snapshot-interval 500) will likely see getHealth
return status behind at some point.
Change the underlying mechanism of how health is computed. Instead of
using the accounts hash slots published in gossip, use the latest
optimistically confirmed slot from the cluster. Even when a node is
behind, it is able to observe cluster optimistically confirmed by slots
by viewing votes published in gossip.
Thus, the latest cluster optimistically confirmed slot can be compared
against the latest optimistically confirmed bank from replay to
determine health. This new comparison is much more granular, and not
needing to depend on individual known validators is also a plus.
* Add wen_restart module:
- Implement reading LastVotedForkSlots from blockstore.
- Add proto file to record the intermediate results.
- Also link wen_restart into validator.
- Move recreation of tower outside replay_stage so we can get last_vote.
* Update lock file.
* Fix linter errors.
* Fix depencies order.
* Update wen_restart explanation and small fixes.
* Generate tower outside tvu.
* Update validator/src/cli.rs
Co-authored-by: Tyera <teulberg@gmail.com>
* Update wen-restart/protos/wen_restart.proto
Co-authored-by: Tyera <teulberg@gmail.com>
* Update wen-restart/build.rs
Co-authored-by: Tyera <teulberg@gmail.com>
* Update wen-restart/src/wen_restart.rs
Co-authored-by: Tyera <teulberg@gmail.com>
* Rename proto directory.
* Rename InitRecord to MyLastVotedForkSlots, add imports.
* Update wen-restart/Cargo.toml
Co-authored-by: Tyera <teulberg@gmail.com>
* Update wen-restart/src/wen_restart.rs
Co-authored-by: Tyera <teulberg@gmail.com>
* Move prost-build dependency to project toml.
* No need to continue if the distance between slot and last_vote is
already larger than MAX_SLOTS_ON_VOTED_FORKS.
* Use 16k slots instead of 81k slots, a few more wording changes.
* Use AncestorIterator which does the same thing.
* Update Cargo.lock
* Update Cargo.lock
---------
Co-authored-by: Tyera <teulberg@gmail.com>
* Revert "Add an RPC API that can query the list of Top N secondary index keys and their sizes (#28887)"
This reverts commit 1e3d6349aa.
* Revert "Add Admin RPC Front End for Top N Secondary Index Key Sizes Query. (#29352)"
This reverts commit aa353e4b83.
* fix test uses
* fmt
GHA: Update Upstream From Fork
steviez
Trent Nelson
Dmitri Makarov
Yihau Chen
Ryo Onodera
Brooks
Pankaj Garg
Kirill Fomichev
Tyera Eulberg
Brennan
Nick Frostbutter
Andrew Fitzgerald
ryleung-solana
Lucas Steuernagel
Trent Nelson
steviez
Jeff Washington (jwash)
Wen
DimAn