There are lots of string literals in the /validator directory,
including many for CLI help and error messages. Any string literals
that exceed 100 characters prevent rustfmt from running properly.
This change temporarily set format_string = true in rustfmt.toml, and
then ran the linter on the validator directory. This was followed up
by manually tweaking several strings that were already well crafted
for readability (and within the 100 character limit)
* Check feature_set for enable_partitioned_epoch_reward
* Keep common variable outside if case
* Keep common early return out of if case, since the first_slot_in_epoch must exist for partiion PDA to exist
* Get and parse epoch partition data PDA
* Find partition index for all addresses
* Pull relevant blocks and get rewards
* Reuse ordering and reformatting
* Remove feature deactivation from TestValidator
* Restore rewards iteration in first block in epoch for feature case to catch Voting rewards
* Add fn get_reward_map helper to dedupe code
* No need to start 2nd get_block_with_limit call with first block again
* Replace filter_map to parameterize RewardType filter expression
* Weird thing to make clippy and compiler agree (https://github.com/rust-lang/rust-clippy/issues/8098)
* Use activated_slot to ensure the right approach for past rewards epochs
* Update proto files with tonic-build v0.9.2
* Manually ignore invalid doc-tests
* Add new ReadRowsRequest fields
* Add LedgerStorageConfig::max_message_size and default value
* Add BigtableConnection::max_message_size and use on client creation
* Add max_message_size to RpcBigtableConfig and make const pub
* Add solana-validator cli arg
```
warning: in a `match` scrutinee, avoid complex blocks or closures with blocks; instead, move the block or closure higher and bind it with a `let`
--> validator/src/dashboard.rs:223:38
|
223 | match async move {
| ______________________________________^
224 | | let rpc_addr = admin_client.rpc_addr().await?;
225 | | let start_time = admin_client.start_time().await?;
226 | | Ok::<_, jsonrpc_core_client::RpcError>((rpc_addr, start_time))
227 | | }
| |_____________________^
|
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#blocks_in_conditions
= note: `#[warn(clippy::blocks_in_conditions)]` on by default
warning: `solana-validator` (lib) generated 1 warning
```
```
error: unnecessary use of `.write(true)` because there is `.append(true)`
--> logger/src/lib.rs:61:9
|
61 | .write(true)
| ^^^^^^^^^^^^ help: remove `.write(true)`
|
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#ineffective_open_options
= note: `-D clippy::ineffective-open-options` implied by `-D warnings`
= help: to override `-D warnings` add `#[allow(clippy::ineffective_open_options)]`
```
These services currently live in core/; however, they operate on the
ledger. Mores so, these two services operate on the blockstore only,
and not necessarily the entire ledger. So, it makes sense to move these
services out of core and into ledger. We've recently been doing similar
changes with breaking things out into individual crates in order to
reduce the scope of core.
So, this change moves the services from core/ to ledger/, and replaces
ledger with blockstore.
* allow test feature to skip rewrites
* hook up cli arg for test skip rewrites, update tests
* fix sanity checker
* add account hash to abi to fix a test
* reviews
* use hashmap to collect skip_rewrites. exclude skip_rewrites from dirty
pubkey set
* accumulate skipped_rewrite in reduce
* mutex
* fmt
* skip hash verify for this test flag
* add skipped rewrites num stat
* skip bank hash verify not account hash verify
* reviews
---------
Co-authored-by: HaoranYi <haoran.yi@solana.com>
* Initialize fork graph in program cache during bank_forks creation
* rename BankForks::new to BankForks::new_rw_arc
* fix compilation
* no need to set fork_graph on insert()
* fix partition tests
The current getHealth mechanism checks a local accounts hash slot vs.
those of other nodes as specified by --known-validator. This is a
very coarse comparison given that the default for this value is 100
slots. More so, any nodes using a value larger than the default
(ie --incremental-snapshot-interval 500) will likely see getHealth
return status behind at some point.
Change the underlying mechanism of how health is computed. Instead of
using the accounts hash slots published in gossip, use the latest
optimistically confirmed slot from the cluster. Even when a node is
behind, it is able to observe cluster optimistically confirmed by slots
by viewing votes published in gossip.
Thus, the latest cluster optimistically confirmed slot can be compared
against the latest optimistically confirmed bank from replay to
determine health. This new comparison is much more granular, and not
needing to depend on individual known validators is also a plus.
* Add wen_restart module:
- Implement reading LastVotedForkSlots from blockstore.
- Add proto file to record the intermediate results.
- Also link wen_restart into validator.
- Move recreation of tower outside replay_stage so we can get last_vote.
* Update lock file.
* Fix linter errors.
* Fix depencies order.
* Update wen_restart explanation and small fixes.
* Generate tower outside tvu.
* Update validator/src/cli.rs
Co-authored-by: Tyera <teulberg@gmail.com>
* Update wen-restart/protos/wen_restart.proto
Co-authored-by: Tyera <teulberg@gmail.com>
* Update wen-restart/build.rs
Co-authored-by: Tyera <teulberg@gmail.com>
* Update wen-restart/src/wen_restart.rs
Co-authored-by: Tyera <teulberg@gmail.com>
* Rename proto directory.
* Rename InitRecord to MyLastVotedForkSlots, add imports.
* Update wen-restart/Cargo.toml
Co-authored-by: Tyera <teulberg@gmail.com>
* Update wen-restart/src/wen_restart.rs
Co-authored-by: Tyera <teulberg@gmail.com>
* Move prost-build dependency to project toml.
* No need to continue if the distance between slot and last_vote is
already larger than MAX_SLOTS_ON_VOTED_FORKS.
* Use 16k slots instead of 81k slots, a few more wording changes.
* Use AncestorIterator which does the same thing.
* Update Cargo.lock
* Update Cargo.lock
---------
Co-authored-by: Tyera <teulberg@gmail.com>
* Revert "Add an RPC API that can query the list of Top N secondary index keys and their sizes (#28887)"
This reverts commit 1e3d6349aa.
* Revert "Add Admin RPC Front End for Top N Secondary Index Key Sizes Query. (#29352)"
This reverts commit aa353e4b83.
* fix test uses
* fmt
* allow pedantic invalid cast lint
* allow lint with false-positive triggered by `test-case` crate
* nightly `fmt` correction
* adapt to rust layout changes
* remove dubious test
* Use transmute instead of pointer cast and de/ref when check_aligned is false.
* Renames clippy::integer_arithmetic to clippy::arithmetic_side_effects.
* bump rust nightly to 2023-08-25
* Upgrades Rust to 1.72.0
---------
Co-authored-by: Trent Nelson <trent@solana.com>
* sdk: Add concurrent support for rand 0.7 and 0.8
* Update rand, rand_chacha, and getrandom versions
* Run command to replace `gen_range`
Run `git grep -l gen_range | xargs sed -i'' -e 's/gen_range(\(\S*\), /gen_range(\1../'
* sdk: Fix users of older `gen_range`
* Replace `hash::new_rand` with `hash::new_with_thread_rng`
Run:
```
git grep -l hash::new_rand | xargs sed -i'' -e 's/hash::new_rand([^)]*/hash::new_with_thread_rng(/'
```
* perf: Use `Keypair::new()` instead of `generate`
* Use older rand version in zk-token-sdk
* program-runtime: Inline random key generation
* bloom: Fix clippy warnings in tests
* streamer: Scope rng usage correctly
* perf: Fix clippy warning
* accounts-db: Map to char to generate a random string
* Remove `from_secret_key_bytes`, it's just `keypair_from_seed`
* ledger: Generate keypairs by hand
* ed25519-tests: Use new rand
* runtime: Use new rand in all tests
* gossip: Clean up clippy and inline keypair generators
* core: Inline keypair generation for tests
* Push sbf lockfile change
* sdk: Sort dependencies correctly
* Remove `hash::new_with_thread_rng`, use `Hash::new_unique()`
* Use Keypair::new where chacha isn't used
* sdk: Fix build by marking rand 0.7 optional
* Hardcode secret key length, add static assertion
* Unify `getrandom` crate usage to fix linking errors
* bloom: Fix tests that require a random hash
* Remove some dependencies, try to unify others
* Remove unnecessary uses of rand and rand_core
* Update lockfiles
* Add back some dependencies to reduce rebuilds
* Increase max rebuilds from 14 to 15
* frozen-abi: Remove `getrandom`
* Bump rebuilds to 17
* Remove getrandom from zk-token-proof
When a consensus divergance occurs, the current workflow involves a
handful of manual steps to hone in on the offending slot and
transaction. This process isn't overly difficult to execute; however, it
is tedious and currently involves creating and parsing logs.
This change introduces functionality to output a debug file that
contains the components go into the bank hash. The file can be generated
in two ways:
- Via solana-validator when the node realizes it has diverged
- Via solana-ledger-tool verify by passing a flag
When a divergance occurs now, the steps to debug would be:
- Grab the file from the node that diverged
- Generate a file for the same slot with ledger-tool with a known good
version
- Diff the files, they are pretty-printed json
Ryo Onodera
steviez
Brooks
Pankaj Garg
Kirill Fomichev
Tyera Eulberg
Brennan
Nick Frostbutter
Andrew Fitzgerald
ryleung-solana
Lucas Steuernagel
Trent Nelson
steviez
Jeff Washington (jwash)
Wen
DimAn
Alexander Meißner
Jon Cinque
behzad nouri