Commit Graph

3837 Commits

Author SHA1 Message Date
steviez 3eae980293
Minor cleanup on some snapshot related tests (#33485) 2023-10-02 20:25:30 +02:00
Andrew Fitzgerald 660e41a8e1
Remove entry shuffling (#33378) 2023-10-02 09:03:12 -07:00
carllin ec2e1241a1
Cleanup select_vote_and_reset_forks() (#33421) 2023-09-29 15:11:25 -07:00
Andrew Fitzgerald e3cd13e49d
Add new received forwarded packets metric to banking stage (#33414) 2023-09-28 09:25:10 -07:00
Tyera ddd029774a
Add geyser block-metadata notification with entry count (#33359)
* Add new ReplicaBlockInfoVersions variant

* Use new variant to return entry count
2023-09-26 10:13:17 -06:00
Tao Zhu a41c15e47e
Separate vote cost (#33230)
* Separate simple-vote transaction cost from non-vote transaction cost

* remove is_simple_vote flag from transaction UsageCostDetails

* update test and comment

* set static usage cost for SimpleVote transaction
2023-09-25 15:02:08 -05:00
Ashwin Sekar 85cc6ace05
Update is_locked_out cache when adopting on chain vote state (#33341)
* Update is_locked_out cache when adopting on chain vote state

* extend to all cached tower checks

* upgrade error to panic
2023-09-25 12:33:38 -07:00
Pankaj Garg f50342a790
Split vote related code from runtime to its own crate (#32882)
* Move vote related code to its own crate

* Update imports in code and tests

* update programs/sbf/Cargo.lock

* fix check errors

* update abi_digest

* rebase fixes

* fixes after rebase
2023-09-19 10:46:37 -07:00
Andrew Fitzgerald 22338f5472
TransactionScheduler: InFlightTracker (#33206) 2023-09-19 09:08:42 -07:00
Brooks bc2b372762
Adds DCOU to verify_snapshot_archive() (#33298) 2023-09-18 19:11:28 -04:00
Andrew Fitzgerald 86dd18bfb5
TransactionScheduler: Id Generators (#33207) 2023-09-18 10:07:40 -07:00
Andrew Fitzgerald e860019687
TransactionScheduler: Pipe BlockProductionMethod (#33217) 2023-09-18 10:05:27 -07:00
Brooks c40e88aef9
Adds metrics for fastboot storages kept alive (#33222) 2023-09-13 09:28:19 -04:00
Brooks acd7ad96c3
Purges old accounts hash cache dirs (#33183) 2023-09-12 13:10:22 -04:00
behzad nouri e01269a9de
sends repair requests over QUIC protocol (#33016)
The commit implements client-side of serve-repair and
ancestor-hash-service over QUIC protocol.
2023-09-11 22:22:04 +00:00
Alexander Meißner bbb57be0a5
Refactor - Move interfaces of address-lookup-table into the program SDK (#33165)
* Adds a module `address_lookup_table` to the SDK.

* Adds a module `address_lookup_table::instruction` to the SDK.

* Adds a module `address_lookup_table::error` to the SDK.

* Adds a module `address_lookup_table::state` to the SDK.

* Moves AddressLookupTable into SDK as well.

* Moves AddressLookupTableAccount into address_lookup_table.

* Adds deprecation messages.

* Disentangles dependencies across cargo files.
2023-09-11 21:10:40 +02:00
behzad nouri 7fc6fea8d8
serves remote repair requests from QUIC endpoint (#33069)
The commit implements server-side of repair using QUIC protocol.

UDP repair requests are adapted as RemoteRequest and sent down the same
channel as remote requests arriving over QUIC, and the rest of the
server code is update to process over RemoteRequest type.
2023-09-11 16:57:10 +00:00
Andrew Fitzgerald 297ffad797
set_forwarded needs mut ref (#33203) 2023-09-11 09:34:48 -07:00
Andrew Fitzgerald 527a4bbf00
TransactionScheduler: TransactionStateContainer (#33002)
Co-authored-by: Tao Zhu <82401714+taozhu-chicago@users.noreply.github.com>
2023-09-09 09:51:23 -07:00
behzad nouri 9ff0b35f29
adds QUIC endpoint for repair service (#33057)
Working towards using QUIC protocol for repair, the commit adds a QUIC
endpoint for repair service.

Outgoing local requests are sent as

    struct LocalRequest {
        remote_address: SocketAddr,
        bytes: Vec<u8>,
        num_expected_responses: usize,
        response_sender: Sender<(SocketAddr, Vec<u8>)>,
    }

to the client-side of the endpoint. The client opens a bidirectional
stream with the LocalRequest.remote_address and once received the
response, sends it down the LocalRequest.response_sender channel.

Incoming requests from remote nodes are received from bidirectional
streams and sent as

    struct RemoteRequest {
        remote_pubkey: Option<Pubkey>,
        remote_address: SocketAddr,
        bytes: Vec<u8>,
        response_sender: Option<OneShotSender<Vec<Vec<u8>>>>,
    }

to the repair-service. The response is received from the receiver end of
RemoteRequest.response_sender channel and send back to the remote node
using the send side of the bidirectional stream.
2023-09-07 18:00:25 +00:00
behzad nouri 528a03f32a
removes outdated matches crate from dependencies (#33172)
removes outdated matches crate from the dependencies

std::matches has been stable since rust 1.42.0.
Other use-cases are covered by assert_matches crate.
2023-09-07 12:52:57 +00:00
Ashwin Sekar a8e83c8720
replay: send duplicate proofs from blockstore to state machine (#32962)
* replay: send duplicate proofs from blockstore to state machine

* pr feedback: bank.slot() -> slot

* pr feedback
2023-09-05 21:29:53 -07:00
Andrew Fitzgerald f8d304c610
Drop poh_service to avoid unwanted ticking (#33150) 2023-09-05 16:08:48 -07:00
steviez 3b108564f9
Demote Arc<Bank> parameter to &Bank (#33130) 2023-09-05 21:04:39 +02:00
Andrew Fitzgerald d36ded20fc
Make TimedTracedEvent accessible outside banking_trace.rs (#32985) 2023-09-05 08:54:32 -07:00
steviez ad33c68ce9
Update ShredFetchStage::modify_packets to drop root bank quicker (#33105)
This function used to contain feature gate activation checks that
required access to a bank. Those checks have been cleaned up, so we no
longer need access to a full Bank. Rather, we can momentarily get a Bank
from BankForks, calculate the necessary results and then drop the Bank
along with the BankForks read lock.
2023-09-01 23:08:49 +02:00
Alessandro Decina 16ea141402
sdk: put AccountSharedData::set_data() behind feature="dev-context-only-utils" (#33086)
set_data() is deprecated in favor of set_data_from_slice().
2023-09-01 21:02:14 +07:00
Alexander Meißner 9e703f85de
Upgrades Rust to 1.72.0 & nightly-2023-08-25 (#32961)
* allow pedantic invalid cast lint

* allow lint with false-positive triggered by `test-case` crate

* nightly `fmt` correction

* adapt to rust layout changes

* remove dubious test

* Use transmute instead of pointer cast and de/ref when check_aligned is false.

* Renames clippy::integer_arithmetic to clippy::arithmetic_side_effects.

* bump rust nightly to 2023-08-25

* Upgrades Rust to 1.72.0

---------

Co-authored-by: Trent Nelson <trent@solana.com>
2023-09-01 07:26:13 +00:00
behzad nouri 1431275328
removes outdated check for merkle shreds (#33088) 2023-08-31 20:01:54 +00:00
Ashwin Sekar 6d2dae6ab0
send duplicate shred proofs for conflicting shred scenarios (#32965)
* send duplicate shred proofs for conflicting shred scenarios

The scenarios are multiple last_shred_in_slot shreds and
coding shreds with conflicting erasure metas.

* Pr feedback: deprecate shred_index for gossip duplicate shred proofs

* Pr feedback

* Remove extraneous dead code
2023-08-31 12:08:10 -07:00
Ashwin Sekar 025651e0d4
ff cleanup: allow_votes_to_directly_update_vote_state and compact_vot… (#32967)
ff cleanup: allow_votes_to_directly_update_vote_state and compact_vote_state_updates
2023-08-30 17:00:19 -07:00
behzad nouri 4ec5ea6f7b
replaces assert!(matches!(...)) with assert_matches!(...) (#33068)
assert_matches!(...) provides more informative error message when it
fails and it is part of nightly rust:
https://doc.rust-lang.org/std/assert_matches/macro.assert_matches.html
2023-08-30 13:48:27 -04:00
steviez 611b7527e5
Cleanup BankForks access code in ShredFetchStage (#33066)
- BankForks is not an optional argument, so remove dated comment
- Given that BankForks is always present, no need for special values to
  initialize variables before the loop
- Root slot can be retrieved from root bank, no need to call
  BankForks::root() which will load the underlying atomic a second time
- Use BankForks::highest_slot() instead of .slot() on .working_bank() to
  avoid the extra clone that .working_bank() performs
- Move several operations outside of BankForks read lock scope to
  minimize lock time
2023-08-30 15:03:02 +02:00
Trent Nelson b8dc5daedb
preliminaries for bumping nightly to 2023-08-25 (#33047)
* remove unnecessary hashes around raw string literals

* remove unncessary literal `unwrap()`s

* remove panicking `unwrap()`

* remove unnecessary `unwrap()`

* use `[]` instead of `vec![]` where applicable

* remove (more) unnecessary explicit `into_iter()` calls

* remove redundant pattern matching

* don't cast to same type and constness

* do not `cfg(any(...` a single item

* remove needless pass by `&mut`

* prefer `or_default()` to `or_insert_with(T::default())`

* `filter_map()` better written as `filter()`

* incorrect `PartialOrd` impl on `Ord` type

* replace "slow zero-filled `Vec` initializations"

* remove redundant local bindings

* add required lifetime to associated constant
2023-08-29 23:05:35 +00:00
behzad nouri 1171002f46
adds repair-protocol identifying serve-repair socket (#33028)
Working towards migrating repair over QUIC protocol, the commit adds
repair-protocol argument for identifying right serve-repair socket.
2023-08-28 21:34:09 +00:00
Pankaj Garg dbe4017143
Prune programs deployed in duplicate unconfirmed slot (#32999)
* Prune programs deployed in duplicate unconfirmed slot

* unit test
2023-08-25 11:02:20 -07:00
Alexander Meißner a8be70fa7a
General cleanup (#32980)
* Fixes `cargo::` in build.rs

* Fixes overshadowing reexports.

* Removes unnecessary `as *const u8`.

* Removes unnecessary `.into_iter()`.

* cargo clippy
2023-08-24 21:44:19 +02:00
Ashwin Sekar 329c6f131b
tower: when syncing from vote state, update last_vote (#32944)
* tower: when syncing from vote state, update last_vote

* pr: bubble error through unchecked
2023-08-23 13:15:57 -07:00
HaoranYi 2098230d8f
Improve Blockstore error logging (#32929)
* improve Blockstore error logging

* reviews

---------

Co-authored-by: HaoranYi <haoran.yi@solana.com>
2023-08-22 13:56:03 -05:00
Lijun Wang 98e19af5eb
Do not do send in cache warmer -- just establish connections (#32898)
* Do not do send in cache warmer -- just establish connections
2023-08-21 12:29:45 -07:00
Jon Cinque 0fe902ced7
Bump rand to 0.8, rand_chacha to 0.3, getrandom to 0.2 (#32871)
* sdk: Add concurrent support for rand 0.7 and 0.8

* Update rand, rand_chacha, and getrandom versions

* Run command to replace `gen_range`

Run `git grep -l gen_range | xargs sed -i'' -e 's/gen_range(\(\S*\), /gen_range(\1../'

* sdk: Fix users of older `gen_range`

* Replace `hash::new_rand` with `hash::new_with_thread_rng`

Run:
```
git grep -l hash::new_rand | xargs sed -i'' -e 's/hash::new_rand([^)]*/hash::new_with_thread_rng(/'
```

* perf: Use `Keypair::new()` instead of `generate`

* Use older rand version in zk-token-sdk

* program-runtime: Inline random key generation

* bloom: Fix clippy warnings in tests

* streamer: Scope rng usage correctly

* perf: Fix clippy warning

* accounts-db: Map to char to generate a random string

* Remove `from_secret_key_bytes`, it's just `keypair_from_seed`

* ledger: Generate keypairs by hand

* ed25519-tests: Use new rand

* runtime: Use new rand in all tests

* gossip: Clean up clippy and inline keypair generators

* core: Inline keypair generation for tests

* Push sbf lockfile change

* sdk: Sort dependencies correctly

* Remove `hash::new_with_thread_rng`, use `Hash::new_unique()`

* Use Keypair::new where chacha isn't used

* sdk: Fix build by marking rand 0.7 optional

* Hardcode secret key length, add static assertion

* Unify `getrandom` crate usage to fix linking errors

* bloom: Fix tests that require a random hash

* Remove some dependencies, try to unify others

* Remove unnecessary uses of rand and rand_core

* Update lockfiles

* Add back some dependencies to reduce rebuilds

* Increase max rebuilds from 14 to 15

* frozen-abi: Remove `getrandom`

* Bump rebuilds to 17

* Remove getrandom from zk-token-proof
2023-08-21 19:11:21 +02:00
Brooks a8c1dabf6a
Renames AccountsPackageType to AccountsPackageKind (#32908) 2023-08-21 13:00:00 -04:00
Brooks c6989189ef
Renames CalcAccountsHashFlavor to CalcAccountsHashKind (#32905) 2023-08-21 15:41:07 +00:00
Brooks a563b4cccb
Renames AccountsHashEnum to AccountsHashKind (#32904) 2023-08-21 10:34:56 -04:00
Brooks f9bc3cec24
Renames SnapshotType to SnapshotKind (#32906) 2023-08-19 19:55:29 -04:00
behzad nouri 7bd7410592
uses typed Pubkey instead of opaque [u8; 32] (#32842) 2023-08-18 23:28:08 +00:00
steviez a4c8cc3ce0
Remove improper uses of &Arc<Bank> (#32802)
In most cases, either a &Bank or an Arc<Bank> is more proper.
- &Bank is used if the function only needs a momentary reference
- Arc<Bank> is used if the function needs its' own copy

This PR leaves several instances of &Arc<Bank> around; these instances
are situations where a clone may only happen conditionally.
2023-08-18 16:46:34 -05:00
Jeff Biseda 13b7ae1432
legacy repair type cleanup (#32849) 2023-08-17 11:11:35 -07:00
behzad nouri 0de8ccfda9
adds socket address for repair service over QUIC (#32834)
Working towards migrating repair to QUIC.
2023-08-15 17:09:09 +00:00
steviez 6bbf514e78
Add ability to output components that go into Bank hash (#32632)
When a consensus divergance occurs, the current workflow involves a
handful of manual steps to hone in on the offending slot and
transaction. This process isn't overly difficult to execute; however, it
is tedious and currently involves creating and parsing logs.

This change introduces functionality to output a debug file that
contains the components go into the bank hash. The file can be generated
in two ways:
- Via solana-validator when the node realizes it has diverged
- Via solana-ledger-tool verify by passing a flag

When a divergance occurs now, the steps to debug would be:
- Grab the file from the node that diverged
- Generate a file for the same slot with ledger-tool with a known good
  version
- Diff the files, they are pretty-printed json
2023-08-15 00:12:05 -05:00