Commit Graph

405 Commits

Author SHA1 Message Date
Greg Cusack 1261b3d496
gossip test update (#33431)
fix bug in gossip test
2023-09-29 08:57:32 -07:00
Pankaj Garg f50342a790
Split vote related code from runtime to its own crate (#32882)
* Move vote related code to its own crate

* Update imports in code and tests

* update programs/sbf/Cargo.lock

* fix check errors

* update abi_digest

* rebase fixes

* fixes after rebase
2023-09-19 10:46:37 -07:00
behzad nouri 528a03f32a
removes outdated matches crate from dependencies (#33172)
removes outdated matches crate from the dependencies

std::matches has been stable since rust 1.42.0.
Other use-cases are covered by assert_matches crate.
2023-09-07 12:52:57 +00:00
Alexander Meißner 9e703f85de
Upgrades Rust to 1.72.0 & nightly-2023-08-25 (#32961)
* allow pedantic invalid cast lint

* allow lint with false-positive triggered by `test-case` crate

* nightly `fmt` correction

* adapt to rust layout changes

* remove dubious test

* Use transmute instead of pointer cast and de/ref when check_aligned is false.

* Renames clippy::integer_arithmetic to clippy::arithmetic_side_effects.

* bump rust nightly to 2023-08-25

* Upgrades Rust to 1.72.0

---------

Co-authored-by: Trent Nelson <trent@solana.com>
2023-09-01 07:26:13 +00:00
behzad nouri 39615bd075
removes manual let...else (#33089)
https://rust-lang.github.io/rust-clippy/master/index.html#/manual_let_else
2023-08-31 22:35:47 +00:00
Ashwin Sekar 6d2dae6ab0
send duplicate shred proofs for conflicting shred scenarios (#32965)
* send duplicate shred proofs for conflicting shred scenarios

The scenarios are multiple last_shred_in_slot shreds and
coding shreds with conflicting erasure metas.

* Pr feedback: deprecate shred_index for gossip duplicate shred proofs

* Pr feedback

* Remove extraneous dead code
2023-08-31 12:08:10 -07:00
Trent Nelson b13589b588
round two preliminaries for bumping nightly to 2023-08-25 (#33064)
* work around nightly ICE

seems to be something related to `const ref`s in generic contexts...

* allow lint false-positive on ignored trait bounds
2023-08-30 21:46:20 +02:00
Trent Nelson b8dc5daedb
preliminaries for bumping nightly to 2023-08-25 (#33047)
* remove unnecessary hashes around raw string literals

* remove unncessary literal `unwrap()`s

* remove panicking `unwrap()`

* remove unnecessary `unwrap()`

* use `[]` instead of `vec![]` where applicable

* remove (more) unnecessary explicit `into_iter()` calls

* remove redundant pattern matching

* don't cast to same type and constness

* do not `cfg(any(...` a single item

* remove needless pass by `&mut`

* prefer `or_default()` to `or_insert_with(T::default())`

* `filter_map()` better written as `filter()`

* incorrect `PartialOrd` impl on `Ord` type

* replace "slow zero-filled `Vec` initializations"

* remove redundant local bindings

* add required lifetime to associated constant
2023-08-29 23:05:35 +00:00
Alexander Meißner a8be70fa7a
General cleanup (#32980)
* Fixes `cargo::` in build.rs

* Fixes overshadowing reexports.

* Removes unnecessary `as *const u8`.

* Removes unnecessary `.into_iter()`.

* cargo clippy
2023-08-24 21:44:19 +02:00
Jon Cinque 0fe902ced7
Bump rand to 0.8, rand_chacha to 0.3, getrandom to 0.2 (#32871)
* sdk: Add concurrent support for rand 0.7 and 0.8

* Update rand, rand_chacha, and getrandom versions

* Run command to replace `gen_range`

Run `git grep -l gen_range | xargs sed -i'' -e 's/gen_range(\(\S*\), /gen_range(\1../'

* sdk: Fix users of older `gen_range`

* Replace `hash::new_rand` with `hash::new_with_thread_rng`

Run:
```
git grep -l hash::new_rand | xargs sed -i'' -e 's/hash::new_rand([^)]*/hash::new_with_thread_rng(/'
```

* perf: Use `Keypair::new()` instead of `generate`

* Use older rand version in zk-token-sdk

* program-runtime: Inline random key generation

* bloom: Fix clippy warnings in tests

* streamer: Scope rng usage correctly

* perf: Fix clippy warning

* accounts-db: Map to char to generate a random string

* Remove `from_secret_key_bytes`, it's just `keypair_from_seed`

* ledger: Generate keypairs by hand

* ed25519-tests: Use new rand

* runtime: Use new rand in all tests

* gossip: Clean up clippy and inline keypair generators

* core: Inline keypair generation for tests

* Push sbf lockfile change

* sdk: Sort dependencies correctly

* Remove `hash::new_with_thread_rng`, use `Hash::new_unique()`

* Use Keypair::new where chacha isn't used

* sdk: Fix build by marking rand 0.7 optional

* Hardcode secret key length, add static assertion

* Unify `getrandom` crate usage to fix linking errors

* bloom: Fix tests that require a random hash

* Remove some dependencies, try to unify others

* Remove unnecessary uses of rand and rand_core

* Update lockfiles

* Add back some dependencies to reduce rebuilds

* Increase max rebuilds from 14 to 15

* frozen-abi: Remove `getrandom`

* Bump rebuilds to 17

* Remove getrandom from zk-token-proof
2023-08-21 19:11:21 +02:00
behzad nouri a9ecdc0ae5
asserts that socket addresses set in new_with_external_ip are valid (#32860)
new_with_external_ip silently ignores errors when setting socket
addresses which makes it harder to catch bugs:
https://github.com/solana-labs/solana/blob/7beeb8310/gossip/src/cluster_info.rs#L3054-L3063
2023-08-16 20:41:40 +00:00
behzad nouri 7beeb83104
patches set_serve_repair instead of set_serve_repair_quic (#32856) 2023-08-16 18:26:00 +00:00
behzad nouri 0de8ccfda9
adds socket address for repair service over QUIC (#32834)
Working towards migrating repair to QUIC.
2023-08-15 17:09:09 +00:00
behzad nouri 35ed7122f6
reuses old TVU-forwards socket tag for TVU over QUIC (#32828)
The commit revises socket tag for TVU over QUIC to reuse old
TVU-forwards socket tag. This avoids gaps in socket tags which wastes
space in ContactInfo.cache.
The change is backward compatible because TVU-forwards is not used any
more and TVU over QUIC is not released yet.
2023-08-15 12:19:10 +00:00
behzad nouri 52616cf7aa
removes repair socket from gossip ContactInfo (#32831)
Repair responses are sent back to the address the repair request came
from and there is no need to gossip repair socket address.
https://github.com/solana-labs/solana/blob/9212ac347/core/src/repair/serve_repair.rs#L519
2023-08-14 20:01:18 +00:00
behzad nouri b06500ee66
adds extensions to contact-info (#32309)
The commit adds a Vec<Extension> to ContactInfo so that future additions
to ContactInfo can be done by only adding new Extensions instead of
modifying the entire ContactInfo.
2023-08-09 22:01:20 +00:00
behzad nouri b7c2ad5b67
repurposes tvu_forwards socket for TVU over QUIC (#32737)
LegacyContactInfo.tvu_forwards is unused.
Repurposing the field for TVU over QUIC will avoid QUIC_PORT_OFFSET hack
in a backward compatible way.
2023-08-07 22:02:41 +00:00
behzad nouri 0511753276
moves DATA_PLANE_FANOUT to turbine (#32744) 2023-08-07 20:53:49 +00:00
Greg Cusack 5c86f89bc7
Add from field for message tracking (#32725)
* we only want to report received message signatures on PUSH requests, not PULL requests

* woops accidently had it has LocalMessage not PushMessage

* switch from match to if let statement

* convert if let to matches macro

* add in from field in PushMessage for message tracking

* update with cargo fmt

* remove display for gossip route and add lifetime param to pubkey reference in gossiproute enum

* forgot to run fmt
2023-08-07 11:40:18 -07:00
Greg Cusack 849525735f
report received message signatures only on PUSH requests (#32708)
* we only want to report received message signatures on PUSH requests, not PULL requests

* woops accidently had it has LocalMessage not PushMessage

* switch from match to if let statement

* convert if let to matches macro
2023-08-06 20:36:23 -07:00
Greg Cusack cd7f03bc88
remove duplicate metric reporting in cluster_info_stats3 (#32566)
process_gossip_packets_iterations_since_last_report measures the same loop count as gossip_listen_loop_iterations_since_last_report which already exists
2023-07-24 09:01:07 -06:00
Greg Cusack 80f708298b
Report new received crds signatures and their respective origins to metrics (#32504)
* screwed up old branch and syncing with upstream branch

* add fixed size ring buff instead of variable sized vecdeque for reporting signatures

* modify difficulty to take in n 0 bits and check if signature ending ends in n 0 bits

* update to only push every 18 trailing zero bits. and clean up

* report origin with signature. and set trailing 0s to 8 for local testing

* change back to 18 trailing zeros and rm unused imports

* run cargo rmt

* run ./scripts/cargo-for-all-lock-files.sh tree

* allow integer arithmetic for bit comparison

* rm unused lifetime

* rm duplicate entry?

* re implement ring buf

* put ringbuf in sorted order

* ringbuf in cargo.toml now in sorted order

* rm ring buf, refactor, fix trailing zero bug

* fix bug in trailing zeros. was comparing wrong ones

* fix needless range loop bug

* fix bug

* change trailing zero checking to build in methods and only report first 8 bytes of signature and origin pubkey

* report full origin string and first 8 bytes of signature

* set SIGNATURE_SAMPLE_TRAILING_ZEROS back to 18

* forgot to run cargo tree

* avoid panic and change working

* rm bs58

* pass in Option<String> into datapoint_info

* shorten metric names
2023-07-20 14:33:40 -07:00
steviez d8516eea4c
Remove unnecessary clone of shred payload (#32540) 2023-07-19 15:34:16 -05:00
behzad nouri d54b6204be
removes instances of clippy::manual_let_else (#32417) 2023-07-09 21:41:36 +00:00
behzad nouri 0da01270ef
removes redundant recycler clones (#32401) 2023-07-06 18:25:20 +00:00
Greg Cusack b8222b230c
Increase push fanout from 6 to 9 (#32323)
* increase push fanout from 6 to 9. leave active set size at 12

* forgot to change comment to reflect push fanout increase
2023-06-28 19:03:12 -05:00
behzad nouri 469661d217
removes outdated tvu_forward socket (#32101)
Shreds are no longer sent to tvu_forward socket.
2023-06-20 20:50:16 +00:00
behzad nouri 987e8eeeaf
removes feature gate code dropping redundant turbine path (#32075) 2023-06-16 19:53:05 +00:00
behzad nouri 3ddb8babc8
adds const SOCKET_ADDR_UNSPECIFIED (#32102)
const socket-addr is stable since rust 1.69.0
2023-06-13 19:34:59 +00:00
behzad nouri aed4ecb633
adds quic receiver to shred-fetch-stage (#31576)
Working towards migrating turbine to QUIC.
2023-06-12 13:16:27 +00:00
Illia Bobyr 4353ac6797
Pass Arc<AtomicBool> by value, not by reference. (#31916)
`Arc` is already a reference internally, so it does not seem to be
beneficial to pass a reference to it.  Just adds an extra layer of
indirection.

Functions that need to be able to increment `Arc` reference count need
to take `Arc<AtomicBool>`, but those that just want to read the
`AtomicBool` value can accept `&AtomicBool`, making them a bit more
generic.

This change focuses specifically on `Arc<AtomicBool>`.  There are other
uses of `&Arc<T>` in the code base that could be converted in a similar
manner.  But it would make the change even larger.
2023-06-01 17:25:48 -07:00
behzad nouri 5178d4d49b
adds quic tvu port to contact-info (#31614)
Working towards migrating turbine to QUIC.
2023-05-15 15:13:21 +00:00
behzad nouri 4e34abbf3d
specifies protocol in contact-info get-socket api (#31602) 2023-05-12 16:16:20 +00:00
Jon Cinque c0e107a97d
gossip-service: Spy multiple nodes by pubkey at once (#31597)
* gossip: Spy multiple pubkeys

* Address behzad's feedback
2023-05-11 18:24:35 +02:00
behzad nouri 6a4a0418a6
removes hard-coded QUIC_PORT_OFFSET from connection-cache (#31541)
New ContactInfo has api identifying QUIC vs UDP ports; no need to hard-code
port-offset deep in connection-cache.
2023-05-09 13:46:17 +00:00
Illia Bobyr 2d090d4547
test_gossip_node: Use random port (#31490)
Using a fixed port could cause a false negative, if the port is
currently in use.  We actually see this test failing regularly with an
error that port `1111` is already in use.

Quick search did not show any tests that hardcode port 1111, so it is
unclear why is this happening.  But using hardcoded ports is not a good
practice anyways.
2023-05-05 18:47:24 -07:00
Brooks c5e071c7fe
Upgrades nightly Rust to 2023-03-04 (#31487) 2023-05-05 08:28:23 -04:00
behzad nouri aafcac27d8
removes pubkey from LegacyContactInfo public interface (#31375)
Working towards LegacyContactInfo => ContactInfo migration, the commit
adds more api parity between the two.
2023-04-28 12:05:15 +00:00
DimAn bd782634dc
Add ability to manage public TPU Forwards address (#31201)
A node operator can manage a public TPU address (at node startup and while it's running), but doesn't have the ability to manage the TPU Forwards address as well.

This PR adds that functionality.

Added the start argument --public-tpu-forwards-address
Reworked the set-public-tpu-address subcommand into the set-public-address subcommand
2023-04-24 23:04:36 +00:00
behzad nouri 1b08d01a80
removes shred_version from LegacyContactInfo public interface (#31304)
Working towards LegacyContactInfo => ContactInfo migration, the commit
adds more api parity between the two.
2023-04-24 15:19:33 +00:00
behzad nouri 33bde55bbd
adds tpu_quic field to RpcContactInfo (#31310) 2023-04-22 20:18:58 +00:00
behzad nouri a88024e295
removes wallclock from LegacyContactInfo public interface (#31303) 2023-04-22 20:18:39 +00:00
behzad nouri cb65a785bc
makes sockets in LegacyContactInfo private (#31248)
Working towards LegacyContactInfo => ContactInfo migration, the commit
hides some implementation details of LegacyContactInfo and expands API
parity with the new ContactInfo.
2023-04-21 15:39:16 +00:00
DimAn b81b7ebf03
validator: `--tpu-host-addr` -> `--public-tpu-address` (#31137) 2023-04-12 13:33:09 +00:00
DimAn 9136f80d36
validator: add `set-public-tpu-address` command (#30452) 2023-04-12 13:32:22 +00:00
Brooks e1cb8a672b
Renames SnapshotHashes in cluster info metrics (#31145) 2023-04-11 17:35:00 +00:00
Brooks 453f272698
Rename IncrementalSnapshotHashes to SnapshotHashes (#31136) 2023-04-11 10:30:29 -04:00
Brooks f3083ad2e0
Rename SnapshotHashes to LegacySnapshotHashes (#31086) 2023-04-10 17:52:20 -04:00
Brooks aa8d7de2d1
Uses impl_crds_entry! for SnapshotHashes (#31032) 2023-04-04 14:44:05 -04:00
Brooks e8ea722061
Uses AccountsHashes type for AccountsHashes CrdsData variant (#31003) 2023-04-03 16:42:21 -04:00