Commit Graph

389 Commits

Author SHA1 Message Date
behzad nouri b7c2ad5b67
repurposes tvu_forwards socket for TVU over QUIC (#32737)
LegacyContactInfo.tvu_forwards is unused.
Repurposing the field for TVU over QUIC will avoid QUIC_PORT_OFFSET hack
in a backward compatible way.
2023-08-07 22:02:41 +00:00
behzad nouri 0511753276
moves DATA_PLANE_FANOUT to turbine (#32744) 2023-08-07 20:53:49 +00:00
Greg Cusack 5c86f89bc7
Add from field for message tracking (#32725)
* we only want to report received message signatures on PUSH requests, not PULL requests

* woops accidently had it has LocalMessage not PushMessage

* switch from match to if let statement

* convert if let to matches macro

* add in from field in PushMessage for message tracking

* update with cargo fmt

* remove display for gossip route and add lifetime param to pubkey reference in gossiproute enum

* forgot to run fmt
2023-08-07 11:40:18 -07:00
Greg Cusack 849525735f
report received message signatures only on PUSH requests (#32708)
* we only want to report received message signatures on PUSH requests, not PULL requests

* woops accidently had it has LocalMessage not PushMessage

* switch from match to if let statement

* convert if let to matches macro
2023-08-06 20:36:23 -07:00
Greg Cusack cd7f03bc88
remove duplicate metric reporting in cluster_info_stats3 (#32566)
process_gossip_packets_iterations_since_last_report measures the same loop count as gossip_listen_loop_iterations_since_last_report which already exists
2023-07-24 09:01:07 -06:00
Greg Cusack 80f708298b
Report new received crds signatures and their respective origins to metrics (#32504)
* screwed up old branch and syncing with upstream branch

* add fixed size ring buff instead of variable sized vecdeque for reporting signatures

* modify difficulty to take in n 0 bits and check if signature ending ends in n 0 bits

* update to only push every 18 trailing zero bits. and clean up

* report origin with signature. and set trailing 0s to 8 for local testing

* change back to 18 trailing zeros and rm unused imports

* run cargo rmt

* run ./scripts/cargo-for-all-lock-files.sh tree

* allow integer arithmetic for bit comparison

* rm unused lifetime

* rm duplicate entry?

* re implement ring buf

* put ringbuf in sorted order

* ringbuf in cargo.toml now in sorted order

* rm ring buf, refactor, fix trailing zero bug

* fix bug in trailing zeros. was comparing wrong ones

* fix needless range loop bug

* fix bug

* change trailing zero checking to build in methods and only report first 8 bytes of signature and origin pubkey

* report full origin string and first 8 bytes of signature

* set SIGNATURE_SAMPLE_TRAILING_ZEROS back to 18

* forgot to run cargo tree

* avoid panic and change working

* rm bs58

* pass in Option<String> into datapoint_info

* shorten metric names
2023-07-20 14:33:40 -07:00
steviez d8516eea4c
Remove unnecessary clone of shred payload (#32540) 2023-07-19 15:34:16 -05:00
behzad nouri d54b6204be
removes instances of clippy::manual_let_else (#32417) 2023-07-09 21:41:36 +00:00
behzad nouri 0da01270ef
removes redundant recycler clones (#32401) 2023-07-06 18:25:20 +00:00
Greg Cusack b8222b230c
Increase push fanout from 6 to 9 (#32323)
* increase push fanout from 6 to 9. leave active set size at 12

* forgot to change comment to reflect push fanout increase
2023-06-28 19:03:12 -05:00
behzad nouri 469661d217
removes outdated tvu_forward socket (#32101)
Shreds are no longer sent to tvu_forward socket.
2023-06-20 20:50:16 +00:00
behzad nouri 987e8eeeaf
removes feature gate code dropping redundant turbine path (#32075) 2023-06-16 19:53:05 +00:00
behzad nouri 3ddb8babc8
adds const SOCKET_ADDR_UNSPECIFIED (#32102)
const socket-addr is stable since rust 1.69.0
2023-06-13 19:34:59 +00:00
behzad nouri aed4ecb633
adds quic receiver to shred-fetch-stage (#31576)
Working towards migrating turbine to QUIC.
2023-06-12 13:16:27 +00:00
Illia Bobyr 4353ac6797
Pass Arc<AtomicBool> by value, not by reference. (#31916)
`Arc` is already a reference internally, so it does not seem to be
beneficial to pass a reference to it.  Just adds an extra layer of
indirection.

Functions that need to be able to increment `Arc` reference count need
to take `Arc<AtomicBool>`, but those that just want to read the
`AtomicBool` value can accept `&AtomicBool`, making them a bit more
generic.

This change focuses specifically on `Arc<AtomicBool>`.  There are other
uses of `&Arc<T>` in the code base that could be converted in a similar
manner.  But it would make the change even larger.
2023-06-01 17:25:48 -07:00
behzad nouri 5178d4d49b
adds quic tvu port to contact-info (#31614)
Working towards migrating turbine to QUIC.
2023-05-15 15:13:21 +00:00
behzad nouri 4e34abbf3d
specifies protocol in contact-info get-socket api (#31602) 2023-05-12 16:16:20 +00:00
Jon Cinque c0e107a97d
gossip-service: Spy multiple nodes by pubkey at once (#31597)
* gossip: Spy multiple pubkeys

* Address behzad's feedback
2023-05-11 18:24:35 +02:00
behzad nouri 6a4a0418a6
removes hard-coded QUIC_PORT_OFFSET from connection-cache (#31541)
New ContactInfo has api identifying QUIC vs UDP ports; no need to hard-code
port-offset deep in connection-cache.
2023-05-09 13:46:17 +00:00
Illia Bobyr 2d090d4547
test_gossip_node: Use random port (#31490)
Using a fixed port could cause a false negative, if the port is
currently in use.  We actually see this test failing regularly with an
error that port `1111` is already in use.

Quick search did not show any tests that hardcode port 1111, so it is
unclear why is this happening.  But using hardcoded ports is not a good
practice anyways.
2023-05-05 18:47:24 -07:00
Brooks c5e071c7fe
Upgrades nightly Rust to 2023-03-04 (#31487) 2023-05-05 08:28:23 -04:00
behzad nouri aafcac27d8
removes pubkey from LegacyContactInfo public interface (#31375)
Working towards LegacyContactInfo => ContactInfo migration, the commit
adds more api parity between the two.
2023-04-28 12:05:15 +00:00
DimAn bd782634dc
Add ability to manage public TPU Forwards address (#31201)
A node operator can manage a public TPU address (at node startup and while it's running), but doesn't have the ability to manage the TPU Forwards address as well.

This PR adds that functionality.

Added the start argument --public-tpu-forwards-address
Reworked the set-public-tpu-address subcommand into the set-public-address subcommand
2023-04-24 23:04:36 +00:00
behzad nouri 1b08d01a80
removes shred_version from LegacyContactInfo public interface (#31304)
Working towards LegacyContactInfo => ContactInfo migration, the commit
adds more api parity between the two.
2023-04-24 15:19:33 +00:00
behzad nouri 33bde55bbd
adds tpu_quic field to RpcContactInfo (#31310) 2023-04-22 20:18:58 +00:00
behzad nouri a88024e295
removes wallclock from LegacyContactInfo public interface (#31303) 2023-04-22 20:18:39 +00:00
behzad nouri cb65a785bc
makes sockets in LegacyContactInfo private (#31248)
Working towards LegacyContactInfo => ContactInfo migration, the commit
hides some implementation details of LegacyContactInfo and expands API
parity with the new ContactInfo.
2023-04-21 15:39:16 +00:00
DimAn b81b7ebf03
validator: `--tpu-host-addr` -> `--public-tpu-address` (#31137) 2023-04-12 13:33:09 +00:00
DimAn 9136f80d36
validator: add `set-public-tpu-address` command (#30452) 2023-04-12 13:32:22 +00:00
Brooks e1cb8a672b
Renames SnapshotHashes in cluster info metrics (#31145) 2023-04-11 17:35:00 +00:00
Brooks 453f272698
Rename IncrementalSnapshotHashes to SnapshotHashes (#31136) 2023-04-11 10:30:29 -04:00
Brooks f3083ad2e0
Rename SnapshotHashes to LegacySnapshotHashes (#31086) 2023-04-10 17:52:20 -04:00
Brooks aa8d7de2d1
Uses impl_crds_entry! for SnapshotHashes (#31032) 2023-04-04 14:44:05 -04:00
Brooks e8ea722061
Uses AccountsHashes type for AccountsHashes CrdsData variant (#31003) 2023-04-03 16:42:21 -04:00
Jeff Biseda 0e2c6c8989
ammend clarifying comment for duplicate instance detection (#31008) 2023-03-31 12:50:04 -07:00
behzad nouri ff9a42a354
uses Duration type instead of untyped ..._ms: u64 (#30971) 2023-03-31 15:42:49 +00:00
Jeff Biseda 5ada25b4e7
add clarifying comment for duplicate instance detection (#30895) 2023-03-30 17:13:41 -07:00
behzad nouri d4b30adffe
reworks gossip crds timeouts (#30468)
CrdsGossipPull::make_timeouts iterates over the stakes hashmap and
creates a new hashmap which is unnecessary:
https://github.com/solana-labs/solana/blob/c032dc275/gossip/src/crds_gossip_pull.rs#L517-L539

The commit instead keeps a reference to the stakes hashmap.
2023-03-27 21:52:48 +00:00
Ryo Onodera 721719d776
Add solana_clap_utils::hidden_unless_forced() (#30843)
* Add solana_clap_utils::hidden_unless_forced()

* Use more descriptive env name

Co-authored-by: mvines <mvines@gmail.com>

---------

Co-authored-by: mvines <mvines@gmail.com>
2023-03-22 08:59:17 +00:00
Yihau Chen df3ef111f7
chore: workspace inheritance (#29893)
* introduce workspace.package

* introduce workspace.dependencies

* read version from root cargo.toml

* pass check when version = { workspace = true }

* don't bump version when version = { workspace = true }

* including workspace Cargo.toml when bump version

* programs/sbf use workspace inheritance

* fix increasing cargo version ignore program/sbf/Cargo.toml
2023-02-23 22:01:54 +08:00
Michael Vines 5136ed3448
Update homepage value for all crates (#30444) 2023-02-23 02:20:18 +00:00
kirill lykov ee1717b24b
Make clippy to be happy (#30394)
* replace default implementation with default enum

* fix format to make clippy happy
2023-02-17 20:51:18 +01:00
Brennan b678ee3583
Clean up socket binding reuse (#30367) 2023-02-16 11:07:22 -08:00
Brooks febaf36e6d
Apply clippy fixes for future rust upgrade (#30363) 2023-02-16 16:12:51 +00:00
behzad nouri 4892a6a910
removes redundant CrdsGossipPull.msg_timeout (#30334) 2023-02-16 00:23:27 +00:00
behzad nouri eb6d6e9bca
evicts old gossip entries by origin's wallclock (#30331)
Local-timestamp used in current gossip eviction code:
https://github.com/solana-labs/solana/blob/a36e1b211/gossip/src/crds.rs#L469-L511
does not indicate how old the entry is but how recently it was received.

The commit instead uses origin's wallclock to identify old values. In
order to avoid cases where the wallclock on the entry is bogus, it is
capped by local-timestamp.
2023-02-15 20:03:13 +00:00
behzad nouri eede50c868
removes unused code from duplicate-shreds (#30329) 2023-02-15 17:37:15 +00:00
behzad nouri ded457cd73
embeds the new gossip ContactInfo in ClusterInfo (#30022)
Working towards replacing the legacy gossip contact-info with the new
one, the commit updates the respective field in gossip cluster-info.
2023-02-10 20:07:45 +00:00
behzad nouri ceb225f36e
caches blockstore duplicate shreds slots (#30241)
In order to reduce number of calls to blockstore, the commit locally caches
blockstore.has_duplicate_shreds_in_slot in DuplicateShredHandler.consumed.
2023-02-10 18:19:57 +00:00
behzad nouri b3887af7c6
flattens the internal buffer in the duplicate-shreds handler (#30196)
Duplicate-shreds handler is using a nested hash-map for the incomplete
chunks buffered. This is resulting in a convoluted logic to limit the
number of entries:
https://github.com/solana-labs/solana/blob/427bd6264/gossip/src/duplicate_shred_handler.rs#L62

This commit instead uses a flat buffer mapping (Slot, Pubkey) pairs to
the respective duplicate shreds chunks. The buffer is allowed to grow to
twice the intended capacity, at which point the extraneous entries are
removed in linear time, resulting an amortized O(1) performance.
2023-02-09 16:33:20 +00:00