behzad nouri
677b6d6458
removes LegacyContactInfo::is_valid_tvu_address ( #29570 )
...
Since
https://github.com/solana-labs/solana/pull/20480
turbine includes all epoch staked nodes in tree construction and no
longer relies on obtaining their contact-info from gossip; and so
distinguishing between is_valid_address and is_valid_tvu_address is no
longer necessary and the latter can be removed.
2023-01-08 22:53:45 +00:00
behzad nouri
8c212f59ad
renames ContactInfo to LegacyContactInfo ( #29566 )
...
Working towards adding a new ContactInfo where new sockets can be
added in a backward compatible way.
2023-01-08 16:00:55 +00:00
behzad nouri
283a2b1540
removes #[allow(clippy::same_item_push)] ( #29543 )
2023-01-06 17:32:26 +00:00
behzad nouri
e5323166b3
dedups gossip addresses, taking the one with highest weight ( #29421 )
...
dedups gossip addresses, keeping only the one with the highest weight
In order to avoid traffic congestion or sending duplicate packets, when
sampling gossip nodes if several nodes have the same gossip address
(because they are behind a relayer or whatever), they need to be
deduplicated into one.
2023-01-03 19:37:43 +00:00
behzad nouri
2d849a2eae
indexes duplicate-shreds in gossip crds table ( #29317 )
...
Also adding Crds::get_duplicate_shreds which retrieves all upserted
duplicate-shreds since a given cursor using the index.
2022-12-20 13:48:05 +00:00
behzad nouri
78a04ed432
ignores pubkey in Protocol::PruneMessage ( #29280 )
...
Protocol::PruneMessage(Pubkey, _) is the same as PruneData.pubkey and so
is redundant and can be ignored:
https://github.com/solana-labs/solana/blob/95d339300/gossip/src/cluster_info.rs#LL277-L279
https://github.com/solana-labs/solana/blob/95d339300/gossip/src/cluster_info.rs#L361-L367
2022-12-15 17:51:12 +00:00
behzad nouri
a5c8c7c536
locks crds table only once to process push messages ( #29218 )
...
Processing push messages is locking and unlocking crds table for each
push message:
https://github.com/solana-labs/solana/blob/536b879aa/gossip/src/cluster_info.rs#L2266-L2276
https://github.com/solana-labs/solana/blob/536b879aa/gossip/src/crds_gossip_push.rs#L215C9-L260
This commit instead locks the crds table once for all the received push
messages.
2022-12-15 16:02:46 +00:00
behzad nouri
95d3393008
prunes gossip nodes based on timeliness of delivered messages
...
As described here:
https://github.com/solana-labs/solana/issues/28642#issuecomment-1337449607
current gossip pruning code fails to maintain spanning trees across
cluster.
This commit instead implements a pruning code based on timeliness of
delivered messages. If a messages is delivered timely enough (in terms
of number of duplicates already observed for that value), it counts
towards the respective node's score. Once there are enough many CRDS
upserts from a specific origin, redundant nodes are pruned based on the
tracked score.
Since the pruning leaves some configurable redundancy and the scores are
reset frequently, it should better tolerate active-set rotations.
2022-12-15 13:28:27 +00:00
behzad nouri
b06656cbba
tracks number of gossip push duplicates
...
The commit tracks number of times duplicates of a CRDS value is received
from gossip push. Following commits will utilize this metric to score
gossip nodes in terms of timeliness of their push messages, in order to
better pick which nodes to prune.
2022-12-15 13:28:27 +00:00
behzad nouri
8ea5dd8b28
removes metric for process_push_success ( #29211 )
...
This is already tracked in CrdsDataStats:
https://github.com/solana-labs/solana/blob/5e799ad56/gossip/src/crds.rs#L96-L106
https://github.com/solana-labs/solana/blob/5e799ad56/gossip/src/cluster_info_metrics.rs#L652-L656
and is so duplicated.
Removing the metric would simplify this code path for upcoming commits.
2022-12-12 22:10:38 +00:00
behzad nouri
9524c9dbff
patches errors from clippy::uninlined_format_args
...
https://rust-lang.github.io/rust-clippy/master/index.html#uninlined_format_args
2022-12-06 19:32:15 +00:00
Jon Cinque
b1340d77a2
sdk: Make Packet::meta private, use accessor functions ( #29092 )
...
sdk: Make packet meta private
2022-12-06 12:54:49 +01:00
behzad nouri
718f433206
adds metrics for gossip push fanout ( #29065 )
2022-12-04 15:20:51 +00:00
Tyera
c32377b5af
Split out quic- and udp-client definitions ( #28762 )
...
* Move ConnectionCache back to solana-client, and duplicate ThinClient, TpuClient there
* Dedupe thin_client modules
* Dedupe tpu_client modules
* Move TpuClient to TpuConnectionCache
* Move ThinClient to TpuConnectionCache
* Move TpuConnection and quic/udp trait implementations back to solana-client
* Remove enum_dispatch from solana-tpu-client
* Move udp-client to its own crate
* Move quic-client to its own crate
2022-11-18 12:21:45 -07:00
Brooks Prumo
d1ba42180d
clippy for rust 1.65.0 ( #28765 )
2022-11-09 19:39:38 +00:00
behzad nouri
f703275fc4
pings peers before sending push messages ( #28537 )
2022-10-25 00:01:23 +00:00
behzad nouri
e283461d99
enforces hash domain for ping-pong protocol ( #28433 )
...
https://github.com/solana-labs/solana/pull/27193
added hash domain to ping-pong protocol.
For backward compatibility responses both with and without domain were
generated and accepted.
Now that all clusters are upgraded, this commit enforces the hash domain
by removing the response without the domain.
2022-10-18 18:17:12 +00:00
Brooks Prumo
12df0f234d
Upgrade to Rust 1.64.0 ( #28034 )
2022-09-29 09:32:24 -04:00
behzad nouri
abfaf06e87
counts gossip packets received before excess packets are dropped ( #28086 )
...
Currently, gossip packets are counted after excess packets are dropped.
This makes it difficult to debug gossip traffic spikes if the majority
of the packets are dropped.
This commit instead counts gossip packets received before excess packets
are dropped
2022-09-27 13:43:35 +00:00
Jeff Biseda
8b0f9b4917
make ping cache rate limit delay configurable ( #27955 )
2022-09-26 14:16:56 -07:00
behzad nouri
f49beb0cbc
caches reed-solomon encoder/decoder instance ( #27510 )
...
ReedSolomon::new(...) initializes a matrix and a data-decode-matrix cache:
https://github.com/rust-rse/reed-solomon-erasure/blob/273ebbced/src/core.rs#L460-L466
In order to cache this computation, this commit caches the reed-solomon
encoder/decoder instance for each (data_shards, parity_shards) pair.
2022-09-25 18:09:47 +00:00
behzad nouri
97c9af4c6b
plumbs through flag to generate merkle variant of shreds
2022-09-23 16:45:18 +00:00
behzad nouri
9a57c64f21
patches clippy errors from new rust nightly release ( #27996 )
2022-09-22 22:23:03 +00:00
dependabot[bot]
c4fa849844
chore: bump itertools from 0.10.3 to 0.10.5 ( #27962 )
...
* chore: bump itertools from 0.10.3 to 0.10.5
Bumps [itertools](https://github.com/rust-itertools/itertools ) from 0.10.3 to 0.10.5.
- [Release notes](https://github.com/rust-itertools/itertools/releases )
- [Changelog](https://github.com/rust-itertools/itertools/blob/master/CHANGELOG.md )
- [Commits](https://github.com/rust-itertools/itertools/commits )
---
updated-dependencies:
- dependency-name: itertools
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2022-09-21 15:28:49 -06:00
Will Hickey
c0e4379f43
Whickey/version v1.15 ( #27739 )
...
* Bump version to v1.13.0
* Bump version to v1.14.0
* Bump version to v1.15.0
2022-09-13 09:06:15 -05:00
dependabot[bot]
f338aa62ba
chore: bump serde from 1.0.143 to 1.0.144 ( #27511 )
...
* chore: bump serde from 1.0.143 to 1.0.144
Bumps [serde](https://github.com/serde-rs/serde ) from 1.0.143 to 1.0.144.
- [Release notes](https://github.com/serde-rs/serde/releases )
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.143...v1.0.144 )
---
updated-dependencies:
- dependency-name: serde
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2022-09-07 16:54:44 -06:00
Will Hickey
4aa2a42cc7
Fix test_bench_tps_local_cluster_solana ( #27448 )
...
* Fix test_bench_tps_local_cluster_solana
* Remove #[ignore] annotations from dos tests (which are also fixed by this change)
* Remove #[ignore] annotations from local cluster tests (which are also fixed by this change)
2022-08-30 13:04:31 -05:00
Tyera Eulberg
7bd08ad3ae
Featurize spinner in rpc-client and tpu-client ( #27381 )
...
* Add spinner features to rpc-client and tpu-client, and disable where unneeded
* Add doc comment
2022-08-30 09:56:56 -06:00
Tyera Eulberg
b8b3d723da
Use new client crates ( #27360 )
...
* Update ancillary cli crates
* Update cli
* Update command-line tools
* Update rpc, etc
* Update client-test
* Update core, validator
* Update local-cluster
2022-08-24 10:47:02 -06:00
Brennan Watt
e4a7d01e10
Rust v1.63 ( #27303 )
...
* Upgrade to Rust v1.63.0
* Add nightly_clippy_allows
* Resolve some new clippy nightly lints
* Increase QUIC packets completion timeout
* Update quinn-udp crate
Co-authored-by: Michael Vines <mvines@gmail.com>
2022-08-22 18:01:03 -07:00
Michael Vines
3f4731b37f
Standardize thread names
...
Tenets:
1. Limit thread names to 15 characters
2. Prefix all Solana-controlled threads with "sol"
3. Use Camel case. It's more character dense than Snake or Kebab case
2022-08-20 07:49:39 -07:00
Brennan Watt
7573000d87
Revert "Rust v1.63.0 ( #27148 )" ( #27245 )
...
This reverts commit a2e7bdf50a
.
2022-08-19 09:19:44 +01:00
behzad nouri
6928b2a5af
adds hash domain to ping-pong protocol ( #27193 )
...
In order to maintain backward compatibility, for now the responding node
will hash the token both with and without domain so that the other node
will accept the response regardless of its upgrade status.
Once the cluster has upgraded to the new code, we will remove the legacy
domain = false case.
2022-08-18 22:39:31 +00:00
HaoranYi
4634fb944c
Use from_secs api to create duration ( #27222 )
...
use from_secs api to create duration
2022-08-18 11:06:52 -05:00
Brennan Watt
a2e7bdf50a
Rust v1.63.0 ( #27148 )
...
* Upgrade to Rust v1.63.0
* Add nightly_clippy_allows
* Resolve some new clippy nightly lints
* Increase QUIC packets completion timeout
Co-authored-by: Michael Vines <mvines@gmail.com>
2022-08-17 15:48:33 -07:00
behzad nouri
fea66c8b63
derives Error trait for ClusterInfoError and core::result::Error ( #27208 )
2022-08-17 22:01:51 +00:00
dependabot[bot]
a0d1f4ef88
chore: bump serial_test from 0.8.0 to 0.9.0 ( #27097 )
...
Bumps [serial_test](https://github.com/palfrey/serial_test ) from 0.8.0 to 0.9.0.
- [Release notes](https://github.com/palfrey/serial_test/releases )
- [Commits](https://github.com/palfrey/serial_test/compare/v0.8.0...v0.9.0 )
---
updated-dependencies:
- dependency-name: serial_test
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-08-15 15:53:12 -06:00
Brennan Watt
521c550ccd
Sleep between vote refreshes ( #27115 )
...
* Sleep between vote refreshes in unit test
2022-08-12 13:45:00 -07:00
Tyera Eulberg
45c0da8597
Fix quic client on TestValidator, alternative ( #27046 )
...
Add new method to enable custom offset
2022-08-10 15:27:12 +00:00
dependabot[bot]
ae5b680c6f
chore: bump serde from 1.0.138 to 1.0.143 ( #27015 )
...
* chore: bump serde from 1.0.138 to 1.0.143
Bumps [serde](https://github.com/serde-rs/serde ) from 1.0.138 to 1.0.143.
- [Release notes](https://github.com/serde-rs/serde/releases )
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.138...v1.0.143 )
---
updated-dependencies:
- dependency-name: serde
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2022-08-10 05:08:43 +00:00
Will Hickey
ed8c224374
Bump version to v1.12 ( #26967 )
2022-08-06 13:20:30 -05:00
github-actions[bot]
fbf1bf6d86
Bump Version to 1.11.6 ( #26906 )
...
Co-authored-by: willhickey <willhickey@users.noreply.github.com>
2022-08-03 12:48:43 -05:00
Jeff Biseda
857be1e237
sign repair requests ( #26833 )
2022-07-31 15:48:51 -07:00
github-actions[bot]
5d038b9d2a
Bump Version to 1.11.5 ( #26758 )
...
Co-authored-by: willhickey <willhickey@users.noreply.github.com>
2022-07-25 13:05:14 -06:00
github-actions[bot]
fd5df1cf25
Bump Version to 1.11.4 ( #26578 )
...
Co-authored-by: willhickey <willhickey@users.noreply.github.com>
2022-07-11 23:30:38 -05:00
behzad nouri
128226c6cc
patches flaky test_push_votes_with_tower ( #26554 )
...
cargo test --package solana-gossip --release test_push_votes_with_tower
occasionally fails because with --release all votes are generated at
the same wallclock (milliseconds resolution) and so the new ones will
not necessarily override existing entries in the table.
The commit ensures that the new vote is pushed with a wallclock later
than existing entries.
2022-07-11 12:56:31 +00:00
behzad nouri
ba785cf8ab
removes erroneous uses of std::mem::swap ( #26536 )
...
All instances should be replace by std::mem::{replace,take},
or just plain assignment.
2022-07-11 11:33:15 +00:00
behzad nouri
df616a0dda
removes redundant clone in gossip PruneData::signable_data ( #26510 )
...
PruneData::signable_data redundantly clones inner fields, while only
references suffice:
https://github.com/solana-labs/solana/blob/d1370f2c7/gossip/src/cluster_info.rs#L219-L233
2022-07-10 13:13:07 +00:00
github-actions[bot]
9d937fb8a0
Bump Version to 1.11.3 ( #26481 )
...
Co-authored-by: willhickey <willhickey@users.noreply.github.com>
2022-07-07 14:39:46 -05:00
dependabot[bot]
5e8e1beeb5
chore: bump serial_test from 0.6.0 to 0.8.0 ( #26463 )
...
Bumps [serial_test](https://github.com/palfrey/serial_test ) from 0.6.0 to 0.8.0.
- [Release notes](https://github.com/palfrey/serial_test/releases )
- [Commits](https://github.com/palfrey/serial_test/compare/v0.6.0...v0.8.0 )
---
updated-dependencies:
- dependency-name: serial_test
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-07-07 10:52:20 -06:00