blockworks-foundation
/
solana-with-rpc-optimizations
mirror of https://github.com/blockworks-foundation/solana-with-rpc-optimizations.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
21,325 Commits 1 Branch 0 Tags 365 MiB
Commit Graph

315 Commits

Author SHA1 Message Date
behzad nouri 677b6d6458
removes LegacyContactInfo::is_valid_tvu_address (#29570) 
Since
https://github.com/solana-labs/solana/pull/20480
turbine includes all epoch staked nodes in tree construction and no
longer relies on obtaining their contact-info from gossip; and so
distinguishing between is_valid_address and is_valid_tvu_address is no
longer necessary and the latter can be removed.
 2023-01-08 22:53:45 +00:00
behzad nouri 8c212f59ad
renames ContactInfo to LegacyContactInfo (#29566) 
Working towards adding a new ContactInfo where new sockets can be
added in a backward compatible way.
 2023-01-08 16:00:55 +00:00
behzad nouri 283a2b1540
removes #[allow(clippy::same_item_push)] (#29543) 2023-01-06 17:32:26 +00:00
behzad nouri e5323166b3
dedups gossip addresses, taking the one with highest weight (#29421) 
dedups gossip addresses, keeping only the one with the highest weight

In order to avoid traffic congestion or sending duplicate packets, when
sampling gossip nodes if several nodes have the same gossip address
(because they are behind a relayer or whatever), they need to be
deduplicated into one.
 2023-01-03 19:37:43 +00:00
behzad nouri 2d849a2eae
indexes duplicate-shreds in gossip crds table (#29317) 
Also adding Crds::get_duplicate_shreds which retrieves all upserted
duplicate-shreds since a given cursor using the index.
 2022-12-20 13:48:05 +00:00
behzad nouri 78a04ed432
ignores pubkey in Protocol::PruneMessage (#29280) 
Protocol::PruneMessage(Pubkey, _) is the same as PruneData.pubkey and so
is redundant and can be ignored:
https://github.com/solana-labs/solana/blob/95d339300/gossip/src/cluster_info.rs#LL277-L279
https://github.com/solana-labs/solana/blob/95d339300/gossip/src/cluster_info.rs#L361-L367
 2022-12-15 17:51:12 +00:00
behzad nouri a5c8c7c536
locks crds table only once to process push messages (#29218) 
Processing push messages is locking and unlocking crds table for each
push message:
https://github.com/solana-labs/solana/blob/536b879aa/gossip/src/cluster_info.rs#L2266-L2276
https://github.com/solana-labs/solana/blob/536b879aa/gossip/src/crds_gossip_push.rs#L215C9-L260

This commit instead locks the crds table once for all the received push
messages.
 2022-12-15 16:02:46 +00:00
behzad nouri 95d3393008 prunes gossip nodes based on timeliness of delivered messages 
As described here:
https://github.com/solana-labs/solana/issues/28642#issuecomment-1337449607
current gossip pruning code fails to maintain spanning trees across
cluster.

This commit instead implements a pruning code based on timeliness of
delivered messages. If a messages is delivered timely enough (in terms
of number of duplicates already observed for that value), it counts
towards the respective node's score. Once there are enough many CRDS
upserts from a specific origin, redundant nodes are pruned based on the
tracked score.

Since the pruning leaves some configurable redundancy and the scores are
reset frequently, it should better tolerate active-set rotations.
 2022-12-15 13:28:27 +00:00
behzad nouri b06656cbba tracks number of gossip push duplicates 
The commit tracks number of times duplicates of a CRDS value is received
from gossip push. Following commits will utilize this metric to score
gossip nodes in terms of timeliness of their push messages, in order to
better pick which nodes to prune.
 2022-12-15 13:28:27 +00:00
behzad nouri 8ea5dd8b28
removes metric for process_push_success (#29211) 
This is already tracked in CrdsDataStats:
https://github.com/solana-labs/solana/blob/5e799ad56/gossip/src/crds.rs#L96-L106
https://github.com/solana-labs/solana/blob/5e799ad56/gossip/src/cluster_info_metrics.rs#L652-L656
and is so duplicated.
Removing the metric would simplify this code path for upcoming commits.
 2022-12-12 22:10:38 +00:00
behzad nouri 9524c9dbff patches errors from clippy::uninlined_format_args 
https://rust-lang.github.io/rust-clippy/master/index.html#uninlined_format_args
 2022-12-06 19:32:15 +00:00
Jon Cinque b1340d77a2
sdk: Make Packet::meta private, use accessor functions (#29092) 
sdk: Make packet meta private
 2022-12-06 12:54:49 +01:00
behzad nouri 718f433206
adds metrics for gossip push fanout (#29065) 2022-12-04 15:20:51 +00:00
Tyera c32377b5af
Split out quic- and udp-client definitions (#28762) 
* Move ConnectionCache back to solana-client, and duplicate ThinClient, TpuClient there

* Dedupe thin_client modules

* Dedupe tpu_client modules

* Move TpuClient to TpuConnectionCache

* Move ThinClient to TpuConnectionCache

* Move TpuConnection and quic/udp trait implementations back to solana-client

* Remove enum_dispatch from solana-tpu-client

* Move udp-client to its own crate

* Move quic-client to its own crate
 2022-11-18 12:21:45 -07:00
Brooks Prumo d1ba42180d
clippy for rust 1.65.0 (#28765) 2022-11-09 19:39:38 +00:00
behzad nouri f703275fc4
pings peers before sending push messages (#28537) 2022-10-25 00:01:23 +00:00
behzad nouri e283461d99
enforces hash domain for ping-pong protocol (#28433) 
https://github.com/solana-labs/solana/pull/27193
added hash domain to ping-pong protocol.
For backward compatibility responses both with and without domain were
generated and accepted.
Now that all clusters are upgraded, this commit enforces the hash domain
by removing the response without the domain.
 2022-10-18 18:17:12 +00:00
Brooks Prumo 12df0f234d
Upgrade to Rust 1.64.0 (#28034) 2022-09-29 09:32:24 -04:00
behzad nouri abfaf06e87
counts gossip packets received before excess packets are dropped (#28086) 
Currently, gossip packets are counted after excess packets are dropped.
This makes it difficult to debug gossip traffic spikes if the majority
of the packets are dropped.

This commit instead counts gossip packets received before excess packets
are dropped
 2022-09-27 13:43:35 +00:00
Jeff Biseda 8b0f9b4917
make ping cache rate limit delay configurable (#27955) 2022-09-26 14:16:56 -07:00
behzad nouri f49beb0cbc
caches reed-solomon encoder/decoder instance (#27510) 
ReedSolomon::new(...) initializes a matrix and a data-decode-matrix cache:
https://github.com/rust-rse/reed-solomon-erasure/blob/273ebbced/src/core.rs#L460-L466

In order to cache this computation, this commit caches the reed-solomon
encoder/decoder instance for each (data_shards, parity_shards) pair.
 2022-09-25 18:09:47 +00:00
behzad nouri 97c9af4c6b plumbs through flag to generate merkle variant of shreds 2022-09-23 16:45:18 +00:00
behzad nouri 9a57c64f21
patches clippy errors from new rust nightly release (#27996) 2022-09-22 22:23:03 +00:00
dependabot[bot] c4fa849844
chore: bump itertools from 0.10.3 to 0.10.5 (#27962) 
* chore: bump itertools from 0.10.3 to 0.10.5

Bumps [itertools](https://github.com/rust-itertools/itertools) from 0.10.3 to 0.10.5.
- [Release notes](https://github.com/rust-itertools/itertools/releases)
- [Changelog](https://github.com/rust-itertools/itertools/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-itertools/itertools/commits)

---
updated-dependencies:
- dependency-name: itertools
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
 2022-09-21 15:28:49 -06:00
Will Hickey c0e4379f43
Whickey/version v1.15 (#27739) 
* Bump version to v1.13.0
* Bump version to v1.14.0
* Bump version to v1.15.0
 2022-09-13 09:06:15 -05:00
dependabot[bot] f338aa62ba
chore: bump serde from 1.0.143 to 1.0.144 (#27511) 
* chore: bump serde from 1.0.143 to 1.0.144

Bumps [serde](https://github.com/serde-rs/serde) from 1.0.143 to 1.0.144.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.143...v1.0.144)

---
updated-dependencies:
- dependency-name: serde
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
 2022-09-07 16:54:44 -06:00
Will Hickey 4aa2a42cc7
Fix test_bench_tps_local_cluster_solana (#27448) 
* Fix test_bench_tps_local_cluster_solana
* Remove #[ignore] annotations from dos tests (which are also fixed by this change)
* Remove #[ignore] annotations from local cluster tests (which are also fixed by this change)
 2022-08-30 13:04:31 -05:00
Tyera Eulberg 7bd08ad3ae
Featurize spinner in rpc-client and tpu-client (#27381) 
* Add spinner features to rpc-client and tpu-client, and disable where unneeded

* Add doc comment
 2022-08-30 09:56:56 -06:00
Tyera Eulberg b8b3d723da
Use new client crates (#27360) 
* Update ancillary cli crates

* Update cli

* Update command-line tools

* Update rpc, etc

* Update client-test

* Update core, validator

* Update local-cluster
 2022-08-24 10:47:02 -06:00
Brennan Watt e4a7d01e10
Rust v1.63 (#27303) 
* Upgrade to Rust v1.63.0

* Add nightly_clippy_allows

* Resolve some new clippy nightly lints

* Increase QUIC packets completion timeout

* Update quinn-udp crate

Co-authored-by: Michael Vines <mvines@gmail.com>
 2022-08-22 18:01:03 -07:00
Michael Vines 3f4731b37f Standardize thread names 
Tenets:
1. Limit thread names to 15 characters
2. Prefix all Solana-controlled threads with "sol"
3. Use Camel case. It's more character dense than Snake or Kebab case
 2022-08-20 07:49:39 -07:00
Brennan Watt 7573000d87
Revert "Rust v1.63.0 (#27148)" (#27245) 
This reverts commit a2e7bdf50a.
 2022-08-19 09:19:44 +01:00
behzad nouri 6928b2a5af
adds hash domain to ping-pong protocol (#27193) 
In order to maintain backward compatibility, for now the responding node
will hash the token both with and without domain so that the other node
will accept the response regardless of its upgrade status.
Once the cluster has upgraded to the new code, we will remove the legacy
domain = false case.
 2022-08-18 22:39:31 +00:00
HaoranYi 4634fb944c
Use from_secs api to create duration (#27222) 
use from_secs api to create duration
 2022-08-18 11:06:52 -05:00
Brennan Watt a2e7bdf50a
Rust v1.63.0 (#27148) 
* Upgrade to Rust v1.63.0

* Add nightly_clippy_allows

* Resolve some new clippy nightly lints

* Increase QUIC packets completion timeout

Co-authored-by: Michael Vines <mvines@gmail.com>
 2022-08-17 15:48:33 -07:00
behzad nouri fea66c8b63
derives Error trait for ClusterInfoError and core::result::Error (#27208) 2022-08-17 22:01:51 +00:00
dependabot[bot] a0d1f4ef88
chore: bump serial_test from 0.8.0 to 0.9.0 (#27097) 
Bumps [serial_test](https://github.com/palfrey/serial_test) from 0.8.0 to 0.9.0.
- [Release notes](https://github.com/palfrey/serial_test/releases)
- [Commits](https://github.com/palfrey/serial_test/compare/v0.8.0...v0.9.0)

---
updated-dependencies:
- dependency-name: serial_test
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-08-15 15:53:12 -06:00
Brennan Watt 521c550ccd
Sleep between vote refreshes (#27115) 
* Sleep between vote refreshes in unit test
 2022-08-12 13:45:00 -07:00
Tyera Eulberg 45c0da8597
Fix quic client on TestValidator, alternative (#27046) 
Add new method to enable custom offset
 2022-08-10 15:27:12 +00:00
dependabot[bot] ae5b680c6f
chore: bump serde from 1.0.138 to 1.0.143 (#27015) 
* chore: bump serde from 1.0.138 to 1.0.143

Bumps [serde](https://github.com/serde-rs/serde) from 1.0.138 to 1.0.143.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.138...v1.0.143)

---
updated-dependencies:
- dependency-name: serde
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
 2022-08-10 05:08:43 +00:00
Will Hickey ed8c224374
Bump version to v1.12 (#26967) 2022-08-06 13:20:30 -05:00
github-actions[bot] fbf1bf6d86
Bump Version to 1.11.6 (#26906) 
Co-authored-by: willhickey <willhickey@users.noreply.github.com>
 2022-08-03 12:48:43 -05:00
Jeff Biseda 857be1e237
sign repair requests (#26833) 2022-07-31 15:48:51 -07:00
github-actions[bot] 5d038b9d2a
Bump Version to 1.11.5 (#26758) 
Co-authored-by: willhickey <willhickey@users.noreply.github.com>
 2022-07-25 13:05:14 -06:00
github-actions[bot] fd5df1cf25
Bump Version to 1.11.4 (#26578) 
Co-authored-by: willhickey <willhickey@users.noreply.github.com>
 2022-07-11 23:30:38 -05:00
behzad nouri 128226c6cc
patches flaky test_push_votes_with_tower (#26554) 
cargo test --package solana-gossip --release test_push_votes_with_tower

occasionally fails because with --release all votes are generated at
the same wallclock (milliseconds resolution) and so the new ones will
not necessarily override existing entries in the table.

The commit ensures that the new vote is pushed with a wallclock later
than existing entries.
 2022-07-11 12:56:31 +00:00
behzad nouri ba785cf8ab
removes erroneous uses of std::mem::swap (#26536) 
All instances should be replace by std::mem::{replace,take},
or just plain assignment.
 2022-07-11 11:33:15 +00:00
behzad nouri df616a0dda
removes redundant clone in gossip PruneData::signable_data (#26510) 
PruneData::signable_data redundantly clones inner fields, while only
references suffice:
https://github.com/solana-labs/solana/blob/d1370f2c7/gossip/src/cluster_info.rs#L219-L233
 2022-07-10 13:13:07 +00:00
github-actions[bot] 9d937fb8a0
Bump Version to 1.11.3 (#26481) 
Co-authored-by: willhickey <willhickey@users.noreply.github.com>
 2022-07-07 14:39:46 -05:00
dependabot[bot] 5e8e1beeb5
chore: bump serial_test from 0.6.0 to 0.8.0 (#26463) 
Bumps [serial_test](https://github.com/palfrey/serial_test) from 0.6.0 to 0.8.0.
- [Release notes](https://github.com/palfrey/serial_test/releases)
- [Commits](https://github.com/palfrey/serial_test/compare/v0.6.0...v0.8.0)

---
updated-dependencies:
- dependency-name: serial_test
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-07-07 10:52:20 -06:00