Commit Graph

50 Commits

Author SHA1 Message Date
steviez 4753dcae71 Rename and uniquify QUIC thread names (#28)
When viewing in various tools such as gdb and perf, it is not easy to
distinguish which threads are serving which function (TPU or TPU FWD)
2024-03-09 13:23:06 -06:00
Lijun Wang 8fde8d26c7
don't sign X.509 certs (#34896)
This get rid of 3rd party components rcgen in the path of private key access to make the code more secure.
2024-01-28 16:17:46 -08:00
Pankaj Garg f92275bcaa
Fix determination of staked QUIC connections (#34760)
* Fix determination of staked QUIC connections

* address review comments

* review comments

* treat connections with zero stake as unstaked
2024-01-13 18:38:31 -08:00
Pankaj Garg 6bbd3661e1
Throttle unstaked quic streams for a given connection (#34562)
* Throttle unstaked quic streams for a given connection

* Fix interval duration check

* move wait to handle_chunk

* set max unistreams to 0

* drop new streams

* cleanup

* some more cleanup

* fix tests

* update test and stop code

* fix bench-tps
2023-12-21 18:47:52 -08:00
GoodDaisy 03386cc7b9
Fix typos (#34459)
* Fix typos

* Fix typos

* fix typo
2023-12-21 13:06:00 -07:00
ryleung-solana 132c910f81
Quic update identity (#33865)
Update the Quic transport layer keypair and identity when the Validator's identity keypair is updated
2023-12-08 14:53:19 +08:00
Lijun Wang 21b2fce6f4
Correct num_packets stats (#33630)
Do not count the empty packets from cache warmer in num_packets stats as they are not sent.
2023-10-11 14:00:30 -07:00
Lijun Wang 344e466e12
Async connection creation in connection cache (#33302)
If there is a connection in the cache available, use it and create the additional connection asynchronously.
2023-09-25 18:17:47 -07:00
Alexander Meißner 9e703f85de
Upgrades Rust to 1.72.0 & nightly-2023-08-25 (#32961)
* allow pedantic invalid cast lint

* allow lint with false-positive triggered by `test-case` crate

* nightly `fmt` correction

* adapt to rust layout changes

* remove dubious test

* Use transmute instead of pointer cast and de/ref when check_aligned is false.

* Renames clippy::integer_arithmetic to clippy::arithmetic_side_effects.

* bump rust nightly to 2023-08-25

* Upgrades Rust to 1.72.0

---------

Co-authored-by: Trent Nelson <trent@solana.com>
2023-09-01 07:26:13 +00:00
Lijun Wang 96138b919a
improve quic-client packet send metrics (#32864)
Added the following metrics on quic-client send_packet:

average_send_packet_us
average_prepare_connection_us
successful_packets
2023-08-22 21:07:53 -07:00
Lijun Wang 98e19af5eb
Do not do send in cache warmer -- just establish connections (#32898)
* Do not do send in cache warmer -- just establish connections
2023-08-21 12:29:45 -07:00
Jon Cinque 4d3bcc55a1
quic-client: Return error when `send_data` fails (#32918)
quic-client: Return error when `send_data` fails
2023-08-21 15:02:50 +02:00
behzad nouri 868e086d75
upgrades quinn and rustls crates (#32499) 2023-07-14 17:30:57 +00:00
behzad nouri 5a80dc0d73
adds QUIC endpoint specific for turbine connections (#32294)
Working towards separating out turbine QUIC from TPU.
2023-07-03 18:57:18 +00:00
behzad nouri ec0001ef85
adds code-path broadcasting shreds using QUIC (#31610)
adds quic connection cache to turbine

Working towards migrating turbine to QUIC.
2023-06-12 22:58:27 +00:00
behzad nouri f2f1dab5ea
patches QuicConnectionManager::new_connection_config (#32031)
QuicConnectionManager::new_connection_config is returning an invalid new
QuicConfig instead of the value it is initialized with.
2023-06-08 16:46:22 +00:00
behzad nouri 5760390d3b
removes redundant NewConnectionConfig trait (#31979)
Working towards removing hard-coded TPU specific configurations from
QUIC code; NewConnectionConfig is redundant and gets in the way.
2023-06-06 21:28:29 +00:00
behzad nouri 0db4f3f263
removes Option wrapper from QuicConfig in QuicConnectionManager (#31995)
QuicConfig isn't really optional and allowing it to be None results in
misconfigured connections.
2023-06-06 21:27:56 +00:00
behzad nouri f1ebc5b5c3
separates out quic streamer connection stats from different servers (#31797) 2023-05-25 16:54:24 +00:00
behzad nouri 6a4a0418a6
removes hard-coded QUIC_PORT_OFFSET from connection-cache (#31541)
New ContactInfo has api identifying QUIC vs UDP ports; no need to hard-code
port-offset deep in connection-cache.
2023-05-09 13:46:17 +00:00
Brooks 6e342ded42
clippy: Removes redundant async blocks (#31526) 2023-05-09 09:35:38 -04:00
behzad nouri 34da001cda
passes through concrete QUIC connection errors up the call stack (#31168) 2023-04-12 19:53:25 +00:00
behzad nouri 557e4c47e1
uses OnceCell for lazy endpoint initialization in quic-client (#31149)
RwLock<Option<...>> lacks the api and adds unnecessary overhead for lazy
initialization. OnceCell instead provides the intended api for this purpose.
2023-04-11 19:53:25 +00:00
behzad nouri ce21a58b65
reworks streamer::StakedNodes (#31082)
{min,max}_stake are computed but never assigned:
https://github.com/solana-labs/solana/blob/4564bcdc1/core/src/staked_nodes_updater_service.rs#L54-L57

The updater code is also inefficient and verbose.
2023-04-10 17:07:40 +00:00
behzad nouri ff9a42a354
uses Duration type instead of untyped ..._ms: u64 (#30971) 2023-03-31 15:42:49 +00:00
Yihau Chen 2c842e9932
chore: workspace inheritance (#30939)
* client/Cargo.toml

* udp-client/Cargo.toml

* tpu-client/Cargo.toml

* thin-client/Cargo.toml

* storage-proto/Cargo.toml

* quic-client/Cargo.toml

* dos/Cargo.toml

* entry/Cargo.toml

* perf/Cargo.toml

* program-runtime/Cargo.toml

* program-test/Cargo.toml

* programs/address-lookup-table/Cargo.toml

* programs/loader-v3/Cargo.toml

* connection-cache/Cargo.toml

* programs/sbf/rust/big_mod_exp/Cargo.toml

* programs/zk-token-proof-tests/Cargo.toml
2023-03-29 03:28:56 +00:00
behzad nouri 75abfc79a6
removes unused dependencies (#30917) 2023-03-28 17:25:44 +00:00
ryleung-solana 0ed9f62602
Quic server batching (#30330) 2023-03-16 21:50:57 +08:00
Yihau Chen df3ef111f7
chore: workspace inheritance (#29893)
* introduce workspace.package

* introduce workspace.dependencies

* read version from root cargo.toml

* pass check when version = { workspace = true }

* don't bump version when version = { workspace = true }

* including workspace Cargo.toml when bump version

* programs/sbf use workspace inheritance

* fix increasing cargo version ignore program/sbf/Cargo.toml
2023-02-23 22:01:54 +08:00
Michael Vines 5136ed3448
Update homepage value for all crates (#30444) 2023-02-23 02:20:18 +00:00
behzad nouri 1ad69cfc38
removes dynamic cast and dynamic dispatch from connection-cache (#30128)
Dynamic dispatch forces heap allocation and adds extra overhead.
Dynamic casting as in the ones below, lacks compile-time type safety:
https://github.com/solana-labs/solana/blob/eeb622c4e/quic-client/src/lib.rs#L172-L175
https://github.com/solana-labs/solana/blob/eeb622c4e/udp-client/src/lib.rs#L52-L55

The commit removes all instances of Any, Box<dyn ...>, and Arc<dyn ...>,
and instead uses generic and associated types.

There are only two protocols QUIC and UDP; and the code which has to
work with both protocols can use a trivial thin enum wrapper.

With respect to connection-cache specifically:
* connection-cache/ConnectionCache is a single protocol cache which
  allows to use either QUIC or UDP without any build dependency on the
  other protocol.
* client/ConnectionCache is an enum wrapper around both protocols and
  can be used in the code which has to work with both QUIC and UDP.

Co-authored-by: Tyera Eulberg <tyera@solana.com>
2023-02-09 00:50:44 +00:00
Lijun Wang ada6136a6c
Refactor connection cache to support generic msgs (#29774)
tpu-client/tpu_connection_cache is refactored out the module and moved to connection-cache/connection_cache and the logic in client/connection_cache is consolidated to connection-cache/connection_cache as well. client/connection_cache only has a thin wrapper which forward calls to connection-cache/connection_cache and deal with constructions of quic/udp connection cache for clients using them both.2.

The TpuConnection is refactored to ClientConnection to make it generic and functions renamed to be proper for other workflows. eg. tpu_addr -> server_addr, send_transaction --> send_data and etc...

The enum dispatch is removed so that we can make the bulk of code of quic and udp agnostic of each other. The client is possible to load quic or udp only into its runtime.

The generic type parameter in the tpu-client/tpu_connection_cache is removed in order to create both quic and udp connection cache and use the object to send transactions with multiple branching when sending data. The generic type parameters and associated types are dropped in other types in order to make the trait "object safe" for this purpose.

I have annotated the code explaining the reasoning and the refactoring source -> destination.

There is no functional changes

bench-tps has been performed for rpc-client, thin-client and tpu-client. And it is found the performance number largely match the ones before the refactoring.
2023-02-01 18:10:06 -08:00
Will Hickey 04a6a631bc
Bump version to v1.16 (#30028) 2023-01-31 17:48:33 -06:00
joeaba a12bf8c003
Update maintainers references (#29997)
* update maintainers references

* chore: update maintainers reference
2023-01-31 08:07:13 -05:00
Yihau Chen 9193b4221d
Revert "chore: workspace inheritance (#29509)" (#29892)
This reverts commit a67d239dde.
2023-01-25 15:50:41 +08:00
Yihau Chen a67d239dde
chore: workspace inheritance (#29509)
* introduce workspace.package

* introduce workspace.dependencies

* read version from root cargo.toml

* pass check when version = { workspace = true }

* don't bump version when version = { workspace = true }

* including workspace Cargo.toml when bump version

* programs/sbf use workspace inheritance

* fix increasing cargo version ignore program/sbf/Cargo.toml
2023-01-25 13:59:59 +08:00
Kevin Ji dd92f225bb
Use Ipv4Addr::{LOCALHOST, UNSPECIFIED} constants (#29813) 2023-01-23 16:49:51 -06:00
Pankaj Garg 164c929b27
Cleanup QUIC single signed client cert code (#29686) 2023-01-12 15:24:02 -08:00
Richard Patel 1b6024a101
disable rustls tls12 feature (#28956)
Co-authored-by: Richard Patel <ripatel@jumptrading.com>
2023-01-12 08:37:06 -08:00
Lijun Wang 7c8b846344
Update quinn versions (#29603)
* chore: bump quinn-udp from 0.1.3 to 0.3.2

Bumps [quinn-udp](https://github.com/quinn-rs/quinn) from 0.1.3 to 0.3.2.
- [Release notes](https://github.com/quinn-rs/quinn/releases)
- [Commits](https://github.com/quinn-rs/quinn/commits)

---
updated-dependencies:
- dependency-name: quinn-udp
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* Try to use quinn 0.9.3 and quinn-proto 0.9.2

* Update streamer and client for quic to support qunn 0.9.3

* Update Cargo.lock

* Fixed unit test failure for quic tests

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-11 10:08:22 -08:00
Brennan Watt dbf97daa78
Increase wait for chunk timeout in QUIC tests (#29629) 2023-01-11 07:16:41 -08:00
Brooks Prumo 76bfd1a9a1
Actually fixes non-binding `let` on a future (quic-client) (#29326) 2022-12-19 17:27:21 -05:00
Brooks Prumo d1722b7a7a
Fixes non-binding `let` on a future (quic-client) (#29300) 2022-12-17 03:01:22 -05:00
Lijun Wang ecea802fe6
Bidirectional quic communication support (#29155)
* Support bi-directional quic communication, use the same endpoint for the quic server and client
This is needed for supporting using quic for repair

* Added comments on the bi-directional communication tests

* Removed some debug logs

* clippy issue
2022-12-09 10:59:43 -08:00
behzad nouri 9524c9dbff patches errors from clippy::uninlined_format_args
https://rust-lang.github.io/rust-clippy/master/index.html#uninlined_format_args
2022-12-06 19:32:15 +00:00
Jon Cinque b1340d77a2
sdk: Make Packet::meta private, use accessor functions (#29092)
sdk: Make packet meta private
2022-12-06 12:54:49 +01:00
Tyera c32377b5af
Split out quic- and udp-client definitions (#28762)
* Move ConnectionCache back to solana-client, and duplicate ThinClient, TpuClient there

* Dedupe thin_client modules

* Dedupe tpu_client modules

* Move TpuClient to TpuConnectionCache

* Move ThinClient to TpuConnectionCache

* Move TpuConnection and quic/udp trait implementations back to solana-client

* Remove enum_dispatch from solana-tpu-client

* Move udp-client to its own crate

* Move quic-client to its own crate
2022-11-18 12:21:45 -07:00
Tyera Eulberg 8b63f73100
Remove Default impls for TpuConnectionCache and ConnectionPool clients (#28788)
* Remove TpuConnectionCache impl Default

* Add fallible ctors for Quic/UdpClients

* Add Quic/TpuClient ctor trait

* Remove Quic/UdpClient impl Default
2022-11-17 20:13:43 -07:00
ryleung-solana 842fa993c8
Quic fix calculation of staked number of concurrent streams (#28705)
* Fix calculation of staked number of concurrent streams
2022-11-03 20:45:44 +08:00
Tyera Eulberg 10db560278
Add TpuConnectionCache and begin disentangling quic and udp (#28080)
* Add new empty crates and tpu_connection_cache module

* Add BaseTpuConnection trait and impl for udp and quic

* Add ConnectionPool trait and impl for udp and quic (quic-client doesn't build)

* Add a couple quic-specific apis to QuicConfig (from ConnectionCache)

* Re-export quic- and udp-client modules to prep for move

* Move ConnectionCacheStats to new module

* Move consts

* Duplicate ConnectionCache into tpu_connection_cache (doesn't build)

* Move methods to QuicConfig and remove unneeded methods (doesn't build)

* Genericize new ConnectionCache

* Rename new struct

* Copy unit tests (tests don't build)

* Fixup unit tests

* Move quic-specific test to quic-client crate

* DRY by using ConnectionPool method

* Prevent evicting pool about to be used
2022-09-30 19:51:50 -06:00