Commit Graph

510 Commits

Author SHA1 Message Date
Richard Patel 048b9f670b
Set solana-tpu ALPN protocol ID (#26587) 2022-07-13 20:55:13 +02:00
Pankaj Garg 14ee87d793
Move QUIC TLS certificate code in its own file (#26569) 2022-07-12 13:34:37 -07:00
Pankaj Garg ea7448c568
Use client certs in QUIC to get peer's stake (#26477)
* Use client certs in QUIC to get peer's stake

* fixes to cert processing

* integrate the code

* clippy

* more cleanup

* sort cargo deps

* test fixes

* info -> debug
2022-07-11 18:06:40 +00:00
ryleung-solana 4772a93109
Quic handshake timeout (#26306)
* Implement timeout for the quic client and server handshake
2022-07-11 14:10:12 +08:00
Tyera Eulberg b8b521535c
Add Pubsub getVersion, and support programSubscribe filter mapping (#26482)
* Add pubsub getVersion api

* Generalize maybe_map_filters

* Add filter mapping to blocking PubsubClient

* Add version tracking to nonblocking PubsubClient

* Add filter mapping to nonblocking PubsubClient
2022-07-07 20:55:18 -06:00
Tyera Eulberg f2abbcaf9c
Enable base58 and base64 encoding parameters for Memcmp filters (#26437)
* Minor refactoring of client RpcProgramAccountsConfig handling

* Enable explicit base58/base64 encoding of Memcmp filters, including client backward compatibility with nodes on old software

* Deprecate Memcmp::Encoding

* Remove deprecation warnings in rpc

* Remove deprecation warnings in cli

* Update docs

* Make variants self-documenting
2022-07-06 22:39:03 -06:00
behzad nouri 6f5857a5db
removes feature gate code separating durable nonce from blockhash domain (#26055) 2022-07-06 12:03:13 +00:00
Nick Rempel 7e4a5de99c
Refactor ConnectionCache::use_quic (#26235)
* Remove UseQuic type

Move to storing the UdpSocket on ConnectionCache and accepting a bool

* Remove use_quic from ConnectionCache constructor

Replace with separate with_udp constructor to force callers to choose
2022-07-05 10:49:42 -07:00
Michael Vines f14922c204 Fix new 1.62 clippy complaint 2022-06-30 13:20:03 -07:00
Steven Luscher e17ed6b2b9
feat: web3.js RPC errors now hold the error `code` and `data` on the error object (#26318)
feat: web3.js RPC errors now hold the error code on the error object
2022-06-30 13:08:10 -07:00
ryleung-solana 826f3312cc
Replace unwraps with expects in quic-client to aid debugging (#26283) 2022-06-29 15:37:27 +08:00
steveluscher eb12983785 feat: add custom version HTTP header string to RPC requests 2022-06-29 06:59:44 +00:00
Jon Cinque 2436a2bea8
client: Use async connection in async TPU client (#25969)
* client: Add nonblocking QuicTpuConnection implementation

* Remove integer arithmetic

* client: Support sync and async connections in cache

* client: Use async connection in async TPU client

* Address feedback

* Rename Connection -> BaseTpuConnection
2022-06-28 11:01:49 -04:00
steveluscher 558dd53025 Add user-agent header to Solana rust client requests 2022-06-27 22:29:44 +00:00
Michael Vines f3639b76ce Remove some clippy lints 2022-06-22 09:23:22 -07:00
Pankaj Garg 43ff65ece9
Use single send socket in UdpTpuConnection (#26105) 2022-06-21 14:56:21 -07:00
Lijun Wang 61946a49c3
Weight concurrent streams by stake (#25993)
Weight concurrent streams by stake for staked nodes
Ported changes from #25056 after address merge conflicts and some refactoring
2022-06-21 12:06:44 -07:00
Michael Vines dd0852bc49 UdpTpuConnection no longer restricts its local port to the 8000-10000 range 2022-06-17 17:57:07 -04:00
Lijun Wang 29b597cea5
Connection pool support in connection cache and QUIC connection reliability improvement (#25793)
* Connection pool in connection cache and handle connection errors

1. The connection not has a pool of connections per address, configurable, default 4
2. The connections per address share a lazy initialized endpoint
3. Handle connection issues better, avoid race conditions
4. Various log improvement for help debug connection issues
2022-06-10 09:25:24 -07:00
behzad nouri 3c1ce3cc93
permanently disables durable nonces with chain blockhash domain (#25788)
https://github.com/solana-labs/solana/pull/25744
separated durable nonce and blockhash domains, which will stop double
execution going forward. However it is possible that a durable
transaction has *already* been executed once as a normal transaction and
it is now a valid durable transaction. #25744 cannot stop such
transactions to be re-executed until the nonce accounts are advanced.

This commit adds a new nonce version indicating that the nonce is moved
out of the blockhash domain, and permanently disables durable
transactions for legacy nonces which are in the blockhash domain.
2022-06-09 15:28:37 +00:00
Jon Cinque 79a8ecd0ac
client: Remove static connection cache, plumb it instead (#25667)
* client: Remove static connection cache, plumb it instead

* Add TpuClient::new_with_connection_cache to not break downstream

* Refactor get_connection and RwLock into ConnectionCache

* Fix merge conflicts from new async TpuClient

* Remove `ConnectionCache::set_use_quic`

* Move DEFAULT_TPU_USE_QUIC to client, use ConnectionCache::default()
2022-06-08 13:57:12 +02:00
Jon Cinque 67a11ce4b1
client: Add nonblocking udp client and connection trait (#25775)
* client: Add nonblocking udp client

* Address feedback
2022-06-06 19:28:31 +02:00
behzad nouri 5ee157f43d separates durable nonce and blockhash domains
AdvanceNonceAccount instruction updates nonce to blockhash. This makes it
possible that a durable transaction is executed twice both as a normal
transaction and a nonce transaction if it uses blockhash (as opposed to nonce)
for its recent_blockhash field.

The commit prevents this double execution by separating nonce and blockhash
domains; when advancing nonce account, blockhash is hashed with a fixed string.
As a result a blockhash cannot be a valid nonce value; and if transaction was
once executed as a normal transaction it cannot be re-executed as a durable
transaction again and vice-versa.
2022-06-04 12:39:31 +00:00
Pankaj Garg 1c2ae470c5
Fix forwarding of transactions over QUIC (#25674)
* Spawn QUIC server to receive forwarded txs

* Update validator port range

* forward votes using UDP

* no forwarding from unstaked nodes

* forwarding stats in banking stage

* fix test builds

* fix lifetime of forward sender
2022-06-02 11:14:58 -07:00
segfaultdoctor edefaa437f
use tokio sleep inside async (#25680)
Co-authored-by: Zano <segfaultdoctor@protonmail.com>
2022-06-02 00:49:51 -06:00
Jon Cinque 10afdfee90
client: Refactor quic_client to nonblocking module (#25501) 2022-06-02 01:20:11 +02:00
Jon Cinque 6349a6dc97
tpu-client: Add "fake" async version with sync sending (#25499)
* tpu-client: Add "fake" async for sending

* Add Drop implementation

* Rebase
2022-06-01 23:41:35 +02:00
ryleung-solana 1ca5c3a7bd
Switch to using enum-dispatch to switch between UDP and Quic (#24713) 2022-05-26 11:21:16 -04:00
Michael Vines b05c7d91ed Fix derive_partial_eq_without_eq clippy lint 2022-05-22 22:22:21 -07:00
Jon Cinque 09ea899a2d
tpu-client: Refactor to prep for async client (#25432) 2022-05-21 20:22:30 +02:00
Lijun Wang 4e9b189154
Remove noisy metrics on open_uni (#25430) 2022-05-20 16:05:42 -07:00
Michael Vines c54e06355f
voteSubscribe pubsub notification now includes the vote transaction signature (#25291) 2022-05-19 18:28:46 -07:00
ryleung-solana b857528a03
Implement updating the connection cache stats in the client (#25308) 2022-05-19 15:29:27 -04:00
Lijun Wang 2266dfc696
Remove unnecessary async marker for create_endpoint (#25290)
* Add metrics for open_uni

* Remove unnecessary async marker for create_endpoint

* Removed noisy metrics
2022-05-19 01:17:29 -07:00
Lijun Wang 81d9a7585d
Add make_connection metrics (#25280)
* Add metrics for open_uni

* Added metrics for make_connection

* measure make_connection success or fail
2022-05-18 13:23:29 -07:00
Brooks Prumo dff089e0dd
Fix RpcResponseContext in rpc_client test (#25296) 2022-05-17 12:44:43 -05:00
Brooks Prumo e608580040
Add get_stake_minimum_delegation() to rpc_client (#25200) 2022-05-17 11:46:34 -05:00
Justin Starry 4690956849
RPC: Add minContextSlot param to ensure consistent responses (#25161)
* RPC: Add minContextSlot param to ensure consistent responses

* revert changes to deprecated endpoints

* update docs
2022-05-17 14:46:02 +08:00
Jack May 21e066ef26
Merge pull request from GHSA-wxrh-ff9f-fc6w 2022-05-16 12:30:37 -07:00
Lijun Wang 196de1c043
Refactor make_connection (#25225)
1. Move logics related to creating endpoint, creating new connection and retry 0rtt connections into a wrapper construct QuicNewConnection to put the logic together.
2. get_or_add_connection: logic is much simplified to allow manage the connection cache -- the QUIC connection is lazy constructed. get_or_add_connection should no longer be a global hotspot.
3.  Per connection stats update is moved out of get_or_add_connection to avoid bad connection impacting good connections.
2022-05-16 11:13:18 -07:00
Lijun Wang abd4ef889e
Fixed a race condition where in multiple thread environment that wasteful (#25176)
Duplicate and useless connections are being created which overloads the server and cause unreliable
connections on the client side.
2022-05-14 15:02:13 -07:00
Justin Starry a118af069e
RPC: Add API version to context response (#25134)
* RPC: Add API version to context response

* restore backwards compatibility
2022-05-12 12:17:21 +08:00
Brennan Watt 301a655f06
Enable 0rtt (#25055)
* Enable 0rtt connections
2022-05-10 09:44:07 -07:00
Pankaj Garg b4c3b66f1a
Replace HashMap and Vec with IndexMap in connection cache (#24967) 2022-05-04 12:38:03 -07:00
Brian Anderson 4d7e231891
Make nonblocking RpcClient examples use nonblocking RpcClient (#24916)
* Make nonblocking RpcClient examples use nonblocking RpcClient

* fmt

Co-authored-by: Tyera Eulberg <tyera@solana.com>
2022-05-03 16:06:57 -06:00
Pankaj Garg 3dad27d84d
Optimize connection cache eviction logic (#24911)
* Optimize connection cache eviction logic

* add eviction count and timing to metrics

* use swap_remove
2022-05-03 02:27:40 +00:00
Pankaj Garg de027a895b
Use RwLock instead of Mutex in QUIC connection cache (#24857)
* Use RwLock instead of Mutex in QUIC connection cache

* replace LruCache with HashMap

* fix tests

* fix tests

* refactor

* add cache eviction for a random connection on reaching upperbound

* cleanup
2022-05-02 13:02:49 -07:00
Pankaj Garg 21ee012c18
Add some timing measure for get_connection() (#24814)
* Add some timing measure for get_connection()

* fix build issue
2022-04-29 09:58:01 -07:00
Tyera Eulberg b42f34a7b8
Update Rpc filters to support token-2022 (#24541)
* Add TokenAccountState rpc_filter

* Use a method on RpcFliterType instead of helper
2022-04-20 19:18:12 -06:00
sakridge b64791ef79
Add packets and batches sent to quic client stats (#24442) 2022-04-20 13:59:54 +02:00
steveluscher 6c5a3ca4a0 docs: document the limits on RPC API inputs and outputs 2022-04-16 09:01:33 -07:00
sakridge 1b7d1f78de
Implement QUIC connection warmup service for future leaders (#24054)
* Increase connection timeouts

* Bump quic connection cache to 1024

* Use constant for quic connection timeout and add warm cache service

* Fixes to QUIC warmup service

* fix check failure

* fixes after rebase

* fix timeout test

Co-authored-by: Pankaj Garg <pankaj@solana.com>
2022-04-15 12:09:24 -07:00
Tyera Eulberg f7d557d5ae
Update simulateTransaction rpc handling of return_data, and update docs (#24355)
* Stringify return_data program_id; also camel-case all fields

* Update simulateTransaction json-rpc docs

* Base64-encode return data in simulation RPC responses
2022-04-14 23:42:08 -06:00
ryleung-solana 8cfc010b84
Send async quic batch of txs (#24298)
Add an interface send_wire_transaction_batch_async to TpuConnection to allow for sending batches without waiting for completion

Co-authored-by: Anatoly Yakovenko <anatoly@solana.com>
2022-04-14 22:20:34 -04:00
Tyera Eulberg 26899359d1
Support quic in bench-tps (#24295)
* Update comment

* Use connection_cache in tpu_client

* Add --tpu-use-quic to bench-tps

* Use connection_cache async send
2022-04-13 12:17:10 -06:00
sakridge e7fcda1424
Quic client stats (#24195)
* Add metrics to connection-cache to measure cache hits and misses

* Add congestion stats

* Add more client stats

* Review comments

Co-authored-by: Ryan Leung <ryan.leung@solana.com>
2022-04-13 05:04:40 +02:00
Tyera Eulberg 8487030ea6
Add TpuClient support to bench-tps (#24227)
* Add fallible send methods, and rpc_client helper

* Add helper to return RpcClient url

* Implement BenchTpsClient for TpuClient

* Add cli rpc and identity handling

* Handle different kinds of clients in main, use TpuClient

* Add tpu_client integration test
2022-04-12 09:43:29 -06:00
anatoly yakovenko 474080608a
Async send for send transaction service (#24265)
* async send
2022-04-12 07:15:59 -07:00
Tyera Eulberg 3871c85fd7
Add BenchTpsClient trait (#24208)
* Add BenchTpsClient

* Impl BenchTpsClient for used clients

* Use BenchTpsClient in do_bench

* Update integration test to use faucet via rpc

* Support keypairs from file that are not prefunded

* Remove old perf-utils
2022-04-11 13:45:40 -06:00
BG Zhu 22224127e0
Refactor thin_client::create_client (#24067)
Refactor the thin_client::create_client to take addresses separately instead of as a tuple

Co-authored-by: Bijie Zhu <bijiezhu@Bijies-MBP.cable.rcn.com>
2022-04-06 11:03:38 -04:00
ryleung-solana a38bd4acc8
Use LRU in connection-cache (#24109)
Switch to using LRU for connection-cache
2022-04-06 10:58:32 -04:00
hana 41f2fd7fca
Implement get_account_with_config (#23997). (#24095) 2022-04-04 22:58:58 +00:00
Justin Starry 792bbf75ab
Support sending versioned txs in AsyncClient (#23982) 2022-04-02 11:12:02 +08:00
ryleung-solana 8b72200afb
Thin client quic (#23973)
Change thin-client to use connection-cache
2022-03-31 15:47:00 -04:00
Justin Starry 88326533ed
Add SDK support for creating transactions with address table lookups (#23728)
* Add SDK support for creating transactions with address table lookups

* fix bpf compilation

* rename compile error variants to indicate overflow

* Add doc tests

* fix bpf compatibility

* use constant for overflow tests

* Use cfg_attr for dead code attribute

* resolve merge conflict
2022-03-31 17:44:20 +08:00
Brian Anderson 210d98bc06 Document APIs related to durable transaction nonces 2022-03-30 22:49:29 -06:00
ryleung-solana 82945ba973
Optimize TpuConnection and its implementations and refactor connection-cache to not use dyn in order to enable those changes (#23877) 2022-03-24 11:40:26 -04:00
Jon Cinque 7af48465fa
transaction-status: Add return data to meta (#23688)
* transaction-status: Add return data to meta

* Add return data to simulation results

* Use pretty-hex for printing return data

* Update arg name, make TransactionRecord struct

* Rename TransactionRecord -> ExecutionRecord
2022-03-22 23:17:05 +01:00
Justin Starry 016d3c450a
Update TpuConnection interface to be compatible with versioned txs (#23760)
* Update TpuConnection interface to be compatible with versioned txs

* Add convenience method for sending txs

* use parallel iterator to serialize transactions
2022-03-22 09:45:22 +08:00
Lijun Wang ae76fe2bd7
Made connection cache configurable. (#23783)
Added command-line argument tpu-use-quic argument.
Changed connection cache to return different connections based on the config.
2022-03-21 09:31:37 -07:00
Lijun Wang 8b230b86cc
Use borrow instead of move in interfaces defined by TpuConnection (#23734)
* Use borrow instead of move in interfaces defined by TpuConnection to avoid data copy

* Removed a few more unnecessary whole array slicing.
2022-03-17 13:31:11 -07:00
ryleung-solana 9b46f9b2da
Quic Connection Cache (#23598)
Add a connection cache to allow add modules that send data to get or create connections (e.g. for quic) associated with a certain SocketAddr
2022-03-15 18:16:35 -04:00
Brian Anderson 72af687aa6
Fix broken links in solana-client (#23503)
* Fix broken links in solana-client

* Revise docs for RpcClient::new_sender

* Add docs for RpcClient::new_mock etc

* Fix doc warnings on RpcSender

* rustfmt
2022-03-09 21:42:23 -07:00
ryleung-solana 17b00ad3a4
Add quic-client module (#23166)
* Add quic-client module to send transactions via quic, abstracted behind the TpuConnection trait (along with a legacy UDP implementation of TpuConnection) and change thin-client to use TpuConnection
2022-03-09 21:33:05 -05:00
Brian Anderson 176fd23002
Continue making it possible to implement RpcSender (#23561)
* Make nonblocking RpcClient::new_sender pub

As is the blocking RpcClient::new_sender, per #23503.

* Make solana_client::rpc_sender pub

Per #17631 it is supposed to be possible to implement RpcSender in
order to call RpcClient::new_sender. As of now though it is
not possible to name RpcSender in order to implement it.
2022-03-09 14:07:51 -07:00
Justin Starry 249d926d1b
Refactor: Add transaction binary encoding enum (#23546) 2022-03-09 16:09:08 +08:00
Justin Starry 3114c199bd
Add RPC support for versioned transactions (#22530)
* Add RPC support for versioned transactions

* fix doc tests

* Add rpc test for versioned txs

* Switch to preflight bank
2022-03-08 15:20:34 +08:00
Justin Starry 38db1dead4
Refactor RPC tests (#23483) 2022-03-04 18:20:11 +08:00
Trent Nelson 6666f23c01 client: expose creating `RpcClient`s with custom `RpcSender` impls 2022-02-28 22:04:15 +00:00
dependabot[bot] 9d2232306e
chore: bump tungstenite from 0.16.0 to 0.17.2 (#23268)
* chore: bump tungstenite from 0.16.0 to 0.17.2

Bumps [tungstenite](https://github.com/snapview/tungstenite-rs) from 0.16.0 to 0.17.2.
- [Release notes](https://github.com/snapview/tungstenite-rs/releases)
- [Changelog](https://github.com/snapview/tungstenite-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/snapview/tungstenite-rs/compare/v0.16.0...v0.17.2)

---
updated-dependencies:
- dependency-name: tungstenite
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* Bump tokio-tungstenite in tandem

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
2022-02-24 01:39:55 -07:00
Bruno 2e3eafaa11
RPC Client: Use Tokio's RwLock instead of std to make using in async easier (#23299) 2022-02-23 09:33:32 -08:00
Trent Nelson aeda27433f rpc: genericize client constructors 2022-02-16 10:05:46 -07:00
Michael Vines 331b953551 Add vote account address to vote subscription 2022-01-27 08:22:29 -08:00
Michael Vines 9d477d45c7 Propagate subscription failures to the caller 2022-01-27 08:19:03 -08:00
Michael Vines 8c376f58cb Drop feature, move to nonblocking mod 2022-01-27 08:19:03 -08:00
Kirill Fomichev a6a8a712e5 Add async WebSocket PubsubClient 2022-01-27 08:19:03 -08:00
Michael Vines 85e8bece2e Add solana_client::nonblocking::RpcClient 2022-01-26 17:58:00 -08:00
Michael Vines 6d5bbca630 Pacify clippy 2022-01-21 19:12:57 -08:00
segfaultdoctor 7171b3a3ac
Bugfix/block subscribe (#22516)
* use correct operation name

* require enable_rpc_transaction_history flag when enabling block_subscription

Co-authored-by: Zano <segfaultdoctor@protonmail.com>
2022-01-14 19:53:34 -07:00
Michael Vines e14ae33e86 Improve nonce-related error messages 2022-01-14 12:59:01 -08:00
Justin Starry f804ccdece
Store address table lookups in blockstore and bigtable (#22402) 2022-01-14 15:24:41 +08:00
Jeff Biseda 8b66625c95
convert std::sync::mpsc to crossbeam_channel (#22264) 2022-01-11 02:44:46 -08:00
Justin Starry 2a00382d71
Refactor: cleanup solana_transaction_status crate (#22230) 2022-01-03 15:45:18 +00:00
Kirill Fomichev 0a0fc85282 Add PubsubClient::vote_subscribe 2021-12-25 13:21:05 -08:00
segfaultdoctor 76098dd42a
RPC Block Subscription (#21787)
* add stuff

* compiling

* add notify block

* wip

* feat: add blockSubscribe pubsub method

* address PR comments

Co-authored-by: Lucas B <buffalu@jito.network>
Co-authored-by: Zano <segfaultdoctor@protonmail.com>
2021-12-17 16:03:09 -07:00
Tyera Eulberg 3e5a5a834f
Bump RpcClient node versions (#21612)
* Bump blockhash/fee api check versions

* Bump snapshot api check version
2021-12-04 21:51:43 +00:00
Michael Vines b8837c04ec Reformat imports to a consistent style for imports
rustfmt.toml configuration:
  imports_granularity = "One"
  group_imports = "One"
2021-12-03 09:19:13 -08:00
Tyera Eulberg 96e5342292
Update deprecated variant in RpcClient doctests (#21121) 2021-11-01 20:58:01 -06:00
Tyera Eulberg 1b47f7f2cd
Fixup filter deprecated since (#21123) 2021-11-01 19:06:38 -06:00
Jack May bced07a099
Update fee api to use blockhash (#21054) 2021-10-29 13:52:59 -07:00