Commit Graph

478 Commits

Author SHA1 Message Date
ryleung-solana b857528a03
Implement updating the connection cache stats in the client (#25308) 2022-05-19 15:29:27 -04:00
Lijun Wang 2266dfc696
Remove unnecessary async marker for create_endpoint (#25290)
* Add metrics for open_uni

* Remove unnecessary async marker for create_endpoint

* Removed noisy metrics
2022-05-19 01:17:29 -07:00
Lijun Wang 81d9a7585d
Add make_connection metrics (#25280)
* Add metrics for open_uni

* Added metrics for make_connection

* measure make_connection success or fail
2022-05-18 13:23:29 -07:00
Brooks Prumo dff089e0dd
Fix RpcResponseContext in rpc_client test (#25296) 2022-05-17 12:44:43 -05:00
Brooks Prumo e608580040
Add get_stake_minimum_delegation() to rpc_client (#25200) 2022-05-17 11:46:34 -05:00
Justin Starry 4690956849
RPC: Add minContextSlot param to ensure consistent responses (#25161)
* RPC: Add minContextSlot param to ensure consistent responses

* revert changes to deprecated endpoints

* update docs
2022-05-17 14:46:02 +08:00
Jack May 21e066ef26
Merge pull request from GHSA-wxrh-ff9f-fc6w 2022-05-16 12:30:37 -07:00
Lijun Wang 196de1c043
Refactor make_connection (#25225)
1. Move logics related to creating endpoint, creating new connection and retry 0rtt connections into a wrapper construct QuicNewConnection to put the logic together.
2. get_or_add_connection: logic is much simplified to allow manage the connection cache -- the QUIC connection is lazy constructed. get_or_add_connection should no longer be a global hotspot.
3.  Per connection stats update is moved out of get_or_add_connection to avoid bad connection impacting good connections.
2022-05-16 11:13:18 -07:00
Lijun Wang abd4ef889e
Fixed a race condition where in multiple thread environment that wasteful (#25176)
Duplicate and useless connections are being created which overloads the server and cause unreliable
connections on the client side.
2022-05-14 15:02:13 -07:00
Justin Starry a118af069e
RPC: Add API version to context response (#25134)
* RPC: Add API version to context response

* restore backwards compatibility
2022-05-12 12:17:21 +08:00
Brennan Watt 301a655f06
Enable 0rtt (#25055)
* Enable 0rtt connections
2022-05-10 09:44:07 -07:00
Pankaj Garg b4c3b66f1a
Replace HashMap and Vec with IndexMap in connection cache (#24967) 2022-05-04 12:38:03 -07:00
Brian Anderson 4d7e231891
Make nonblocking RpcClient examples use nonblocking RpcClient (#24916)
* Make nonblocking RpcClient examples use nonblocking RpcClient

* fmt

Co-authored-by: Tyera Eulberg <tyera@solana.com>
2022-05-03 16:06:57 -06:00
Pankaj Garg 3dad27d84d
Optimize connection cache eviction logic (#24911)
* Optimize connection cache eviction logic

* add eviction count and timing to metrics

* use swap_remove
2022-05-03 02:27:40 +00:00
Pankaj Garg de027a895b
Use RwLock instead of Mutex in QUIC connection cache (#24857)
* Use RwLock instead of Mutex in QUIC connection cache

* replace LruCache with HashMap

* fix tests

* fix tests

* refactor

* add cache eviction for a random connection on reaching upperbound

* cleanup
2022-05-02 13:02:49 -07:00
Pankaj Garg 21ee012c18
Add some timing measure for get_connection() (#24814)
* Add some timing measure for get_connection()

* fix build issue
2022-04-29 09:58:01 -07:00
Tyera Eulberg b42f34a7b8
Update Rpc filters to support token-2022 (#24541)
* Add TokenAccountState rpc_filter

* Use a method on RpcFliterType instead of helper
2022-04-20 19:18:12 -06:00
sakridge b64791ef79
Add packets and batches sent to quic client stats (#24442) 2022-04-20 13:59:54 +02:00
steveluscher 6c5a3ca4a0 docs: document the limits on RPC API inputs and outputs 2022-04-16 09:01:33 -07:00
sakridge 1b7d1f78de
Implement QUIC connection warmup service for future leaders (#24054)
* Increase connection timeouts

* Bump quic connection cache to 1024

* Use constant for quic connection timeout and add warm cache service

* Fixes to QUIC warmup service

* fix check failure

* fixes after rebase

* fix timeout test

Co-authored-by: Pankaj Garg <pankaj@solana.com>
2022-04-15 12:09:24 -07:00
Tyera Eulberg f7d557d5ae
Update simulateTransaction rpc handling of return_data, and update docs (#24355)
* Stringify return_data program_id; also camel-case all fields

* Update simulateTransaction json-rpc docs

* Base64-encode return data in simulation RPC responses
2022-04-14 23:42:08 -06:00
ryleung-solana 8cfc010b84
Send async quic batch of txs (#24298)
Add an interface send_wire_transaction_batch_async to TpuConnection to allow for sending batches without waiting for completion

Co-authored-by: Anatoly Yakovenko <anatoly@solana.com>
2022-04-14 22:20:34 -04:00
Tyera Eulberg 26899359d1
Support quic in bench-tps (#24295)
* Update comment

* Use connection_cache in tpu_client

* Add --tpu-use-quic to bench-tps

* Use connection_cache async send
2022-04-13 12:17:10 -06:00
sakridge e7fcda1424
Quic client stats (#24195)
* Add metrics to connection-cache to measure cache hits and misses

* Add congestion stats

* Add more client stats

* Review comments

Co-authored-by: Ryan Leung <ryan.leung@solana.com>
2022-04-13 05:04:40 +02:00
Tyera Eulberg 8487030ea6
Add TpuClient support to bench-tps (#24227)
* Add fallible send methods, and rpc_client helper

* Add helper to return RpcClient url

* Implement BenchTpsClient for TpuClient

* Add cli rpc and identity handling

* Handle different kinds of clients in main, use TpuClient

* Add tpu_client integration test
2022-04-12 09:43:29 -06:00
anatoly yakovenko 474080608a
Async send for send transaction service (#24265)
* async send
2022-04-12 07:15:59 -07:00
Tyera Eulberg 3871c85fd7
Add BenchTpsClient trait (#24208)
* Add BenchTpsClient

* Impl BenchTpsClient for used clients

* Use BenchTpsClient in do_bench

* Update integration test to use faucet via rpc

* Support keypairs from file that are not prefunded

* Remove old perf-utils
2022-04-11 13:45:40 -06:00
BG Zhu 22224127e0
Refactor thin_client::create_client (#24067)
Refactor the thin_client::create_client to take addresses separately instead of as a tuple

Co-authored-by: Bijie Zhu <bijiezhu@Bijies-MBP.cable.rcn.com>
2022-04-06 11:03:38 -04:00
ryleung-solana a38bd4acc8
Use LRU in connection-cache (#24109)
Switch to using LRU for connection-cache
2022-04-06 10:58:32 -04:00
hana 41f2fd7fca
Implement get_account_with_config (#23997). (#24095) 2022-04-04 22:58:58 +00:00
Justin Starry 792bbf75ab
Support sending versioned txs in AsyncClient (#23982) 2022-04-02 11:12:02 +08:00
ryleung-solana 8b72200afb
Thin client quic (#23973)
Change thin-client to use connection-cache
2022-03-31 15:47:00 -04:00
Justin Starry 88326533ed
Add SDK support for creating transactions with address table lookups (#23728)
* Add SDK support for creating transactions with address table lookups

* fix bpf compilation

* rename compile error variants to indicate overflow

* Add doc tests

* fix bpf compatibility

* use constant for overflow tests

* Use cfg_attr for dead code attribute

* resolve merge conflict
2022-03-31 17:44:20 +08:00
Brian Anderson 210d98bc06 Document APIs related to durable transaction nonces 2022-03-30 22:49:29 -06:00
ryleung-solana 82945ba973
Optimize TpuConnection and its implementations and refactor connection-cache to not use dyn in order to enable those changes (#23877) 2022-03-24 11:40:26 -04:00
Jon Cinque 7af48465fa
transaction-status: Add return data to meta (#23688)
* transaction-status: Add return data to meta

* Add return data to simulation results

* Use pretty-hex for printing return data

* Update arg name, make TransactionRecord struct

* Rename TransactionRecord -> ExecutionRecord
2022-03-22 23:17:05 +01:00
Justin Starry 016d3c450a
Update TpuConnection interface to be compatible with versioned txs (#23760)
* Update TpuConnection interface to be compatible with versioned txs

* Add convenience method for sending txs

* use parallel iterator to serialize transactions
2022-03-22 09:45:22 +08:00
Lijun Wang ae76fe2bd7
Made connection cache configurable. (#23783)
Added command-line argument tpu-use-quic argument.
Changed connection cache to return different connections based on the config.
2022-03-21 09:31:37 -07:00
Lijun Wang 8b230b86cc
Use borrow instead of move in interfaces defined by TpuConnection (#23734)
* Use borrow instead of move in interfaces defined by TpuConnection to avoid data copy

* Removed a few more unnecessary whole array slicing.
2022-03-17 13:31:11 -07:00
ryleung-solana 9b46f9b2da
Quic Connection Cache (#23598)
Add a connection cache to allow add modules that send data to get or create connections (e.g. for quic) associated with a certain SocketAddr
2022-03-15 18:16:35 -04:00
Brian Anderson 72af687aa6
Fix broken links in solana-client (#23503)
* Fix broken links in solana-client

* Revise docs for RpcClient::new_sender

* Add docs for RpcClient::new_mock etc

* Fix doc warnings on RpcSender

* rustfmt
2022-03-09 21:42:23 -07:00
ryleung-solana 17b00ad3a4
Add quic-client module (#23166)
* Add quic-client module to send transactions via quic, abstracted behind the TpuConnection trait (along with a legacy UDP implementation of TpuConnection) and change thin-client to use TpuConnection
2022-03-09 21:33:05 -05:00
Brian Anderson 176fd23002
Continue making it possible to implement RpcSender (#23561)
* Make nonblocking RpcClient::new_sender pub

As is the blocking RpcClient::new_sender, per #23503.

* Make solana_client::rpc_sender pub

Per #17631 it is supposed to be possible to implement RpcSender in
order to call RpcClient::new_sender. As of now though it is
not possible to name RpcSender in order to implement it.
2022-03-09 14:07:51 -07:00
Justin Starry 249d926d1b
Refactor: Add transaction binary encoding enum (#23546) 2022-03-09 16:09:08 +08:00
Justin Starry 3114c199bd
Add RPC support for versioned transactions (#22530)
* Add RPC support for versioned transactions

* fix doc tests

* Add rpc test for versioned txs

* Switch to preflight bank
2022-03-08 15:20:34 +08:00
Justin Starry 38db1dead4
Refactor RPC tests (#23483) 2022-03-04 18:20:11 +08:00
Trent Nelson 6666f23c01 client: expose creating `RpcClient`s with custom `RpcSender` impls 2022-02-28 22:04:15 +00:00
dependabot[bot] 9d2232306e
chore: bump tungstenite from 0.16.0 to 0.17.2 (#23268)
* chore: bump tungstenite from 0.16.0 to 0.17.2

Bumps [tungstenite](https://github.com/snapview/tungstenite-rs) from 0.16.0 to 0.17.2.
- [Release notes](https://github.com/snapview/tungstenite-rs/releases)
- [Changelog](https://github.com/snapview/tungstenite-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/snapview/tungstenite-rs/compare/v0.16.0...v0.17.2)

---
updated-dependencies:
- dependency-name: tungstenite
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* Bump tokio-tungstenite in tandem

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
2022-02-24 01:39:55 -07:00
Bruno 2e3eafaa11
RPC Client: Use Tokio's RwLock instead of std to make using in async easier (#23299) 2022-02-23 09:33:32 -08:00
Trent Nelson aeda27433f rpc: genericize client constructors 2022-02-16 10:05:46 -07:00
Michael Vines 331b953551 Add vote account address to vote subscription 2022-01-27 08:22:29 -08:00
Michael Vines 9d477d45c7 Propagate subscription failures to the caller 2022-01-27 08:19:03 -08:00
Michael Vines 8c376f58cb Drop feature, move to nonblocking mod 2022-01-27 08:19:03 -08:00
Kirill Fomichev a6a8a712e5 Add async WebSocket PubsubClient 2022-01-27 08:19:03 -08:00
Michael Vines 85e8bece2e Add solana_client::nonblocking::RpcClient 2022-01-26 17:58:00 -08:00
Michael Vines 6d5bbca630 Pacify clippy 2022-01-21 19:12:57 -08:00
segfaultdoctor 7171b3a3ac
Bugfix/block subscribe (#22516)
* use correct operation name

* require enable_rpc_transaction_history flag when enabling block_subscription

Co-authored-by: Zano <segfaultdoctor@protonmail.com>
2022-01-14 19:53:34 -07:00
Michael Vines e14ae33e86 Improve nonce-related error messages 2022-01-14 12:59:01 -08:00
Justin Starry f804ccdece
Store address table lookups in blockstore and bigtable (#22402) 2022-01-14 15:24:41 +08:00
Jeff Biseda 8b66625c95
convert std::sync::mpsc to crossbeam_channel (#22264) 2022-01-11 02:44:46 -08:00
Justin Starry 2a00382d71
Refactor: cleanup solana_transaction_status crate (#22230) 2022-01-03 15:45:18 +00:00
Kirill Fomichev 0a0fc85282 Add PubsubClient::vote_subscribe 2021-12-25 13:21:05 -08:00
segfaultdoctor 76098dd42a
RPC Block Subscription (#21787)
* add stuff

* compiling

* add notify block

* wip

* feat: add blockSubscribe pubsub method

* address PR comments

Co-authored-by: Lucas B <buffalu@jito.network>
Co-authored-by: Zano <segfaultdoctor@protonmail.com>
2021-12-17 16:03:09 -07:00
Tyera Eulberg 3e5a5a834f
Bump RpcClient node versions (#21612)
* Bump blockhash/fee api check versions

* Bump snapshot api check version
2021-12-04 21:51:43 +00:00
Michael Vines b8837c04ec Reformat imports to a consistent style for imports
rustfmt.toml configuration:
  imports_granularity = "One"
  group_imports = "One"
2021-12-03 09:19:13 -08:00
Tyera Eulberg 96e5342292
Update deprecated variant in RpcClient doctests (#21121) 2021-11-01 20:58:01 -06:00
Tyera Eulberg 1b47f7f2cd
Fixup filter deprecated since (#21123) 2021-11-01 19:06:38 -06:00
Jack May bced07a099
Update fee api to use blockhash (#21054) 2021-10-29 13:52:59 -07:00
Manuel Gil a0f9e0e8ee
Add missing websocket methods to rust RPC PubSub client (#21065)
- Added accountSubscribe,  programSubscribe, slotSubscribe and rootSubscribe to rust RpcClient
 - Removed duplication on cleanup threads
 - Moved RPCVote from rpc/ to client/rpc_response
2021-10-29 17:11:20 +00:00
sakridge 700e42d556
Different error if block status is not yet available (#20407)
* Different error if block is not available

* Add slot to error message

* Make and use helper function

* Check finalized path as well

Co-authored-by: Tyera Eulberg <tyera@solana.com>
2021-10-27 13:11:27 -06:00
Tyera Eulberg b2f6cfb9ff
Restore getProgramAccounts spl-token secondary-index functionality (#20993)
* Allow get_spl_token_X_filters to match on any encoding, and optimize earlier

* Remove redundant optimize calls

* Compress match statements

* Add method docs, including note to use optimize_filters before spl-token checks

* Add logs
2021-10-26 21:44:24 +00:00
Jon Cinque 5f7b60576f
tpu-client: Move `send_messages_with_spinner` from program (#20960)
We have too many ways of sending transactions, and too many
reimplementations of the same logic all over the place.

The program deploy logic and stake-o-matic currently make the
most use of the TPU client, so this merges their implementations into
one place to be reused by both.  Yay for consolidation!
2021-10-26 22:54:26 +02:00
Jack May 2515f6a04f
Update deprecation versions (#20959) 2021-10-26 09:06:41 -07:00
Tyera Eulberg fbc0919a1f
Bump fee-api check version (#20831) 2021-10-20 22:18:50 +00:00
Kirill Fomichev e9a427b9c8
Rpc: filters performance improvement (#20185)
* Add Base58,Base64,Bytes to MemcmpEncodedBytes

* Rpc: decode memcmp before filtering accounts

* Add deprecated attribute

* Add Memcmp::bytes

* Fix clippy for deprecated

* Another clippy fix

* merge RpcFilterError::DataTooLarge

* add deprecation for Base58DataTooLarge

* change filter data size limit

* strict data size len for base58

* add magic numbers

* fix tests
2021-10-14 12:43:27 -06:00
Jack May da45be366a
Remove blockhash from fee calculation (#20641) 2021-10-13 13:10:58 -07:00
sakridge 60fe29b63e
Move transaction executor (#19946) 2021-10-13 13:06:23 +02:00
Brian Anderson 1417c1456d
Reorder RpcClient method defs for more logical docs. (#20549)
Methods follow this order:

- Constructors
- send_and_confirm variations
- send variations
- confirm variations
- simulate variations
- queries
2021-10-11 12:12:05 -06:00
Michael Vines 7027d56064 Resolve nightly-2021-10-05 clippy complaints 2021-10-06 10:37:58 -07:00
Brian Anderson 082d5dc5b2
Add more docs for RpcClient (#19771)
* Add more docs for RpcClient

* Use custom mocks in rpc_client examples

* Move create_rpc_client_mocks into rpc_client module

Signed-off-by: Brian Anderson <andersrb@gmail.com>

* Update client/src/rpc_client.rs

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>

* Update RpcClient docs per review feedback

* Consistently link 'commitment level' in RpcClient docs

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
2021-09-27 14:59:25 -06:00
Kirill Fomichev ac79ae6848 Rpc: use rust convenient methods 2021-09-27 11:05:39 -07:00
Kirill Fomichev 9542bae56e Rpc: remove not required clone 2021-09-26 09:17:21 -07:00
Jon Cinque e9b066d497
client: Add retry logic on Pubsub 429s (#19990) 2021-09-18 12:40:43 +02:00
Pavel Strakhov 65227f44dc
Optimize RPC pubsub for multiple clients with the same subscription (#18943)
* reimplement rpc pubsub with a broadcast queue

* update tests for new pubsub implementation

* fix: fix review suggestions

* chore(rpc): add additional pubsub metrics

* integrate max subscriptions check into SubscriptionTracker to reduce locking

* separate subscription control from tracker

* limit memory usage of items in pubsub broadcast queue, improve error handling

* add more pubsub metrics

* add final count metrics to pubsub

* add metric for total number of subscriptions

* fix small review suggestions

* remove by_params from SubscriptionTracker and add node_progress_watchers map instead

* add subscription tracker tests

* add metrics for number of pubsub notifications as a counter

* ignore clippy lint in TokenCounter

* fix underflow in token counter

* reduce queue capacity in pubsub tests

* fix(rpc): fix test timeouts

* fix race in account subscription test

* Add RpcSubscriptions::new_for_tests

Co-authored-by: Pavel Strakhov <p.strakhov@iconic.vc>
Co-authored-by: Nikita Podoliako <n.podoliako@zubr.io>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
2021-09-17 13:40:14 -06:00
sakridge c8f76b8bd0
Fix tpu service fanout (#19944) 2021-09-17 01:01:56 +02:00
Michael Vines 21f4606212 Add RpcClient::get_transport_stats() 2021-09-09 20:11:15 -07:00
Michael Vines b83284c9f6 TpuClient now uses the processed slot to seed its leader schedule cache 2021-09-08 16:51:25 -05:00
Michael Vines 007fb3abe5 Improve comments, error handling, and retry delay handling 2021-09-08 10:51:55 -05:00
Michael Vines ae43ca3bfb Add RpcClient::get_multiple_accounts_with_config() 2021-09-03 22:41:38 +00:00
Brooks Prumo 38e1f7c4ba
Add backwards compatibility support to get_highest_snapshot_slot() (#19591)
Co-authored-by: Michael Vines <mvines@gmail.com>
2021-09-02 20:47:35 -05:00
Brooks Prumo 8ac94b2cf4
Add Incremental Snapshot support to RPC (#19559)
#### Problem

There's no way to get incremental snapshot information from RPC.

#### Summary of Changes

- Add new RPC method, `getHighestSnapshotSlot` that returns a `SnapshotSlotInfo`, which contains both the highest full snapshot slot, and the highest incremental snapshot slot _based on_ the full snapshot.
- Deprecate old RPC method, `getSnapshotSlot`
- Update API docs

Fixes #19579
2021-09-02 15:25:42 -05:00
Jack May f81bfc8462
Limit message encoding to base-64 (#19468) 2021-08-29 19:53:37 -07:00
Tyera Eulberg 7482861f4b
Add parameter to allow setting max-retries for SendTransaction rpc (#19387)
* Add parameter to cap rpc send retries for a tx

* Add parameter to docs
2021-08-24 22:44:13 -06:00
Brian Anderson 628ad58912
Document more RpcClient methods (#19236)
* Document more RpcClient methods

* Update client/src/rpc_client.rs

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>

* Update RpcClient docs per feedback

* Address review feedback

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
2021-08-23 18:49:37 -06:00
Tyera Eulberg cf3fc61821
Add backward compatibility for new RpcClient methods (#19383) 2021-08-24 00:09:00 +00:00
Justin Starry c053df143f
RPC: add option to exclude accounts from get_supply (#19270) 2021-08-17 23:32:58 +00:00
dependabot[bot] 74cc2e5ffa
chore: bump tungstenite from 0.14.0 to 0.15.0 (#19222)
* chore: bump tungstenite from 0.14.0 to 0.15.0

Bumps [tungstenite](https://github.com/snapview/tungstenite-rs) from 0.14.0 to 0.15.0.
- [Release notes](https://github.com/snapview/tungstenite-rs/releases)
- [Changelog](https://github.com/snapview/tungstenite-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/snapview/tungstenite-rs/compare/v0.14.0...v0.15.0)

---
updated-dependencies:
- dependency-name: tungstenite
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

* Update stream type

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <anatoly+githubjenkins@solana.io>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
2021-08-13 11:57:22 -06:00
Jack May 0b50bb2b20
Deprecate FeeCalculator returning APIs (#19120) 2021-08-13 09:08:20 -07:00
Tyera Eulberg 5970083b4d
Use last_valid_block_height in services and client apps (#19163)
* Add deprecated tag to Bank::get_blockhash_last_valid_slot

* Update SendTransactionService to use last_valid_block_height

* Update solana-tokens to use last_valid_block_height

* Remove dangling file

* Update solana program to use last_valid_block_height

* Update Banks crates to use last_valid_block_height
2021-08-11 01:04:00 -06:00
Brian Anderson e4b66a5913
Fix deserialization of RPC errors in HttpSender (#19110)
Fixes #15576
2021-08-09 10:45:00 -07:00