Commit Graph

93 Commits

Author SHA1 Message Date
Andrew Fitzgerald 2c869ef778
QoS refactor: Allow pre-filtering (#31542) 2023-05-12 08:53:22 -07:00
Tao Zhu 49f44f5ded
Refactor pass feature status to deserialized packet via packet meta (#31549)
Add a flag to packet, set its value by packet_deserializer when received by banking_stage with working_bank
2023-05-11 09:31:05 -05:00
steviez 18a118b438
Condense banking_stage counters into existing datapoint (#31564)
Counters incur additional overhead in sending points to the MetricsAgent
over a crossbeam channel. Additionally, some of these counters would be
submitted by non-voting nodes which is just extra overhead and noise.

This change condenses several updates of a counter into a field of the
existing BankingStageStats metrics struct.
2023-05-10 15:44:42 -05:00
Tao Zhu fb7ba97afc
refactor: move test and bench only code from main code to where they are needed (#31550)
* refactor: move test and bench only code to specific location
* remove inactive bench test
2023-05-09 16:39:23 -05:00
behzad nouri 6a4a0418a6
removes hard-coded QUIC_PORT_OFFSET from connection-cache (#31541)
New ContactInfo has api identifying QUIC vs UDP ports; no need to hard-code
port-offset deep in connection-cache.
2023-05-09 13:46:17 +00:00
Andrew Fitzgerald 886aea21cb
Internal structs for ThreadAware AccountRead/WriteLocks (#31431) 2023-05-04 09:38:34 -07:00
Andrew Fitzgerald 18cd4311af
remove counters in hotpath (#31398) 2023-05-04 09:36:28 -07:00
Andrew Fitzgerald 7a393e479d
Scheduler Messages (#30976) 2023-04-19 15:14:47 -07:00
Andrew Fitzgerald 10d637d2e6
PohRecorder take Arc not &Arc for blockstore (#31234) 2023-04-19 11:41:18 -07:00
Andrew Fitzgerald 748220c9d3
Forwarder: Add common setup for tests (#31232) 2023-04-19 09:08:13 -07:00
Andrew Fitzgerald b657004141
store slot on BlockBatchUpdate (#31190) 2023-04-14 13:15:31 -07:00
Andrew Fitzgerald c847236147
decision maker perf (#30618) 2023-04-12 21:40:59 -07:00
Andrew Fitzgerald 01659edd16
Forwarder: forward_packets w/o metrics (#30925) 2023-04-12 14:09:24 -07:00
Ryo Onodera f0432ec50f
Avoid overflow in ThreadSet::any() and nits (#31098)
Avoid overflow in ThreadSet::any and etc
2023-04-07 12:45:29 +09:00
Andrew Fitzgerald 00250819b8
ThreadAwareAccountLocks (#30422) 2023-04-06 10:12:03 -07:00
Andrew Fitzgerald 8e910b494f
Forwarder: separate get_leader_and_addr (#30922) 2023-03-28 16:34:36 -07:00
Andrew Fitzgerald b72be0f086
Forwarder: clean up packet_vec filter (#30921) 2023-03-28 14:54:04 -07:00
Andrew Fitzgerald 2dfc46c71e
Forwarder: separate update_data_budget (#30920) 2023-03-28 12:58:39 -07:00
Andrew Fitzgerald f226a34f48
Only need bank reference for update (#30879) 2023-03-24 09:48:04 -07:00
Tao Zhu 52e63e2ffa
Allow banking_stage to update prioritization_fee_cache (#30853)
* Allow banking_stage to update prioritization_fee_cache

* Update core/src/banking_stage/committer.rs

Co-authored-by: Andrew Fitzgerald <apfitzge@gmail.com>

* move use to top

---------

Co-authored-by: Andrew Fitzgerald <apfitzge@gmail.com>
2023-03-24 00:05:54 +00:00
Andrew Fitzgerald ac8c31b5d6
PohRecorder::recorder -> new_recorder (#30838) 2023-03-22 13:19:20 +09:00
Andrew Fitzgerald b7e76c752f
Separate stats updates from decision_maker (#30481)
* Separate stats updates from decision_maker

* BufferedPacketsDecision::bank_start

* BufferedPacketsDecision: bank_start() doc-comment

* remove unnecessary clone
2023-03-15 19:39:48 -07:00
Andrew Fitzgerald 5f6755f58b
remove test fn (#30616)
kill banking_stage::Consumer test_fn
2023-03-10 09:30:25 -08:00
Andrew Fitzgerald b0112a5f43
BankingStage Consumer: test_buffered_packets* reworking (#30615)
* refactor test_consume_buffered_packets_interrupted without test_fn

* Fix comment

* Also check retries

* Add retryable test case
2023-03-09 09:13:04 -08:00
Andrew Fitzgerald bba0ed702f
BankingStage Refactor: Consumer State (#30288)
* BankingStage Refactor: Consumer add state

* remove trailing comma
2023-03-06 09:13:28 -08:00
Andrew Fitzgerald 50f553e245
Clean up: consumer saturating add assign (#30347)
Use saturating_add_assign where appropriate in Consumer
2023-02-16 15:19:43 -08:00
Andrew Fitzgerald 4194661bcf
Rewrite accumulate_execute_units_and_time without allocation (#30338) 2023-02-15 17:22:24 -08:00
Andrew Fitzgerald 1cefb90271
BankingStage Refactor: Simplify Consumer (#30253)
* measure! to measure_us!

* Consistent naming of transaction_recorder

* Remove outdated comment - Instant cannot be None

* use local

* Remove measure! import
2023-02-15 17:20:55 -08:00
Andrew Fitzgerald b86bfbb5c5
measure_us! use Instant and duration_to_us internally (#30339) 2023-02-15 12:43:47 -08:00
Andrew Fitzgerald beb3cd5ed9
BankingStage Refactor: Separate Consumer Module (#30238) 2023-02-15 08:52:13 -08:00
Andrew Fitzgerald 4b17acf64e
BankingStage Refactor: Add state to Committer (#30107) 2023-02-09 13:22:42 -08:00
behzad nouri 1ad69cfc38
removes dynamic cast and dynamic dispatch from connection-cache (#30128)
Dynamic dispatch forces heap allocation and adds extra overhead.
Dynamic casting as in the ones below, lacks compile-time type safety:
https://github.com/solana-labs/solana/blob/eeb622c4e/quic-client/src/lib.rs#L172-L175
https://github.com/solana-labs/solana/blob/eeb622c4e/udp-client/src/lib.rs#L52-L55

The commit removes all instances of Any, Box<dyn ...>, and Arc<dyn ...>,
and instead uses generic and associated types.

There are only two protocols QUIC and UDP; and the code which has to
work with both protocols can use a trivial thin enum wrapper.

With respect to connection-cache specifically:
* connection-cache/ConnectionCache is a single protocol cache which
  allows to use either QUIC or UDP without any build dependency on the
  other protocol.
* client/ConnectionCache is an enum wrapper around both protocols and
  can be used in the code which has to work with both QUIC and UDP.

Co-authored-by: Tyera Eulberg <tyera@solana.com>
2023-02-09 00:50:44 +00:00
Andrew Fitzgerald 7fb2fc6f27
Add comment on the closure (#30091) 2023-02-06 08:24:36 -08:00
Andrew Fitzgerald 8914d1af27
BankingStage Refactor: Add state to PacketReceiver (#30090) 2023-02-03 11:35:43 -08:00
Andrew Fitzgerald 8fa396a321
BankingStage Refactor: Add state to Forwarder (#29403) 2023-02-02 11:09:08 -08:00
Andrew Fitzgerald fd3f26380e
BankingStage Refactor: Simplify PacketReceiver (#29784) 2023-02-02 07:58:55 -08:00
Lijun Wang ada6136a6c
Refactor connection cache to support generic msgs (#29774)
tpu-client/tpu_connection_cache is refactored out the module and moved to connection-cache/connection_cache and the logic in client/connection_cache is consolidated to connection-cache/connection_cache as well. client/connection_cache only has a thin wrapper which forward calls to connection-cache/connection_cache and deal with constructions of quic/udp connection cache for clients using them both.2.

The TpuConnection is refactored to ClientConnection to make it generic and functions renamed to be proper for other workflows. eg. tpu_addr -> server_addr, send_transaction --> send_data and etc...

The enum dispatch is removed so that we can make the bulk of code of quic and udp agnostic of each other. The client is possible to load quic or udp only into its runtime.

The generic type parameter in the tpu-client/tpu_connection_cache is removed in order to create both quic and udp connection cache and use the object to send transactions with multiple branching when sending data. The generic type parameters and associated types are dropped in other types in order to make the trait "object safe" for this purpose.

I have annotated the code explaining the reasoning and the refactoring source -> destination.

There is no functional changes

bench-tps has been performed for rpc-client, thin-client and tpu-client. And it is found the performance number largely match the ones before the refactoring.
2023-02-01 18:10:06 -08:00
Andrew Fitzgerald c549129974
BankingStage Refactor: Committer Simplify (#29958) 2023-02-01 15:44:53 -08:00
Andrew Fitzgerald c06053f505
BankingStage Refactor: Add state to DecisionMaker (#29806) 2023-02-01 09:18:40 -08:00
Andrew Fitzgerald fbb90603a9
BankingStage Refactor: Separate transaction commiting module (#29808)
Separate transaction commiting module
2023-01-25 19:02:21 -08:00
Andrew Fitzgerald 704472ae13
BankingStage Refactor: Separate Forwarder Module (#29402)
Separate Forwarder module
2023-01-25 12:31:59 -08:00
apfitzge 8c793da7d0
BankingStage Refactor: Move decision making functions to new module (#29788)
Move decision making functions to new module
2023-01-20 10:10:47 -08:00
apfitzge 2c347ac0a5
BankingStage Refactor: Move packet receiving and buffering functions to separate module (#29761)
Move packet receiving and buffering functions to separate module
2023-01-19 08:52:32 -08:00