Commit Graph

343 Commits

Author SHA1 Message Date
Michael Vines b05c7d91ed Fix derive_partial_eq_without_eq clippy lint 2022-05-22 22:22:21 -07:00
Pankaj Garg 7fb0ef1fa5
Use async send for forwarding transactions (#25435) 2022-05-20 21:20:47 -07:00
buffalu 971748b335
fix banking stage starvation (#25245) 2022-05-18 22:37:47 +02:00
Tao Zhu b1b3702e6d
Prioritize transactions in banking stage by their compute unit price (#25178)
* - get prioritization fee from compute_budget instruction;
- update compute_budget::process_instruction function to take instruction iter to support sanitized versioned message;
- updated runtime.md

* update transaction fee calculation for prioritization fee rate as lamports per 10K CUs

* review changes

* fix test

* fix a bpf test

* fix bpf test

* patch feedback

* fix clippy

* fix bpf test

* feedback

* rename prioritization fee rate to compute unit price

* feedback

Co-authored-by: Justin Starry <justin@solana.com>
2022-05-16 12:06:33 +08:00
sakridge 52db2e19bc
Lower default batch size to 64 and add 2 banking threads (#25226) 2022-05-15 16:52:47 +02:00
Pankaj Garg 71dd95e842
Tune banking_stage receive loop timing (#25172) 2022-05-13 03:42:08 +00:00
Pankaj Garg bcf4d54235
Update test_banking_stage_entryfication to be more deterministic (#25146)
* Update test_banking_stage_entryfication to be more deterministic

* revert to original test with updated checks
2022-05-12 15:36:19 +00:00
Justin Starry e3bdc38f0a
Add sanitized types for use in banking stage (#25067) 2022-05-11 00:30:48 +08:00
Pankaj Garg 362b0526cd
Greedy receive in banking stage (#25060)
* Greedy receive in banking stage

* add upperbound to batch size and batching time

* update test_banking_stage_entryfication test
2022-05-08 10:47:55 -07:00
Justin Starry 082502d4f3
Fail tx sanitization when ix program id uses lookup table (#25035)
* Fail tx sanitization when ix program id uses lookup table

* feedback
2022-05-07 03:19:50 +08:00
carllin 870ac80b79
Prioritize BankingStage packets individually in min-max heap (#24187) 2022-05-04 21:50:56 -05:00
Justin Starry a61652104b
Avoid holding lock guards in match expressions (#24805)
* Avoid holding bank forks read lock for RPC requests

* Avoid using lock guards in temporaries

* revert fetch stage change
2022-04-29 16:32:46 +08:00
carllin 8a062273de
Move error counters to be reported by leader only at end of slot (#24581)
* Add error counters to leader metrics only

* Add dependencies
2022-04-23 18:10:47 -05:00
Jon Cinque 0d51596224
sim: Override slot hashes account on simulation bank (#24543)
* sim: Override slot hashes during simulation

* Add simulation test program

* Address feedback

* Add AccountOverrides explicit type

* Cargo fmt
2022-04-22 12:32:31 +02:00
Justin Starry 02bfb85c16
Refactor transaction processing in banking stage (#24336)
* Refactor transaction processing in banking stage

* feedback

* more feedback
2022-04-21 21:06:26 +08:00
Tao Zhu a21fc3f303
Apply transaction actual execution units to cost_tracker (#24311)
* Pass the sum of consumed compute units to cost_tracker

* cost model tracks builtins and bpf programs separately, enabling adjust block cost by actual bpf programs execution costs

* Copied nightly-only experimental `checked_add_(un)signed` implementation to sdk

* Add function to update cost tracker with execution cost adjustment

* Review suggestion - using enum instead of struct for CommitTransactionDetails
Co-authored-by: Justin Starry <justin.m.starry@gmail.com>

* review - rename variable to distinguish accumulated_consumed_units from individual compute_units_consumed

* not to use signed integer operations

* Review - using saturating_add_assign!(), and checked_*().unwrap_or()

* Review - using Ordering enum to cmp

* replace checked_ with saturating_

* review - remove unnecessary Option<>

* Review - add function to report number of non-zero units account to metrics
2022-04-21 07:38:07 +00:00
Tao Zhu 94b0186a96
Cost model tracks builtins and bpf programs separately (#24468)
* Cost model tracks builtins and bpf programs separatele (enables adjusting block cost by actual bpf programs execution costs)

* Address reviews: expand test; add metrics stat
2022-04-19 13:25:47 -05:00
Christian Kamm d2c6c04d3e banking-bench: Add and rearrange options
- Add write-lock-contention option, replacing same_payer
- write-lock-contention also has a same-batch-only value, where
  contention happens only inside batches, not between them
- Rename num-threads to batches-per-iteration, which is closer to what
  it is actually doing.
- Add num-banking-threads as a new option
- Rename packets-per-chunk to packets-per-batch, because this is closer
  to what's happening; and it was previously confusing that num-chunks
  had little to do with packets-per-chunk.

Example output for a iterations=100 and a permutation of inputs:

contention,threads,batchsize,batchcount,tps
none,           3,192, 4,65290.30
none,           4,192, 4,77358.06
none,           5,192, 4,86436.65
none,           3, 12,64,43944.57
none,           4, 12,64,65852.15
none,           5, 12,64,70674.37
same-batch-only,3,192, 4,3928.21
same-batch-only,4,192, 4,6460.15
same-batch-only,5,192, 4,7242.85
same-batch-only,3, 12,64,11377.58
same-batch-only,4, 12,64,19582.79
same-batch-only,5, 12,64,24648.45
full,           3,192, 4,3914.26
full,           4,192, 4,2102.99
full,           5,192, 4,3041.87
full,           3, 12,64,11316.17
full,           4, 12,64,2224.99
full,           5, 12,64,5240.32
2022-04-18 09:43:46 -05:00
Tao Zhu 6bc6384f8e refactor to consolidate info into single return field 2022-04-17 19:26:24 -05:00
Tao Zhu 9dadfb2e2c Add checked_add_signed() to apply cost adjustment to cost_tracker 2022-04-17 19:26:24 -05:00
Tao Zhu 810b1dff40 undo cost of executed-but-not-recorded transactions from cost_tracker 2022-04-17 19:26:24 -05:00
Tao Zhu 23d365d02f Address review comment: extract transaction was_executed status to avoid cloning execution_results 2022-04-17 19:26:24 -05:00
Tao Zhu 29ca21ed78 undo transaction cost from cost_tracker if it was not executed successfully 2022-04-17 19:26:24 -05:00
Christian Kamm 97f2eb8e65 Banking stage: Deserialize packets only once
Benchmarks show roughly a 6% improvement. The impact could be more
significant when transactions need to be retried a lot.

after patch:
{'name': 'banking_bench_total', 'median': '72767.43'}
{'name': 'banking_bench_tx_total', 'median': '80240.38'}
{'name': 'banking_bench_success_tx_total', 'median': '72767.43'}
test bench_banking_stage_multi_accounts
... bench:   6,137,264 ns/iter (+/- 1,364,111)
test bench_banking_stage_multi_programs
... bench:  10,086,435 ns/iter (+/- 2,921,440)

before patch:
{'name': 'banking_bench_total', 'median': '68572.26'}
{'name': 'banking_bench_tx_total', 'median': '75704.75'}
{'name': 'banking_bench_success_tx_total', 'median': '68572.26'}
test bench_banking_stage_multi_accounts
... bench:   6,521,007 ns/iter (+/- 1,926,741)
test bench_banking_stage_multi_programs
... bench:  10,526,433 ns/iter (+/- 2,736,530)
2022-04-15 00:57:11 -06:00
Christian Kamm a058f348a2 Address review comments 2022-04-08 14:37:55 -05:00
Christian Kamm 2ed29771f2 Unittest for cost tracker after process_and_record_transactions 2022-04-08 14:37:55 -05:00
Christian Kamm 924b8ea1eb Adjustments to cost_tracker updates
- don't store pending tx signatures and costs in CostTracker
- apply tx costs to global state immediately again
- go from commit_or_cancel to update_or_remove, where the cost tracker
  is either updated with the true costs for successful tx, or the costs
  of a retryable tx is removed
- move the function into qos_service and hold the cost tracker lock for
  the whole loop
2022-04-08 14:37:55 -05:00
Tao Zhu 9e07272af8 - Only commit successfully executed transactions' cost to cost_tracker;
- In-fly transactions are pended in cost_tracker until being committed
  or cancelled;
2022-04-08 14:37:55 -05:00
ryleung-solana 6b85c2104c
Implement forwarding via TpuConnection (#23817) 2022-03-25 11:31:40 -04:00
Jon Cinque 7af48465fa
transaction-status: Add return data to meta (#23688)
* transaction-status: Add return data to meta

* Add return data to simulation results

* Use pretty-hex for printing return data

* Update arg name, make TransactionRecord struct

* Rename TransactionRecord -> ExecutionRecord
2022-03-22 23:17:05 +01:00
HaoranYi 45a7c6edfb
Fix typos and a small refactor (#23805)
* fix typo

* remove packet_has_more_unprocessed_transactions function
2022-03-21 18:35:31 -05:00
Tao Zhu eb73dacd58 harden banking tests 2022-03-15 09:58:08 -05:00
Justin Starry 8c8f9694e0
Refactor: Sanitized transaction creation (#23558)
* Refactor: SanitizedTransaction::try_create optionally computes hash

* Refactor: Add SimpleAddressLoader
2022-03-15 12:02:22 +08:00
Tao Zhu 8590911b0a Replace type alias with newtype for UnprocesedPacketBatches 2022-03-14 13:14:27 -05:00
Brooks Prumo 7758c32035
Banking Stage drops transactions that'll exceed the total account data size limit (#23537) 2022-03-13 15:58:57 +00:00
Tao Zhu 35d1235ed0
- move `unprocessed_packet_batches` from `BankingStage` to its own (#23508)
module
- deserialize packets during receving and buffering
2022-03-10 18:47:46 +00:00
carllin 588414a776
Report even if slot begins and ends in process_buffered_packets() (#23549) 2022-03-09 23:42:35 -05:00
Carl Lin 0a17edcc1f Revert "fix tests after merge"
This reverts commit ba2d83f580.
2022-03-08 17:18:44 -08:00
Justin Starry 3114c199bd
Add RPC support for versioned transactions (#22530)
* Add RPC support for versioned transactions

* fix doc tests

* Add rpc test for versioned txs

* Switch to preflight bank
2022-03-08 15:20:34 +08:00
HaoranYi 86e2f728c3
Fix a batch limits bug in banking (#23327)
* add thread index in thread name for debugging

* fix batch_limit

* use NUM_VOTE_THREAD instead of hardcoded number (credit to carllin)
2022-03-02 09:08:08 -06:00
carllin 619335df1a
Add execute timings (#23097) 2022-02-17 01:14:32 -05:00
Jeff Biseda 115d71536b
forward_buffered_packets return packet count in error path (#23167) 2022-02-16 07:46:32 -08:00
Tao Zhu 03bf66a51b flag end-of-slot when poh bank is gone 2022-02-15 15:01:27 -06:00
Lijun Wang c04438be4b
Retaining transaction logs when transaction plugin is loaded. (#22874)
Transaction logs are not being saved to the database through the plugin interface.

Summary of Changes

Retain the transaction logs when transaction notification plugin is loaded.

Fixes #
lijunwangs/solana-accountsdb-plugin-postgres#6
2022-02-11 20:29:07 -08:00
carllin 2f9e30a1f7
Introduce slot-specific packet metrics (#22906) 2022-02-11 03:07:45 -05:00
Justin Starry d5dec989b9
Enforce tx metadata upload with static types (#23028) 2022-02-10 13:28:18 +08:00
Tao Zhu ba2d83f580 fix tests after merge 2022-02-08 16:18:23 -06:00
Tao Zhu e52e48076e
bench should update leader schedule cache (#22991) 2022-02-08 02:28:28 +00:00
Tao Zhu 4bec182b32
Allow buffered packets be consumed if bank is active, regardless leader schedule (#22913) 2022-02-03 21:29:41 +00:00
Justin Starry 60af1a4cce
Refactor: Add trait for loading addresses (#22903) 2022-02-03 11:00:12 +00:00