4f86c0b74a
Rate limit transaction counters ( #5447 )
...
* Rate limit transaction counters
* @sakridge feedback
* Set default high metrics rate for multinode demo
* Fix tests
* Swap defaults and fix env var tests
* Only set metrics rate if not already set
2019-08-08 17:05:06 -04:00
a7a10e12c7
Forward transactions as packets instead of blobs ( #5334 )
...
* Forward transactions as packets instead of blobs
* clippy
2019-07-30 14:50:02 -07:00
aef7bae60d
Let grace ticks to roll over into multiple leader slots ( #5268 )
...
* Let grace ticks to roll over into multiple leader slots
* address review comments
2019-07-26 11:33:51 -07:00
afa05acb32
more replay_stage grooming ( #5163 )
2019-07-18 14:54:27 -07:00
fe87c05423
fix transaction_count ( #5110 )
...
* fix transaction_count
* add sig count to bank hash
2019-07-15 13:42:59 -07:00
22ef3c7c54
Blob verify ( #4951 )
...
* Ensure signable data is not out of range
* Add a broadcast stage that puts bad sizes in blobs
* Resign blob after modifyign size
* Remove assertions that fail when size != meta.size
2019-07-08 18:21:52 -07:00
c1953dca8f
Cleanup some of banking stage ( #4878 )
...
* Add committable transactions that cause errors like InstructionErrors back to retryable list on MaxHeightReached
* Remove unnecessary logic
* Add comments/renaming for clarity
2019-07-01 12:14:40 -07:00
a89589a1d5
Add Measure abstraction over measuring time intervals ( #4851 )
...
Allows one to swap in different implementations. This provides
the normal Insant::now() -> .elapsed() path.
2019-06-29 15:34:49 +02:00
0fa1af5d47
Cleanup num_threads() and batch_limit numbers ( #4852 )
2019-06-28 10:55:24 +02:00
66552d7047
Credit-Only Accounts: Cache account balance for thread-safe load/store ( #4691 )
...
* Implement CreditOnlyLocks
* Update credit-only atomic on account load
* Update credit-only atomic after bank.freeze_lock; store credits if all credit-only lock references are dropped
* Commit credit-only credits on bank freeze
* Update core to CreditAccountLocks
* Impl credit-only in System Transfer
* Rework CreditAccountLocks, test, and fix bugs
* Review comments: Pass CreditAccountLocks by reference; Tighten up insert block
* Only store credits on completed slot
* Check balance in bench_exchange funding to ensure commit_credits has completed
* Add is_debitable info to KeyedAccount meta to pass into programs
* Reinstate CreditOnlyLocks check on lock_account
* Rework CreditAccountLocks to remove strong_count usage
* Add multi-threaded credit-only locks test
* Improve RwLocks usage
* Review comments: panic if bad things happen; tighter code
* Assert lock_accounts race does not happen
* Revert panic if bad things happen; not a bad thing
2019-06-27 17:25:10 -04:00
fbea9d8621
Page-pin packet memory for cuda ( #4250 )
...
* Page-pin packet memory for cuda
Bring back recyclers and pin offset buffers
* Add packet recycler to streamer
* Add set_pinnable to sigverify vecs to pin them
* Add packets reset test
* Add test for recycler and reduce the gc lock critical section
* Add comments/tests to cuda_runtime
* Add recycler to recv_blobs path.
* Add trace/names for debug and PacketsRecycler to bench-streamer
* Predict realloc and unpin beforehand.
* Add helper to reserve and pin
* Cap buffered packets length
* Call cuda wrapper functions
2019-06-27 09:32:32 +02:00
97c97db97e
Fix early exit clearing all buffered packets ( #4810 )
2019-06-26 22:39:50 -07:00
9a52b01171
Change to crossbeam channel in banking_threads VerifiedReceiver ( #4822 )
...
* Add crossbeam channel instead of channel in banking_stage
2019-06-26 18:42:27 -07:00
74a06e4230
Update thinclient to resend the same tx until its blockhash expires ( #4807 )
2019-06-24 16:46:34 -07:00
3f8ff23125
Forward transactions to the leader for next Nth slot ( #4806 )
...
* review comments
2019-06-24 15:56:50 -07:00
06ba0b7279
Remove holding cluster_info lock while forwarding packets ( #4773 )
2019-06-21 15:21:49 -07:00
a4035a3c65
Remove record locks and parent locks from accounts ( #4633 )
...
* Revert "Fix parent record locks usage in child banks (#4159 )"
This reverts commit 69eeb7cf08#3819 )"
This reverts commit 083090817a
2019-06-10 22:05:46 -07:00
5bd3eb4557
Up number of threads ( #4541 )
2019-06-04 18:01:28 -07:00
1f71d05299
remove copying of forwarded packets ( #4425 )
...
automerge
2019-05-24 17:35:09 -07:00
cfe5afd34c
_id => _pubkey variable renaming ( #4419 )
...
* wallet: rename *_account_id to *_account_pubkey
* s/from_id/from_pubkey/g
* s/node_id/node_pubkey/g
* s/stake_id/stake_pubkey/g
* s/voter_id/voter_pubkey/g
* s/vote_id/vote_pubkey/g
* s/delegate_id/delegate_pubkey/g
* s/account_id/account_pubkey/g
* s/to_id/to_pubkey/g
* s/my_id/my_pubkey/g
* cargo fmt
* s/staker_id/staker_pubkey/g
* s/mining_pool_id/mining_pool_pubkey/g
* s/leader_id/leader_pubkey/g
* cargo fmt
* s/funding_id/funding_pubkey/g
2019-05-23 23:20:04 -07:00
35e8f966e3
add freeze_lock() and fix par_process_entries() failure to detect self conflict ( #4415 )
...
* add freeze_lock and fix par_process_entries failure to detect self conflict
* fixup
* fixup
2019-05-23 17:35:15 -07:00
2ed77b040a
create_genesis_block() now returns a struct ( #4403 )
2019-05-22 20:39:00 -07:00
b8f6c17dee
Don't filter transactions if we are buffering it locally ( #4395 )
...
automerge
2019-05-22 17:54:28 -07:00
7153abd483
Revert "Performance tweaks ( #4340 )" ( #4350 )
...
* Revert "Performance tweaks (#4340 )"
This reverts commit 55cee5742f
2019-05-20 17:48:42 -07:00
7024c73e9b
Revert banking threads increase which seems to be slower in testing. ( #4349 )
2019-05-20 11:42:37 -07:00
f079a78c5e
Remove fee arg from system_transaction::* ( #4346 )
...
automerge
2019-05-20 10:03:19 -07:00
55cee5742f
Performance tweaks ( #4340 )
...
* Use Rc to prevent clone of Packets
* Fix min => max in banking_stage threads.
Coalesce packet buffers better since a larger batch will
be faster through banking and sigverify.
Deconstruct batches into banking_stage from sigverify since
sigverify likes to accumulate batches but then a single banking_stage
thread will be stuck with a large batch. Maximize parallelism by
creating more chunks of work for banking_stage.
2019-05-20 09:15:00 -07:00
392a39dd54
Poh subsystem cleanup, genesis plumbing, enable real PoH on edge testnet ( #4292 )
...
* Remove unused PohServiceConfig::Step
* Clarify variable name
* Poh::hash() now takes an iteration counter
* man -> max
* Inline functions with single call site
* Move PohServiceConfig into GenesisBlock
* Add plumbing to enable real PoH on testnets
* Batch hashes to improve PoH hash rate
* Ensure a constant hashes_per_tick
* Remove PohEntry mixin field
* Poh/PohEntry no longer maintains tick_height
* Ensure a constant hashes_per_tick
* ci/localnet-sanity.sh: Use real PoH
* Rework Poh/PohService to keep PohRecorder unlocked as much as possible while hashing
2019-05-18 14:01:36 -07:00
b2ce5dc9f5
Adjust log level for counter metrics ( #4323 )
2019-05-17 07:00:06 -07:00
c9b86018c6
Filter out all unprocessed transactions before forwarding them ( #4266 )
...
* Filter out all unprocessed transactions before forwarding them
* fix clippy
2019-05-13 14:40:05 -07:00
5e91f8f59d
Update reported tx count to exclude errors ( #4201 )
2019-05-13 13:23:52 -07:00
133be2df51
Check for transaction forwarding delay to detect an expired transaction before forwarding it ( #4249 )
...
Also refactored code for forwarding packets, and added test for it
2019-05-10 14:28:38 -07:00
f567877d1d
Cleanup metrics ( #4230 )
2019-05-10 08:33:58 -07:00
9cdffc7d64
Don't push empty vecs into the unprocessed buffers ( #4214 )
2019-05-08 17:58:07 -07:00
1a2b131ceb
Don't forward transactions that are expired or failed signature check ( #4199 )
2019-05-08 10:32:25 -07:00
69eeb7cf08
Fix parent record locks usage in child banks ( #4159 )
...
* Introduce record locks on txs that will be recorded
* Add tests for LockedAccountsResults
* Fix broken bench
* Exit process_entries on detecting conflicting locks within same entry
2019-05-07 15:51:35 -07:00
2c78a93001
GenesisBlock::new(X) => create_genesis_block(X)
2019-05-07 12:34:17 -07:00
8e400fc4bd
rework genesis (passive staking groundwork) ( #4187 )
...
* rework genesis
* fixup
2019-05-07 11:16:22 -07:00
29c2a63c8b
Retry transactions that failed due to account lock ( #4184 )
...
* added test
2019-05-07 10:23:02 -07:00
e9f80e5542
Don't use default ticks per slot in calculating next slot leader ( #4169 )
2019-05-06 11:23:06 -07:00
441e76ebeb
Index buffered transactions at the correct offset ( #4126 )
...
* tests
2019-05-02 19:05:53 -07:00
c2dfb9900e
Revert "Change forwarded metrics to be in line with fetch stage metrics ( #4068 )" ( #4135 )
...
automerge
2019-05-02 17:36:19 -07:00
3eec3cfac2
Cleanup banking stage in lieu of recent transaction forwarding changes ( #4101 )
2019-05-01 15:13:10 -07:00
4f18fc836f
Forward transactions to the next slot leader ( #4092 )
...
- this ensures that transactions will reach in time for the next node to process them
2019-05-01 11:37:29 -07:00
1a259d4a3f
Prevent Requests/Caching of leader schedules for epochs beyond confirmed roots ( #4033 )
...
automerge
2019-04-30 13:23:21 -07:00
3b2adbc9df
Change forwarded metrics to be in line with fetch stage metrics ( #4068 )
...
automerge
2019-04-29 13:50:14 -07:00
a056c1f18f
Don't send packets when msgs len is 0 ( #4030 )
...
And don't send metrics every iteration.
2019-04-26 17:27:31 -07:00
b912ee7fdf
Nit: Spelling ( #4025 )
2019-04-26 16:02:37 -07:00
9c7ccc0e2b
More metrics to track memory usage ( #3966 )
...
automerge
2019-04-24 11:11:30 -07:00
ab11327e34
Fix mismatch between leader/validator bank votability ( #3942 )
...
* Fix mismatch between leader/validator bank votability
2019-04-23 15:32:19 -07:00
7372ec9e1a
Use poh would_be_leader check in banking stage to hold or forward txs ( #3948 )
2019-04-23 11:56:30 -07:00
95a16426f3
Cleanup bench-exchange ( #3919 )
...
* bench-exchange changes
Generate new keypair for each bench-exchange
Add metrics
Tweak network sleep parameters for better reliability.
* Bench-exchange contract metrics
2019-04-22 13:16:28 -07:00
512bfc93cb
Add a cache for leader schedules ( #3841 )
...
* Add a cache for leader schedules
2019-04-19 02:39:44 -07:00
67b8ad6a0f
fix entries.to_blobs() ( #3882 )
...
* * rename Entry::serialized_size() to Entry::to_blob_size() to better
reduce confusion with bincode, et al. and to better reflect its
real meaning
* fix implementation of to_blob_size() to actually return what happens
when we do entries.to_blobs() (i.e. we serialize Vec<Entry>, not Entry)
* update tests to be more rigorous
* clippy
2019-04-18 14:45:41 -07:00
f2187780d2
Do not forward vote transactions ( #3871 )
2019-04-18 11:18:49 -07:00
2f4a3ed190
Use a separate channel to process votes in banking stage ( #3861 )
...
- This will help expedite the vote processing on peer nodes
2019-04-17 21:07:45 -07:00
aa21f5343a
Remove broken read().unwrap() call to a non-shared packet
2019-04-17 21:24:40 -05:00
9c2809db21
Delete SharedPackets ( #3843 )
...
* Delete SharedPackets
* Fix bench and sigverify
2019-04-17 18:15:50 -07:00
a2c8e3952f
Fixes to TPS calculation and reporting ( #3836 )
...
Fixes to TPS calculations and reporting
2019-04-17 15:37:01 -07:00
2c745ce108
Shorten recv wait when there are buffered packets in banking stage ( #3757 )
...
- packets are buffered on leader rotation, when the next leader is
unknown
- shortening the wait allows the banking stage to poll for next
leader more frequently
2019-04-14 12:34:07 -07:00
ee35ed5250
Refactored buffered packet forwarding code ( #3750 )
...
- Added unit tests
- Don't consume packets if bank is not known
2019-04-13 23:19:54 -07:00
44ebfa736a
Don't forward buffered packet to the same node ( #3712 )
...
- instead, process the packets
2019-04-11 17:23:45 -07:00
2c93062f54
Improve banking_stage performance messages
...
Use transaction count instead of batch count,
and set the recv_start from when we finished processing
the previous batch to get a more accurate number.
2019-04-09 14:54:12 -07:00
46e6911ec1
Add get_signature_status() to SyncClient
...
And move bank::Result to transaction module.
2019-04-05 10:22:05 -07:00
35298e01a8
Remove Instruction wrapper structs and name functions after enum fields
2019-04-03 13:34:27 -07:00
867f6f107b
Rename SystemInstruction::Move to SystemInstruction::Transfer
2019-04-03 08:35:57 -06:00
43bb813cbe
Rename 'new_account' to 'new_user_account'
...
And 'new_program_account' to 'new_account'
2019-04-02 21:24:42 -06:00
d228b6467c
Implement finalizer so that all locked accounts are dropped ( #3585 )
...
* Implement finalizer so that all locked accounts are dropped when finalizer goes out of scope
* Add test for tx error with lock conflict
* Fix double unlock from destructor running after a call to unlock
2019-04-02 03:55:42 -07:00
fcef54d062
Add a constructor to generate random pubkeys
2019-03-31 16:23:18 -06:00
f886b3b12b
Fix resetting PohRecorder to wrong bank ( #3553 )
...
* Check whether future slot already has transmission
2019-03-29 20:00:36 -07:00
0887832b00
Early exit if buffered packets is empty
2019-03-29 13:40:07 -07:00
2ab50cbae8
Move untested code out of SDK
...
verify_signature() was only used in a test that was testing
binary layout. It only worked because the test transaction only
had one signature.
from() was only used by verify_signature() and that's something
we'd typically called `pubkey()`.
hash() didn't return the hash of the Transaction, as you might
guess. It's only used for PoH, so move it into Entry.
2019-03-28 14:24:59 -06:00
0482f153d0
Lower a bunch of debug
...
Can't afford to be printing on every transaction error, it will slow
the system down.
2019-03-28 12:24:47 -07:00
6bfe497ab5
remove leader confirmaiton
2019-03-27 08:41:41 -07:00
acedf4ca5a
Move Instruction into its own module
2019-03-23 20:31:55 -07:00
e1a3708844
Ensure accounts are unlocked ( #3458 )
2019-03-23 13:30:56 -07:00
52f6c33ff9
Make sure banking stage is recording with the same bank that it read ( #3447 )
...
* make sure banking stage is recording with the same bank that it read with
2019-03-22 14:17:39 -07:00
d7ff6645a9
change pubkey to ref
2019-03-20 16:55:01 -07:00
1824e09d0a
find next leader slot before resetting working bank in Poh recorder
2019-03-20 16:55:01 -07:00
0dc364c17a
Relocate transaction reference verification to join the other validity checks
2019-03-20 07:46:01 -07:00
1be7ee51be
Fix potential crash in banking stage
2019-03-19 12:06:42 -07:00
c70412d7bb
move core tests to core ( #3355 )
...
* move core tests to core
* remove window
* fix up flaky tests
* test_entryfication needs a singly-threaded banking_stage
* move core benches to core
* remove unnecessary dependencies
* remove core as a member for now, test it like runtime
* stop running tests twice
* remove duplicate runs of tests in perf
2019-03-18 22:08:21 -07:00
8d032aba9d
Merge InstructionError and ProgramError
...
From the user's perspective, it's just an instruction error.
For program-specific errors, we still have
InstructionError::CustomError.
2019-03-18 10:39:20 -06:00
66fb1bbb2e
Give last leader some grace ticks to catch up ( #3299 )
...
* Wait for last leader for some ticks
* New tests and fixed existing tests
2019-03-15 13:22:16 -07:00
36763d0802
Cleanup entry.rs packing code ( #3303 )
2019-03-15 12:48:32 -07:00
e58220282a
Move TransactionError into the SDK
2019-03-13 21:26:57 -06:00
4ca4038d54
Rename BankError to TransactionError
2019-03-13 21:26:57 -06:00
6a89c68a1d
Add utility function to help get System error out of ProgramError
2019-03-13 21:26:57 -06:00
c14cce4c85
Add InstructionError for runtime instruction errors
2019-03-13 21:26:57 -06:00
959961b596
Modified test
2019-03-13 18:18:27 -07:00
cb3eeace56
Replay Stage start_leader() can use wrong parent fork() ( #3238 )
...
* Make sure start_leader starts on the last voted block, not necessarily the biggest indexed bank in frozen_slots()
* Fix tvu test
2019-03-12 17:42:53 -07:00
f8e07ef5a3
banking_stage_entryfication fails when run as cargo test
...
Add some retry for getting entries from the channel.
2019-03-11 14:13:32 -07:00
a4b6d181a2
rename forwarder ports to tpu_via_blobs
2019-03-11 14:07:17 -07:00
0b8c5d807d
code cleanup
2019-03-11 14:07:17 -07:00
e201136eee
more review comments
2019-03-11 14:07:17 -07:00
55f660d5f9
address review comments
2019-03-11 14:07:17 -07:00
a4acc631ee
Refactor packing packets into blobs into separate packets_to_blob() function in packets.rs
2019-03-11 14:07:17 -07:00
cd1a9faacd
Batch packet forwarding in banking stage
2019-03-11 14:07:17 -07:00
fe1f67ea9a
clippy errors
2019-03-11 14:07:17 -07:00
069ce71256
fix clippy
2019-03-11 14:07:17 -07:00
e3cacb9296
Buffer unprocessed packets if next leader is the current node
2019-03-11 14:07:17 -07:00
195a880576
pass Pubkeys as refs, copy only where values needed ( #3213 )
...
* pass Pubkeys as refs, copy only where values needed
* Pubkey is pervasive
* fixup
2019-03-09 19:28:43 -08:00
bb93504965
Rename ClusterInfo::new() to ClusterInfo::new_with_invalid_keypair()
2019-03-07 13:05:42 -08:00
0252bf2f46
fix fmt
2019-03-06 12:25:28 -08:00
283bb84134
Create UDP socket once per process_loop for forwarding transactions
2019-03-06 12:25:28 -08:00
f1d1852691
Rename tokens to lamports in core/
2019-03-05 17:28:06 -08:00
53f09c44f3
Rename tokens to lamports in sdk/
2019-03-05 17:28:06 -08:00
6ab6e6cb9b
Clean up exit flag handing across TVU
2019-03-04 21:26:50 -08:00
906df5e20e
Exit signal cleanup: pass in references, make the receiver clone as needed
2019-03-04 18:43:21 -08:00
1654199b23
Use PohRecorder to synchronize instead of rotate. ( #3080 )
2019-03-03 16:44:06 -08:00
a9a7fc56eb
Purge MAX_RECENT_TICK_HASHES
2019-03-02 17:04:42 -08:00
d22a13257e
Refactor bank get vote accounts ( #3052 )
2019-03-02 16:44:36 -08:00
a94880574b
block_hash => blockhash
2019-03-02 12:13:30 -07:00
258cf21416
Purge remaining last_id (now called block_hash)
2019-03-02 12:13:30 -07:00
2bfad87a5f
Rename Bank.last_id() to Bank.last_block_hash()
2019-03-02 12:13:30 -07:00
5f5d779ee1
Move src/ into core/src. Top-level crate is now called solana-workspace
2019-03-02 09:52:18 -08:00
Justin Starry
Pankaj Garg
Rob Walker
carllin
sakridge
Tyera Eulberg
Sagar Dhawan
Michael Vines
Jack May
sakridge
Mark E. Sinclair
Greg Fitzgerald
Anatoly Yakovenko
Sathish
Carl