carllin
f1699721ef
Bench RaptorQ ( #10886 )
...
Co-authored-by: Carl <carl@solana.com>
2020-07-02 18:31:32 -07:00
Greg Fitzgerald
1c498369b5
Remove fee-payer guesswork from Message and Transaction ( #10776 )
...
* Make Message::new_with_payer the default constructor
* Remove Transaction::new_[un]signed_instructions
These guess the fee-payer instead of stating it explicitly
2020-06-24 14:52:38 -06:00
Greg Fitzgerald
6ee222363e
Move BankForks to solana_runtime ( #10637 )
...
* Move BankForks to solana_runtime
* Update imports
2020-06-17 15:27:03 +00:00
sakridge
0de6c444d6
Simd poh ( #10604 )
...
* Simd poh
* Fix poh verify bench
2020-06-16 23:03:26 -07:00
Greg Fitzgerald
2eb6f498a8
Remove redundant BankForks parameter ( #10537 )
2020-06-12 11:04:17 -06:00
Kristofer Peterson
e23340d89e
Clippy cleanup for all targets and nighly rust (also support 1.44.0) ( #10445 )
...
* address warnings from 'rustup run beta cargo clippy --workspace'
minor refactoring in:
- cli/src/cli.rs
- cli/src/offline/blockhash_query.rs
- logger/src/lib.rs
- runtime/src/accounts_db.rs
expect some performance improvement AccountsDB::clean_accounts()
* address warnings from 'rustup run beta cargo clippy --workspace --tests'
* address warnings from 'rustup run nightly cargo clippy --workspace --all-targets'
* rustfmt
* fix warning stragglers
* properly fix clippy warnings test_vote_subscribe()
replace ref-to-arc with ref parameters where arc not cloned
* Remove lock around JsonRpcRequestProcessor (#10417 )
automerge
* make ancestors parameter optional to avoid forcing construction of empty hash maps
Co-authored-by: Greg Fitzgerald <greg@solana.com>
2020-06-09 09:38:14 +09:00
sakridge
2cf719ac2c
Cache tvu peers for broadcast ( #10373 )
2020-06-03 08:24:05 -07:00
carllin
97f2bcff69
master: Add nonce to shreds repairs, add shred data size to header ( #10109 )
...
* Add nonce to shreds/repairs
* Add data shred size to header
Co-authored-by: Carl <carl@solana.com>
2020-05-19 12:38:18 -07:00
Jack May
eb1acaf927
Remove archiver and storage program ( #9992 )
...
automerge
2020-05-14 18:22:47 -07:00
Greg Fitzgerald
76b1c2baf0
One less alloc per transaction ( #9705 )
...
* One less alloc per transaction
* Fix benches
* Fix compiler warnings in bench build
* Fix move build
* Fix bench
2020-04-24 13:03:46 -06:00
carllin
bab3502260
Push down cluster_info lock ( #9594 )
...
* Push down cluster_info lock
* Rework budget decrement
Co-authored-by: Carl <carl@solana.com>
2020-04-21 12:54:45 -07:00
sakridge
69f1e487b3
Reduce cluster-info metrics. ( #9465 )
2020-04-14 21:21:58 -07:00
sakridge
4677cdb4c2
Optimize broadcast cluster_info critical section ( #9327 )
2020-04-06 17:36:22 -07:00
carllin
7b68628e6c
Remove write lock ( #9311 )
...
* Remove write lock
Co-authored-by: Carl <carl@solana.com>
2020-04-05 15:18:45 -07:00
Jack May
d61191db40
fix bench warnings ( #9277 )
2020-04-02 21:56:38 -07:00
Michael Vines
18c1f0dfe9
Remove stub core/src/genesis_utils.rs ( #8999 )
2020-03-21 10:54:40 -07:00
anatoly yakovenko
9cedeb0a8d
Pull streamer out into its own module. ( #8917 )
...
automerge
2020-03-17 23:30:23 -07:00
Tyera Eulberg
ab361a8073
Rename KeypairUtil to Signer ( #8360 )
...
automerge
2020-02-20 13:28:55 -08:00
Greg Fitzgerald
b5dba77056
Rename blocktree to blockstore ( #7757 )
...
automerge
2020-01-13 13:13:52 -08:00
anatoly yakovenko
97589f77f8
Pipeline broadcast socket transmit and blocktree record ( #7481 )
...
automerge
2019-12-16 17:11:18 -08:00
Pankaj Garg
d94041e98d
Allow coding shred index to be different than data shred index ( #7438 )
...
* Allow coding shred index to be different than data shred index
* move fec_set_index to shred's common header
* fix bench
2019-12-12 16:50:29 -08:00
Tyera Eulberg
97ca6858b7
Write transaction status and fee into persistent store ( #7030 )
...
* Pass blocktree into execute_batch, if persist_transaction_status
* Add validator arg to enable persistent transaction status store
* Pass blocktree into banking_stage, if persist_transaction_status
* Add validator params to bash scripts
* Expose actual transaction statuses outside Bank; add tests
* Fix benches
* Offload transaction status writes to a separate thread
* Enable persistent transaction status along with rpc service
* nudge
* Review comments
2019-11-20 16:43:10 -07:00
Jack May
d2ed921bc6
Cleanup nightly warnings ( #7055 )
2019-11-19 20:15:37 -08:00
Sagar Dhawan
6bfe0fca1f
Add a version field to shreds ( #7023 )
...
* Add a version field to shreds
* Clippy
* Fix Chacha Golden
* Fix shredder bench compile
* Fix blocktree bench compile
2019-11-18 18:05:02 -08:00
carllin
43e2301e2c
Fix roots overrunning broadcast ( #6884 )
...
* Add trusted pathway for insert_shreds to avoid checks
2019-11-14 00:32:07 -08:00
Greg Fitzgerald
a3a830e1ab
Delete Service trait ( #6921 )
2019-11-13 11:12:09 -07:00
Greg Fitzgerald
30a08f4282
Cleanup ledger macros ( #6916 )
...
automerge
2019-11-13 07:14:09 -08:00
Justin Starry
9807f47d4e
Rename genesis block to genesis config ( #6816 )
2019-11-08 23:56:57 -05:00
Pankaj Garg
0ace79939b
Add reference tick to data shreds ( #6772 )
...
* Add reference tick to data shreds
* fix tests
2019-11-06 13:27:58 -08:00
anatoly yakovenko
67f636545a
Refactor sigverify to stage for signing shreds on the GPU ( #6635 )
...
automerge
2019-11-06 10:52:30 -08:00
anatoly yakovenko
b825d04597
Pull perf into a separate module. ( #6718 )
...
automerge
2019-11-04 20:13:43 -08:00
Michael Vines
50a17fc00b
Use Slot and Epoch type aliases instead of raw u64 ( #6693 )
...
automerge
2019-11-02 00:38:30 -07:00
anatoly yakovenko
385b4ce959
Get rid of verified packets and use the Meta::discard flag ( #6674 )
...
* get rid of verified packets and use the disabled meta field everywhere
2019-11-01 14:23:03 -07:00
Justin Starry
e8e5ddc55d
Verify number of hashes for each block of entries ( #6262 )
...
* Verify number of hashes for each block of entries
* Fix blocktree processor tick check
* Rebase once more
2019-10-31 16:38:50 -04:00
anatoly yakovenko
34a9619806
SigVerify stage for shreds. ( #6563 )
2019-10-28 16:07:51 -07:00
Rob Walker
b4119c454a
credit_only credits forwarding ( #6509 )
...
* credit_only_credits_forwarding
* whack transfer_now()
* fixup
* bench should retry the airdrop TX
* fixup
* try to make bench-exchange a bit more robust, informative
2019-10-23 22:01:22 -07:00
Pankaj Garg
84e911361a
Use constants instead of lazy_static for shred header sizes ( #6472 )
2019-10-21 12:46:16 -07:00
Greg Fitzgerald
2636418659
Move blocktree_processor to solana_ledger ( #6460 )
...
* Drop core::result dependency in bank_forks
* Move blocktree_processor into solana_ledger
2019-10-20 09:54:38 -06:00
Rob Walker
e2c316d2d0
system_instruction_processor updates ( #6448 )
...
* zero lamport account creation
* whack create_user_account, take 2
* target->to
* ..
* ..
* update chacha golden
* update chacha golden
* ..
* ..
2019-10-19 18:23:27 -07:00
Pankaj Garg
badeb4d31a
Rework shred headers to fix position of signature ( #6451 )
...
* Rework shred headers to fix position of signature
* fix clippy
2019-10-18 22:55:59 -07:00
sakridge
32bfced6a4
Add offset checks for sigverify ( #6452 )
...
* Add offset checks for sigverify
* decode_len returning error instead of unwrap
2019-10-18 17:52:59 -07:00
Greg Fitzgerald
5468be2ef9
Add solana-ledger crate ( #6415 )
...
automerge
2019-10-18 09:28:51 -07:00
Ryo Onodera
e267dfacdd
Stabilize some banking stage tests ( #6251 )
...
* Stabilize some banking stage tests
Fixes #5660
* Fix CI...
* clean up
* Fix ci
* Address review nits
* Use bank.max_tick_height due to off-by-one for no PohRecord's clearing bank
* Fix CI...
* Use bank.max_tick_height() instead for clarity
2019-10-16 12:37:27 -07:00
Pankaj Garg
33052c1dd2
Cleanup shred header structures ( #6378 )
...
automerge
2019-10-15 20:48:45 -07:00
Pankaj Garg
20e800230f
Don't deserialize coding header for data shreds ( #6367 )
...
* Don't deserialize coding hdr for data shreds
* review comments
* fix tests
2019-10-15 15:18:23 -07:00
Pankaj Garg
364781366a
Use sendmmsg for broadcasting shreds ( #6325 )
...
* Replace packet with slice of data in sendmmsg
* fixes
* fix bench
2019-10-10 19:38:48 -07:00
carllin
1960ea8ed7
Add benches for shredding and poh ( #6307 )
...
* Add benches for shredding and poh
* ignore poh bench
* Factor out Poh bench as separate function
2019-10-10 14:00:24 -07:00
sakridge
1b775044f7
Use multiple retransmit stage threads/sockets ( #6279 )
2019-10-10 13:24:03 -07:00
carllin
dd66d16fdb
Broadcast final shred for slots that are interrupted ( #6269 )
...
* Broadcast final shred for slots that are interrupted
2019-10-09 16:07:18 -07:00
Justin Starry
5e31565574
Expand blocktree processor options ( #6248 )
...
* Refactor blocktree processor args and support full leader cache
* Add entry callback option
* Rename num_threads to override_num_threads
* Add test for entry callback
* Refactor cached leader schedule changes
* Add tests for blocktree process options
* Refactor test
* @mvines feedback
2019-10-08 17:58:49 -04:00
carllin
ac2374e9a1
Shred entries in parallel ( #6180 )
...
* Make shredding more parallel
* Fix erasure tests
* Fix replicator test
* Remove UnfinishedSlotInfo
2019-10-08 00:42:51 -07:00
sakridge
da7d94d0f0
Retransmit stage benchmark ( #6249 )
2019-10-06 12:56:17 -07:00
sakridge
ae7700296d
broadcast_shreds opt ( #6175 )
...
* Don't clone/copy/sort ContactInfo vec
2019-10-01 09:38:29 -07:00
Pankaj Garg
783e8672e7
Removed Shred enum ( #5963 )
...
* Remove shred enum and it's references
* rename ShredInfo to Shred
* clippy
2019-09-18 16:24:30 -07:00
Sagar Dhawan
b757294864
Add minor performance bump to shredding ( #5956 )
2019-09-18 12:35:52 -07:00
Rob Walker
0d4a2c5eb0
simplify poh recorder => broadcast channel ( #5940 )
...
* simplify poh recorder broadcast channel
* fixup
* fixup
2019-09-18 12:16:22 -07:00
Rob Walker
a2595b44c6
test randomize with error ( #5916 )
...
* test randomize with error
* update magic numbers
* fixup
* fixup
* fixup
* no more blobs
* fixup
2019-09-17 15:11:29 -07:00
Pankaj Garg
3d3b03a123
Verify signature of recovered shred before adding them to blocktree ( #5811 )
...
* Verify signature of recovered shred before adding them to blocktree
* fix failing tests, and review comments
2019-09-05 18:20:30 -07:00
Pankaj Garg
3b0d48e3b8
Remove blocktree blob references ( #5691 )
...
* Remove blocktree blob references
* fixes and cleanup
* replace uninitialized() call with MaybeUninit
* fix bench
2019-09-03 21:32:51 -07:00
Parth
7dfb735db9
randomize tx ordering ( #4978 )
...
Summary of Changes:
This change adds functionality to randomize tx execution for every entry. It does this by implementing OrderedIterator that iterates tx slice as per the order specified. The order is generated randomly for every entry.
2019-08-28 21:08:32 +05:30
Michael Vines
3450b9a44d
Rename solana to solana-core ( #5583 )
2019-08-21 10:23:33 -07:00
Pankaj Garg
4798e7fa73
Integrate data shreds ( #5541 )
...
* Insert data shreds in blocktree and database
* Integrate data shreds with rest of the code base
* address review comments, and some clippy fixes
* Fixes to some tests
* more test fixes
* ignore some local cluster tests
* ignore replicator local cluster tests
2019-08-20 17:16:06 -07:00
Michael Vines
c78db6a94b
ledger path reform: use Path/PathBuf instead of strings ( #5344 )
2019-07-30 15:53:41 -07:00
sakridge
94db9cd412
Reduce banking_stage bench copy-paste code and fix programs bench ( #4926 )
2019-07-16 18:28:18 -07:00
sakridge
0fa1af5d47
Cleanup num_threads() and batch_limit numbers ( #4852 )
2019-06-28 10:55:24 +02:00
sakridge
a9b044f0ab
Fix banking_stage benchmark sends ( #4850 )
...
Only one big batch was being sent so only 1
thread active at a time in the benchmark.
2019-06-27 10:37:33 +02:00
sakridge
fbea9d8621
Page-pin packet memory for cuda ( #4250 )
...
* Page-pin packet memory for cuda
Bring back recyclers and pin offset buffers
* Add packet recycler to streamer
* Add set_pinnable to sigverify vecs to pin them
* Add packets reset test
* Add test for recycler and reduce the gc lock critical section
* Add comments/tests to cuda_runtime
* Add recycler to recv_blobs path.
* Add trace/names for debug and PacketsRecycler to bench-streamer
* Predict realloc and unpin beforehand.
* Add helper to reserve and pin
* Cap buffered packets length
* Call cuda wrapper functions
2019-06-27 09:32:32 +02:00
carllin
9a52b01171
Change to crossbeam channel in banking_threads VerifiedReceiver ( #4822 )
...
* Add crossbeam channel instead of channel in banking_stage
2019-06-26 18:42:27 -07:00
sakridge
9cafd1f85e
Change check_txs to ignore recv errors and re-enable test ( #4593 )
...
Use more chunks to avoid duplicate signature failures:
Duplicate signatures can occur because bank.clear_signatures()
can occur before the bank has actually committed the signatures
to the status cache and then error out on the next set of transactions.
2019-06-17 19:04:21 -07:00
TristanDebrunner
0c4cb76acf
Add GPU based PoH verification ( #4524 )
...
* Add GPU poh verify
* Switch to single PoH verify function
* Add EntrySlice verify tests with hashes and txs
* Add poh-verify benchmarks
2019-06-08 10:21:43 -06:00
Michael Vines
cfab54511b
Ignore bench_banking_stage_multi_accounts ( #4590 )
...
automerge
2019-06-06 21:47:07 -07:00
Pankaj Garg
1f71d05299
remove copying of forwarded packets ( #4425 )
...
automerge
2019-05-24 17:35:09 -07:00
Michael Vines
cfe5afd34c
_id => _pubkey variable renaming ( #4419 )
...
* wallet: rename *_account_id to *_account_pubkey
* s/from_id/from_pubkey/g
* s/node_id/node_pubkey/g
* s/stake_id/stake_pubkey/g
* s/voter_id/voter_pubkey/g
* s/vote_id/vote_pubkey/g
* s/delegate_id/delegate_pubkey/g
* s/account_id/account_pubkey/g
* s/to_id/to_pubkey/g
* s/my_id/my_pubkey/g
* cargo fmt
* s/staker_id/staker_pubkey/g
* s/mining_pool_id/mining_pool_pubkey/g
* s/leader_id/leader_pubkey/g
* cargo fmt
* s/funding_id/funding_pubkey/g
2019-05-23 23:20:04 -07:00
Michael Vines
2ed77b040a
create_genesis_block() now returns a struct ( #4403 )
2019-05-22 20:39:00 -07:00
Pankaj Garg
b8f6c17dee
Don't filter transactions if we are buffering it locally ( #4395 )
...
automerge
2019-05-22 17:54:28 -07:00
sakridge
1d7a758c97
Bump perf libs version to fix out buffer sizing ( #4385 )
2019-05-22 13:00:03 -07:00
sakridge
7153abd483
Revert "Performance tweaks ( #4340 )" ( #4350 )
...
* Revert "Performance tweaks (#4340 )"
This reverts commit 55cee5742f
.
* Revert Rc change
2019-05-20 17:48:42 -07:00
Rob Walker
d4da2fbacd
fix bench warnings ( #4356 )
2019-05-20 14:32:23 -07:00
Michael Vines
f079a78c5e
Remove fee arg from system_transaction::* ( #4346 )
...
automerge
2019-05-20 10:03:19 -07:00
sakridge
55cee5742f
Performance tweaks ( #4340 )
...
* Use Rc to prevent clone of Packets
* Fix min => max in banking_stage threads.
Coalesce packet buffers better since a larger batch will
be faster through banking and sigverify.
Deconstruct batches into banking_stage from sigverify since
sigverify likes to accumulate batches but then a single banking_stage
thread will be stuck with a large batch. Maximize parallelism by
creating more chunks of work for banking_stage.
2019-05-20 09:15:00 -07:00
Michael Vines
392a39dd54
Poh subsystem cleanup, genesis plumbing, enable real PoH on edge testnet ( #4292 )
...
* Remove unused PohServiceConfig::Step
* Clarify variable name
* Poh::hash() now takes an iteration counter
* man -> max
* Inline functions with single call site
* Move PohServiceConfig into GenesisBlock
* Add plumbing to enable real PoH on testnets
* Batch hashes to improve PoH hash rate
* Ensure a constant hashes_per_tick
* Remove PohEntry mixin field
* Poh/PohEntry no longer maintains tick_height
* Ensure a constant hashes_per_tick
* ci/localnet-sanity.sh: Use real PoH
* Rework Poh/PohService to keep PohRecorder unlocked as much as possible while hashing
2019-05-18 14:01:36 -07:00
sakridge
b114bc3674
Add benchmark for sigverify stage ( #4320 )
2019-05-17 11:09:42 -07:00
sakridge
5d0d467287
fix banking_stage benches ( #4231 )
2019-05-09 11:20:26 -07:00
Rob Walker
8e400fc4bd
rework genesis (passive staking groundwork) ( #4187 )
...
* rework genesis
* fixup
2019-05-07 11:16:22 -07:00
Pankaj Garg
3eec3cfac2
Cleanup banking stage in lieu of recent transaction forwarding changes ( #4101 )
2019-05-01 15:13:10 -07:00
carllin
512bfc93cb
Add a cache for leader schedules ( #3841 )
...
* Add a cache for leader schedules
2019-04-19 02:39:44 -07:00
Pankaj Garg
2f4a3ed190
Use a separate channel to process votes in banking stage ( #3861 )
...
- This will help expedite the vote processing on peer nodes
2019-04-17 21:07:45 -07:00
Sagar Dhawan
9c2809db21
Delete SharedPackets ( #3843 )
...
* Delete SharedPackets
* Fix bench and sigverify
2019-04-17 18:15:50 -07:00
Greg Fitzgerald
878a842611
Move append_vec bench to the crate with append_vec ( #3650 )
...
* Move append_vec bench to the crate with append_vec
* Use black_box to tell the compiler not to optimize away test data
```
pub fn black_box<T>(dummy: T) -> T {
unsafe {
let ret = std::ptr::read_volatile(&dummy);
std::mem::forget(dummy);
ret
}
}
```
* Revert "Use black_box to tell the compiler not to optimize away test data"
This reverts commit 5610b8ee957f1d8bf6e270e392859e8b23b1e472.
* Use black_box to tell the compiler not to optimize away test data
* Create bench directories
2019-04-06 07:18:56 -06:00
Greg Fitzgerald
35298e01a8
Remove Instruction wrapper structs and name functions after enum fields
2019-04-03 13:34:27 -07:00
Greg Fitzgerald
867f6f107b
Rename SystemInstruction::Move to SystemInstruction::Transfer
2019-04-03 08:35:57 -06:00
Greg Fitzgerald
fcef54d062
Add a constructor to generate random pubkeys
2019-03-31 16:23:18 -06:00
carllin
f886b3b12b
Fix resetting PohRecorder to wrong bank ( #3553 )
...
* Check whether future slot already has transmission
2019-03-29 20:00:36 -07:00
Greg Fitzgerald
31f8b6d352
Integrate Message into Transaction
2019-03-29 13:03:29 -07:00
Rob Walker
c70412d7bb
move core tests to core ( #3355 )
...
* move core tests to core
* remove window
* fix up flaky tests
* test_entryfication needs a singly-threaded banking_stage
* move core benches to core
* remove unnecessary dependencies
* remove core as a member for now, test it like runtime
* stop running tests twice
* remove duplicate runs of tests in perf
2019-03-18 22:08:21 -07:00
Mark
656fb173f9
Extract kvstore into separate crate ( #3327 )
...
* extract kvstore into new crate
* add kvstore crate to CI publishing list
2019-03-15 18:42:47 -05:00
Mark
56b0ba2601
KvStore - A data-store to support BlockTree ( #2897 )
...
* Mostly implement key-value store and add integration points
Essential key-value store functionality is implemented, needs more work to be integrated, tested, and activated.
Behind the `kvstore` feature.
2019-03-11 17:53:14 -05:00