e6280fc1fa
Add additional checks for should_retransmit_and_persist() ( #20672 )
...
Add additional checks to should_retransmit_and_persist()
- Check invalid shred index
- Update cases that check if node was leader
- Some comments and variable rename for clarity
2021-11-03 02:01:07 -05:00
0f03971c3c
adds counters for errors in window-service run_insert ( #20670 )
2021-10-15 14:13:26 +00:00
588168b99d
Add check for shred data header size ( #20668 )
2021-10-14 05:56:14 +02:00
7a8807b8bb
retransmits shreds recovered from erasure codes
...
Shreds recovered from erasure codes have not been received from turbine
and have not been retransmitted to other nodes downstream. This results
in more repairs across the cluster which is slower.
This commit channels through recovered shreds to retransmit stage in
order to further broadcast the shreds to downstream nodes in the tree.
2021-08-17 13:44:10 +00:00
3efccbffab
sends shreds (instead of packets) to retransmit stage
...
Working towards channelling through shreds recovered from erasure codes
to retransmit stage.
2021-08-17 13:44:10 +00:00
8198a7eae1
adds packet/shred count stats to window-service
...
Adding back these metrics from the earlier commit which removed them
from retransmit stage.
2021-08-17 13:44:10 +00:00
b64eeb7729
removes erroneous uses of &Arc<...> from window-service
2021-08-13 17:26:31 +00:00
d57398a959
removes repeated bank-forks locking in window-service
...
Window service is repeatedly locking bank-forks to look-up working-bank
for every single shred:
https://github.com/solana-labs/solana/blob/5fde4ee3a/core/src/window_service.rs#L597-L606
This commit updates shred_filter signature in recv_window so that where
we already obtain the lock on bank-forks, we can also look-up
working-bank once for all packets:
https://github.com/solana-labs/solana/blob/5fde4ee3a/core/src/window_service.rs#L256-L277
2021-08-13 17:26:31 +00:00
14361906ca
for all tests, bank::new -> bank::new_for_tests ( #19064 )
2021-08-05 08:42:38 -05:00
9255ae334d
drop outstanding_requests lock before sending repair requests ( #18893 )
2021-07-28 19:30:43 -07:00
c0704d4ec9
Plumb signal from replay to ancestor hashes service ( #18880 )
2021-07-26 20:59:00 -07:00
1ee64afb12
Introduce AncestorHashesService ( #18812 )
2021-07-23 16:54:47 -07:00
d2d5f36a3c
adds validator flag to allow private ip addresses ( #18850 )
2021-07-23 15:25:03 +00:00
7d56fa8363
sends packets in batches from sigverify-stage ( #18446 )
...
sigverify-stage is breaking batches to single-item vectors before
sending them down the channel:
https://github.com/solana-labs/solana/blob/d451363dc/core/src/sigverify_stage.rs#L88-L92
Also simplifying window-service code, reducing number of nested branches.
2021-07-22 14:49:21 +00:00
588c0464b8
Add sampling logic and DuplicateSlotRepairStatus module ( #18721 )
2021-07-21 11:15:08 -07:00
8a846b048e
Add AncestorHashesRepair type ( #18681 )
2021-07-15 19:29:53 -07:00
7f2254225e
Move entry/poh to own crate to speed up poh bench build ( #18225 )
2021-07-14 14:16:29 +02:00
a86ced0bac
generate deterministic seeds for shreds ( #17950 )
...
* generate shred seed from leader pubkey
* clippy
* clippy
* review
* review 2
* fmt
* review
* check
* review
* cleanup
* fmt
2021-07-07 08:21:12 -07:00
8d9a6deda4
Add repair number per slot ( #18082 )
2021-06-30 18:20:07 +02:00
84b9de8c18
Shredder no longer holds a keypair
2021-06-21 21:29:52 -07:00
4a12c715a3
Drop Error suffix from enum values to avoid the enum_variant_names clippy lint
2021-06-18 23:02:13 +00:00
6514096a67
chore: cargo +nightly clippy --fix -Z unstable-options
2021-06-18 10:42:46 -07:00
52dccc656a
Purge slots greater than new last index ( #16071 )
2021-05-26 16:12:57 -07:00
9a5330b7eb
Move gossip modules into solana-gossip crate ( #17352 )
...
* Move gossip modules to solana-gossip
* Update Protocol abi digest due to move
* Move gossip benches and hook up CI
* Remove unneeded Result entries
* Single use statements
2021-05-26 09:15:46 -06:00
0781fe1b4f
Upgrade Rust to 1.52.0 ( #17096 )
...
* Upgrade Rust to 1.52.0
update nightly_version to newly pushed docker image
fix clippy lint errors
1.52 comes with grcov 0.8.0, include this version to script
* upgrade to Rust 1.52.1
* disabling Serum from downstream projects until it is upgraded to Rust 1.52.1
2021-05-19 09:31:47 -05:00
81ad795d46
removes position field in coding-shred-header
...
CodingShredHeader.position is equal to
ShredCommonHeader.index - ShredCommonHeader.fec_set_index
and is so redundant. The extra position field can add bugs if not
consistent with index and fec_set_index.
2021-05-10 13:20:56 +00:00
37b8587d4e
expands number of erasure coding shreds in the last batch in slots ( #16484 )
...
Number of parity coding shreds is always less than the number of data
shreds in FEC blocks:
https://github.com/solana-labs/solana/blob/6907a2366/ledger/src/shred.rs#L719
Data shreds are batched in chunks of 32 shreds each:
https://github.com/solana-labs/solana/blob/6907a2366/ledger/src/shred.rs#L714
However the very last batch of data shreds in a slot can be small, in
which case the loss rate can be exacerbated.
This commit expands the number of coding shreds in the last FEC block in
slots to: 64 - number of data shreds; so that FEC blocks are always 64
data and parity coding shreds each.
As a consequence of this, the last FEC block has more parity coding
shreds than data shreds. So for some shred indices we will have a coding
shred but no data shreds. This should not cause any kind of overlapping
FEC blocks as in:
https://github.com/solana-labs/solana/pull/10095
since this is done only for the very last batch in a slot, and the next
slot will reset the shred index.
2021-04-21 12:47:50 +00:00
8e69dd42c1
Add non-default repair nonce values ( #16512 )
...
* Track outstanding nonces in repair
* Rework outstanding requests to use lru cache and randomize nonces
Co-authored-by: Carl <carl@solana.com>
2021-04-20 09:37:33 -07:00
52703badfa
Setup ReplayStage confirmation scaffolding for duplicate slots ( #9698 )
2021-03-24 23:41:52 -07:00
491b059755
broadcasts duplicate shreds through gossip ( #14699 )
2021-01-24 15:47:43 +00:00
aa2751e614
Check shred type in is_duplicate ( #14050 )
2020-12-10 18:20:08 -08:00
7bc073defe
Run `codemod --extensions rs Pubkey::new_rand solana_sdk::pubkey::new_rand`
2020-10-21 19:08:13 -07:00
ce98088457
Track inserted repair shreds ( #12455 )
2020-09-29 14:13:21 -07:00
1c1a3f979d
Detect and notify when deserializable shreds are available ( #11816 )
...
* Add logic to check for complete data ranges
* Add RPC signature notification
Co-authored-by: Carl <carl@solana.com>
2020-09-01 22:06:06 -07:00
7a14e359d7
Switch to using weighted repair in RepairService ( #10735 )
...
* Plumb votes into repair service
* Remove refactoring
* Fix tests
* Switch to using RepairWeight for generating repairs
* Revert "Weight repair slots based on vote stake (#10741 )"
This reverts commit cabd0a09c3
2020-07-10 05:52:54 +00:00
cabd0a09c3
Weight repair slots based on vote stake ( #10741 )
...
* Weight repair slots based on vote stake
* Add test
2020-06-22 20:27:45 -07:00
6ee222363e
Move BankForks to solana_runtime ( #10637 )
...
* Move BankForks to solana_runtime
* Update imports
2020-06-17 15:27:03 +00:00
e23340d89e
Clippy cleanup for all targets and nighly rust (also support 1.44.0) ( #10445 )
...
* address warnings from 'rustup run beta cargo clippy --workspace'
minor refactoring in:
- cli/src/cli.rs
- cli/src/offline/blockhash_query.rs
- logger/src/lib.rs
- runtime/src/accounts_db.rs
expect some performance improvement AccountsDB::clean_accounts()
* address warnings from 'rustup run beta cargo clippy --workspace --tests'
* address warnings from 'rustup run nightly cargo clippy --workspace --all-targets'
* rustfmt
* fix warning stragglers
* properly fix clippy warnings test_vote_subscribe()
replace ref-to-arc with ref parameters where arc not cloned
* Remove lock around JsonRpcRequestProcessor (#10417 )
automerge
* make ancestors parameter optional to avoid forcing construction of empty hash maps
Co-authored-by: Greg Fitzgerald <greg@solana.com>
2020-06-09 09:38:14 +09:00
97f2bcff69
master: Add nonce to shreds repairs, add shred data size to header ( #10109 )
...
* Add nonce to shreds/repairs
* Add data shred size to header
Co-authored-by: Carl <carl@solana.com>
2020-05-19 12:38:18 -07:00
58ef02f02b
9951 clippy errors in the test suite ( #10030 )
...
automerge
2020-05-15 09:35:43 -07:00
eb1acaf927
Remove archiver and storage program ( #9992 )
...
automerge
2020-05-14 18:22:47 -07:00
bab3502260
Push down cluster_info lock ( #9594 )
...
* Push down cluster_info lock
* Rework budget decrement
Co-authored-by: Carl <carl@solana.com>
2020-04-21 12:54:45 -07:00
f3afe5c99c
Remove dead code ( #9404 )
...
automerge
2020-04-09 13:09:59 -07:00
ed036b978d
Accumulate blockstore metrics and submit every 2s ( #9075 )
2020-03-26 12:51:41 -07:00
076fef5e57
Update Cluster Slots to support multiple threads ( #9071 )
...
Co-authored-by: Carl <carl@solana.com>
2020-03-25 18:09:19 -07:00
18c1f0dfe9
Remove stub core/src/genesis_utils.rs ( #8999 )
2020-03-21 10:54:40 -07:00
9cedeb0a8d
Pull streamer out into its own module. ( #8917 )
...
automerge
2020-03-17 23:30:23 -07:00
ab361a8073
Rename KeypairUtil to Signer ( #8360 )
...
automerge
2020-02-20 13:28:55 -08:00
76e20015a4
Add separate thread to check for and store duplicate slot proofs ( #7834 )
2020-01-16 15:27:54 -08:00
b5dba77056
Rename blocktree to blockstore ( #7757 )
...
automerge
2020-01-13 13:13:52 -08:00
steviez
behzad nouri
sakridge
Jeff Washington (jwash)
Jeff Biseda
carllin
Michael Vines
Alexander Meißner
Tyera Eulberg
Tao Zhu
carllin
Greg Fitzgerald
Kristofer Peterson
Jack May
anatoly yakovenko