Commit Graph

24 Commits

Author SHA1 Message Date
steviez bb24318ef0
Document shreds (#16514)
No functionality changes from this commit
2021-04-16 14:04:46 -05:00
behzad nouri 4f82b897bc
buffers data shreds to make larger erasure coded sets (#15849)
Broadcast stage batches up to 8 entries:
https://github.com/solana-labs/solana/blob/79280b304/core/src/broadcast_stage/broadcast_utils.rs#L26-L29
which will be serialized into some number of shreds and chunked into FEC
sets of at most 32 shreds each:
https://github.com/solana-labs/solana/blob/79280b304/ledger/src/shred.rs#L576-L597
So depending on the size of entries, FEC sets can be small, which may
aggravate loss rate.
For example 16 FEC sets of 2:2 data/code shreds each have higher loss
rate than one 32:32 set.

This commit broadcasts data shreds immediately, but also buffers them
until it has a batch of 32 data shreds, at which point 32 coding shreds
are generated and broadcasted.
2021-03-23 14:52:38 +00:00
Trent Nelson 7f7370c306 Re-allow clippy::integer_arithmetic at crate-level 2021-02-17 13:55:08 -07:00
sakridge c5fe076432
Better dupe detection (#13992) 2020-12-09 23:14:31 -08:00
sakridge b4cf968e14
Add back shredding broadcast stats (#13463) 2020-11-09 23:04:27 -08:00
carllin f1699721ef
Bench RaptorQ (#10886)
Co-authored-by: Carl <carl@solana.com>
2020-07-02 18:31:32 -07:00
carllin 97f2bcff69
master: Add nonce to shreds repairs, add shred data size to header (#10109)
* Add nonce to shreds/repairs

* Add data shred size to header

Co-authored-by: Carl <carl@solana.com>
2020-05-19 12:38:18 -07:00
Jack May d61191db40
fix bench warnings (#9277) 2020-04-02 21:56:38 -07:00
Tyera Eulberg ab361a8073
Rename KeypairUtil to Signer (#8360)
automerge
2020-02-20 13:28:55 -08:00
Pankaj Garg d94041e98d
Allow coding shred index to be different than data shred index (#7438)
* Allow coding shred index to be different than data shred index

* move fec_set_index to shred's common header

* fix bench
2019-12-12 16:50:29 -08:00
Sagar Dhawan 6bfe0fca1f
Add a version field to shreds (#7023)
* Add a version field to shreds

* Clippy

* Fix Chacha Golden

* Fix shredder bench compile

* Fix blocktree bench compile
2019-11-18 18:05:02 -08:00
Pankaj Garg 0ace79939b
Add reference tick to data shreds (#6772)
* Add reference tick to data shreds

* fix tests
2019-11-06 13:27:58 -08:00
anatoly yakovenko 67f636545a Refactor sigverify to stage for signing shreds on the GPU (#6635)
automerge
2019-11-06 10:52:30 -08:00
Justin Starry e8e5ddc55d
Verify number of hashes for each block of entries (#6262)
* Verify number of hashes for each block of entries

* Fix blocktree processor tick check

* Rebase once more
2019-10-31 16:38:50 -04:00
Pankaj Garg 84e911361a
Use constants instead of lazy_static for shred header sizes (#6472) 2019-10-21 12:46:16 -07:00
Pankaj Garg badeb4d31a
Rework shred headers to fix position of signature (#6451)
* Rework shred headers to fix position of signature

* fix clippy
2019-10-18 22:55:59 -07:00
Greg Fitzgerald 5468be2ef9 Add solana-ledger crate (#6415)
automerge
2019-10-18 09:28:51 -07:00
Pankaj Garg 33052c1dd2 Cleanup shred header structures (#6378)
automerge
2019-10-15 20:48:45 -07:00
Pankaj Garg 20e800230f
Don't deserialize coding header for data shreds (#6367)
* Don't deserialize coding hdr for data shreds

* review comments

* fix tests
2019-10-15 15:18:23 -07:00
carllin 1960ea8ed7
Add benches for shredding and poh (#6307)
* Add benches for shredding and poh

* ignore poh bench

* Factor out Poh bench as separate function
2019-10-10 14:00:24 -07:00
carllin dd66d16fdb
Broadcast final shred for slots that are interrupted (#6269)
* Broadcast final shred for slots that are interrupted
2019-10-09 16:07:18 -07:00
carllin ac2374e9a1
Shred entries in parallel (#6180)
* Make shredding more parallel

* Fix erasure tests

* Fix replicator test

* Remove UnfinishedSlotInfo
2019-10-08 00:42:51 -07:00
Pankaj Garg 783e8672e7
Removed Shred enum (#5963)
* Remove shred enum and it's references

* rename ShredInfo to Shred

* clippy
2019-09-18 16:24:30 -07:00
Sagar Dhawan b757294864
Add minor performance bump to shredding (#5956) 2019-09-18 12:35:52 -07:00