blockworks-foundation
/
solana
mirror of https://github.com/blockworks-foundation/solana.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
17,107 Commits 168 Branches 497 Tags 539 MiB
Commit Graph

144 Commits

Author SHA1 Message Date
behzad nouri 65d59f4ef0
tracks erasure coding shreds' indices explicitly (#21822) 
The indices for erasure coding shreds are tied to data shreds:
https://github.com/solana-labs/solana/blob/90f41fd9b/ledger/src/shred.rs#L921

However with the upcoming changes to erasure schema, there will be more
erasure coding shreds than data shreds and we can no longer infer coding
shreds indices from data shreds.

The commit adds constructs to track coding shreds indices explicitly.
 2021-12-19 22:37:55 +00:00
behzad nouri 7476dfeec0
removes Select in favor of recv_timeout/try_iter (#21981) 
crossbeam_channel::Select::ready_timeout might return with success spuriously.
 2021-12-18 17:39:07 +00:00
carllin 7f6fb6937a
Ensure AncestorHashesSerice selects an open port (#21919) 2021-12-18 00:44:01 -05:00
behzad nouri 4ceb2689f5
adds ShredId uniquely identifying each shred (#21820) 2021-12-14 17:34:02 +00:00
Justin Starry 254ef3e7b6
Rename Packets to PacketBatch (#21794) 2021-12-11 09:44:15 -05:00
behzad nouri 8063273d09
adds more sanity checks to shreds (#21675) 2021-12-09 16:43:57 +00:00
behzad nouri cd17f63d81
adds back position field to coding-shred-header (#21600) 
https://github.com/solana-labs/solana/pull/17004
removed position field from coding-shred-header because as it stands the
field is redundant and unused.
However, with the upcoming changes to erasure coding schema this field
will no longer be redundant and needs to be populated.
 2021-12-05 14:42:09 +00:00
Michael Vines b8837c04ec Reformat imports to a consistent style for imports 
rustfmt.toml configuration:
  imports_granularity = "One"
  group_imports = "One"
 2021-12-03 09:19:13 -08:00
behzad nouri dd338b6c9f
changes Shred::parent return type to Option<Slot> (#21370) 
Shred::parent can return garbage if the struct fields are invalid:
https://github.com/solana-labs/solana/blob/8a50b6302/ledger/src/shred.rs#L446-L453

The commit adds more sanity checks and changes the return type to Option<Slot>.
 2021-11-23 14:45:26 +00:00
behzad nouri 48dfdfb4d5 changes Blockstore::is_shred_duplicate arg type to ShredType 2021-11-19 14:16:39 +00:00
behzad nouri 57057f8d39 uses enum for shred type 
Current code is using u8 which does not have any type-safety and can
contain invalid values:
https://github.com/solana-labs/solana/blob/66fa062f1/ledger/src/shred.rs#L167

Checks for invalid shred-types are scattered through the code:
https://github.com/solana-labs/solana/blob/66fa062f1/ledger/src/blockstore.rs#L849-L851
https://github.com/solana-labs/solana/blob/66fa062f1/ledger/src/shred.rs#L346-L348

The commit uses enum for shred type with #[repr(u8)]. Backward
compatibility is maintained by implementing Serialize and Deserialize
compatible with u8, and adding a test to assert that.
 2021-11-19 14:16:39 +00:00
steviez e6280fc1fa
Add additional checks for should_retransmit_and_persist() (#20672) 
Add additional checks to should_retransmit_and_persist()

- Check invalid shred index
- Update cases that check if node was leader
- Some comments and variable rename for clarity
 2021-11-03 02:01:07 -05:00
behzad nouri 0f03971c3c
adds counters for errors in window-service run_insert (#20670) 2021-10-15 14:13:26 +00:00
sakridge 588168b99d
Add check for shred data header size (#20668) 2021-10-14 05:56:14 +02:00
behzad nouri 7a8807b8bb retransmits shreds recovered from erasure codes 
Shreds recovered from erasure codes have not been received from turbine
and have not been retransmitted to other nodes downstream. This results
in more repairs across the cluster which is slower.

This commit channels through recovered shreds to retransmit stage in
order to further broadcast the shreds to downstream nodes in the tree.
 2021-08-17 13:44:10 +00:00
behzad nouri 3efccbffab sends shreds (instead of packets) to retransmit stage 
Working towards channelling through shreds recovered from erasure codes
to retransmit stage.
 2021-08-17 13:44:10 +00:00
behzad nouri 8198a7eae1 adds packet/shred count stats to window-service 
Adding back these metrics from the earlier commit which removed them
from retransmit stage.
 2021-08-17 13:44:10 +00:00
behzad nouri b64eeb7729 removes erroneous uses of &Arc<...> from window-service 2021-08-13 17:26:31 +00:00
behzad nouri d57398a959 removes repeated bank-forks locking in window-service 
Window service is repeatedly locking bank-forks to look-up working-bank
for every single shred:
https://github.com/solana-labs/solana/blob/5fde4ee3a/core/src/window_service.rs#L597-L606

This commit updates shred_filter signature in recv_window so that where
we already obtain the lock on bank-forks, we can also look-up
working-bank once for all packets:
https://github.com/solana-labs/solana/blob/5fde4ee3a/core/src/window_service.rs#L256-L277
 2021-08-13 17:26:31 +00:00
Jeff Washington (jwash) 14361906ca
for all tests, bank::new -> bank::new_for_tests (#19064) 2021-08-05 08:42:38 -05:00
Jeff Biseda 9255ae334d
drop outstanding_requests lock before sending repair requests (#18893) 2021-07-28 19:30:43 -07:00
carllin c0704d4ec9
Plumb signal from replay to ancestor hashes service (#18880) 2021-07-26 20:59:00 -07:00
carllin 1ee64afb12
Introduce AncestorHashesService (#18812) 2021-07-23 16:54:47 -07:00
behzad nouri d2d5f36a3c
adds validator flag to allow private ip addresses (#18850) 2021-07-23 15:25:03 +00:00
behzad nouri 7d56fa8363
sends packets in batches from sigverify-stage (#18446) 
sigverify-stage is breaking batches to single-item vectors before
sending them down the channel:
https://github.com/solana-labs/solana/blob/d451363dc/core/src/sigverify_stage.rs#L88-L92

Also simplifying window-service code, reducing number of nested branches.
 2021-07-22 14:49:21 +00:00
carllin 588c0464b8
Add sampling logic and DuplicateSlotRepairStatus module (#18721) 2021-07-21 11:15:08 -07:00
carllin 8a846b048e
Add AncestorHashesRepair type (#18681) 2021-07-15 19:29:53 -07:00
sakridge 7f2254225e
Move entry/poh to own crate to speed up poh bench build (#18225) 2021-07-14 14:16:29 +02:00
jbiseda a86ced0bac
generate deterministic seeds for shreds (#17950) 
* generate shred seed from leader pubkey

* clippy

* clippy

* review

* review 2

* fmt

* review

* check

* review

* cleanup

* fmt
 2021-07-07 08:21:12 -07:00
sakridge 8d9a6deda4
Add repair number per slot (#18082) 2021-06-30 18:20:07 +02:00
Michael Vines 84b9de8c18 Shredder no longer holds a keypair 2021-06-21 21:29:52 -07:00
Michael Vines 4a12c715a3 Drop Error suffix from enum values to avoid the enum_variant_names clippy lint 2021-06-18 23:02:13 +00:00
Alexander Meißner 6514096a67 chore: cargo +nightly clippy --fix -Z unstable-options 2021-06-18 10:42:46 -07:00
carllin 52dccc656a
Purge slots greater than new last index (#16071) 2021-05-26 16:12:57 -07:00
Tyera Eulberg 9a5330b7eb
Move gossip modules into solana-gossip crate (#17352) 
* Move gossip modules to solana-gossip

* Update Protocol abi digest due to move

* Move gossip benches and hook up CI

* Remove unneeded Result entries

* Single use statements
 2021-05-26 09:15:46 -06:00
Tao Zhu 0781fe1b4f
Upgrade Rust to 1.52.0 (#17096) 
* Upgrade Rust to 1.52.0
update nightly_version to newly pushed docker image
fix clippy lint errors
1.52 comes with grcov 0.8.0, include this version to script

* upgrade to Rust 1.52.1

* disabling Serum from downstream projects until it is upgraded to Rust 1.52.1
 2021-05-19 09:31:47 -05:00
behzad nouri 81ad795d46 removes position field in coding-shred-header 
CodingShredHeader.position is equal to
  ShredCommonHeader.index - ShredCommonHeader.fec_set_index
and is so redundant. The extra position field can add bugs if not
consistent with index and fec_set_index.
 2021-05-10 13:20:56 +00:00
behzad nouri 37b8587d4e
expands number of erasure coding shreds in the last batch in slots (#16484) 
Number of parity coding shreds is always less than the number of data
shreds in FEC blocks:
https://github.com/solana-labs/solana/blob/6907a2366/ledger/src/shred.rs#L719

Data shreds are batched in chunks of 32 shreds each:
https://github.com/solana-labs/solana/blob/6907a2366/ledger/src/shred.rs#L714

However the very last batch of data shreds in a slot can be small, in
which case the loss rate can be exacerbated.

This commit expands the number of coding shreds in the last FEC block in
slots to: 64 - number of data shreds; so that FEC blocks are always 64
data and parity coding shreds each.

As a consequence of this, the last FEC block has more parity coding
shreds than data shreds. So for some shred indices we will have a coding
shred but no data shreds. This should not cause any kind of overlapping
FEC blocks as in:
https://github.com/solana-labs/solana/pull/10095
since this is done only for the very last batch in a slot, and the next
slot will reset the shred index.
 2021-04-21 12:47:50 +00:00
sakridge 8e69dd42c1
Add non-default repair nonce values (#16512) 
* Track outstanding nonces in repair

* Rework outstanding requests to use lru cache and randomize nonces

Co-authored-by: Carl <carl@solana.com>
 2021-04-20 09:37:33 -07:00
carllin 52703badfa
Setup ReplayStage confirmation scaffolding for duplicate slots (#9698) 2021-03-24 23:41:52 -07:00
behzad nouri 491b059755
broadcasts duplicate shreds through gossip (#14699) 2021-01-24 15:47:43 +00:00
sakridge aa2751e614
Check shred type in is_duplicate (#14050) 2020-12-10 18:20:08 -08:00
Michael Vines 7bc073defe Run `codemod --extensions rs Pubkey::new_rand solana_sdk::pubkey::new_rand` 2020-10-21 19:08:13 -07:00
sakridge ce98088457
Track inserted repair shreds (#12455) 2020-09-29 14:13:21 -07:00
carllin 1c1a3f979d
Detect and notify when deserializable shreds are available (#11816) 
* Add logic to check for complete data ranges

* Add RPC signature notification

Co-authored-by: Carl <carl@solana.com>
 2020-09-01 22:06:06 -07:00
carllin 7a14e359d7
Switch to using weighted repair in RepairService (#10735) 
* Plumb votes into repair service

* Remove refactoring

* Fix tests

* Switch to using RepairWeight for generating repairs

* Revert "Weight repair slots based on vote stake (#10741)"

This reverts commit cabd0a09c3.

* Update logging

Co-authored-by: Carl <carl@solana.com>
 2020-07-10 05:52:54 +00:00
sakridge cabd0a09c3
Weight repair slots based on vote stake (#10741) 
* Weight repair slots based on vote stake

* Add test
 2020-06-22 20:27:45 -07:00
Greg Fitzgerald 6ee222363e
Move BankForks to solana_runtime (#10637) 
* Move BankForks to solana_runtime

* Update imports
 2020-06-17 15:27:03 +00:00
Kristofer Peterson e23340d89e
Clippy cleanup for all targets and nighly rust (also support 1.44.0) (#10445) 
* address warnings from 'rustup run beta cargo clippy --workspace'

minor refactoring in:
- cli/src/cli.rs
- cli/src/offline/blockhash_query.rs
- logger/src/lib.rs
- runtime/src/accounts_db.rs

expect some performance improvement AccountsDB::clean_accounts()

* address warnings from 'rustup run beta cargo clippy --workspace --tests'

* address warnings from 'rustup run nightly cargo clippy --workspace --all-targets'

* rustfmt

* fix warning stragglers

* properly fix clippy warnings test_vote_subscribe()
replace ref-to-arc with ref parameters where arc not cloned

* Remove lock around JsonRpcRequestProcessor (#10417)

automerge

* make ancestors parameter optional to avoid forcing construction of empty hash maps

Co-authored-by: Greg Fitzgerald <greg@solana.com>
 2020-06-09 09:38:14 +09:00
carllin 97f2bcff69
master: Add nonce to shreds repairs, add shred data size to header (#10109) 
* Add nonce to shreds/repairs

* Add data shred size to header

Co-authored-by: Carl <carl@solana.com>
 2020-05-19 12:38:18 -07:00