HaoranYi
8a1c7614f1
typos ( #30604 )
2023-03-07 11:08:46 -06:00
behzad nouri
ded457cd73
embeds the new gossip ContactInfo in ClusterInfo ( #30022 )
...
Working towards replacing the legacy gossip contact-info with the new
one, the commit updates the respective field in gossip cluster-info.
2023-02-10 20:07:45 +00:00
behzad nouri
1c7662a37f
asserts that cluster-info keypair is consistent with contact-info id ( #29818 )
2023-01-24 16:57:55 +00:00
behzad nouri
8c212f59ad
renames ContactInfo to LegacyContactInfo ( #29566 )
...
Working towards adding a new ContactInfo where new sockets can be
added in a backward compatible way.
2023-01-08 16:00:55 +00:00
behzad nouri
283a2b1540
removes #[allow(clippy::same_item_push)] ( #29543 )
2023-01-06 17:32:26 +00:00
behzad nouri
f49beb0cbc
caches reed-solomon encoder/decoder instance ( #27510 )
...
ReedSolomon::new(...) initializes a matrix and a data-decode-matrix cache:
https://github.com/rust-rse/reed-solomon-erasure/blob/273ebbced/src/core.rs#L460-L466
In order to cache this computation, this commit caches the reed-solomon
encoder/decoder instance for each (data_shards, parity_shards) pair.
2022-09-25 18:09:47 +00:00
behzad nouri
97c9af4c6b
plumbs through flag to generate merkle variant of shreds
2022-09-23 16:45:18 +00:00
behzad nouri
88599fd760
skips shreds deserialization before retransmit ( #26230 )
...
Fully deserializing shreds in window-service before sending them to
retransmit stage adds latency to shreds propagation.
This commit instead channels through the payload and relies on only
partial deserialization of a few required fields: slot, shred-index,
shred-type.
2022-06-30 12:13:00 +00:00
behzad nouri
f534b8981b
maps number of data shreds to erasure batch size ( #25917 )
...
In prepration of
https://github.com/solana-labs/solana/pull/25807
which reworks erasure batch sizes, this commit:
* adds a helper function mapping the number of data shreds to the
erasure batch size.
* adds ProcessShredsStats to Shredder::entries_to_shreds in order to
replace and remove entries_to_data_shreds from the public interface.
2022-06-23 13:27:54 +00:00
Jeff Biseda
8b66625c95
convert std::sync::mpsc to crossbeam_channel ( #22264 )
2022-01-11 02:44:46 -08:00
behzad nouri
65d59f4ef0
tracks erasure coding shreds' indices explicitly ( #21822 )
...
The indices for erasure coding shreds are tied to data shreds:
https://github.com/solana-labs/solana/blob/90f41fd9b/ledger/src/shred.rs#L921
However with the upcoming changes to erasure schema, there will be more
erasure coding shreds than data shreds and we can no longer infer coding
shreds indices from data shreds.
The commit adds constructs to track coding shreds indices explicitly.
2021-12-19 22:37:55 +00:00
behzad nouri
01a7ec8198
uses rayon thread-pool for retransmit-stage parallelization ( #19486 )
2021-09-07 15:15:01 +00:00
behzad nouri
3efccbffab
sends shreds (instead of packets) to retransmit stage
...
Working towards channelling through shreds recovered from erasure codes
to retransmit stage.
2021-08-17 13:44:10 +00:00
behzad nouri
6e413331b5
removes erroneous uses of Arc<...> from retransmit stage
2021-08-17 13:44:10 +00:00
Jeff Washington (jwash)
bde9b4de94
Bank::new -> Bank::new_for_benches ( #19063 )
2021-08-04 17:30:43 -05:00
behzad nouri
d06dc6c8a6
shares cluster-nodes between retransmit threads ( #18947 )
...
cluster_nodes and last_peer_update are not shared between retransmit
threads, as each thread have its own value:
https://github.com/solana-labs/solana/blob/65ccfed86/core/src/retransmit_stage.rs#L476-L477
Additionally, with shared references, this code:
https://github.com/solana-labs/solana/blob/0167daa11/core/src/retransmit_stage.rs#L315-L328
has a concurrency bug where the thread which does compare_and_swap,
updates cluster_nodes much later after other threads have run with
outdated cluster_nodes for a while. In particular, the write-lock there
may block.
2021-07-29 16:20:15 +00:00
behzad nouri
d2d5f36a3c
adds validator flag to allow private ip addresses ( #18850 )
2021-07-23 15:25:03 +00:00
sakridge
7f2254225e
Move entry/poh to own crate to speed up poh bench build ( #18225 )
2021-07-14 14:16:29 +02:00
Michael Vines
84b9de8c18
Shredder no longer holds a keypair
2021-06-21 21:29:52 -07:00
behzad nouri
161838655c
removes port-based forwarding logic from turbine retransmit ( #17716 )
...
Turbine retransmit logic is based on which socket it received the packet
from (i.e `packet.meta.forward`):
https://github.com/solana-labs/solana/blob/708bbcb00/core/src/retransmit_stage.rs#L467-L470
This can leave the cluster vulnerable to spoofing and selective
propagation of packets; see
https://github.com/solana-labs/solana/issues/6672
https://github.com/solana-labs/solana/pull/7774
This commit identifies if the node is on the "critical path" based on
its index in the shuffled cluster. If so, it forwards the packet to both
neighbors and children; otherwise, the packet is only forwarded to the
children.
The metrics added in
https://github.com/solana-labs/solana/pull/17351
shows that the number of times the index does not match the port is very
rare, and therefore this change should be safe.
2021-06-15 13:19:41 +00:00
Tyera Eulberg
9a5330b7eb
Move gossip modules into solana-gossip crate ( #17352 )
...
* Move gossip modules to solana-gossip
* Update Protocol abi digest due to move
* Move gossip benches and hook up CI
* Remove unneeded Result entries
* Single use statements
2021-05-26 09:15:46 -06:00
Tyera Eulberg
827355a6b1
Create solana-rpc crate and move subscriptions ( #17320 )
...
* Move non_circulating_supply to runtime
* Add solana-rpc crate and move max_slots
* Move subscriptions to solana-rpc
* Single use statements
2021-05-19 00:54:28 -06:00
behzad nouri
37b8587d4e
expands number of erasure coding shreds in the last batch in slots ( #16484 )
...
Number of parity coding shreds is always less than the number of data
shreds in FEC blocks:
https://github.com/solana-labs/solana/blob/6907a2366/ledger/src/shred.rs#L719
Data shreds are batched in chunks of 32 shreds each:
https://github.com/solana-labs/solana/blob/6907a2366/ledger/src/shred.rs#L714
However the very last batch of data shreds in a slot can be small, in
which case the loss rate can be exacerbated.
This commit expands the number of coding shreds in the last FEC block in
slots to: 64 - number of data shreds; so that FEC blocks are always 64
data and parity coding shreds each.
As a consequence of this, the last FEC block has more parity coding
shreds than data shreds. So for some shred indices we will have a coding
shred but no data shreds. This should not cause any kind of overlapping
FEC blocks as in:
https://github.com/solana-labs/solana/pull/10095
since this is done only for the very last batch in a slot, and the next
slot will reset the shred index.
2021-04-21 12:47:50 +00:00
carllin
ae96ba3459
Plumb slot update pubsub notifications ( #15488 )
2021-02-28 23:29:11 -08:00
sakridge
1b59b163dd
Add max retransmit and shred insert slot ( #15475 )
2021-02-23 13:06:33 -08:00
sakridge
5c95d8e963
Shred filter ( #14030 )
2020-12-10 07:54:15 -08:00
Michael Vines
959880db60
Remove unused pubkey::Pubkey imports
2020-10-21 19:08:13 -07:00
Michael Vines
7bc073defe
Run `codemod --extensions rs Pubkey::new_rand solana_sdk::pubkey::new_rand`
2020-10-21 19:08:13 -07:00
Michael Vines
d15173ad9d
Address latest nightly clippy lints, but globally disable stable_sort_primitive
2020-08-17 22:36:10 -07:00
Greg Fitzgerald
6ee222363e
Move BankForks to solana_runtime ( #10637 )
...
* Move BankForks to solana_runtime
* Update imports
2020-06-17 15:27:03 +00:00
Greg Fitzgerald
2eb6f498a8
Remove redundant BankForks parameter ( #10537 )
2020-06-12 11:04:17 -06:00
Kristofer Peterson
e23340d89e
Clippy cleanup for all targets and nighly rust (also support 1.44.0) ( #10445 )
...
* address warnings from 'rustup run beta cargo clippy --workspace'
minor refactoring in:
- cli/src/cli.rs
- cli/src/offline/blockhash_query.rs
- logger/src/lib.rs
- runtime/src/accounts_db.rs
expect some performance improvement AccountsDB::clean_accounts()
* address warnings from 'rustup run beta cargo clippy --workspace --tests'
* address warnings from 'rustup run nightly cargo clippy --workspace --all-targets'
* rustfmt
* fix warning stragglers
* properly fix clippy warnings test_vote_subscribe()
replace ref-to-arc with ref parameters where arc not cloned
* Remove lock around JsonRpcRequestProcessor (#10417 )
automerge
* make ancestors parameter optional to avoid forcing construction of empty hash maps
Co-authored-by: Greg Fitzgerald <greg@solana.com>
2020-06-09 09:38:14 +09:00
Greg Fitzgerald
76b1c2baf0
One less alloc per transaction ( #9705 )
...
* One less alloc per transaction
* Fix benches
* Fix compiler warnings in bench build
* Fix move build
* Fix bench
2020-04-24 13:03:46 -06:00
carllin
bab3502260
Push down cluster_info lock ( #9594 )
...
* Push down cluster_info lock
* Rework budget decrement
Co-authored-by: Carl <carl@solana.com>
2020-04-21 12:54:45 -07:00
Michael Vines
18c1f0dfe9
Remove stub core/src/genesis_utils.rs ( #8999 )
2020-03-21 10:54:40 -07:00
anatoly yakovenko
9cedeb0a8d
Pull streamer out into its own module. ( #8917 )
...
automerge
2020-03-17 23:30:23 -07:00
Justin Starry
9807f47d4e
Rename genesis block to genesis config ( #6816 )
2019-11-08 23:56:57 -05:00
anatoly yakovenko
67f636545a
Refactor sigverify to stage for signing shreds on the GPU ( #6635 )
...
automerge
2019-11-06 10:52:30 -08:00
Greg Fitzgerald
2636418659
Move blocktree_processor to solana_ledger ( #6460 )
...
* Drop core::result dependency in bank_forks
* Move blocktree_processor into solana_ledger
2019-10-20 09:54:38 -06:00
Greg Fitzgerald
5468be2ef9
Add solana-ledger crate ( #6415 )
...
automerge
2019-10-18 09:28:51 -07:00
sakridge
1b775044f7
Use multiple retransmit stage threads/sockets ( #6279 )
2019-10-10 13:24:03 -07:00
sakridge
da7d94d0f0
Retransmit stage benchmark ( #6249 )
2019-10-06 12:56:17 -07:00