steviez
ec7ca411dd
Make PacketBatch packets vector non-public ( #25413 )
...
Upcoming changes to PacketBatch to support variable sized packets will
modify the internals of PacketBatch. So, this change removes usage of
the internal packet struct and instead uses accessors (which are
currently just wrappers of Vector functions but will change down the
road).
2022-05-23 15:30:15 -05:00
behzad nouri
c248fb3f51
renames Packet Meta::{,set_}addr methods to {,set_}socket_addr ( #25478 )
...
In order to distinguish between Meta.addr field which is an IpAddr and
the methods which refer to a SocketAddr.
2022-05-23 15:48:59 +00:00
Michael Vines
3608801a54
Avoid clippy::significant_drop_in_scrutinee
2022-05-22 22:22:21 -07:00
Michael Vines
c54e06355f
voteSubscribe pubsub notification now includes the vote transaction signature ( #25291 )
2022-05-19 18:28:46 -07:00
steviez
b48fd4eec2
Construct PacketBatches from PongMessages directly ( #24708 )
...
Serialize pongs directly into PacketBatch to save copying the data from
intermediate packets into PacketBatch.
2022-04-26 21:30:00 -05:00
behzad nouri
895f76a93c
hides implementation details of shred from its public interface ( #24563 )
...
Working towards embedding versioning into shreds binary, so that a new
variant of shred struct can include merkle tree hashes of the erasure
set.
2022-04-25 12:43:22 +00:00
behzad nouri
1d50832389
replaces counters with datapoints in gossip metrics ( #24451 )
2022-04-18 23:14:59 +00:00
behzad nouri
7cb3b6cbe2
demotes WeightedShuffle failures to error metrics ( #24079 )
...
Since call-sites are calling unwrap anyways, panicking seems too punitive
for our use cases.
2022-04-03 16:20:06 +00:00
ryleung-solana
8b72200afb
Thin client quic ( #23973 )
...
Change thin-client to use connection-cache
2022-03-31 15:47:00 -04:00
Michael Vines
7ef18f220a
Update Version CrdsData on node identity changes
2022-03-28 15:57:16 -07:00
sakridge
a4f4ac5279
add plumbing to allow for arbitrary tpu address in gossip ( #22703 )
...
* add plumbing to allow for arbitrary tpu address in gossip
* make clippy happy
* Review comments
Co-authored-by: CherryWorm <nico.gruendel@web.de>
2022-03-02 09:42:14 +01:00
sakridge
514aab46d9
Search for consecutive ports ( #22979 )
2022-02-07 17:53:40 +01:00
sakridge
5a230f418d
Add quic port for accepting transactions ( #22753 )
...
using quinn library
streamer: Sign TLS cert with validator identity key
Handle multiple incoming chunks
2022-02-04 15:27:09 +01:00
behzad nouri
45e09664b8
removes Rng field from WeightedShuffle struct ( #22850 )
2022-02-01 15:27:23 +00:00
Michael Vines
6d5bbca630
Pacify clippy
2022-01-21 19:12:57 -08:00
Justin Starry
7f20c6149e
Refactor: move simple vote parsing to runtime ( #22537 )
2022-01-20 10:39:21 +08:00
anatoly yakovenko
d343713f61
Optimize packet dedup ( #22571 )
...
* Use bloom filter to dedup packets
* dedup first
* Update bloom/src/bloom.rs
Co-authored-by: Trent Nelson <trent.a.b.nelson@gmail.com>
* Update core/src/sigverify_stage.rs
Co-authored-by: Trent Nelson <trent.a.b.nelson@gmail.com>
* Update core/src/sigverify_stage.rs
Co-authored-by: Trent Nelson <trent.a.b.nelson@gmail.com>
* Update core/src/sigverify_stage.rs
Co-authored-by: Trent Nelson <trent.a.b.nelson@gmail.com>
* fixup
* fixup
* fixup
Co-authored-by: Trent Nelson <trent.a.b.nelson@gmail.com>
2022-01-19 13:58:20 -08:00
Jeff Biseda
8b66625c95
convert std::sync::mpsc to crossbeam_channel ( #22264 )
2022-01-11 02:44:46 -08:00
behzad nouri
c9c78622a8
discards serialized gossip crds votes if cannot parse tx ( #22129 )
2021-12-29 19:31:26 +00:00
carllin
7f6fb6937a
Ensure AncestorHashesSerice selects an open port ( #21919 )
2021-12-18 00:44:01 -05:00
Justin Starry
254ef3e7b6
Rename Packets to PacketBatch ( #21794 )
2021-12-11 09:44:15 -05:00
Ashwin Sekar
f0acf7681e
Add vote instructions that directly update on chain vote state ( #21531 )
...
* Add vote state instructions
UpdateVoteState and UpdateVoteStateSwitch
* cargo tree
* extract vote state version conversion to common fn
2021-12-07 16:47:26 -08:00
Michael Vines
b8837c04ec
Reformat imports to a consistent style for imports
...
rustfmt.toml configuration:
imports_granularity = "One"
group_imports = "One"
2021-12-03 09:19:13 -08:00
behzad nouri
9886366977
exempts AccountsHashes from stake check ( #21565 )
...
Otherwise getHealth fails if account hashes are not propagated.
2021-12-02 18:01:32 +00:00
behzad nouri
57057f8d39
uses enum for shred type
...
Current code is using u8 which does not have any type-safety and can
contain invalid values:
https://github.com/solana-labs/solana/blob/66fa062f1/ledger/src/shred.rs#L167
Checks for invalid shred-types are scattered through the code:
https://github.com/solana-labs/solana/blob/66fa062f1/ledger/src/blockstore.rs#L849-L851
https://github.com/solana-labs/solana/blob/66fa062f1/ledger/src/shred.rs#L346-L348
The commit uses enum for shred type with #[repr(u8)]. Backward
compatibility is maintained by implementing Serialize and Deserialize
compatible with u8, and adding a test to assert that.
2021-11-19 14:16:39 +00:00
carllin
b30c94ce55
ClusterInfoVoteListener send only missing votes to BankingStage ( #20873 )
2021-11-18 15:20:41 -08:00
behzad nouri
43168e6365
doubles crds unique pubkey capacity ( #20947 )
2021-10-26 13:06:55 +00:00
behzad nouri
1297a13586
adds metrics tracking crds writes and votes ( #20953 )
2021-10-26 13:02:30 +00:00
behzad nouri
0c0384ec32
revises turbine peers shuffling order ( #20480 )
...
Turbine randomly shuffles cluster nodes on a broadcast tree for each
shred. This requires knowing the stakes and nodes' contact-infos (from
gossip).
However gossip is subject to partitioning and propogation delays.
Additionally unstaked nodes may join and leave the cluster at any
moment, changing the cluster view from one node to another.
This commit:
* Always arranges the unstaked nodes at the bottom of turbine broadcast
tree.
* Staked nodes are always included regardless of if their contact-info
is available in gossip or not.
* Uses the unbiased WeightedShuffle construct for shuffling nodes.
2021-10-14 15:09:36 +00:00
Brooks Prumo
1fcfbfccbb
Add fn to push IncrementalSnapshotHashes to cluster via gossip ( #20395 )
2021-10-08 08:20:35 -05:00
Brooks Prumo
57592e463e
Add get_incremental_snapshot_hash_for_node() to gossip ( #20394 )
2021-10-07 19:47:14 -05:00
Tao Zhu
177a375479
Tpu vote 1.7 ( #20187 ) ( #20494 )
...
* Add separate vote processing tpu port
* Add feature to send to tpu vote port
* Add vote rejecting sigverify mode
* use packet.meta.is_simple_vote_tx in place of deserialization
* consolidate code that identifies vote tx atcommon path for cpu and gpu
* new key for feature set
* banking forward tpu vote
* add tpu vote port to dockerfile and other review changes
* Simplify thread id compare
* fix a test; updated cluster_info ABI change
Co-authored-by: Tao Zhu <tao@solana.com>
Co-authored-by: sakridge <sakridge@gmail.com>
2021-10-07 09:38:23 +00:00
Brooks Prumo
4e3818e5c1
Add CrdsData::IncrementalSnapshotHashes ( #20374 )
2021-10-05 09:57:46 -05:00
Brooks Prumo
5d141fe01d
Rename CRDS SnapshotHash to SnapshotHashes ( #20421 )
2021-10-04 19:03:28 -05:00
carllin
ee8621a8bd
Add metric measuring number of successfully inserted push messages ( #20275 )
...
* Add number of successfully inserted push messages
2021-09-28 21:41:17 -07:00
sakridge
013e1d9d49
Limit transaction forwarding from banking_stage ( #19940 )
2021-09-21 08:49:41 -07:00
behzad nouri
d7051b0d21
adds logs when push-vote panics with invalid vote-index ( #19485 )
...
In order to debug this panic on the clusters:
panicked at 'assertion failed: (vote_index as usize) <
MAX_LOCKOUT_HISTORY', core/src/cluster_info.rs:1012:9
2021-08-31 12:15:07 +00:00
behzad nouri
6909a79b6f
removes require-stake-for-gossip feature ( #19476 )
...
The feature is already activated on all clusters.
2021-08-27 21:17:15 +00:00
behzad nouri
3efccbffab
sends shreds (instead of packets) to retransmit stage
...
Working towards channelling through shreds recovered from erasure codes
to retransmit stage.
2021-08-17 13:44:10 +00:00
behzad nouri
140abec6ef
exempts node-instances from shred-version check ( #19190 )
...
Clusters are kept separate using the shred-versions obtained from
contact-infos. However, this mechanism breaks if there are 2 instances
of the same identity key running on different clusters, because then one
of the two contact-infos have the right shred-version.
If a node has the contact-info with the matching shred-version, then it
will pass all associated crds values even if they belong to the other
instance. So the shred-version check breaks.
As a result we cannot support 2 instances of the same identity key
running on different clusters. To prevent that, this commit is exempting
node-instances from shred-version check so that they are always
propagated across clusters and halt one of the running duplicate
instances.
2021-08-14 00:47:44 +00:00
behzad nouri
f302774cf7
implements copy-on-write for staked-nodes ( #19090 )
...
Bank::staked_nodes and Bank::epoch_staked_nodes redundantly clone
staked-nodes HashMap even though an immutable reference will suffice:
https://github.com/solana-labs/solana/blob/a9014cece/runtime/src/vote_account.rs#L77
This commit implements copy-on-write semantics for staked-nodes by
wrapping the underlying HashMap in Arc<...>.
2021-08-10 12:59:12 +00:00
Justin Starry
8817f59b6e
Version transaction message and add new message format ( #18725 )
...
* Version transaction message and add new message format
* Update abi digest due to message path change
* Update v0.rs
Fix comment
* Update original.rs
* Update message versions name and address map indexes field name
* s/original/legacy
* update comment
* cargo fmt
* Update abi digest due to legacy rename
2021-08-09 22:03:39 -07:00
behzad nouri
049fb0417f
allows sendmmsg api taking owned values (as well as references) ( #18999 )
...
Current signature of api in sendmmsg requires a slice of inner
references:
https://github.com/solana-labs/solana/blob/fe1ee4980/streamer/src/sendmmsg.rs#L130-L152
That forces the call-site to convert owned values to references even
though doing so is redundant and adds an extra level of indirection:
https://github.com/solana-labs/solana/blob/fe1ee4980/core/src/repair_service.rs#L291
This commit expands the api using AsRef and Borrow traits to allow
calling the method with owned values (as well as references like
before).
2021-07-30 20:58:49 +00:00
behzad nouri
f1198fc6d5
filters crds values in parallel when responding to gossip pull-requests ( #18877 )
...
When responding to gossip pull-requests, filter_crds_values takes a lot of time
while holding onto read-lock:
https://github.com/solana-labs/solana/blob/f51d64868/gossip/src/crds_gossip_pull.rs#L509-L566
This commit will filter-crds-values in parallel using rayon thread-pools.
2021-07-26 17:13:11 +00:00
behzad nouri
d2d5f36a3c
adds validator flag to allow private ip addresses ( #18850 )
2021-07-23 15:25:03 +00:00
behzad nouri
bbd22f06f4
implements generic lookups into gossip crds table ( #18765 )
...
This commit adds CrdsEntry trait which allows generic lookups into crds
table. For example to get ContactInfo or LowestSlot associated with a
Pubkey, the lookup code would be respectively:
crds.get::<&ContactInfo>(pubkey)
crds.get::<&LowestSlot>(pubkey)
2021-07-21 12:16:26 +00:00
Justin Starry
207c90bd8b
Shorten long SerializeWith type paths in abi digest ( #18734 )
2021-07-20 08:59:50 -05:00
behzad nouri
8da261cf5c
locks crds only once in ClusterInfo::repair_peers ( #18752 )
...
ClusterInfo::repair_peers locks crds table twice, and shows performance
regression if the RwLock is not reader-preferred:
https://github.com/solana-labs/solana/blob/269028360/gossip/src/cluster_info.rs#L1188-L1210
2021-07-18 16:55:58 +00:00
behzad nouri
e316586516
excludes private ip addresses
2021-07-16 20:05:48 -06:00
Jeff Biseda
ae5ad5cf9b
sendmmsg cleanup #18589
...
Rationalize usage of sendmmsg(2). Skip packets which failed to send and track failures.
2021-07-16 14:36:49 -07:00