ec7ca411dd
Make PacketBatch packets vector non-public ( #25413 )
...
Upcoming changes to PacketBatch to support variable sized packets will
modify the internals of PacketBatch. So, this change removes usage of
the internal packet struct and instead uses accessors (which are
currently just wrappers of Vector functions but will change down the
road).
2022-05-23 15:30:15 -05:00
c248fb3f51
renames Packet Meta::{,set_}addr methods to {,set_}socket_addr ( #25478 )
...
In order to distinguish between Meta.addr field which is an IpAddr and
the methods which refer to a SocketAddr.
2022-05-23 15:48:59 +00:00
3608801a54
Avoid clippy::significant_drop_in_scrutinee
2022-05-22 22:22:21 -07:00
c54e06355f
voteSubscribe pubsub notification now includes the vote transaction signature ( #25291 )
2022-05-19 18:28:46 -07:00
b48fd4eec2
Construct PacketBatches from PongMessages directly ( #24708 )
...
Serialize pongs directly into PacketBatch to save copying the data from
intermediate packets into PacketBatch.
2022-04-26 21:30:00 -05:00
895f76a93c
hides implementation details of shred from its public interface ( #24563 )
...
Working towards embedding versioning into shreds binary, so that a new
variant of shred struct can include merkle tree hashes of the erasure
set.
2022-04-25 12:43:22 +00:00
1d50832389
replaces counters with datapoints in gossip metrics ( #24451 )
2022-04-18 23:14:59 +00:00
7cb3b6cbe2
demotes WeightedShuffle failures to error metrics ( #24079 )
...
Since call-sites are calling unwrap anyways, panicking seems too punitive
for our use cases.
2022-04-03 16:20:06 +00:00
8b72200afb
Thin client quic ( #23973 )
...
Change thin-client to use connection-cache
2022-03-31 15:47:00 -04:00
7ef18f220a
Update Version CrdsData on node identity changes
2022-03-28 15:57:16 -07:00
a4f4ac5279
add plumbing to allow for arbitrary tpu address in gossip ( #22703 )
...
* add plumbing to allow for arbitrary tpu address in gossip
* make clippy happy
* Review comments
Co-authored-by: CherryWorm <nico.gruendel@web.de>
2022-03-02 09:42:14 +01:00
514aab46d9
Search for consecutive ports ( #22979 )
2022-02-07 17:53:40 +01:00
5a230f418d
Add quic port for accepting transactions ( #22753 )
...
using quinn library
streamer: Sign TLS cert with validator identity key
Handle multiple incoming chunks
2022-02-04 15:27:09 +01:00
45e09664b8
removes Rng field from WeightedShuffle struct ( #22850 )
2022-02-01 15:27:23 +00:00
6d5bbca630
Pacify clippy
2022-01-21 19:12:57 -08:00
7f20c6149e
Refactor: move simple vote parsing to runtime ( #22537 )
2022-01-20 10:39:21 +08:00
d343713f61
Optimize packet dedup ( #22571 )
...
* Use bloom filter to dedup packets
* dedup first
* Update bloom/src/bloom.rs
Co-authored-by: Trent Nelson <trent.a.b.nelson@gmail.com>
* Update core/src/sigverify_stage.rs
Co-authored-by: Trent Nelson <trent.a.b.nelson@gmail.com>
* Update core/src/sigverify_stage.rs
Co-authored-by: Trent Nelson <trent.a.b.nelson@gmail.com>
* Update core/src/sigverify_stage.rs
Co-authored-by: Trent Nelson <trent.a.b.nelson@gmail.com>
* fixup
* fixup
* fixup
Co-authored-by: Trent Nelson <trent.a.b.nelson@gmail.com>
2022-01-19 13:58:20 -08:00
8b66625c95
convert std::sync::mpsc to crossbeam_channel ( #22264 )
2022-01-11 02:44:46 -08:00
c9c78622a8
discards serialized gossip crds votes if cannot parse tx ( #22129 )
2021-12-29 19:31:26 +00:00
7f6fb6937a
Ensure AncestorHashesSerice selects an open port ( #21919 )
2021-12-18 00:44:01 -05:00
254ef3e7b6
Rename Packets to PacketBatch ( #21794 )
2021-12-11 09:44:15 -05:00
f0acf7681e
Add vote instructions that directly update on chain vote state ( #21531 )
...
* Add vote state instructions
UpdateVoteState and UpdateVoteStateSwitch
* cargo tree
* extract vote state version conversion to common fn
2021-12-07 16:47:26 -08:00
b8837c04ec
Reformat imports to a consistent style for imports
...
rustfmt.toml configuration:
imports_granularity = "One"
group_imports = "One"
2021-12-03 09:19:13 -08:00
9886366977
exempts AccountsHashes from stake check ( #21565 )
...
Otherwise getHealth fails if account hashes are not propagated.
2021-12-02 18:01:32 +00:00
57057f8d39
uses enum for shred type
...
Current code is using u8 which does not have any type-safety and can
contain invalid values:
https://github.com/solana-labs/solana/blob/66fa062f1/ledger/src/shred.rs#L167
Checks for invalid shred-types are scattered through the code:
https://github.com/solana-labs/solana/blob/66fa062f1/ledger/src/blockstore.rs#L849-L851
https://github.com/solana-labs/solana/blob/66fa062f1/ledger/src/shred.rs#L346-L348
The commit uses enum for shred type with #[repr(u8)]. Backward
compatibility is maintained by implementing Serialize and Deserialize
compatible with u8, and adding a test to assert that.
2021-11-19 14:16:39 +00:00
b30c94ce55
ClusterInfoVoteListener send only missing votes to BankingStage ( #20873 )
2021-11-18 15:20:41 -08:00
43168e6365
doubles crds unique pubkey capacity ( #20947 )
2021-10-26 13:06:55 +00:00
1297a13586
adds metrics tracking crds writes and votes ( #20953 )
2021-10-26 13:02:30 +00:00
0c0384ec32
revises turbine peers shuffling order ( #20480 )
...
Turbine randomly shuffles cluster nodes on a broadcast tree for each
shred. This requires knowing the stakes and nodes' contact-infos (from
gossip).
However gossip is subject to partitioning and propogation delays.
Additionally unstaked nodes may join and leave the cluster at any
moment, changing the cluster view from one node to another.
This commit:
* Always arranges the unstaked nodes at the bottom of turbine broadcast
tree.
* Staked nodes are always included regardless of if their contact-info
is available in gossip or not.
* Uses the unbiased WeightedShuffle construct for shuffling nodes.
2021-10-14 15:09:36 +00:00
1fcfbfccbb
Add fn to push IncrementalSnapshotHashes to cluster via gossip ( #20395 )
2021-10-08 08:20:35 -05:00
57592e463e
Add get_incremental_snapshot_hash_for_node() to gossip ( #20394 )
2021-10-07 19:47:14 -05:00
177a375479
Tpu vote 1.7 ( #20187 ) ( #20494 )
...
* Add separate vote processing tpu port
* Add feature to send to tpu vote port
* Add vote rejecting sigverify mode
* use packet.meta.is_simple_vote_tx in place of deserialization
* consolidate code that identifies vote tx atcommon path for cpu and gpu
* new key for feature set
* banking forward tpu vote
* add tpu vote port to dockerfile and other review changes
* Simplify thread id compare
* fix a test; updated cluster_info ABI change
Co-authored-by: Tao Zhu <tao@solana.com>
Co-authored-by: sakridge <sakridge@gmail.com>
2021-10-07 09:38:23 +00:00
4e3818e5c1
Add CrdsData::IncrementalSnapshotHashes ( #20374 )
2021-10-05 09:57:46 -05:00
5d141fe01d
Rename CRDS SnapshotHash to SnapshotHashes ( #20421 )
2021-10-04 19:03:28 -05:00
ee8621a8bd
Add metric measuring number of successfully inserted push messages ( #20275 )
...
* Add number of successfully inserted push messages
2021-09-28 21:41:17 -07:00
013e1d9d49
Limit transaction forwarding from banking_stage ( #19940 )
2021-09-21 08:49:41 -07:00
d7051b0d21
adds logs when push-vote panics with invalid vote-index ( #19485 )
...
In order to debug this panic on the clusters:
panicked at 'assertion failed: (vote_index as usize) <
MAX_LOCKOUT_HISTORY', core/src/cluster_info.rs:1012:9
2021-08-31 12:15:07 +00:00
6909a79b6f
removes require-stake-for-gossip feature ( #19476 )
...
The feature is already activated on all clusters.
2021-08-27 21:17:15 +00:00
3efccbffab
sends shreds (instead of packets) to retransmit stage
...
Working towards channelling through shreds recovered from erasure codes
to retransmit stage.
2021-08-17 13:44:10 +00:00
140abec6ef
exempts node-instances from shred-version check ( #19190 )
...
Clusters are kept separate using the shred-versions obtained from
contact-infos. However, this mechanism breaks if there are 2 instances
of the same identity key running on different clusters, because then one
of the two contact-infos have the right shred-version.
If a node has the contact-info with the matching shred-version, then it
will pass all associated crds values even if they belong to the other
instance. So the shred-version check breaks.
As a result we cannot support 2 instances of the same identity key
running on different clusters. To prevent that, this commit is exempting
node-instances from shred-version check so that they are always
propagated across clusters and halt one of the running duplicate
instances.
2021-08-14 00:47:44 +00:00
f302774cf7
implements copy-on-write for staked-nodes ( #19090 )
...
Bank::staked_nodes and Bank::epoch_staked_nodes redundantly clone
staked-nodes HashMap even though an immutable reference will suffice:
https://github.com/solana-labs/solana/blob/a9014cece/runtime/src/vote_account.rs#L77
This commit implements copy-on-write semantics for staked-nodes by
wrapping the underlying HashMap in Arc<...>.
2021-08-10 12:59:12 +00:00
8817f59b6e
Version transaction message and add new message format ( #18725 )
...
* Version transaction message and add new message format
* Update abi digest due to message path change
* Update v0.rs
Fix comment
* Update original.rs
* Update message versions name and address map indexes field name
* s/original/legacy
* update comment
* cargo fmt
* Update abi digest due to legacy rename
2021-08-09 22:03:39 -07:00
049fb0417f
allows sendmmsg api taking owned values (as well as references) ( #18999 )
...
Current signature of api in sendmmsg requires a slice of inner
references:
https://github.com/solana-labs/solana/blob/fe1ee4980/streamer/src/sendmmsg.rs#L130-L152
That forces the call-site to convert owned values to references even
though doing so is redundant and adds an extra level of indirection:
https://github.com/solana-labs/solana/blob/fe1ee4980/core/src/repair_service.rs#L291
This commit expands the api using AsRef and Borrow traits to allow
calling the method with owned values (as well as references like
before).
2021-07-30 20:58:49 +00:00
f1198fc6d5
filters crds values in parallel when responding to gossip pull-requests ( #18877 )
...
When responding to gossip pull-requests, filter_crds_values takes a lot of time
while holding onto read-lock:
https://github.com/solana-labs/solana/blob/f51d64868/gossip/src/crds_gossip_pull.rs#L509-L566
This commit will filter-crds-values in parallel using rayon thread-pools.
2021-07-26 17:13:11 +00:00
d2d5f36a3c
adds validator flag to allow private ip addresses ( #18850 )
2021-07-23 15:25:03 +00:00
bbd22f06f4
implements generic lookups into gossip crds table ( #18765 )
...
This commit adds CrdsEntry trait which allows generic lookups into crds
table. For example to get ContactInfo or LowestSlot associated with a
Pubkey, the lookup code would be respectively:
crds.get::<&ContactInfo>(pubkey)
crds.get::<&LowestSlot>(pubkey)
2021-07-21 12:16:26 +00:00
207c90bd8b
Shorten long SerializeWith type paths in abi digest ( #18734 )
2021-07-20 08:59:50 -05:00
8da261cf5c
locks crds only once in ClusterInfo::repair_peers ( #18752 )
...
ClusterInfo::repair_peers locks crds table twice, and shows performance
regression if the RwLock is not reader-preferred:
https://github.com/solana-labs/solana/blob/269028360/gossip/src/cluster_info.rs#L1188-L1210
2021-07-18 16:55:58 +00:00
e316586516
excludes private ip addresses
2021-07-16 20:05:48 -06:00
ae5ad5cf9b
sendmmsg cleanup #18589
...
Rationalize usage of sendmmsg(2). Skip packets which failed to send and track failures.
2021-07-16 14:36:49 -07:00
steviez
behzad nouri
Michael Vines
ryleung-solana
sakridge
Justin Starry
anatoly yakovenko
Jeff Biseda
carllin
Ashwin Sekar
carllin
Brooks Prumo
Tao Zhu