57057f8d39
uses enum for shred type
...
Current code is using u8 which does not have any type-safety and can
contain invalid values:
https://github.com/solana-labs/solana/blob/66fa062f1/ledger/src/shred.rs#L167
Checks for invalid shred-types are scattered through the code:
https://github.com/solana-labs/solana/blob/66fa062f1/ledger/src/blockstore.rs#L849-L851
https://github.com/solana-labs/solana/blob/66fa062f1/ledger/src/shred.rs#L346-L348
The commit uses enum for shred type with #[repr(u8)]. Backward
compatibility is maintained by implementing Serialize and Deserialize
compatible with u8, and adding a test to assert that.
2021-11-19 14:16:39 +00:00
b30c94ce55
ClusterInfoVoteListener send only missing votes to BankingStage ( #20873 )
2021-11-18 15:20:41 -08:00
3fc858eb60
adds methods to obtain data/coding shreds indices from ErasureMeta
2021-11-13 17:08:05 +00:00
b0ca335463
Rename "trusted" to "known" in `validators/` ( #21197 )
...
* Replaced trusted with known validator
* Format Convention
2021-11-12 11:57:55 -07:00
eea3fb327f
seeds rng for test_build_crds_filter test ( #21031 )
2021-10-28 18:29:32 +00:00
43168e6365
doubles crds unique pubkey capacity ( #20947 )
2021-10-26 13:06:55 +00:00
1297a13586
adds metrics tracking crds writes and votes ( #20953 )
2021-10-26 13:02:30 +00:00
14af1957d6
Make pub IncrementalSnapshotHashes fields ( #20727 )
2021-10-18 13:38:43 -05:00
0c0384ec32
revises turbine peers shuffling order ( #20480 )
...
Turbine randomly shuffles cluster nodes on a broadcast tree for each
shred. This requires knowing the stakes and nodes' contact-infos (from
gossip).
However gossip is subject to partitioning and propogation delays.
Additionally unstaked nodes may join and leave the cluster at any
moment, changing the cluster view from one node to another.
This commit:
* Always arranges the unstaked nodes at the bottom of turbine broadcast
tree.
* Staked nodes are always included regardless of if their contact-info
is available in gossip or not.
* Uses the unbiased WeightedShuffle construct for shuffling nodes.
2021-10-14 15:09:36 +00:00
1fcfbfccbb
Add fn to push IncrementalSnapshotHashes to cluster via gossip ( #20395 )
2021-10-08 08:20:35 -05:00
57592e463e
Add get_incremental_snapshot_hash_for_node() to gossip ( #20394 )
2021-10-07 19:47:14 -05:00
0da661de62
adds metrics for number of nodes vs number of pubkeys ( #20512 )
2021-10-07 18:56:05 +00:00
177a375479
Tpu vote 1.7 ( #20187 ) ( #20494 )
...
* Add separate vote processing tpu port
* Add feature to send to tpu vote port
* Add vote rejecting sigverify mode
* use packet.meta.is_simple_vote_tx in place of deserialization
* consolidate code that identifies vote tx atcommon path for cpu and gpu
* new key for feature set
* banking forward tpu vote
* add tpu vote port to dockerfile and other review changes
* Simplify thread id compare
* fix a test; updated cluster_info ABI change
Co-authored-by: Tao Zhu <tao@solana.com>
Co-authored-by: sakridge <sakridge@gmail.com>
2021-10-07 09:38:23 +00:00
767f740305
Bump version to 1.9.0
2021-10-06 17:57:41 -07:00
5ceb5ef328
chore: bump lru from 0.6.6 to 0.7.0 ( #20437 )
...
Bumps [lru](https://github.com/jeromefroe/lru-rs ) from 0.6.6 to 0.7.0.
- [Release notes](https://github.com/jeromefroe/lru-rs/releases )
- [Changelog](https://github.com/jeromefroe/lru-rs/blob/master/CHANGELOG.md )
- [Commits](https://github.com/jeromefroe/lru-rs/compare/0.6.6...0.7.0 )
---
updated-dependencies:
- dependency-name: lru
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-10-06 17:32:06 -06:00
4e3818e5c1
Add CrdsData::IncrementalSnapshotHashes ( #20374 )
2021-10-05 09:57:46 -05:00
5d141fe01d
Rename CRDS SnapshotHash to SnapshotHashes ( #20421 )
2021-10-04 19:03:28 -05:00
ee8621a8bd
Add metric measuring number of successfully inserted push messages ( #20275 )
...
* Add number of successfully inserted push messages
2021-09-28 21:41:17 -07:00
43ed727ba7
reverts #17542 ( #20259 )
...
https://github.com/solana-labs/solana/pull/17542
excludes caller's crds values from pull responses.
Reverting that commit so that when a (staked) node restarts, it can
obtain its crds values before restart from other nodes.
2021-09-27 22:03:26 +00:00
013e1d9d49
Limit transaction forwarding from banking_stage ( #19940 )
2021-09-21 08:49:41 -07:00
44c8b1bca2
Remove clippy ( #19793 )
2021-09-13 20:08:28 -07:00
c9261e034e
remove dev deps
2021-09-11 14:33:53 -07:00
d7051b0d21
adds logs when push-vote panics with invalid vote-index ( #19485 )
...
In order to debug this panic on the clusters:
panicked at 'assertion failed: (vote_index as usize) <
MAX_LOCKOUT_HISTORY', core/src/cluster_info.rs:1012:9
2021-08-31 12:15:07 +00:00
6a53ec28e0
chore: bump serde from 1.0.129 to 1.0.130 ( #19497 )
...
* chore: bump serde from 1.0.129 to 1.0.130
Bumps [serde](https://github.com/serde-rs/serde ) from 1.0.129 to 1.0.130.
- [Release notes](https://github.com/serde-rs/serde/releases )
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.129...v1.0.130 )
---
updated-dependencies:
- dependency-name: serde
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <you@example.com>
2021-08-30 18:19:34 +00:00
6909a79b6f
removes require-stake-for-gossip feature ( #19476 )
...
The feature is already activated on all clusters.
2021-08-27 21:17:15 +00:00
46890ac197
chore: bump serde from 1.0.128 to 1.0.129 ( #19395 )
...
* chore: bump serde from 1.0.128 to 1.0.129
Bumps [serde](https://github.com/serde-rs/serde ) from 1.0.128 to 1.0.129.
- [Release notes](https://github.com/serde-rs/serde/releases )
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.128...v1.0.129 )
---
updated-dependencies:
- dependency-name: serde
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <you@example.com>
2021-08-24 12:32:30 -06:00
aea9960c75
chore: bump serde from 1.0.127 to 1.0.128 ( #19370 )
...
* chore: bump serde from 1.0.127 to 1.0.128
Bumps [serde](https://github.com/serde-rs/serde ) from 1.0.127 to 1.0.128.
- [Release notes](https://github.com/serde-rs/serde/releases )
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.127...v1.0.128 )
---
updated-dependencies:
- dependency-name: serde
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <anatoly+githubjenkins@solana.io>
2021-08-23 09:33:34 -06:00
3efccbffab
sends shreds (instead of packets) to retransmit stage
...
Working towards channelling through shreds recovered from erasure codes
to retransmit stage.
2021-08-17 13:44:10 +00:00
140abec6ef
exempts node-instances from shred-version check ( #19190 )
...
Clusters are kept separate using the shred-versions obtained from
contact-infos. However, this mechanism breaks if there are 2 instances
of the same identity key running on different clusters, because then one
of the two contact-infos have the right shred-version.
If a node has the contact-info with the matching shred-version, then it
will pass all associated crds values even if they belong to the other
instance. So the shred-version check breaks.
As a result we cannot support 2 instances of the same identity key
running on different clusters. To prevent that, this commit is exempting
node-instances from shred-version check so that they are always
propagated across clusters and halt one of the running duplicate
instances.
2021-08-14 00:47:44 +00:00
5b9671d01a
chore: bump matches from 0.1.8 to 0.1.9 ( #19220 )
...
* chore: bump matches from 0.1.8 to 0.1.9
Bumps [matches](https://github.com/SimonSapin/rust-std-candidates ) from 0.1.8 to 0.1.9.
- [Release notes](https://github.com/SimonSapin/rust-std-candidates/releases )
- [Commits](https://github.com/SimonSapin/rust-std-candidates/commits )
---
updated-dependencies:
- dependency-name: matches
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <you@example.com>
2021-08-13 16:37:45 +00:00
7a789e0763
filters for recent contact-infos when checking for live stake ( #19204 )
...
Contact-infos are saved to disk:
https://github.com/solana-labs/solana/blob/9dfeee299/gossip/src/cluster_info.rs#L1678-L1683
and restored on validator start-up:
https://github.com/solana-labs/solana/blob/9dfeee299/core/src/validator.rs#L450
Staked nodes entries will not expire until an epoch after. So when the
validator checks for online stake it is erroneously picking up
contact-infos restored from disk, which breaks the entire
wait-for-supermajority logic:
https://github.com/solana-labs/solana/blob/9dfeee299/core/src/validator.rs#L1515-L1561
This commit adds an extra check for the age of contact-info entries and
filters out old ones.
2021-08-13 12:12:40 +00:00
f302774cf7
implements copy-on-write for staked-nodes ( #19090 )
...
Bank::staked_nodes and Bank::epoch_staked_nodes redundantly clone
staked-nodes HashMap even though an immutable reference will suffice:
https://github.com/solana-labs/solana/blob/a9014cece/runtime/src/vote_account.rs#L77
This commit implements copy-on-write semantics for staked-nodes by
wrapping the underlying HashMap in Arc<...>.
2021-08-10 12:59:12 +00:00
8817f59b6e
Version transaction message and add new message format ( #18725 )
...
* Version transaction message and add new message format
* Update abi digest due to message path change
* Update v0.rs
Fix comment
* Update original.rs
* Update message versions name and address map indexes field name
* s/original/legacy
* update comment
* cargo fmt
* Update abi digest due to legacy rename
2021-08-09 22:03:39 -07:00
14361906ca
for all tests, bank::new -> bank::new_for_tests ( #19064 )
2021-08-05 08:42:38 -05:00
c16bf02448
chore: bump serde from 1.0.126 to 1.0.127 ( #19010 )
...
* chore: bump serde from 1.0.126 to 1.0.127
Bumps [serde](https://github.com/serde-rs/serde ) from 1.0.126 to 1.0.127.
- [Release notes](https://github.com/serde-rs/serde/releases )
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.126...v1.0.127 )
---
updated-dependencies:
- dependency-name: serde
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <you@example.com>
2021-08-02 21:16:34 +00:00
049fb0417f
allows sendmmsg api taking owned values (as well as references) ( #18999 )
...
Current signature of api in sendmmsg requires a slice of inner
references:
https://github.com/solana-labs/solana/blob/fe1ee4980/streamer/src/sendmmsg.rs#L130-L152
That forces the call-site to convert owned values to references even
though doing so is redundant and adds an extra level of indirection:
https://github.com/solana-labs/solana/blob/fe1ee4980/core/src/repair_service.rs#L291
This commit expands the api using AsRef and Borrow traits to allow
calling the method with owned values (as well as references like
before).
2021-07-30 20:58:49 +00:00
81026f9ea5
passes through --allow-private-addr to validators in system perf tests ( #18876 )
2021-07-29 19:04:45 +00:00
5f23d8530d
chore: bump lru from 0.6.5 to 0.6.6 ( #18963 )
...
Bumps [lru](https://github.com/jeromefroe/lru-rs ) from 0.6.5 to 0.6.6.
- [Release notes](https://github.com/jeromefroe/lru-rs/releases )
- [Changelog](https://github.com/jeromefroe/lru-rs/blob/master/CHANGELOG.md )
- [Commits](https://github.com/jeromefroe/lru-rs/compare/0.6.5...0.6.6 )
---
updated-dependencies:
- dependency-name: lru
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-07-29 08:33:41 -06:00
f1198fc6d5
filters crds values in parallel when responding to gossip pull-requests ( #18877 )
...
When responding to gossip pull-requests, filter_crds_values takes a lot of time
while holding onto read-lock:
https://github.com/solana-labs/solana/blob/f51d64868/gossip/src/crds_gossip_pull.rs#L509-L566
This commit will filter-crds-values in parallel using rayon thread-pools.
2021-07-26 17:13:11 +00:00
d2d5f36a3c
adds validator flag to allow private ip addresses ( #18850 )
2021-07-23 15:25:03 +00:00
588c0464b8
Add sampling logic and DuplicateSlotRepairStatus module ( #18721 )
2021-07-21 11:15:08 -07:00
bbd22f06f4
implements generic lookups into gossip crds table ( #18765 )
...
This commit adds CrdsEntry trait which allows generic lookups into crds
table. For example to get ContactInfo or LowestSlot associated with a
Pubkey, the lookup code would be respectively:
crds.get::<&ContactInfo>(pubkey)
crds.get::<&LowestSlot>(pubkey)
2021-07-21 12:16:26 +00:00
207c90bd8b
Shorten long SerializeWith type paths in abi digest ( #18734 )
2021-07-20 08:59:50 -05:00
8da261cf5c
locks crds only once in ClusterInfo::repair_peers ( #18752 )
...
ClusterInfo::repair_peers locks crds table twice, and shows performance
regression if the RwLock is not reader-preferred:
https://github.com/solana-labs/solana/blob/269028360/gossip/src/cluster_info.rs#L1188-L1210
2021-07-18 16:55:58 +00:00
e316586516
excludes private ip addresses
2021-07-16 20:05:48 -06:00
ae5ad5cf9b
sendmmsg cleanup #18589
...
Rationalize usage of sendmmsg(2). Skip packets which failed to send and track failures.
2021-07-16 14:36:49 -07:00
37ee0b5599
Eliminate doc warnings and fix some markdown ( #18566 )
...
* Fix link target in doc comment
* Fix formatting of log examples in process_instruction
* Fix doc markdown in solana-gossip
* Fix doc markdown in solana-runtime
* Escape square braces in doc comments to avoid warnings
* Surround 'account references' doc items in code spans to avoid warnings
* Fix code block in loader_upgradeable_instruction
* Fix doctest for loader_upgradable_instruction
2021-07-16 00:40:07 +00:00
cf31afdd6a
makes CrdsGossip thread-safe ( #18615 )
2021-07-14 22:27:17 +00:00
7f2254225e
Move entry/poh to own crate to speed up poh bench build ( #18225 )
2021-07-14 14:16:29 +02:00
c90af3cd63
removes id from push_lowest_slot args ( #18645 )
...
push_lowest_slot cannot sign the new crds-value unless the id (pubkey)
argument passed-in is the same pubkey as in ClusterInfo::keypair(), in
which case the id argument is redundant:
https://github.com/solana-labs/solana/blob/bb41cf346/gossip/src/cluster_info.rs#L824-L845
Additionally, the lookup is done with self.id(), but insert is done with
the id argument, which is logically a bug.
2021-07-13 22:32:59 +00:00
behzad nouri
carllin
Michael Keleti
Brooks Prumo
Tao Zhu
Trent Nelson
dependabot[bot]
sakridge
Giles Cope
Justin Starry
Jeff Washington (jwash)
Jeff Biseda
Brian Anderson