Commit Graph

169 Commits

Author SHA1 Message Date
behzad nouri afd044e296
removes redundant ClusterInfo::drain_push_queue (#33753) 2023-10-18 18:53:58 +00:00
behzad nouri 2465abce5c
simplifies pull-responses handling (#33743)
Following:
https://github.com/solana-labs/solana/pull/33722
from pubkey in PullResponse is no longer used in processing
pull-responses and so the code can be simplified.
2023-10-18 18:06:14 +00:00
Greg Cusack 6efc7ec61d
remove redundant pubkey update record (#33722)
* remove redundant pubkey update record

* from became unused, so removed from all process_pull_response() calls
2023-10-17 10:34:12 -07:00
Wen 0a3810854f
Add RestartLastVotedForkSlots for wen_restart. (#33239)
* Add RestartLastVotedForkSlots and RestartHeaviestFork for wen_restart.

* Fix linter errors.

* Revert RestartHeaviestFork, it will be added in another PR.

* Update frozen abi message.

* Fix wrong number in test generation, change to pub(crate) to limit scope.

* Separate push_epoch_slots and push_restart_last_voted_fork_slots.

* Add RestartLastVotedForkSlots data structure.

* Remove unused parts to make PR smaller.

* Remove unused clone.

* Use CompressedSlotsVec to share code between EpochSlots and RestartLastVotedForkSlots.

* Add total_messages to show how many messages are there.

* Reduce RestartLastVotedForkSlots to one packet (16k slots).

* Replace last_vote_slot with shred_version, revert CompressedSlotsVec.
2023-10-09 15:01:50 -07:00
behzad nouri 1d91b60a57
removes unused legacy-snapshot-hashes api in gossip (#33593)
https://github.com/solana-labs/solana/pull/33576
stops broadcasting legacy snapshot hashes over gossip, and this commit
removes unused legacy snapshot hashed code in gossip.
2023-10-09 15:22:34 +00:00
Pankaj Garg f50342a790
Split vote related code from runtime to its own crate (#32882)
* Move vote related code to its own crate

* Update imports in code and tests

* update programs/sbf/Cargo.lock

* fix check errors

* update abi_digest

* rebase fixes

* fixes after rebase
2023-09-19 10:46:37 -07:00
behzad nouri 39615bd075
removes manual let...else (#33089)
https://rust-lang.github.io/rust-clippy/master/index.html#/manual_let_else
2023-08-31 22:35:47 +00:00
Ashwin Sekar 6d2dae6ab0
send duplicate shred proofs for conflicting shred scenarios (#32965)
* send duplicate shred proofs for conflicting shred scenarios

The scenarios are multiple last_shred_in_slot shreds and
coding shreds with conflicting erasure metas.

* Pr feedback: deprecate shred_index for gossip duplicate shred proofs

* Pr feedback

* Remove extraneous dead code
2023-08-31 12:08:10 -07:00
Trent Nelson b13589b588
round two preliminaries for bumping nightly to 2023-08-25 (#33064)
* work around nightly ICE

seems to be something related to `const ref`s in generic contexts...

* allow lint false-positive on ignored trait bounds
2023-08-30 21:46:20 +02:00
Trent Nelson b8dc5daedb
preliminaries for bumping nightly to 2023-08-25 (#33047)
* remove unnecessary hashes around raw string literals

* remove unncessary literal `unwrap()`s

* remove panicking `unwrap()`

* remove unnecessary `unwrap()`

* use `[]` instead of `vec![]` where applicable

* remove (more) unnecessary explicit `into_iter()` calls

* remove redundant pattern matching

* don't cast to same type and constness

* do not `cfg(any(...` a single item

* remove needless pass by `&mut`

* prefer `or_default()` to `or_insert_with(T::default())`

* `filter_map()` better written as `filter()`

* incorrect `PartialOrd` impl on `Ord` type

* replace "slow zero-filled `Vec` initializations"

* remove redundant local bindings

* add required lifetime to associated constant
2023-08-29 23:05:35 +00:00
Jon Cinque 0fe902ced7
Bump rand to 0.8, rand_chacha to 0.3, getrandom to 0.2 (#32871)
* sdk: Add concurrent support for rand 0.7 and 0.8

* Update rand, rand_chacha, and getrandom versions

* Run command to replace `gen_range`

Run `git grep -l gen_range | xargs sed -i'' -e 's/gen_range(\(\S*\), /gen_range(\1../'

* sdk: Fix users of older `gen_range`

* Replace `hash::new_rand` with `hash::new_with_thread_rng`

Run:
```
git grep -l hash::new_rand | xargs sed -i'' -e 's/hash::new_rand([^)]*/hash::new_with_thread_rng(/'
```

* perf: Use `Keypair::new()` instead of `generate`

* Use older rand version in zk-token-sdk

* program-runtime: Inline random key generation

* bloom: Fix clippy warnings in tests

* streamer: Scope rng usage correctly

* perf: Fix clippy warning

* accounts-db: Map to char to generate a random string

* Remove `from_secret_key_bytes`, it's just `keypair_from_seed`

* ledger: Generate keypairs by hand

* ed25519-tests: Use new rand

* runtime: Use new rand in all tests

* gossip: Clean up clippy and inline keypair generators

* core: Inline keypair generation for tests

* Push sbf lockfile change

* sdk: Sort dependencies correctly

* Remove `hash::new_with_thread_rng`, use `Hash::new_unique()`

* Use Keypair::new where chacha isn't used

* sdk: Fix build by marking rand 0.7 optional

* Hardcode secret key length, add static assertion

* Unify `getrandom` crate usage to fix linking errors

* bloom: Fix tests that require a random hash

* Remove some dependencies, try to unify others

* Remove unnecessary uses of rand and rand_core

* Update lockfiles

* Add back some dependencies to reduce rebuilds

* Increase max rebuilds from 14 to 15

* frozen-abi: Remove `getrandom`

* Bump rebuilds to 17

* Remove getrandom from zk-token-proof
2023-08-21 19:11:21 +02:00
behzad nouri a9ecdc0ae5
asserts that socket addresses set in new_with_external_ip are valid (#32860)
new_with_external_ip silently ignores errors when setting socket
addresses which makes it harder to catch bugs:
https://github.com/solana-labs/solana/blob/7beeb8310/gossip/src/cluster_info.rs#L3054-L3063
2023-08-16 20:41:40 +00:00
behzad nouri 7beeb83104
patches set_serve_repair instead of set_serve_repair_quic (#32856) 2023-08-16 18:26:00 +00:00
behzad nouri 0de8ccfda9
adds socket address for repair service over QUIC (#32834)
Working towards migrating repair to QUIC.
2023-08-15 17:09:09 +00:00
behzad nouri 52616cf7aa
removes repair socket from gossip ContactInfo (#32831)
Repair responses are sent back to the address the repair request came
from and there is no need to gossip repair socket address.
https://github.com/solana-labs/solana/blob/9212ac347/core/src/repair/serve_repair.rs#L519
2023-08-14 20:01:18 +00:00
behzad nouri b06500ee66
adds extensions to contact-info (#32309)
The commit adds a Vec<Extension> to ContactInfo so that future additions
to ContactInfo can be done by only adding new Extensions instead of
modifying the entire ContactInfo.
2023-08-09 22:01:20 +00:00
behzad nouri b7c2ad5b67
repurposes tvu_forwards socket for TVU over QUIC (#32737)
LegacyContactInfo.tvu_forwards is unused.
Repurposing the field for TVU over QUIC will avoid QUIC_PORT_OFFSET hack
in a backward compatible way.
2023-08-07 22:02:41 +00:00
behzad nouri 0511753276
moves DATA_PLANE_FANOUT to turbine (#32744) 2023-08-07 20:53:49 +00:00
Greg Cusack cd7f03bc88
remove duplicate metric reporting in cluster_info_stats3 (#32566)
process_gossip_packets_iterations_since_last_report measures the same loop count as gossip_listen_loop_iterations_since_last_report which already exists
2023-07-24 09:01:07 -06:00
behzad nouri d54b6204be
removes instances of clippy::manual_let_else (#32417) 2023-07-09 21:41:36 +00:00
behzad nouri 0da01270ef
removes redundant recycler clones (#32401) 2023-07-06 18:25:20 +00:00
behzad nouri 469661d217
removes outdated tvu_forward socket (#32101)
Shreds are no longer sent to tvu_forward socket.
2023-06-20 20:50:16 +00:00
behzad nouri 987e8eeeaf
removes feature gate code dropping redundant turbine path (#32075) 2023-06-16 19:53:05 +00:00
behzad nouri aed4ecb633
adds quic receiver to shred-fetch-stage (#31576)
Working towards migrating turbine to QUIC.
2023-06-12 13:16:27 +00:00
behzad nouri 5178d4d49b
adds quic tvu port to contact-info (#31614)
Working towards migrating turbine to QUIC.
2023-05-15 15:13:21 +00:00
behzad nouri 4e34abbf3d
specifies protocol in contact-info get-socket api (#31602) 2023-05-12 16:16:20 +00:00
Illia Bobyr 2d090d4547
test_gossip_node: Use random port (#31490)
Using a fixed port could cause a false negative, if the port is
currently in use.  We actually see this test failing regularly with an
error that port `1111` is already in use.

Quick search did not show any tests that hardcode port 1111, so it is
unclear why is this happening.  But using hardcoded ports is not a good
practice anyways.
2023-05-05 18:47:24 -07:00
Brooks c5e071c7fe
Upgrades nightly Rust to 2023-03-04 (#31487) 2023-05-05 08:28:23 -04:00
behzad nouri aafcac27d8
removes pubkey from LegacyContactInfo public interface (#31375)
Working towards LegacyContactInfo => ContactInfo migration, the commit
adds more api parity between the two.
2023-04-28 12:05:15 +00:00
DimAn bd782634dc
Add ability to manage public TPU Forwards address (#31201)
A node operator can manage a public TPU address (at node startup and while it's running), but doesn't have the ability to manage the TPU Forwards address as well.

This PR adds that functionality.

Added the start argument --public-tpu-forwards-address
Reworked the set-public-tpu-address subcommand into the set-public-address subcommand
2023-04-24 23:04:36 +00:00
behzad nouri 1b08d01a80
removes shred_version from LegacyContactInfo public interface (#31304)
Working towards LegacyContactInfo => ContactInfo migration, the commit
adds more api parity between the two.
2023-04-24 15:19:33 +00:00
behzad nouri a88024e295
removes wallclock from LegacyContactInfo public interface (#31303) 2023-04-22 20:18:39 +00:00
behzad nouri cb65a785bc
makes sockets in LegacyContactInfo private (#31248)
Working towards LegacyContactInfo => ContactInfo migration, the commit
hides some implementation details of LegacyContactInfo and expands API
parity with the new ContactInfo.
2023-04-21 15:39:16 +00:00
DimAn b81b7ebf03
validator: `--tpu-host-addr` -> `--public-tpu-address` (#31137) 2023-04-12 13:33:09 +00:00
DimAn 9136f80d36
validator: add `set-public-tpu-address` command (#30452) 2023-04-12 13:32:22 +00:00
Brooks 453f272698
Rename IncrementalSnapshotHashes to SnapshotHashes (#31136) 2023-04-11 10:30:29 -04:00
Brooks f3083ad2e0
Rename SnapshotHashes to LegacySnapshotHashes (#31086) 2023-04-10 17:52:20 -04:00
Brooks e8ea722061
Uses AccountsHashes type for AccountsHashes CrdsData variant (#31003) 2023-04-03 16:42:21 -04:00
behzad nouri d4b30adffe
reworks gossip crds timeouts (#30468)
CrdsGossipPull::make_timeouts iterates over the stakes hashmap and
creates a new hashmap which is unnecessary:
https://github.com/solana-labs/solana/blob/c032dc275/gossip/src/crds_gossip_pull.rs#L517-L539

The commit instead keeps a reference to the stakes hashmap.
2023-03-27 21:52:48 +00:00
kirill lykov ee1717b24b
Make clippy to be happy (#30394)
* replace default implementation with default enum

* fix format to make clippy happy
2023-02-17 20:51:18 +01:00
Brennan b678ee3583
Clean up socket binding reuse (#30367) 2023-02-16 11:07:22 -08:00
behzad nouri 4892a6a910
removes redundant CrdsGossipPull.msg_timeout (#30334) 2023-02-16 00:23:27 +00:00
behzad nouri ded457cd73
embeds the new gossip ContactInfo in ClusterInfo (#30022)
Working towards replacing the legacy gossip contact-info with the new
one, the commit updates the respective field in gossip cluster-info.
2023-02-10 20:07:45 +00:00
behzad nouri 544fbded07
removes wallclock from duplicate-shreds handler (#30187) 2023-02-08 17:29:30 +00:00
Kevin Ji cd51499ab9
Use Ipv4Addr constants in socketaddr! (#30095) 2023-02-02 16:48:21 -07:00
behzad nouri 4cc07a176e
reduces number of gossip pull requests/responses (#29974) 2023-01-30 17:59:56 +00:00
behzad nouri d6fbf3fb17
adds new contact-info with forward compatible sockets (#29596)
The commit implement new ContactInfo where
* Ports and IP addresses are specified separately so that unique IP
  addresses can only be specified once.
* Different sockets (tvu, tpu, etc) are specified by opaque u8 tags so
  that adding and removing sockets is backward and forward compatible.
* solana_version::Version is also embedded in so that it won't need to
  be gossiped separately.
* NodeInstance is also rolled in by adding a field identifying when the
  instance was first created so that it won't need to be gossiped
  separately.

Update plan:
* Once the cluster is able to ingest the new type (i.e. this patch), a
  2nd patch will start gossiping the new ContactInfo along with the
  LegacyContactInfo.
* Once all nodes in the cluster gossip the new ContactInfo, a 3rd patch
  will start solely using the new ContactInfo while still gossiping the
  old LegacyContactInfo.
* Once all nodes in the cluster solely use the new ContactInfo, a 4th
  patch will stop gossiping the old LegacyContactInfo.
2023-01-26 17:02:18 +00:00
behzad nouri 1c7662a37f
asserts that cluster-info keypair is consistent with contact-info id (#29818) 2023-01-24 16:57:55 +00:00
Kevin Ji dd92f225bb
Use Ipv4Addr::{LOCALHOST, UNSPECIFIED} constants (#29813) 2023-01-23 16:49:51 -06:00
behzad nouri 590b75140f
removes legacy retransmit tests (#29817)
Retransmit code has moved to core/src/cluster_nodes.rs and has been
significantly revised.
gossip/tests/cluster_info.rs is testing the old code which is no longer
relevant.
2023-01-21 22:28:48 +00:00