Commit Graph

63 Commits

Author SHA1 Message Date
steviez ec7ca411dd
Make PacketBatch packets vector non-public (#25413)
Upcoming changes to PacketBatch to support variable sized packets will
modify the internals of PacketBatch. So, this change removes usage of
the internal packet struct and instead uses accessors (which are
currently just wrappers of Vector functions but will change down the
road).
2022-05-23 15:30:15 -05:00
behzad nouri c248fb3f51
renames Packet Meta::{,set_}addr methods to {,set_}socket_addr (#25478)
In order to distinguish between Meta.addr field which is an IpAddr and
the methods which refer to a SocketAddr.
2022-05-23 15:48:59 +00:00
Michael Vines b05c7d91ed Fix derive_partial_eq_without_eq clippy lint 2022-05-22 22:22:21 -07:00
sakridge a56ce8283a
quic: Fix prune_oldest table size calculation (#25431) 2022-05-22 00:06:05 +02:00
Justin Starry 773b2f23f4
Fix flakey streamer_send_test (#25440) 2022-05-22 01:09:26 +08:00
Jon Cinque a5792885ca
streamer: Add nonblocking versions of sendmmsg / recvmmsg (#25415) 2022-05-20 22:40:40 +02:00
sakridge 3d96a1ab76
Block packets in vote-only mode (#24906) 2022-05-14 17:53:37 +02:00
sakridge 5be138826b
Add sender stake to quic packets (#25054) 2022-05-07 16:45:03 +02:00
Justin Starry 7100f1c94b
Collect stats in streamer receiver and report fetch stage metrics (#25010) 2022-05-06 02:56:18 +08:00
Pankaj Garg 0910d9a209
Increase worker thread limits in quic streamer runtime (#24751)
* Resurrect "Increase worker thread limits in quic streamer runtime (#24535)""

* Fix test_quic_server_block_multiple_connections

* remove sleep
2022-04-27 20:50:08 +00:00
Pankaj Garg ded6ac6696
Swap quic counters with 0 while reporting (#24754) 2022-04-27 20:17:40 +00:00
Justin Starry 2f62fb2031 Revert "Increase worker thread limits in quic streamer runtime (#24535)"
This reverts commit f244a2e141.
2022-04-28 00:46:42 +08:00
Pankaj Garg f244a2e141
Increase worker thread limits in quic streamer runtime (#24535)
* Increase worker thread limits in quic streamer runtime

* fix Cargo.lock

* fix test, and use a constant for controlling thread count
2022-04-26 21:25:02 +00:00
Pankaj Garg 564bd4b34b
Add more datapoints in QUIC streamer (#24611) 2022-04-25 10:12:11 -07:00
dependabot[bot] 8d4e53d23c
chore: bump nix from 0.23.1 to 0.24.0 (#24548)
* chore: bump nix from 0.23.1 to 0.24.0

Bumps [nix](https://github.com/nix-rust/nix) from 0.23.1 to 0.24.0.
- [Release notes](https://github.com/nix-rust/nix/releases)
- [Changelog](https://github.com/nix-rust/nix/blob/master/CHANGELOG.md)
- [Commits](https://github.com/nix-rust/nix/commits)

---
updated-dependencies:
- dependency-name: nix
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

* allow deprecated

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
2022-04-22 01:41:11 +00:00
Pankaj Garg 5d0dcca1e7
Add some datapoints in quic for error conditions (#24489) 2022-04-19 13:05:02 -07:00
kirill lykov 5c7060eaeb
fix test compilation error (#24413) 2022-04-16 14:41:33 +02:00
sakridge d71986cecf
Separate staked and un-staked on quic tpu port (#24339) 2022-04-16 10:54:22 +02:00
sakridge 1b7d1f78de
Implement QUIC connection warmup service for future leaders (#24054)
* Increase connection timeouts

* Bump quic connection cache to 1024

* Use constant for quic connection timeout and add warm cache service

* Fixes to QUIC warmup service

* fix check failure

* fixes after rebase

* fix timeout test

Co-authored-by: Pankaj Garg <pankaj@solana.com>
2022-04-15 12:09:24 -07:00
ryleung-solana 82945ba973
Optimize TpuConnection and its implementations and refactor connection-cache to not use dyn in order to enable those changes (#23877) 2022-03-24 11:40:26 -04:00
Stephen Akridge 976b138e76 Add tx weighting stage 2022-03-17 19:31:28 -05:00
ryleung-solana 17b00ad3a4
Add quic-client module (#23166)
* Add quic-client module to send transactions via quic, abstracted behind the TpuConnection trait (along with a legacy UDP implementation of TpuConnection) and change thin-client to use TpuConnection
2022-03-09 21:33:05 -05:00
sakridge 7a9884c831
Quic limit connections (#23283)
* quic server limit connections

* bump per_ip

* Review comments

* Make the connections per port
2022-03-09 10:52:31 +01:00
Trent Nelson d4292774c5 checks 2022-02-25 08:05:28 +00:00
buffalu 70ebab2c82
Add rustfmt.toml and `cargo fmt` (#23238)
* fmt

* formatted

Co-authored-by: Lucas B <buffalu@jito.network>
2022-02-19 13:32:29 +08:00
behzad nouri c078ca3fb3
uses sendmmsg in streamer (#23062)
packet::send_to sends packets one by one:
https://github.com/solana-labs/solana/blob/9213fcb11/streamer/src/packet.rs#L63-L75

sendmmsg uses a single system call for multiple messages:
https://github.com/solana-labs/solana/blob/9213fcb11/streamer/src/sendmmsg.rs#L94
2022-02-11 13:36:50 +00:00
sakridge 5a230f418d
Add quic port for accepting transactions (#22753)
using quinn library

streamer: Sign TLS cert with validator identity key

Handle multiple incoming chunks
2022-02-04 15:27:09 +01:00
Michael Vines 6d5bbca630 Pacify clippy 2022-01-21 19:12:57 -08:00
Jeff Biseda 8b66625c95
convert std::sync::mpsc to crossbeam_channel (#22264) 2022-01-11 02:44:46 -08:00
behzad nouri 379feecae5 patches bug in recv_mmsg when npkts != nrecv
If recv_mmsg receives 2 packets where the first one is filtered out,
then it returns npkts == 1:
https://github.com/solana-labs/solana/blob/01a096adc/streamer/src/recvmmsg.rs#L104-L115

But then streamer::packet::recv_from will erroneously keep the 1st
packet and drop the 2nd one:
https://github.com/solana-labs/solana/blob/01a096adc/streamer/src/packet.rs#L34-L49

To avoid this bug, this commit updates recv_mmsg to always return total
number of received packets. If socket address cannot be correctly
obtained, it is left as the default value which is UNSPECIFIED:
https://github.com/solana-labs/solana/blob/01a096adc/sdk/src/packet.rs#L145
2022-01-04 21:06:59 +00:00
behzad nouri 4b24499916 removes total-size from return value of recv_mmsg 2022-01-04 21:06:59 +00:00
behzad nouri 73a7741c49 uses std::net::IpAddr type for Packet.Meta.addr 2022-01-04 13:53:40 +00:00
steviez 20b61e28b6
Flip iter operations to keep associated address/header/packets together (#22245)
Flip iter operations to keep associated address/header/packets together

Before this change, if cast_socket_addr() returned a None for any
address/header pair, the subsequent zip() would misalign the
address/header pair and packet. So, this change zips all three together,
then does filter_map() so keep things aligned.

Additionally, compute total_size inline to avoid running through packets
a second time.
2022-01-03 17:15:50 -06:00
Jeff Biseda 97a1fa10a6
streamer send destination metrics for repair, gossip (#21564) 2021-12-17 15:21:05 -08:00
Justin Starry 254ef3e7b6
Rename Packets to PacketBatch (#21794) 2021-12-11 09:44:15 -05:00
Michael Vines b8837c04ec Reformat imports to a consistent style for imports
rustfmt.toml configuration:
  imports_granularity = "One"
  group_imports = "One"
2021-12-03 09:19:13 -08:00
Michael Vines 7027d56064 Resolve nightly-2021-10-05 clippy complaints 2021-10-06 10:37:58 -07:00
sakridge 94668c95c2
Prune sigverify queue (#20331) 2021-09-30 05:41:05 +02:00
Jeff Biseda 640e93187c
periodically report sigverify_stage stats (#19674) 2021-09-21 10:37:58 -07:00
behzad nouri 8229a4fbf6
removes raw indexing from streamer (#19183)
Raw indexing is verbose and error-prone. This same code had an indexing
bug causing validator nodes panic just a few months ago:
https://github.com/solana-labs/solana/commit/482b8c6be
2021-08-12 01:42:12 +00:00
Jeff Biseda 0b7ed18cfa
recvmmsg IPv6 awareness (#18957) 2021-08-03 20:35:50 -07:00
behzad nouri 049fb0417f
allows sendmmsg api taking owned values (as well as references) (#18999)
Current signature of api in sendmmsg requires a slice of inner
references:
https://github.com/solana-labs/solana/blob/fe1ee4980/streamer/src/sendmmsg.rs#L130-L152

That forces the call-site to convert owned values to references even
though doing so is redundant and adds an extra level of indirection:
https://github.com/solana-labs/solana/blob/fe1ee4980/core/src/repair_service.rs#L291

This commit expands the api using AsRef and Borrow traits to allow
calling the method with owned values (as well as references like
before).
2021-07-30 20:58:49 +00:00
behzad nouri d2d5f36a3c
adds validator flag to allow private ip addresses (#18850) 2021-07-23 15:25:03 +00:00
behzad nouri e316586516 excludes private ip addresses 2021-07-16 20:05:48 -06:00
Jeff Biseda ae5ad5cf9b
sendmmsg cleanup #18589
Rationalize usage of sendmmsg(2). Skip packets which failed to send and track failures.
2021-07-16 14:36:49 -07:00
Michael Vines 4a12c715a3 Drop Error suffix from enum values to avoid the enum_variant_names clippy lint 2021-06-18 23:02:13 +00:00
sakridge eeee75c5be
Don't use pinned memory when unnecessary (#17832)
Reports of excessive GPU memory usage and errors
from cudaHostRegister. There are some cases where pinning is
not required.
2021-06-14 16:10:04 +02:00
behzad nouri e405747409 Revert "Add limit and shrink policy for recycler (#15320)"
This reverts commit c2e8814dce.
2021-04-18 19:29:24 +00:00
Christian Drappi 54a04bac3d
Apple M1 compatibility (#16346)
Co-authored-by: Christian Drappi <christiandrappi@Christians-MacBook-Pro.local>
2021-04-09 17:21:01 -07:00
behzad nouri 482b8c6be9
Merge pull request from GHSA-qgrq-w56v-hr62 2021-04-01 09:54:45 -07:00