quic server: disable GSO
The server only accepts inbound unidirectional streams initiated by
clients, which means that reply data never exceeds one MTU. By disabling
GSO, we make quinn_proto::Connection::poll_transmit allocate only 1 MTU
vs 10 * MTU for _each_ transmit. This reduces allocations 10x.
* use rate limit on connectings
use rate limit on connectings; missing file
* Change connection rate limit to 8/min instead of 4/s
* Addressed some feedback from Trent
* removed some comments
* fix test failures which are opening connections more frequently
* moved the flag up
* turn off rate limiting to debug CI
* Fix CI test failures
* differentiate of the two throttling cases in stats: across connections or per ip addr
* fmt issues
* Addressed some feedback from Trent
* Added unit tests
Cleanup connection cache rate limiter if exceeding certain threshold
missing files
CONNECITON_RATE_LIMITER_CLEANUP_THRESHOLD to 100_000
clippy issue
clippy issue
sort crates
* revert Cargo.lock changes
* Addressed some feedback from Pankaj
* sleep instead of drop when stream rate exceeded limit;
Consider connection count of staked nodes when calculating allowed PPS
remove rtt from throttle_duration calculation
removed connection count in StreamerCounter -- we do not need it at this point
* remove connection count related changes -- they are unrelated to this PR
* revert unintended changes
* quic: don't call connection.set_max_concurrent_uni_streams if we're going to drop a connection
Avoids taking a mutex and waking a task.
* quic: don't increase the receive window before we've actually accepted a connection
quic: use smallvec, save one allocation per packet
Use smallvec to hold chunks. Streams are packet-sized so we don't expect
them to have many chunks. This saves us an allocation for each packet.
quic: switch to ordered reads
Unordered reads cause a BTreeMap allocation for each packet inside quinn
in Assembler::ensure_ordering.
Most streams will fit in one datagram and will therefore be ordered by
definition. Switch to ordered reads to avoid the allocation.
* add PacketFlags::FROM_STAKED_NODE
* Only forward packets from staked node
* fix local-cluster test forwarding
* review comment
* tpu_votes get marked as from_staked_node
* corrected to not use hardcoded connections count for unstaked
* Fixed a math problem on max_unstaked_load_in_throttling_window
* Fixed a unit test failure
The name was previously hard-coded to solReceiver. The use of the same
name makes it hard to figure out which thread is which when these
threads are handling many services (Gossip, Tvu, etc).
* Use EMA to compute QUIC streamer load for staked connections
* change min load to 25% of max load
* reduce max PPS from 500K to 250K
* update ema_function to account for missing intervals
* replace f64 math with u128
* track streams across all connections for a peer
* u128 -> u64
* replace ' as ' type conversion to from and try_from
* add counter for u64 overflow
* reset recent stream load on ema interval
* do not use same counter for unstaked connections from a peer IP
* Throttle unstaked quic streams for a given connection
* Fix interval duration check
* move wait to handle_chunk
* set max unistreams to 0
* drop new streams
* cleanup
* some more cleanup
* fix tests
* update test and stop code
* fix bench-tps
* allow pedantic invalid cast lint
* allow lint with false-positive triggered by `test-case` crate
* nightly `fmt` correction
* adapt to rust layout changes
* remove dubious test
* Use transmute instead of pointer cast and de/ref when check_aligned is false.
* Renames clippy::integer_arithmetic to clippy::arithmetic_side_effects.
* bump rust nightly to 2023-08-25
* Upgrades Rust to 1.72.0
---------
Co-authored-by: Trent Nelson <trent@solana.com>
* remove unnecessary hashes around raw string literals
* remove unncessary literal `unwrap()`s
* remove panicking `unwrap()`
* remove unnecessary `unwrap()`
* use `[]` instead of `vec![]` where applicable
* remove (more) unnecessary explicit `into_iter()` calls
* remove redundant pattern matching
* don't cast to same type and constness
* do not `cfg(any(...` a single item
* remove needless pass by `&mut`
* prefer `or_default()` to `or_insert_with(T::default())`
* `filter_map()` better written as `filter()`
* incorrect `PartialOrd` impl on `Ord` type
* replace "slow zero-filled `Vec` initializations"
* remove redundant local bindings
* add required lifetime to associated constant
* sdk: Add concurrent support for rand 0.7 and 0.8
* Update rand, rand_chacha, and getrandom versions
* Run command to replace `gen_range`
Run `git grep -l gen_range | xargs sed -i'' -e 's/gen_range(\(\S*\), /gen_range(\1../'
* sdk: Fix users of older `gen_range`
* Replace `hash::new_rand` with `hash::new_with_thread_rng`
Run:
```
git grep -l hash::new_rand | xargs sed -i'' -e 's/hash::new_rand([^)]*/hash::new_with_thread_rng(/'
```
* perf: Use `Keypair::new()` instead of `generate`
* Use older rand version in zk-token-sdk
* program-runtime: Inline random key generation
* bloom: Fix clippy warnings in tests
* streamer: Scope rng usage correctly
* perf: Fix clippy warning
* accounts-db: Map to char to generate a random string
* Remove `from_secret_key_bytes`, it's just `keypair_from_seed`
* ledger: Generate keypairs by hand
* ed25519-tests: Use new rand
* runtime: Use new rand in all tests
* gossip: Clean up clippy and inline keypair generators
* core: Inline keypair generation for tests
* Push sbf lockfile change
* sdk: Sort dependencies correctly
* Remove `hash::new_with_thread_rng`, use `Hash::new_unique()`
* Use Keypair::new where chacha isn't used
* sdk: Fix build by marking rand 0.7 optional
* Hardcode secret key length, add static assertion
* Unify `getrandom` crate usage to fix linking errors
* bloom: Fix tests that require a random hash
* Remove some dependencies, try to unify others
* Remove unnecessary uses of rand and rand_core
* Update lockfiles
* Add back some dependencies to reduce rebuilds
* Increase max rebuilds from 14 to 15
* frozen-abi: Remove `getrandom`
* Bump rebuilds to 17
* Remove getrandom from zk-token-proof
`Arc` is already a reference internally, so it does not seem to be
beneficial to pass a reference to it. Just adds an extra layer of
indirection.
Functions that need to be able to increment `Arc` reference count need
to take `Arc<AtomicBool>`, but those that just want to read the
`AtomicBool` value can accept `&AtomicBool`, making them a bit more
generic.
This change focuses specifically on `Arc<AtomicBool>`. There are other
uses of `&Arc<T>` in the code base that could be converted in a similar
manner. But it would make the change even larger.
Working towards LegacyContactInfo => ContactInfo migration, the commit
hides some implementation details of LegacyContactInfo and expands API
parity with the new ContactInfo.
Alessandro Decina
Lijun Wang
Yihau Chen
Andrew Fitzgerald
Brooks
steviez
ryleung-solana
dependabot[bot]
Pankaj Garg
Jeff Biseda
Alexander Meißner
behzad nouri
Trent Nelson
Jon Cinque
steviez
Illia Bobyr