* Restrict the usable port range of the validator such that adding QUIC_PORT_OFFSET never gets us an invalid port. Also validate this for incoming ContactInfos
* Require the proper port range in ContactInfo::valid_client_facing_addr
* Use asserts instead of panics, and enforce USABLE_PORT_RANGE for all the ports in ContactInfo
* Fix typo
* Make the quic client return errors on the quinn endpoint connect() call,
not just the result of awaiting the connect() call, as the connect()
call can itself fail realistically (e.g. due to expected/invalid IPs, etc)
* Update USABLE_PORT_RANGE to a better range and use port_range_validator to validate dynamic-port-range rather than a panic
* Fall back on UDP when the remote peer's tpu port is too large to have QUIC_PORT_OFFSET added to it
* Get rid of tpu port sanitization in ContactInfo
* Turn USABLE_PORT_RANGE into a Range and make connnection_cache fall back on UDP when the tpu port is out of range
* Fix build
* Dummy commit
* Reert dummy commit
* dummy commit
* revert dummy commit
* Fix typo
* Fix range validation
* Fix formatting
* Fix USABLE_PORT_RANGE
* Remove USABLE_PORT_RANGE
* Avoid creating a QuicLazyInitializedEndpoint when forcing the use of UDP
* Implement test for connection cache overflow
* Add ability to use a non-default app profile id in bigtable requests
* Only run subcommand once when getting global configs
* Remove unneded scoping on option type
* Connection pool in connection cache and handle connection errors
1. The connection not has a pool of connections per address, configurable, default 4
2. The connections per address share a lazy initialized endpoint
3. Handle connection issues better, avoid race conditions
4. Various log improvement for help debug connection issues
* client: Remove static connection cache, plumb it instead
* Add TpuClient::new_with_connection_cache to not break downstream
* Refactor get_connection and RwLock into ConnectionCache
* Fix merge conflicts from new async TpuClient
* Remove `ConnectionCache::set_use_quic`
* Move DEFAULT_TPU_USE_QUIC to client, use ConnectionCache::default()
#### Problem
Currently, the creation of ShredStorageType::RocksFifo is hard coded in validator/src/main.rs.
But this common code will also need to be used in other places like ledger-tool.
#### Summary of Changes
This PR creates a helper functionShredStorageType::rocks_fifo that takes a total shred_storage_size
and equally allocates to data-shred and coding-shred storage.
Add in some CPU utilization metrics such as: number of vCPUs, clock frequency, average load across different time intervals, and number of total threads
#### Problem
When FIFO compaction is used, the size ratio between data shred and coding
shred is set to 1:1 based on the `--rocksdb_fifo_shred_storage_size` arg.
However, BlockstoreRocksFifoOptions::default() uses a slightly optimized
5:4 ratio instead, and the default() function is only used in benchmarks.
#### Summary of Changes
This PR makes both validator argument and BlockstoreRocksFifoOptions::default()
to use 1:1 ratio between data and coding shred size.
* Spawn QUIC server to receive forwarded txs
* Update validator port range
* forward votes using UDP
* no forwarding from unstaked nodes
* forwarding stats in banking stage
* fix test builds
* fix lifetime of forward sender
#### Problem
blockstore_db.rs has a mutual dependency between blockstore_metrics.rs.
#### Summary of Changes
This PR removes the mutual dependency by moving the option-related stuff
out from blockstore_db.rs to its new home --- blockstore_options.rs.
By doing this, we address the mutual dependency and also make the code cleaner.
dependabot[bot]
Michael Vines
Ahmad
Will Hickey
github-actions[bot]
Brennan Watt
Trent Nelson
sakridge
skrrb
ryleung-solana
Nicholas Clarke
Jeff Washington (jwash)
Brooks Prumo
DimAn
Ivan Mironov
Dan Coombs
Lijun Wang
Jon Cinque
Yueh-Hsuan Chiang
behzad nouri
Pankaj Garg
Justin Starry
HaoranYi