The IP echo server currently spins up a worker thread for every thread
on the machine. Observing some data for nodes,
- MNB validators and RPC nodes look to get several hundred of these
requests per day
- MNB entrypoint nodes look to get 2-3 requests per second on average
In both instances, the current threadpool is severely overprovisioned
which is a waste of resources. This PR plumnbs a flag to control the
number of worker threads for this pool as well as setting a default of
two threads for this server. Two threads allow for one thread to always
listen on the TCP port while the other thread processes requests
* dsiable read cache while populating stakes cache on load
* use struct with drop as api
* use LoadHint
* remove disable_read_cache_updates_count
* add comment
* fmt
* add metric for duplicate push messages
* add in num_total_push
* address comments. don't lock stats each time
* address comments. remove num_total_push
* change dup push message name in code to reflect metric name
When a leader is packing a Bank, transactions costs are added to the
CostTracker and then later updated or removed, depending on if the
tx is committed. However, it is possible for a Bank to be frozen while
there are several tx's in flight.
CostUpdateService submits a metric with cost information almost
immediately after a Bank has been frozen. The result is that we have
observed cost details being submitted before some cost removals take
place, which causes a massive over-reporting of the block cost
compared to actual.
This PR adds a field to track the number of transactions that are
inflight, and adds a simple mechanism to try to allow that value to
settle to zero before submitting the datapoint. The number of inflight
tx's is submitted with the datapoint, so even if the value does not
settle to zero, we can still detect this case and know the metric is
tainted.
Co-authored-by: Andrew Fitzgerald <apfitzge@gmail.com>
* clap-utils: Refactor compute_unit_price into compute_budget
* clap-utils: Validate compute unit price as a u64
* clap-utils: Add compute unit limit arg
* clap-v3-utils: Add compute unit price and limit helpers
* Add deprecation on `pub use` even though it isn't triggered
Previously, entry verification had a dedicated threadpool used to verify
PoH hashes as well as some basic transaction verification via
Bank::verify_transaction(). It should also be noted that the entry
verification code provides logic to offload to a GPU if one is present.
Regardless of whether a GPU is present or not, some of the verification
must be done on a CPU. Moreso, the CPU verification of entries and
transaction execution are serial operations; entry verification finishes
first before moving onto transaction execution.
So, tx execution and entry verification are not competing for CPU cycles
at the same time and can use the same pool.
One exception to the above statement is that if someone is using the
feature to replay forks in parallel, then hypothetically, different
forks may end up competing for the same resources at the same time.
However, that is already true given that we had pools that were shared
between replay of multiple forks. So, this change doesn't really change
much for that case, but will reduce overhead in the single fork case
which is the vast majority of the time.
#### Problem
TieredStorage::drop() currently panic when it fails to delete the
underlying file to raise awareness of possible storage resource
leakage, including io::ErrorKind::NotFound. But sometimes the
TieredStorage (or AccountsFile in general) instance is created
then dropped without any file being created. This causes some
false-alarms including unit-tests.
#### Summary of Changes
This PR excludes NotFound in reporting storage leakage on
TieredStorage::drop().
Jeff Washington (jwash)
steviez
Andrew Fitzgerald
Brooks
dependabot[bot]
sakridge
Yihau Chen
Joe C
samkim-crypto
Greg Cusack
Tyera
Ryo Onodera
Alessandro Decina
Will Hickey
Jon C
Lucas Steuernagel
Kirill Fomichev
Ashwin Sekar
Yueh-Hsuan Chiang
Kevin Heavey