* Allow banking_stage to update prioritization_fee_cache
* Update core/src/banking_stage/committer.rs
Co-authored-by: Andrew Fitzgerald <apfitzge@gmail.com>
* move use to top
---------
Co-authored-by: Andrew Fitzgerald <apfitzge@gmail.com>
* only executed transactions are used to update prioritization_fee_cache
* Update ledger/src/blockstore_processor.rs
Co-authored-by: Andrew Fitzgerald <apfitzge@gmail.com>
In cluster restart scenarios, it is desirable to know the latest
optimistic slot(s), which the subcommand latest-optimistic-slots will
return. However, it is also useful to know whether slots contain only
votes or if they contain votes and user transactions.
This PR adds an extra column of output to show whether an optimistically
confirmed slot is vote only (contains zero non-vote transactions).
Additionally, a flag has been added to enable filtering output to
exclude vote only slots.
* Add --{replaying,banking}-backend flags (noop atm)
* Greatly simplify enums with strum
* Update programs/sbf/Cargo.lock...
* Rely on Display, removing Debug
* constify cli_names()
* Don't allow omitting bankend value
* Rename to --block-{verification,production}-method
* Use more specific name
* Actually support missing value....
* Remove strictly-unnecessary flags
* Use lazy_static! instead of abusing DefaultArgs...
* Construct a bank from a snapshot directory
* Clean up. Remove archiving releated data structures
* Fix auto checks
* remove ArchiveFormat::None
* fix merge error
* remove incremental snapshot dir
* Minor cleanup, remove unused functiond defs
* remove pub fn bank_from_latest_snapshot_dir
* rename bank_from_snapshot_dir to bank_from_snapshot
* Clean up invalid comments
* A few minor review changes
* Removed insert_slot_storage_file
* Add comment explain hardlink symlink
* Skip the whole verify_snapshot_bank call for the from_dir case
* Add bank.set_initial_accounts_hash_verification_completed()
* address review issues: appendvec to append_vec, replace unwrap with expect, etc
* AtomicAppendVecId, remove arc on bank etc
* slice, CI error on &snapshot_version_path
* measure_build_storage
* move snapshot_from
* remove measure_name from build_storage_from_snapshot_dir
* remove from_dir specific next_append_vec_id logic
* revert insert_slot_storage_file change
* init next_append_vec_id to fix the substraction underflow
* remove measure from build_storage_from_snapshot_dir
* make measure name more specific
* refactor status_cache deserialization into a function
* remove reference to pass the ci check
* track next appendvec id
* verify that the next_append_vec_id tracking is correct
* clean up usize
* in build_storage_from_snapshot_dir remove expect
* test max appendvecc id tracking with multiple banks in the test
* cleared expect and unwrap in streaming_snapshot_dir_files
* rebase cleanup
* change to measure!
* dereference arc in the right way
`poh_verify` actually disables transaction signature, tick count and
built in program argument verifications as well. It is somewhat
confusing to call it `poh_verify`.
Shreds arriving at tvu/tvu_forward/repair sockets are each processed in
a separate thread, and since each thread has its own deduper, the
duplicates across these sockets are not filtered out.
Using a common deduper across these threads will require an RwLock
wrapper and may introduce lock contention.
The commit instead moves the shred-deduper to shred-sigverify-stage
where all these shreds arrive through the same channel.
* include loaded accounts data size limit in transaction base fee calculation
* citing compute_budget for heap cost;
* update sbf tests
Co-authored-by: Trent Nelson <trent.a.b.nelson@gmail.com>
Fix SlotMeta is_connected tracking
The tracking of connected status was previously based upon an assumption
that would be practically false for all validators. The connected status
of slots played into whether Blockstore would signal ReplayStage that it
had new shreds ready to be replayed. Prior to the change, we would never
signal and ReplayStage would always wait the entire duration of a 100ms
timeout before restarting its' main processing loop.
This commit introduces a change where we mark snapshot slots as
connected. A validator may not have a path all the way back to genesis
itself; however, snapshots are taken at known roots so we extend the
connected status to these slots. Once a node has been bootstrapped once
to have is connected, the logic persists in Blockstore such that all
children on the main fork also get their connected status updated
properly.
There is some logic related to how timing values are collected that is not immediately obvious. It is better to document it, rather than requiring everyone interested to reverse engineer it from the code.
If turbine_disabled is true, the commit discards turbine packets
earlier in the pipeline so that they won't interfere with the deduper
and the packets can get through once turbine is enabled again.
This is a prerequisite of:
https://github.com/solana-labs/solana/pull/30786
so that local-cluster tests pass.
For duplicate block detection, for each (slot, shred-index, shred-type)
we need to allow 2 different shreds to be retransmitted.
The commit implements this using two bloom-filter dedupers:
* Shreds are deduplicated using the 1st deduper.
* If a shred is not a duplicate, then we check if:
(slot, shred-index, shred-type, k)
is not a duplicate for either k = 0 or k = 1 using the 2nd deduper,
and if so then the shred is retransmitted.
This allows to achieve larger capactiy compared to current LRU-cache.
#### Problem
As we start supporting new storage formats, alignment-related constants and macros
defined in append_vec.rs aren't only specific to AppendVec.
#### Summary of Changes
Move alignment-related constants/macros from append_vec.rs to accounts_file.rs
generalizes Deduper to work with any hashable type
Current Deduper is hard-coded only for Packet type. In order to use
Deduper in retransmit-stage, we need to dedup types other than Packet.
The commit generalizes Deduper to any hashable type.
samkim-crypto
Tao Zhu
behzad nouri
steviez
Andrew Fitzgerald
dependabot[bot]
Alexander Meißner
Tyera
Ryo Onodera
Jeff Biseda
Jeff Washington (jwash)
Xiang Zhu
Illia Bobyr
Dmitri Makarov
Brooks
HaoranYi
Nick Frostbutter
Yueh-Hsuan Chiang