Summary of Changes:
This change adds functionality to randomize tx execution for every entry. It does this by implementing OrderedIterator that iterates tx slice as per the order specified. The order is generated randomly for every entry.
* Insert data shreds in blocktree and database
* Integrate data shreds with rest of the code base
* address review comments, and some clippy fixes
* Fixes to some tests
* more test fixes
* ignore some local cluster tests
* ignore replicator local cluster tests
* Page-pin packet memory for cuda
Bring back recyclers and pin offset buffers
* Add packet recycler to streamer
* Add set_pinnable to sigverify vecs to pin them
* Add packets reset test
* Add test for recycler and reduce the gc lock critical section
* Add comments/tests to cuda_runtime
* Add recycler to recv_blobs path.
* Add trace/names for debug and PacketsRecycler to bench-streamer
* Predict realloc and unpin beforehand.
* Add helper to reserve and pin
* Cap buffered packets length
* Call cuda wrapper functions
Use more chunks to avoid duplicate signature failures:
Duplicate signatures can occur because bank.clear_signatures()
can occur before the bank has actually committed the signatures
to the status cache and then error out on the next set of transactions.
* Use Rc to prevent clone of Packets
* Fix min => max in banking_stage threads.
Coalesce packet buffers better since a larger batch will
be faster through banking and sigverify.
Deconstruct batches into banking_stage from sigverify since
sigverify likes to accumulate batches but then a single banking_stage
thread will be stuck with a large batch. Maximize parallelism by
creating more chunks of work for banking_stage.
* Remove unused PohServiceConfig::Step
* Clarify variable name
* Poh::hash() now takes an iteration counter
* man -> max
* Inline functions with single call site
* Move PohServiceConfig into GenesisBlock
* Add plumbing to enable real PoH on testnets
* Batch hashes to improve PoH hash rate
* Ensure a constant hashes_per_tick
* Remove PohEntry mixin field
* Poh/PohEntry no longer maintains tick_height
* Ensure a constant hashes_per_tick
* ci/localnet-sanity.sh: Use real PoH
* Rework Poh/PohService to keep PohRecorder unlocked as much as possible while hashing
* Move append_vec bench to the crate with append_vec
* Use black_box to tell the compiler not to optimize away test data
```
pub fn black_box<T>(dummy: T) -> T {
unsafe {
let ret = std::ptr::read_volatile(&dummy);
std::mem::forget(dummy);
ret
}
}
```
* Revert "Use black_box to tell the compiler not to optimize away test data"
This reverts commit 5610b8ee957f1d8bf6e270e392859e8b23b1e472.
* Use black_box to tell the compiler not to optimize away test data
* Create bench directories
* move core tests to core
* remove window
* fix up flaky tests
* test_entryfication needs a singly-threaded banking_stage
* move core benches to core
* remove unnecessary dependencies
* remove core as a member for now, test it like runtime
* stop running tests twice
* remove duplicate runs of tests in perf
* Mostly implement key-value store and add integration points
Essential key-value store functionality is implemented, needs more work to be integrated, tested, and activated.
Behind the `kvstore` feature.
sakridge
sakridge
Pankaj Garg
Sagar Dhawan
Rob Walker
Parth
Michael Vines
carllin
TristanDebrunner
Greg Fitzgerald
Mark