* accounts-db: unpack_archive: avoid extra iteration on each path
We used to do a iterator.clone().any(...) followed by
iterator.collect(). Merge the two and avoid an extra iteration and
re-parsing of the path.
* accounts-db: unpack_archive: unpack accounts straight into their final destination
We used to unpack accounts into account_path/accounts/<account> then
rename to account_path/<account>. We now unpack them into their final
destination directly and avoid the rename syscall.
#### Problem
TieredWritableFile currently uses File instead of BufWriter.
This will introduce more syscall when doing file writes.
#### Summary of Changes
This PR makes TieredWritableFile uses BufWriter to allow the
write-call to be more optimized to reduce the number of syscalls.
#### Test Plan
Existing tiered-storage test.
Will run experiments to verify its performance improvement.
#### Dependency
https://github.com/anza-xyz/agave/pull/260
#### Problem
TieredStorageFile struct currently offers new_readonly() and new_writable()
to allow both read and write work-load to share the same struct. However,
as we need the writer to use BufWriter to improve performance as well as
enable Hasher on writes. There is a need to refactor TieredStorageFile to
split its usage for read-only and writable.
#### Summary of Changes
Refactor TieredStorageFile to TieredReadonlyFIle and TieredWritableFile.
#### Test Plan
Existing tiered-storage tests.
* Switch to blockstore.is_full() check because replay thread isn't active.
* Use make_chaining_slot_entries and add first_parent to the method.
Small style fixes.
* Switch to blockstore.is_full() check because replay thread isn't active.
* add `ParseError` in `zk-token-elgamal`
* implement `FromStr` for `ElGamalPubkey` and `ElGamalCiphertext`
* implement `FromStr` for `AeCiphertext`
* fix target
* cargo fmt
* use constants for byte length check
* make `FromStr` functions available on chain
* use macros for the `FromStr` implementations
* restrict `from_str` macro to `pub(crate)`
* decode directly into array
* cargo fmt
* Apply suggestions from code review
Co-authored-by: Jon C <me@jonc.dev>
* remove unnecessary imports
* remove the need for `ParseError` dependency
---------
Co-authored-by: Jon C <me@jonc.dev>
#### Problem
tiered_storage/writer.rs was added when we planned to support multiple
tiers in the tiered-storage (i.e., at least hot and cold). However, as we
changed our plan to handle cold accounts as state-compressed accounts,
we don't need a general purposed tiered-storage writer at this moment.
#### Summary of Changes
Remove tiered_storage/writer.rs as we currently don't have plans to develop cold storage.
#### Test Plan
Existing tiered-storage tests.
#### Problem
As we further optimize the HotStorageMeta in #146, there is a need
for a HotAccount struct that contains all the hot account information.
Meanwhile, we currently don't have plans to develop a cold account
format at this moment. As a result, this makes it desirable to repurpose
TieredReadableAccount to HotAccount.
#### Summary of Changes
Repurpose TieredReadableAccount to HotAccount.
#### Test Plan
Existing tiered-storage tests.
The default value was previously being determined down where the thread
pool is being created. Providing a default value at the CLI level is
consistent with other args, and gives an operator better visibility into
what the default will actually be
Pankaj Garg
Jon C
Tao Zhu
Alessandro Decina
Ashwin Sekar
Dmitri Makarov
Lucas Steuernagel
Brennan
Greg Cusack
Justin Starry
Yueh-Hsuan Chiang
Lijun Wang
ryleung-solana
WGB5445
Jeff Washington (jwash)
听寒
ripatel-fd
Andrew Fitzgerald
Tyera
HaoranYi
Trent Nelson
Wen
Yihau Chen
samkim-crypto
sakridge
Nagaprasad V R
dependabot[bot]
Kirill Fomichev
anwayde
steviez