#### Problem
TieredWritableFile currently uses File instead of BufWriter.
This will introduce more syscall when doing file writes.
#### Summary of Changes
This PR makes TieredWritableFile uses BufWriter to allow the
write-call to be more optimized to reduce the number of syscalls.
#### Test Plan
Existing tiered-storage test.
Will run experiments to verify its performance improvement.
#### Dependency
https://github.com/anza-xyz/agave/pull/260
#### Problem
TieredStorageFile struct currently offers new_readonly() and new_writable()
to allow both read and write work-load to share the same struct. However,
as we need the writer to use BufWriter to improve performance as well as
enable Hasher on writes. There is a need to refactor TieredStorageFile to
split its usage for read-only and writable.
#### Summary of Changes
Refactor TieredStorageFile to TieredReadonlyFIle and TieredWritableFile.
#### Test Plan
Existing tiered-storage tests.
#### Problem
tiered_storage/writer.rs was added when we planned to support multiple
tiers in the tiered-storage (i.e., at least hot and cold). However, as we
changed our plan to handle cold accounts as state-compressed accounts,
we don't need a general purposed tiered-storage writer at this moment.
#### Summary of Changes
Remove tiered_storage/writer.rs as we currently don't have plans to develop cold storage.
#### Test Plan
Existing tiered-storage tests.
#### Problem
As we further optimize the HotStorageMeta in #146, there is a need
for a HotAccount struct that contains all the hot account information.
Meanwhile, we currently don't have plans to develop a cold account
format at this moment. As a result, this makes it desirable to repurpose
TieredReadableAccount to HotAccount.
#### Summary of Changes
Repurpose TieredReadableAccount to HotAccount.
#### Test Plan
Existing tiered-storage tests.
#### Problem
TieredStorage stores account hash as an optional field inside its HotStorage.
However, the field isn't used and we have already decided to deprecate
the account hash.
#### Summary of Changes
Remove account-hash from the tiered-storage.
#### Test Plan
Existing tiered-storage tests.
Running validators w/ tiered-storage in mainnet-beta w/o storing account-hash.
#### Problem
In TieredAccountMeta, RENT_EXEMPT_RENT_EPOCH will be used when
its optional field rent_epoch is None. However, for legacy reasons, 0
should be used for zero-lamport accounts.
#### Summary of Changes
Return 0 for TieredAccountMeta::rent_epoch() for zero-lamport accounts.
#### Test Plan
accounts_db::tests::test_clean_zero_lamport_and_dead_slot
#### Problem
While accounts-db might not invoke appends_account twice
for the same AccountsFile, TieredStorage::write_accounts()
itself isn't thread-safe, and it depends on the above accounts-db
assumption.
#### Summary of Changes
This PR makes TieredStorage::write_accounts() thread-safe.
So only the first thread that successfully updates the already_written
flag can proceed and write the input accounts. All subsequent
calls to write_accounts() will be a no-op and return AttemptToUpdateReadOnly
Error.
#### Problem
There're some test functions that have been used in different
mod in TieredStorage. It's better to have one same place for
all tiere-storage related test functions.
#### Summary of Changes
Created test_utils.rs under /tiered_storage and move test-related
functions into it.
#### Test Plan
Existing tests.
Yueh-Hsuan Chiang
Jeff Washington (jwash)
HaoranYi
Yihau Chen
Brooks
Justin Starry
Brooks
Brooks
Dmitri Makarov