[TieredStorage] Allow HotStorage to handle more account data (#34155)

#### Problem
After we have defined AccountOffset to be u32, it means the address space within
one HotAccountsFile is up to 4GB.  However, since the Accounts Blocks in a
HotAccountsFile is 8-byte aligned, it has the opportunity to store more data by
internally multiplying the AccountOffset by 8.

#### Summary of Changes
This PR allows a HotAccountsFile to store up to 32GB accounts data by using
8 x AccountOffset as its actual offset.

#### Test Plan
Updated existing unit-tests.
This commit is contained in:
Yueh-Hsuan Chiang 2023-12-01 10:39:15 -08:00 committed by GitHub
parent b97b3dd4ab
commit fa0930f254
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 11 additions and 2 deletions

View File

@ -36,6 +36,10 @@ const MAX_HOT_PADDING: u8 = 7;
/// The maximum allowed value for the owner index of a hot account.
const MAX_HOT_OWNER_OFFSET: OwnerOffset = OwnerOffset((1 << 29) - 1);
/// The multiplier for converting AccountOffset to the internal hot account
/// offset. This increases the maximum size of a hot accounts file.
const HOT_ACCOUNT_OFFSET_MULTIPLIER: usize = 8;
#[bitfield(bits = 32)]
#[repr(C)]
#[derive(Debug, Default, Copy, Clone, Eq, PartialEq)]
@ -228,7 +232,9 @@ impl HotStorageReader {
&self,
account_offset: AccountOffset,
) -> TieredStorageResult<&HotAccountMeta> {
let (meta, _) = get_type::<HotAccountMeta>(&self.mmap, account_offset.block as usize)?;
let internal_account_offset = account_offset.block as usize * HOT_ACCOUNT_OFFSET_MULTIPLIER;
let (meta, _) = get_type::<HotAccountMeta>(&self.mmap, internal_account_offset)?;
Ok(meta)
}
@ -468,7 +474,10 @@ pub mod tests {
.map(|meta| {
let prev_offset = current_offset;
current_offset += file.write_type(meta).unwrap() as u32;
AccountOffset { block: prev_offset }
assert_eq!(prev_offset % HOT_ACCOUNT_OFFSET_MULTIPLIER as u32, 0);
AccountOffset {
block: prev_offset / HOT_ACCOUNT_OFFSET_MULTIPLIER as u32,
}
})
.collect();
// while the test only focuses on account metas, writing a footer