64f64dedea
Use regex in get_snapshot_file_kind ( #27728 )
...
refactor get_snapshot_file_kind to use regex
2022-09-22 13:55:48 -05:00
a846d50b6a
Interleaved snapshot unpack versioning ( #27484 )
...
* Issue #27346 - deserialize and check snapshot version before account fields
* Update comment on SnapshotFileKind
Co-authored-by: Brooks Prumo <brooks@prumo.org>
* SnapshotStorageRebuilderResult to RebuiltSnapshotStorage
* better error propagation from rebuild_storage
Co-authored-by: apfitzge <apfitzge@users.noreply.github.com>
Co-authored-by: Brooks Prumo <brooks@prumo.org>
2022-09-22 11:44:25 -05:00
49411aaae0
Feature: Interleaved Snapshot Untar and Indexing - Stage 1 ( #26590 )
...
* Add snapshot_storage_lengths_from_fields
* Add SnapshotUnpacker
* Add SnapshotStorageRebuilder
* Rebuild snapshot storage during unpack
* move snapshot unpacker back into snapshot_utils
* use SerializedAppendVecId in storage rebuilder
* create AccountStorageMap type alias
* Wrap storage and next_append_vec_id in a type
* fixed typo
* move use StorageAndNextAppendVecId
* calculate num_rebuilder_threads more concisely
2022-08-29 13:17:27 -05:00
3e44046a73
Support lz4 for snapshot archives ( #25089 )
...
* add lz4
* add cargo package file
* fix tests
* use fast lz4 compression
* report snapshot archive format
* add test
* code review feedback
* add cargo.lock
* fix var name
* refactor archive format parsing and add default compress cli arg
* clippy
* add from_cli_arg test
* update cargo.lock
* add lz4 support for mmap
* cargo.lock
* clippy
2022-05-16 12:44:15 -05:00
1eddb6d1e9
Move ArchiveFormat into own module ( #23562 )
2022-03-09 16:09:34 -06:00
apfitzge
apfitzge
HaoranYi
Brooks Prumo