carllin
90ef2cd02a
Parse snapshot for bank fields ( #26016 )
2022-07-06 17:30:30 -05:00
Jeff Washington (jwash)
17a99d98dd
Revert "avoid adding to 'uncleaned_roots' when generating index and c… ( #26441 )
...
Revert "avoid adding to 'uncleaned_roots' when generating index and caller passes accounts-db-skip-shrink (#25936 )"
This reverts commit e24cc537a4
.
2022-07-06 11:32:45 -05:00
Jeff Washington (jwash)
38216aa781
update comment for the next weary traveller ( #26413 )
2022-07-05 23:12:35 -05:00
Brooks Prumo
877fedadac
Remove StatusCacheRc type and use StatusCache directly ( #26184 )
2022-06-24 08:38:56 -05:00
Brooks Prumo
23c50a2389
Add StatusCache::root_slot_deltas() and use it ( #26170 )
2022-06-23 15:19:06 -05:00
Brooks Prumo
8a18e02f80
Call StatusCache::append() directly ( #26125 )
2022-06-22 07:42:01 -05:00
Tyera Eulberg
752c851070
Clean up feature preventing new rent-paying accounts ( #26000 )
...
* Clean up require_rent_exempt_accounts feature
* Fix tests in bank_client, system_instruction_processor
* Fix tests in snapshot_utils
* Fix bank tests
* Ignore rent tests
* Fix stake-accounts tests
2022-06-16 15:35:25 -06:00
Jeff Washington (jwash)
e24cc537a4
avoid adding to 'uncleaned_roots' when generating index and caller passes accounts-db-skip-shrink ( #25936 )
2022-06-14 10:10:44 -05:00
apfitzge
3e1fd4dad0
snapshot_utils.rs: remove unnecessary sort ( #25893 )
2022-06-13 11:12:33 -05:00
apfitzge
04969a2282
serialize snapshots dir first ( #25889 )
2022-06-13 09:33:32 -05:00
apfitzge
56d38e4954
Add lz4 to snapshot filename regex for matching files on load ( #25633 )
...
* Add lz4 to snapshot filename regex for matching files on load
* Add tests for lz4 archive format filenames
2022-05-31 11:06:41 -05:00
HaoranYi
85a2e599bb
revert memory map for reading snapshot ( #25259 )
2022-05-17 09:32:18 -05:00
HaoranYi
3e44046a73
Support lz4 for snapshot archives ( #25089 )
...
* add lz4
* add cargo package file
* fix tests
* use fast lz4 compression
* report snapshot archive format
* add test
* code review feedback
* add cargo.lock
* fix var name
* refactor archive format parsing and add default compress cli arg
* clippy
* add from_cli_arg test
* update cargo.lock
* add lz4 support for mmap
* cargo.lock
* clippy
2022-05-16 12:44:15 -05:00
HaoranYi
389d78e424
Don't use mmarinus package ( #25177 )
...
* Revert "Revert "Use memory map to speed up snapshot untar (#24889 )" (#25174 )"
This reverts commit fc793de2dd
.
* not use mmarinus
* enable secondary build
* Revert "enable secondary build"
This reverts commit 5aa43a94bbad59497506f02235597e7adc0dc47b.
* macbuild
* Revert "macbuild"
This reverts commit 0da92945a40dc5431eff4bd91054b407e22418fb.
2022-05-16 08:49:17 -05:00
Jeff Washington (jwash)
fc793de2dd
Revert "Use memory map to speed up snapshot untar ( #24889 )" ( #25174 )
...
This reverts commit 3367e44671
.
2022-05-12 14:48:29 -05:00
HaoranYi
3367e44671
Use memory map to speed up snapshot untar ( #24889 )
...
* mmap
* add no_os_memory_reporting_args to ledger-tool
* use safe memory map package for unix. use regular file for windows
* fallback to untaring with filebuf when mmap fails
* update comments
2022-05-12 13:35:19 -05:00
DimAn
2fa9bc3e70
Add options to store full and/or incremental snapshots in separate locations ( #24247 )
2022-05-10 16:37:41 -04:00
HaoranYi
fd44198c0a
fix typo ( #24937 )
2022-05-03 09:57:15 -05:00
apfitzge
112a0b475a
Revert "Refactor to use EpochSchedule from within RentCollector struct" ( #24893 )
...
* Revert "Ran cargo fmt"
This reverts commit 9052e41b32
.
* Revert "Fix build error introduced by my editor setup, part 2"
This reverts commit 4dfeab3b38
.
* Revert "Fix build error introduced by my editor setup"
This reverts commit 87fb78dc56
.
* Revert "Remove redundant epoch_schedule from AccountsPackage"
This reverts commit c2f7f2fff8
.
* Revert "Fix a test"
This reverts commit 36c0bdaa78
.
* Revert "Fixes to initial code"
This reverts commit ed7813e698
.
* Revert "Removing redundant EpochSchedule param from fns"
This reverts commit 5472d2e605
.
2022-05-02 13:46:17 -05:00
Michael Vines
74b586ae71
Always retain the highest incremental snapshot for all full snapshots
2022-05-02 07:37:55 -07:00
Jeff Washington (jwash)
94d3ec37d2
collect_rent_eagerly can just do rewrites ( #24438 )
2022-04-18 22:44:19 -05:00
Jason Davis
c2f7f2fff8
Remove redundant epoch_schedule from AccountsPackage
2022-04-18 11:57:40 -05:00
Jeff Washington (jwash)
210f6a6fab
move hash calculation out of acct bg svc ( #23689 )
...
* move hash calculation out of acct bg svc
* pr feedback
2022-04-08 10:42:03 -05:00
Jeff Washington (jwash)
c27150b1a3
reserialize_bank_fields_with_hash ( #23916 )
...
* reserialize_bank_with_new_accounts_hash
* Update runtime/src/serde_snapshot.rs
Co-authored-by: Brooks Prumo <brooks@prumo.org>
* Update runtime/src/serde_snapshot/tests.rs
Co-authored-by: Brooks Prumo <brooks@prumo.org>
* Update runtime/src/serde_snapshot/tests.rs
Co-authored-by: Brooks Prumo <brooks@prumo.org>
* pr feedback
Co-authored-by: Brooks Prumo <brooks@prumo.org>
2022-04-07 14:05:57 -05:00
Jeff Washington (jwash)
550ca7bf92
compare contents of serialized banks instead of exact file format ( #24141 )
...
* compare contents of serialized banks instead of exact file format
* Update runtime/src/snapshot_utils.rs
Co-authored-by: Brooks Prumo <brooks@prumo.org>
* Update runtime/src/snapshot_utils.rs
Co-authored-by: Brooks Prumo <brooks@prumo.org>
* pr feedback
* get rid of clone
* pr feedback
Co-authored-by: Brooks Prumo <brooks@prumo.org>
2022-04-06 21:55:44 -05:00
Jeff Washington (jwash)
fddd162645
reserialize bank in ahv by first writing to temp file in abs ( #23947 )
2022-04-06 21:39:26 -05:00
Brooks Prumo
c322842257
Replace channel with Mutex<Option> for AccountsPackage ( #24013 )
2022-04-06 05:47:19 -05:00
Jeff Washington (jwash)
9c8dad33c7
add epoch_schedule and rent_collector to hash calc ( #24012 )
2022-03-31 10:51:18 -05:00
Brooks Prumo
31b707b625
Specify if archive size datapoint is for full or incremental snapshots ( #23941 )
2022-03-26 12:29:13 -05:00
Jeff Washington (jwash)
bc35e1c5f5
snapshot code needs all storages for hash calc ( #23840 )
2022-03-22 21:27:54 -05:00
HaoranYi
8c4f010b8d
fix logging: only walk remote dir if it exist ( #23663 )
2022-03-15 08:56:22 -05:00
HaoranYi
0c684721d8
Separate remotely downloaded snapshot archives ( #23510 )
...
* seperate remotely downloaded snapshot archives
* add str const for snapshot download dir
* only walk remote sub directory
* move directory creation outside of loop
* move is_remote to traits
* clippy simplify
* clippy
* clippy
* add unittest
* fix local cluster tests
* look for remote snapshot archive in remote foler
* create remote dir in tests
* use snapshot download dir constant
* extract build_remote_dir fn
* fix build
* code review - walking snapshot archived dirs explicitly
* fix build
* fix build
* fix comments
* Update runtime/src/snapshot_utils.rs
Co-authored-by: Brooks Prumo <brooks@prumo.org>
* clippy
* borrow to avoid copy
Co-authored-by: Brooks Prumo <brooks@prumo.org>
2022-03-14 14:03:59 -05:00
Giorgio Gambino
ccff006948
Add more logs to `purge_old_snapshot_archives` ( #23531 )
2022-03-11 15:01:48 -06:00
Brooks Prumo
1eddb6d1e9
Move ArchiveFormat into own module ( #23562 )
2022-03-09 16:09:34 -06:00
Brooks Prumo
9bbccbe27c
Use `AsRef<Path>` instead of `PathBuf` for parameters ( #23560 )
2022-03-09 16:08:33 -06:00
Michael Vines
5599bd9442
Remove BankFromArchiveTimings from ledger/
2022-03-08 08:11:50 -08:00
HaoranYi
181fffb916
rename status filename to be consistent ( #23501 )
2022-03-07 17:34:35 +00:00
HaoranYi
463cd564cf
fix typos ( #23495 )
...
* fix typos
2022-03-05 20:46:46 -06:00
Michael Vines
b28acd2d4d
Reduce fn visibility
2022-03-04 09:52:46 +01:00
Michael Vines
72c68695b5
Do not keep oldest full snapshot
2022-02-21 17:10:31 -07:00
Michael Vines
577fa4ec0c
Generate full snapshots 4x faster to keep incremental snapshots nice and small
2022-02-15 21:09:21 -08:00
Brooks Prumo
b82d71d22a
Inline DEFAULT_SNAPSHOT_VERSION constant string ( #22487 )
2022-01-13 18:19:15 +00:00
Brooks Prumo
2756abce39
More serde snapshot cleanup ( #22449 )
2022-01-13 09:20:20 -06:00
mooori
0f6e8d3385
Check file size of `snapshot_version` when unarchiving snapshot ( #21925 )
2021-12-17 12:27:54 -06:00
Michael Vines
b8837c04ec
Reformat imports to a consistent style for imports
...
rustfmt.toml configuration:
imports_granularity = "One"
group_imports = "One"
2021-12-03 09:19:13 -08:00
Michael Vines
ba9dfa0d22
Remove frozen account support
2021-11-29 08:38:11 -08:00
Jack May
bced07a099
Update fee api to use blockhash ( #21054 )
2021-10-29 13:52:59 -07:00
Justin Starry
036d7fcc81
Clean up sanitized tx creation for tests ( #21006 )
2021-10-27 18:09:16 +01:00
Jack May
bfbbc53dac
Divorce the runtime from FeeCalculator ( #20737 )
2021-10-22 14:32:40 -07:00
Brooks Prumo
751b45df64
Make pub get_incremental_snapshot_archives() ( #20725 )
2021-10-18 13:48:50 -05:00