Commit Graph

3867 Commits

Author SHA1 Message Date
Ryo Onodera 0b9c040fcf
Add .codecov.yml with checks disabled (#30666)
* Add .codecov.yml with checks disabled

* Trigger test-coverage.sh...
2023-03-10 20:11:22 +09:00
Yueh-Hsuan Chiang 71b6370426
Abstract out StoredAccountMeta as an Enum (#30478)
#### Problem
The existing StoredAccountMeta requires the file and in-memory layout of
account metadata to match StoredMeta and AccountMeta.  This limits the
flexibility to implement different accounts storage file formats.

#### Summary of Changes
This PR abstracts out StoredAccountMeta as an Enum.
2023-03-09 15:52:51 -08:00
Tao Zhu 3b9438fd49
Replace hardcoded loaded accounts size limit with compute budget instruction (#30506)
1. replace hardcoded loaded accounts data size limit with compute budget instruction;
2. new transaction error for invalid account data size limit
3. test requested limit with combination of features and transactions
2023-03-08 19:41:45 -06:00
Brooks 70c6c7e1f7
Uses strong types for snapshot hashes in SnapshotPackagerService (#30603) 2023-03-06 16:50:45 -05:00
Illia Bobyr 82288d4457
accounts_db::get_all_accounts: Select slots with an iterator (#30581)
As `Range` is a special case of an `Iterator`, this is a generalization
that makes `get_all_accounts()` accept other kinds of ranges.  Such as
`RangeInclusive`.
2023-03-06 12:30:50 -08:00
Yueh-Hsuan Chiang a78ef393bf
Generalize the return value of append_accounts() (#30571)
#### Problem
The existing append_accounts returns a vector of offsets, which accounts-db
later uses it to infer the storage size of each account.  However, such inference
does not apply to the new hot/cold storage format.

#### Summary of Changes
This PR changes the return value of append_accounts() to `Vec<StoredAccountInfo>`,
which includes both offset and storage size information of an account.
2023-03-06 09:52:05 -08:00
Alexander Meißner 38e74325e3
Refactor - Remove `process_instruction` parameter `first_instruction_account` (#30579)
* Stops using first_instruction_account parameter in bpf_loader.

* Removes first_instruction_account parameter from process_instruction().
2023-03-06 17:37:37 +01:00
Jeff Washington (jwash) b26d470b02
support loading snapshots with > 1 append vec per slot (#30570)
* support loading snapshots with > 1 append vec per slot

* pr feedback

* drain to into_values

* unwrap() -> expect()

* loop to find non-conflicting file and add test
2023-03-06 06:08:57 -08:00
Illia Bobyr 14c600aa70
StoredAccountMeta: Relax returned refs lifetimes. (#30576)
All the references returned from a `StoredAccountMeta` instance
actually come from the underlying account.  As written, the lifetimes of
the returned references are constrained by the lifetime of the
`StoredAccountMeta` instance.

While technically correct, it is overly strict.  It is totally fine to
use the returned reference as long as the underlying account is alive.
Which could be longer than the lifetime of the `StoredAccountMeta`
wrapper.
2023-03-03 14:51:37 -08:00
Jeff Washington (jwash) b7d05dd755
consolidate held in mem stats (#30574) 2023-03-03 10:05:05 -08:00
Jeff Washington (jwash) fbcb82dcf8
add index_entries_being_shrunk to ShrinkCollect (#30518) 2023-03-01 17:09:42 -08:00
Jeff Washington (jwash) 590ee58667
wip: throttle correctly (#30547)
throttle correctly
2023-03-01 17:08:46 -08:00
Yueh-Hsuan Chiang 33e179905f
getter functions for StoredAccountMeta (#30447)
This PR makes all the StoredAccountMeta fields pub(crate)
and provides getter functions to access its member fields.

This PR is a preparation step for abstracting out StoredAccountMeta.
2023-02-27 10:03:22 -08:00
Jeff Washington (jwash) 85eaf1e7ab
improve BankCreationFreezingProgress and add tests (#30513) 2023-02-27 07:54:08 -08:00
Tao Zhu cbcd3ef27e
Not allow transactions request zero CU to update prioritization fee cache (#30507)
* filter out transaction requests zero cu from updating prioritization fee cache

* updated comment
2023-02-24 23:50:02 -06:00
Tao Zhu 9ec5e9b866
refactor to remove additional is_simple_vote check (#30521) 2023-02-24 23:49:04 -06:00
Brooks 64c7b401cd
BankIncrementalSnapshotPersistence uses strong accounts hash types (#30502) 2023-02-24 18:15:23 -05:00
Jeff Washington (jwash) 45a34f637b
move BankCreationFreezingProgress to its own file (#30510) 2023-02-24 14:05:39 -08:00
Brooks ffe2d309b2
Adds serde_snapshot::types module (#30501) 2023-02-24 15:27:12 -05:00
Jeff Washington (jwash) 31573beb2d
use slot instead of store_id in clean (#30472)
* use slot instead of store_id in clean

* update debug
2023-02-24 12:01:16 -08:00
Jeff Washington (jwash) 9080b36192
use local for store_id in write_accounts_to_storage (#30500) 2023-02-24 11:50:57 -08:00
Xiang Zhu 510be7bdb5
Add checks when constructing a BankSnapshotInfo from a directory (#30373)
* Read snapshot directories and find the highest

* format check

* Fix test_get_bank_snapshots

* fix test_bank_fields_from_snapshot

* review changes, is_file etc

* removed incremental snapshot

* SnapshotNewFromDirError

* nit and comments issues

* NewFromDir(#[from] SnapshotNewFromDirError)

* change fill to create

* replace unwrap with map_err and ok_or_else

* Remove BankForks, fix the bank loop
2023-02-24 11:50:29 -08:00
Brooks e0abad43c3
Removes checks for bank_hash_stats in test (#30480) 2023-02-24 14:35:36 -05:00
Jeff Washington (jwash) efc5cc4a48
break apart AccountInfo (#30505) 2023-02-24 11:35:27 -08:00
Jeff Washington (jwash) 7c086ca42d
add DiskIndexValue trait for disk index (#30503) 2023-02-24 11:02:06 -08:00
K-anon 3ed63de656
Revert "Revert "Add test for executor cache runtime feature upgrades.… (#30428) 2023-02-24 10:04:51 -07:00
Tao Zhu 66ea750182
add compute budget instruction to set loaded accounts data size limit (#30377)
* add compute budget instruction to set accounts data size limit

* changes names to explicitly for loaded accounts data size
2023-02-24 09:27:49 -06:00
Jeff Washington (jwash) 0a75be399b
dirty_stores key is slot and not AppendVecId (#30474) 2023-02-24 07:17:09 -08:00
Jeff Washington (jwash) 005dde70e2
minor cleanup (#30477) 2023-02-23 15:10:09 -08:00
Jeff Washington (jwash) 2441a06e78
drop default from PhantomData::default() (#30476) 2023-02-23 14:59:08 -08:00
Brooks b7d31e134c
Refactors Accounts::new_from_parent() (#30475) 2023-02-23 16:50:51 -05:00
Jeff Washington (jwash) 2c7d9f988e
use slot with store id in some clean paths (#30473) 2023-02-23 13:50:25 -08:00
Jeff Washington (jwash) 9ddb89226c
remove_dead_accounts uses slot instead of AppendVecId (#30471) 2023-02-23 13:20:33 -08:00
Jeff Washington (jwash) e39626ab14
add second type to accounts index for disk (#30446) 2023-02-23 13:05:06 -08:00
Jeff Washington (jwash) 6df160bedd
get stored_size from StoredAccountMeta (#30456) 2023-02-23 12:33:35 -08:00
Jeff Washington (jwash) a2b0b8e346
rearrange how shrink_in_progress_map is populated (#30466) 2023-02-23 11:35:38 -08:00
Xiang Zhu d6da019ccf
Clean up old account snapshot directories to avoid the file existing hardlink error (#30426)
* Fix the hardlink failure

* minor comment cleanup

* use ? and slot_str

* &slot_str

* Add InvalidAccountPath
2023-02-23 10:53:12 -08:00
Jeff Washington (jwash) 47d95a431c
remove magic numbers in test (#30465) 2023-02-23 09:23:14 -08:00
Jeff Washington (jwash) c032dc2754
add bank progress stats (#30405)
* add bank progress stats

* handle more bank creation code paths

* use wrapping sub
2023-02-23 06:29:02 -08:00
Jeff Washington (jwash) 9ecc4462f8
cleanup test type (#30453) 2023-02-23 06:26:55 -08:00
Yihau Chen df3ef111f7
chore: workspace inheritance (#29893)
* introduce workspace.package

* introduce workspace.dependencies

* read version from root cargo.toml

* pass check when version = { workspace = true }

* don't bump version when version = { workspace = true }

* including workspace Cargo.toml when bump version

* programs/sbf use workspace inheritance

* fix increasing cargo version ignore program/sbf/Cargo.toml
2023-02-23 22:01:54 +08:00
Pankaj Garg 0e13dcca00
Replenish executor cache eagerly during transaction processing (#30425) 2023-02-23 05:13:39 -08:00
Michael Vines 5136ed3448
Update homepage value for all crates (#30444) 2023-02-23 02:20:18 +00:00
Yueh-Hsuan Chiang ac7e7aa8f0
Move account-meta structs from append_vec.rs to account_meta.rs (#30443)
#### Problem
As we start supporting new storage formats, those account-meta
structs will be shared by different storage formats and thus need
a new home.

#### Summary of Changes
This PR creates meta.rs under account_storage and moves all the account-meta
structs out from append_vec.rs.
2023-02-22 16:10:34 -08:00
Yueh-Hsuan Chiang 833cb75c20
Fix the dependency of accounts_file.rs (#30423)
Replace the use of append_vec::* in accounts_file.rs
2023-02-22 11:13:58 -08:00
Jeff Washington (jwash) a78f763896
simplify remove_if_slot_list_empty_value (#30436) 2023-02-22 10:46:12 -08:00
Jeff Washington (jwash) 6a66d4631c
remove 'stored_size' from account info (#30407) 2023-02-22 07:04:21 -08:00
Jeff Washington (jwash) db8764f98d
hold items with ref count != 1 in memory (#30414)
* hold items with ref count != 1 in memory

* fix tests
2023-02-22 06:57:59 -08:00
Brooks 1689586213
Uses a channel for AHV -> SPS (#30406) 2023-02-22 03:36:29 +00:00
K-anon 2e4b8ea8bb
Revert "Add test for executor cache runtime feature upgrades. (#30382)" (#30424)
This reverts commit c5a24e11ba.

Co-authored-by: K-anon <IntokuSatori@users.noreply.github.com>
2023-02-21 15:47:39 -07:00