Commit Graph

4233 Commits

Author SHA1 Message Date
Pankaj Garg 4674b0099f
Move CostModel and CostTracker to its own crate (#32354)
* Move CostModel and CostTracker to its own crate

* compile new crate and update imports

* update sbf Cargo.lock

* fix AbiExample

* fix cargo sort

* Fix AbiExample
2023-07-06 10:08:18 -07:00
Pankaj Garg b060f62004
Move transaction results data structures out of bank.rs (#32386)
* Move transaction results data structures out of bank.rs

* Fix CI check failures
2023-07-06 09:45:01 -07:00
Yueh-Hsuan Chiang fbb1dd5fe5
Replace AccountsFile::cloned_account by ReadableAccount::to_account_shared_data() (#32399)
#### Problem
Both AppendVec::cloned_account() and ReadableAccount::to_account_shard_data()
returns an AccountSharedData instance by cloning itself as pointed out under
a comment in PR #32380.

#### Summary of Changes
This PR removes AccountsFile::cloned_account() and replaces its call-sites by
ReadableAccount::to_account_shared_data().
2023-07-07 00:08:50 +08:00
Yueh-Hsuan Chiang f1836bb10f
[TieredAccountStorage] TieredReadableAccount (#32380)
#### Summary of Changes
This PR introduces TieredReadableAccount, a struct that takes a generic
TieredAccountMeta that implements ReadableAccount and support
StoredAccountMeta.

#### Test Plan
More tests will be added once more reader and writer functions for hot
accounts have been added.
2023-07-07 00:08:09 +08:00
Brooks d08ce116b3
Measures accounts hash calculation time (#32385) 2023-07-05 15:53:36 -04:00
Brooks f1f0f451b7
Adds cache_hash_data_us to HashStats (#32371) 2023-07-05 13:57:07 -04:00
Pankaj Garg 356a827087
Move NonceInfo, NonceFull and NoncePartial out of bank.rs (#32375)
* Move NonceInfo, NonceFull and NoncePartial out of bank.rs

* fix imports

* move test_nonce_info to nonce_info.rs
2023-07-05 10:44:54 -07:00
Brooks c80981070c
Adds mark_time and sort_time to HashStats total_time (#32370) 2023-07-05 12:26:31 -04:00
Brooks 52fe7eb1cf
Renames HashStats "sort" to "sort_us" (#32369) 2023-07-05 09:35:44 -04:00
Yueh-Hsuan Chiang 806d79a643
[TieredStorage] Improve TieredAccountMeta API's readability (#32338)
#### Problem
In TieredAccountMeta, it uses `data` and `account_data` to refer to
the data associated with the account, and this could be confusing.

#### Summary of Changes
This PR makes TieredAccountMeta only use `account_data` to refer
to the data associated with the account.
2023-07-01 01:55:13 +08:00
HaoranYi 9b405b21ff
Test `EpochRewards` sysvar creation, distribution, and burning. (#32339)
* Test `EpochRewards` sysvar creation, distribution, and burning.

* Update runtime/src/bank/tests.rs

Co-authored-by: Brooks <brooks@prumo.org>

---------

Co-authored-by: HaoranYi <haoran.yi@solana.com>
Co-authored-by: Brooks <brooks@prumo.org>
2023-06-30 10:46:31 -05:00
HaoranYi fccc1c75f3
test partitioned reward history update from rewards slice (#32334)
Co-authored-by: HaoranYi <haoran.yi@solana.com>
2023-06-30 09:44:20 -05:00
Yueh-Hsuan Chiang 2850432b57
[TieredStorage] Make data_size related APIs cleaner (#32317)
#### Problem
Function data_size_for_shared_block() is designed to provide
the data_size when its account shares its account block with
other accounts.  However, TieredAccountMeta already has another
function for returning data size.

#### Summary of Changes
This PR removes data_size_for_shared_block() from TieredAccountMeta.
Will follow-up with another PR that includes a better API.
2023-06-30 02:44:57 +08:00
Brooks d62f5ba37b
Removes unnecessary casts calling datapoint! (#32333) 2023-06-29 14:43:07 +00:00
HaoranYi 801bc2c814
Add test for partitioned epoch rewards store (#32319)
add tests for partitioned epoch rewards store

Co-authored-by: HaoranYi <haoran.yi@solana.com>
2023-06-29 08:57:03 -05:00
Ryo Onodera 5b6c3ba81b
Make prepare_simulation_batch() more usable (#32304)
* Make prepare_simulation_batch() more usable

* Rename

* Use `use`
2023-06-29 11:20:42 +09:00
Yueh-Hsuan Chiang 9c9db4a461
[TieredAccountStorage] getter functions for HotAccountMeta (#32296)
#### Summary of Changes
This PR implements HotAccountMeta's getter functions for accessing
optional fields (e.g. rent_epoch, account_hash, etc) and account data.

#### Test Plan
A new unit test is included in this PR.
2023-06-29 04:17:57 +08:00
HaoranYi 05ba9d58fb
Add test for partition out of range (#32312)
add test for partition out of range

Co-authored-by: HaoranYi <haoran.yi@solana.com>
2023-06-28 14:58:37 -05:00
Yueh-Hsuan Chiang 5e830d2692
[TieredAccountStorage] functions for obtaining optional fields offsets (#32311)
#### Summary of Changes
This PR adds several functions to AccountMetaOptionalFields
for obtaining the offset of each optional field.

#### Test Plan
New unit test cases are included in this PR.
2023-06-29 01:30:37 +08:00
Brooks 794f3849a4
Standardizes datapoints in AccountsBackgroundService's StatsManager (#32310) 2023-06-28 13:00:46 -04:00
Yueh-Hsuan Chiang 4cfdb374ae
[TieredStorage] byte_block::read_type (#32295)
#### Summary of Changes
This PR adds byte_block::read_type(), an util function that
reads the raw part of the input byte block at the specified offset
as type T.  This function will be later used by the hot storage.

#### Test Plan
Modified existing unit tests to use byte_block::read_type()
when the input byte block is properly aligned.
2023-06-28 12:33:29 +08:00
Brooks 9fb105c801
Cleanup impl for remove_tmp_snapshot_archives() (#32297) 2023-06-27 15:51:28 -04:00
Brooks 13aff74f82
Cleanup filesystem error handling in snapshot_utils (#32286) 2023-06-27 13:08:47 -04:00
Wen 6f72258e3e
Vote refresh fix when outside slothash (#29948)
* When there are too many pubkeys in one slot, kick the one with lowest
stake out.

* Cache last_root to reduce read locks we need.

* Use slots_in_epoch to limit number of slots in the map.

* Fix lint errors.

* Only cache stake and slots per epoch once per epoch.

* Revert "Only cache stake and slots per epoch once per epoch."

This reverts commit 8658aad0083456794b4c4403adaf9c74d1a71d09.

* Vote at the tip of current fork if last vote is outside SlotHash
of the tip and last vote expired.

* Add unittest when last vote is outside slothash, we should vote at the tip
of the current fork.

* Revert "Use slots_in_epoch to limit number of slots in the map."

This reverts commit 93574f57a48d2a70fbbc0f62fa8810d3b6bee0af.

* Revert "Cache last_root to reduce read locks we need."

This reverts commit bb114ec2b62cb9c0207328b19c415f6116be0f1c.

* Revert "When there are too many pubkeys in one slot, kick the one with lowest"

This reverts commit 711e29a6a025fd4f11fbc97dcbbe90e4832be04c.

* Move new vote generation when last vote is outside slothash into the
main path, this actually makes more sense since we don't select where
to vote in two different places, and all the vote generation logic
is seamlessly inherited.

* - Move vote refresh to be behind select vote and do not refresh vote if a new
  vote is selected.
- Check whether last vote is inside slothash inside select_vote_and_reset_forks
- rename slot_within_slothash to is_in_slothashes_history
- remove one unittest for now, more tests will be added in a separate CL

* Remove new test, it will be in another file.

* Add is_in_slot_hashes_history test in the new file.

* Add unittest for the case when last vote is outside slot hashes.

* Small improvements and more unittests.

* Fix bad merge.

* Update docs/src/terminology.md

Co-authored-by: mvines <mvines@gmail.com>

* Put SwitchForkDecision::FailedSwitchThreshold logic into separate function.

* Make linter happy.

---------

Co-authored-by: mvines <mvines@gmail.com>
2023-06-26 18:21:24 -07:00
Brooks 4547549a73
Uses fs_err instead of SnapshotError::IoWithSourceAndFile (#32267) 2023-06-26 17:10:21 -04:00
Yueh-Hsuan Chiang 1b2b825c1f
[TieredStorage] getter functions for TieredAccountMeta trait (#32276)
#### Summary of Changes
This PR adds getter functions to the TieredAccountMeta for accessing
optional fields and account data.

#### Test Plan
This PR only adds methods to a trait.  More tests will be included in the
follow-up PRs.
2023-06-27 03:01:06 +08:00
steviez 77b587aa4d
Add constant for disabled snapshot interval (#32236)
Slot::MAX was used to specify that a type of snapshots should not be
created; define a constant to be that value and reference the constant
to have a single point of edit.
2023-06-26 12:26:56 -05:00
Brooks 95fae285a4
Introduces fs_err to snapshot_utils (#32266) 2023-06-26 13:14:27 -04:00
Brooks 394fc9b8fc
Adds "archive" to log when loading snapshot archives (#32278) 2023-06-26 12:32:28 -04:00
Brooks 814ea7ae9f
Removes file permission twiddling in snapshot test (#32277) 2023-06-26 12:17:14 -04:00
Brooks 644cdaaa48
Adds comments in BankSnapshotInfo::new_from_dir() when selecting Pre or Post (#32261) 2023-06-26 11:53:25 -04:00
Brooks 4fb6bbf92e
Refactors datapoints and logs in bank_from_xxx fns (#32255) 2023-06-26 11:33:58 -04:00
Pankaj Garg 089fb464a9
Move message_processor to program-runtime (#32268) 2023-06-24 15:55:39 -07:00
Tyera 02eaaae513
Clean up credits_auto_rewind feature (#32211)
* Remove credits_auto_rewind feature logic from stake program

* Obey clean-up orders

* Clippy prefers this match

* Remove credits_auto_rewind feature helpers from Bank

* Comment nit for language, and shrink to fit on one line
2023-06-24 10:26:27 -06:00
Pankaj Garg 0b8af56924
Remove message_processor dependency on solana_runtime (#32256) 2023-06-24 07:55:32 -07:00
Yueh-Hsuan Chiang 9a620b4862
[TieredStorage] HotAccountMeta (2/N) (#32227)
#### Summary of Changes
This PR introduces HotAccountMeta, the storage and in-memory struct of
the metadata struct for a hot account.

#### Test Plan
Unit tests are included in this PR.
Tested in mnb w/ the prototype implementation of the tiered account storage (#30626)
2023-06-24 01:25:54 +08:00
Yueh-Hsuan Chiang d95e976a71
[TieredStorage] AccountMetaOptionalFields::size_from_flags() (#32242)
#### Summary of Changes
This PR adds AccountMetaOptionalFields::size_from_flags that takes
`&AccountMegaFlags` and returns the size of the AccountMetaOptionalFields
based on the input AccountMegaFlags.

This function is needed because the reader of the TieredAccountMeta
directly extract all the Some fields of AccountMetaOptionalFields
from its account block without constructing the AccountMetaOptionalFields
instance.

#### Test plan
Improve existing unit tests that further verify the correctness of the function.
2023-06-23 01:42:02 +08:00
Yueh-Hsuan Chiang 1c618f2479
[TieredStorage] TieredAccountMeta (1/N) (#32220)
#### Summary of Changes
This PR introduces TieredAccountMeta, a trait that allows different implementations
of the account meta for different tiers of the accounts storage.

This PR includes the mutator and getter methods for the common TieredAccountMeta
fields.  Functions that also work with the account data block will be in separate PRs.
2023-06-23 01:40:43 +08:00
Ryo Onodera f62d231886
Make `ci/_` to print to stderr instead of stdout (#32238)
* make _ print to stderr instead...

* Trigger normal build procedure
2023-06-22 15:46:22 +09:00
Brooks ba05cbf4cc
Sets default snapshot archive format to zstd (#32205) 2023-06-21 15:15:53 -04:00
steviez 20a7cdd43d
Restrict access to Bank's HardForks (#32180)
* Restrict access to Bank's HardForks

Callers could previously obtain a a lock to read/write HardForks from
any Bank. This would allow any caller to modify, and creates the
opportunity for inconsistent handling of what is considered a valid hard
fork (ie too old).

This PR adds a function to Bank so consistent sanity checks can be
applied; the caller will already have a Bank as that is where they would
have obtained the HardForks from in the first place. Additionally,
change the getter to return a copy of HardForks (simple Vec).

* Allow hard fork at bank slot if bank is not yet frozen
2023-06-20 23:44:43 -05:00
HaoranYi 203544293b
Add epoch_rewards to sysvar cache (#32155)
add epoch_rewards to sysvar cache

Co-authored-by: HaoranYi <haoran.yi@solana.com>
2023-06-20 15:01:34 -05:00
HaoranYi 2e9aeff8dd
Enable partitioned epoch reward by feature id (#32174)
enable partitioned epoch reward feature by feature id
update stake rewards tests with partitioned epoch rewards feature enable

Co-authored-by: HaoranYi <haoran.yi@solana.com>
2023-06-20 12:12:50 -05:00
Yueh-Hsuan Chiang c56d0bc6a6
[TieredStorage] Make AccountMetaFlags::new_from pub (#32194)
#### Summary of Changes
The newly introduced AccountMetaFlags::new_from must be pub
in order to make the tiered storage writer able to use it.
2023-06-21 00:00:59 +08:00
galactus 2ceabd9368
Simd 47 syscall sysvar last restart slot (#31957)
* add sysvar and logic for last restart slot

* cleanup

* add test for getting last restart slot from account

* format code

* add some basic rustdoc

* copy+paste error

* feature flag for last_restart_slot

* add to sysvars.md

* updated wording in sysvars.md

* rename sol_get_last_restart_slot_sysvar > sol_get_last_restart_slot

* create sbf C header for sol_get_last_restart_slot

* cleanup imports

* reverted hardened_unpack workaround

* cleanup imports

* cleanup logs + blank lines

* Implementing ui changes for last restart slot, nit

* Some more nit change and implementing the UI for sysvar

* fixing the CI

* Minor clippy fix

* format changes

* changes suggested by mvines and lichtso

* increase timeout in local_cluster test

* fix code format

* use keypair for feature flag from mvines

* delete test.json file

* Revert "increase timeout in local_cluster test"

This reverts commit a67465ae22.

* last restart slot should be always less than or equal to current slot

* fixing bug

* changes after  steviez comments

* format issue fixed

* fixing the comment on premature application of future hardfork

* nit change in test

Co-authored-by: steviez <steven@solana.com>

* reverting sysvar_cache.rs because change was not necessary

---------

Co-authored-by: steve-gg <grooviegermanikus@gmail.com>
Co-authored-by: steviez <steven@solana.com>
2023-06-16 20:14:02 +00:00
Jeff Washington (jwash) 174ceba82b
rename to stake_rewards_by_partition for clarity (#32165) 2023-06-16 19:28:53 +00:00
HaoranYi 39d9f8974d
Revert "add test for is_partitioned_rewards_feature_enabled (#32158)" (#32176)
This reverts commit d06b099ecb.

Co-authored-by: HaoranYi <haoran.yi@solana.com>
2023-06-16 11:56:05 -05:00
HaoranYi d06b099ecb
add test for is_partitioned_rewards_feature_enabled (#32158)
* add set_partitioned_rewards_enabled_for_test fn

* reveiws

---------

Co-authored-by: HaoranYi <haoran.yi@solana.com>
2023-06-16 11:07:07 -05:00
HaoranYi af7b7f1436
Refactor reward block calculation fn (#32167)
refactor reward block calculation fn

Co-authored-by: HaoranYi <haoran.yi@solana.com>
2023-06-16 08:40:23 -05:00
Brooks 47ff3cecc9
Enables creating snapshots after booting from local state (#32137) 2023-06-15 22:54:32 -04:00