Commit Graph

23158 Commits

Author SHA1 Message Date
Ryo Onodera f949a95a9e
Define ClusterConfig::new_with_equal_stakes() (#32330)
* Define ClusterConfig::new_with_equal_stakes()

* Remove needless variable assignment
2023-06-30 10:35:26 +09:00
behzad nouri b722ce4519
rolls out merkle shreds to 100% of testnet slots (#32316) 2023-06-29 18:46:00 +00:00
Yueh-Hsuan Chiang 2850432b57
[TieredStorage] Make data_size related APIs cleaner (#32317)
#### Problem
Function data_size_for_shared_block() is designed to provide
the data_size when its account shares its account block with
other accounts.  However, TieredAccountMeta already has another
function for returning data size.

#### Summary of Changes
This PR removes data_size_for_shared_block() from TieredAccountMeta.
Will follow-up with another PR that includes a better API.
2023-06-30 02:44:57 +08:00
Yihau Chen 66233576a1
ci: trigger shellcheck for .buildkite/hooks (#32329) 2023-06-29 17:34:36 +00:00
hana 755adfd0a5
impl Eq for dyn Signer (#32272) 2023-06-29 08:43:56 -07:00
Brooks d62f5ba37b
Removes unnecessary casts calling datapoint! (#32333) 2023-06-29 14:43:07 +00:00
HaoranYi 801bc2c814
Add test for partitioned epoch rewards store (#32319)
add tests for partitioned epoch rewards store

Co-authored-by: HaoranYi <haoran.yi@solana.com>
2023-06-29 08:57:03 -05:00
sakridge 7ff5e463e9
Move is_niceness logic out of clap utils to reduce dependencies (#32331)
Move is_niceness logic out of clap utils to reduce dependencies..

..for solana-keygen
2023-06-29 14:03:36 +02:00
Ashwin Sekar cbb5381a98
feature flag cleanup: vote_state_update_root_fix (#32321) 2023-06-28 21:26:58 -07:00
dependabot[bot] c0d8891d36
Bump serde_yaml from 0.9.21 to 0.9.22 (#32264)
* Bump serde_yaml from 0.9.21 to 0.9.22

Bumps [serde_yaml](https://github.com/dtolnay/serde-yaml) from 0.9.21 to 0.9.22.
- [Release notes](https://github.com/dtolnay/serde-yaml/releases)
- [Commits](https://github.com/dtolnay/serde-yaml/compare/0.9.21...0.9.22)

---
updated-dependencies:
- dependency-name: serde_yaml
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2023-06-29 04:19:21 +00:00
Yihau Chen 3e5ee8de50
ci: only kill containers that are based on the solanalabs/rust* images (#32292)
* ci: only kill containers that are based on the solanalabs/rust* images

* Update .buildkite/hooks/post-checkout

Co-authored-by: Trent Nelson <trent.a.b.nelson@gmail.com>

* Update .buildkite/hooks/post-checkout

Co-authored-by: Trent Nelson <trent.a.b.nelson@gmail.com>

* fix lint

---------

Co-authored-by: Trent Nelson <trent.a.b.nelson@gmail.com>
2023-06-29 12:18:03 +08:00
steviez d5ad29d837
Make Blockstore::scan_and_fix_roots() take optional start/stop slots (#32289)
The optional args allow reuse by ledger-tool repair roots command Also,
hold cleanup lock for duration of Blockstore::scan_and_fix_roots().

This prevents a scenario where scan_and_fix_roots() could identify a
slot as needing to be marked root, that slot getting cleaned by
LedgerCleanupService, and then scan_and_fix_roots() marking the slot as
root on the now purged slot.
2023-06-28 22:32:03 -05:00
dependabot[bot] d6a0406d54
Bump proc-macro2 from 1.0.60 to 1.0.61 (#32271)
* Bump proc-macro2 from 1.0.60 to 1.0.61

Bumps [proc-macro2](https://github.com/dtolnay/proc-macro2) from 1.0.60 to 1.0.61.
- [Release notes](https://github.com/dtolnay/proc-macro2/releases)
- [Commits](https://github.com/dtolnay/proc-macro2/compare/1.0.60...1.0.61)

---
updated-dependencies:
- dependency-name: proc-macro2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2023-06-28 21:25:58 -06:00
dependabot[bot] 169307d405
Bump serde_json from 1.0.97 to 1.0.99 (#32263)
* Bump serde_json from 1.0.97 to 1.0.99

Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.97 to 1.0.99.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.97...v1.0.99)

---
updated-dependencies:
- dependency-name: serde_json
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2023-06-28 21:21:16 -06:00
Ryo Onodera 5b6c3ba81b
Make prepare_simulation_batch() more usable (#32304)
* Make prepare_simulation_batch() more usable

* Rename

* Use `use`
2023-06-29 11:20:42 +09:00
Greg Cusack b8222b230c
Increase push fanout from 6 to 9 (#32323)
* increase push fanout from 6 to 9. leave active set size at 12

* forgot to change comment to reflect push fanout increase
2023-06-28 19:03:12 -05:00
steviez 67f5a42bd5
ledger-tool: Adjust the help messages for force-update-to-open (#32318)
The previous help message was pretty generic as it covered two potential
error scenarios. This change gives a more detailed error message to each
of those errors.
2023-06-28 18:40:20 -05:00
samkim-crypto 91186d3860
[zk-token-sdk] replace hard-coded constants with constant variables (#32274)
* add ristretto and scalar byte length constants

* add serialization and deserialization helper functions

* remove hard-coded constants in the `sigma` module

* remove hard-coded constants in the `encryption` module

* remove hard-coded constants in the `zk-token-elgamal` module

* Apply suggestions from code review

Co-authored-by: Tyera <tyera@solana.com>

* fix docs for range proof constants

* Apply suggestions from code review

Co-authored-by: Tyera <tyera@solana.com>

* clippy

---------

Co-authored-by: Tyera <tyera@solana.com>
2023-06-29 07:03:26 +09:00
buffalu 5dee2e4d0c
Ensure that uncommitted transactions are always removed from QoS (#32285)
Co-authored-by: Tao Zhu <82401714+taozhu-chicago@users.noreply.github.com>
2023-06-28 13:44:58 -07:00
Yueh-Hsuan Chiang 9c9db4a461
[TieredAccountStorage] getter functions for HotAccountMeta (#32296)
#### Summary of Changes
This PR implements HotAccountMeta's getter functions for accessing
optional fields (e.g. rent_epoch, account_hash, etc) and account data.

#### Test Plan
A new unit test is included in this PR.
2023-06-29 04:17:57 +08:00
HaoranYi 05ba9d58fb
Add test for partition out of range (#32312)
add test for partition out of range

Co-authored-by: HaoranYi <haoran.yi@solana.com>
2023-06-28 14:58:37 -05:00
Yueh-Hsuan Chiang 5e830d2692
[TieredAccountStorage] functions for obtaining optional fields offsets (#32311)
#### Summary of Changes
This PR adds several functions to AccountMetaOptionalFields
for obtaining the offset of each optional field.

#### Test Plan
New unit test cases are included in this PR.
2023-06-29 01:30:37 +08:00
Brooks 794f3849a4
Standardizes datapoints in AccountsBackgroundService's StatsManager (#32310) 2023-06-28 13:00:46 -04:00
HaoranYi 906121645c
Test epoch rewards sysvar in program test (#32293)
* Add epoch rewards sysvar test to program test

* Add a test to check epoch rewards sysvar inside/outisde reward interval

---------

Co-authored-by: HaoranYi <haoran.yi@solana.com>
2023-06-28 10:50:11 -05:00
Yihau Chen e60b58258d
ci: remove wasm-pack version lock (#32307)
* ci: remove wasm-pack version lock

* remove comment
2023-06-28 13:21:07 +00:00
Yueh-Hsuan Chiang 4cfdb374ae
[TieredStorage] byte_block::read_type (#32295)
#### Summary of Changes
This PR adds byte_block::read_type(), an util function that
reads the raw part of the input byte block at the specified offset
as type T.  This function will be later used by the hot storage.

#### Test Plan
Modified existing unit tests to use byte_block::read_type()
when the input byte block is properly aligned.
2023-06-28 12:33:29 +08:00
samkim-crypto 5624aaa1e5
[zk-token-sdk](docs) Update docs to include a brief description of how fee sigma proof is computed (#32288)
* function docs to focus on the action they perform

* update docs for fee sigma proof

* add link to zk token proof doc

* Apply suggestions from code review

Co-authored-by: Tyera <tyera@solana.com>

---------

Co-authored-by: Tyera <tyera@solana.com>
2023-06-28 09:55:55 +09:00
samkim-crypto 4bc8f90182
[zk-token-sdk] Add fee sigma proof verification instruction (#32291)
* add fee sigma proof verification instruction

* update proof program processor for fee sigma proof

* update bench for fee sigma proof

* cargo fmt

* clippy

* fix target arch

* add comments to describe fee, delta, and claimed instruction data field
2023-06-28 09:49:56 +09:00
Jeff Biseda 87c1b67d53
refactor core to create consensus module (#32282) 2023-06-27 17:25:08 -07:00
Brooks 9fb105c801
Cleanup impl for remove_tmp_snapshot_archives() (#32297) 2023-06-27 15:51:28 -04:00
Brennan 8ddce4b566
report last slot vote send metrics (#32258)
* report last slot vote send metrics

* push conditional checking down

* fix up unit tests
2023-06-27 19:43:10 +00:00
steviez 0264d50ebf
Remove duplicate SnapshotConfig validation code (#32290)
Instead, use the is_snapshot_config_valid() helper function.
2023-06-27 13:54:55 -05:00
Brooks ee6684b2eb
Adds UseSnapshotArchivesAtStartup::WhenNewest (#32284) 2023-06-27 14:26:40 -04:00
Brooks 13aff74f82
Cleanup filesystem error handling in snapshot_utils (#32286) 2023-06-27 13:08:47 -04:00
steviez 6b013f46eb
Add comments and unit test for Blockstore::scan_and_fix_roots() (#32283) 2023-06-27 10:39:23 -05:00
HaoranYi b12180b863
Add epoch_rewards syscalls (#32159)
Co-authored-by: HaoranYi <haoran.yi@solana.com>
2023-06-27 09:14:43 -05:00
Wen 6f72258e3e
Vote refresh fix when outside slothash (#29948)
* When there are too many pubkeys in one slot, kick the one with lowest
stake out.

* Cache last_root to reduce read locks we need.

* Use slots_in_epoch to limit number of slots in the map.

* Fix lint errors.

* Only cache stake and slots per epoch once per epoch.

* Revert "Only cache stake and slots per epoch once per epoch."

This reverts commit 8658aad0083456794b4c4403adaf9c74d1a71d09.

* Vote at the tip of current fork if last vote is outside SlotHash
of the tip and last vote expired.

* Add unittest when last vote is outside slothash, we should vote at the tip
of the current fork.

* Revert "Use slots_in_epoch to limit number of slots in the map."

This reverts commit 93574f57a48d2a70fbbc0f62fa8810d3b6bee0af.

* Revert "Cache last_root to reduce read locks we need."

This reverts commit bb114ec2b62cb9c0207328b19c415f6116be0f1c.

* Revert "When there are too many pubkeys in one slot, kick the one with lowest"

This reverts commit 711e29a6a025fd4f11fbc97dcbbe90e4832be04c.

* Move new vote generation when last vote is outside slothash into the
main path, this actually makes more sense since we don't select where
to vote in two different places, and all the vote generation logic
is seamlessly inherited.

* - Move vote refresh to be behind select vote and do not refresh vote if a new
  vote is selected.
- Check whether last vote is inside slothash inside select_vote_and_reset_forks
- rename slot_within_slothash to is_in_slothashes_history
- remove one unittest for now, more tests will be added in a separate CL

* Remove new test, it will be in another file.

* Add is_in_slot_hashes_history test in the new file.

* Add unittest for the case when last vote is outside slot hashes.

* Small improvements and more unittests.

* Fix bad merge.

* Update docs/src/terminology.md

Co-authored-by: mvines <mvines@gmail.com>

* Put SwitchForkDecision::FailedSwitchThreshold logic into separate function.

* Make linter happy.

---------

Co-authored-by: mvines <mvines@gmail.com>
2023-06-26 18:21:24 -07:00
Ashwin Sekar 1e12a18e01
Use bank status instead of default hash in state machine (#31699) 2023-06-27 01:14:40 +00:00
Brooks 4547549a73
Uses fs_err instead of SnapshotError::IoWithSourceAndFile (#32267) 2023-06-26 17:10:21 -04:00
Brooks 1574063143
Don't look up snapshot archives twice in load_bank_forks() (#32279) 2023-06-26 16:31:56 -04:00
Yueh-Hsuan Chiang 1b2b825c1f
[TieredStorage] getter functions for TieredAccountMeta trait (#32276)
#### Summary of Changes
This PR adds getter functions to the TieredAccountMeta for accessing
optional fields and account data.

#### Test Plan
This PR only adds methods to a trait.  More tests will be included in the
follow-up PRs.
2023-06-27 03:01:06 +08:00
Jeff Biseda 5cf5edd5fe
unnecessary error conversions in core (#32257) 2023-06-26 10:31:55 -07:00
steviez 77b587aa4d
Add constant for disabled snapshot interval (#32236)
Slot::MAX was used to specify that a type of snapshots should not be
created; define a constant to be that value and reference the constant
to have a single point of edit.
2023-06-26 12:26:56 -05:00
Brooks 95fae285a4
Introduces fs_err to snapshot_utils (#32266) 2023-06-26 13:14:27 -04:00
Brooks 5f1b5b877a
Replace boot_from_local_state with use_snapshot_archives_at_startup (#32260) 2023-06-26 12:44:25 -04:00
Brooks 394fc9b8fc
Adds "archive" to log when loading snapshot archives (#32278) 2023-06-26 12:32:28 -04:00
Brooks 814ea7ae9f
Removes file permission twiddling in snapshot test (#32277) 2023-06-26 12:17:14 -04:00
Jeff Biseda 5ca1b40f11
refactor core to create cluster_slots_service module (#32119) 2023-06-26 08:54:49 -07:00
Brooks 644cdaaa48
Adds comments in BankSnapshotInfo::new_from_dir() when selecting Pre or Post (#32261) 2023-06-26 11:53:25 -04:00
Brooks 4fb6bbf92e
Refactors datapoints and logs in bank_from_xxx fns (#32255) 2023-06-26 11:33:58 -04:00