Commit Graph

24985 Commits

Author SHA1 Message Date
GHA: Update Upstream From Fork 27eff8408b Revert "Allow configuration of replay thread pools from CLI (#236)"
This reverts commit 973d05c098.
2024-03-22 15:58:10 -05:00
steviez 973d05c098 Allow configuration of replay thread pools from CLI (#236)
Bubble up the constants to the CLI that control the sizes of the
following two thread pools:
- The thread pool used to replay multiple forks in parallel
- The thread pool used to execute transactions in parallel
2024-03-20 16:29:43 -05:00
Dmitri Makarov 1d89ea01cc Rename LoadedPrograms to ProgramCache for readability (#339) 2024-03-20 16:28:38 -05:00
Yueh-Hsuan Chiang 2273098c55 [TieredStorage] Store account address range (#172)
#### Problem
The TieredStorageFooter has the min_account_address and
max_account_address fields to describe the account address
range in its file.  But the current implementation hasn't updated
the fields yet.

#### Summary of Changes
This PR enables the TieredStorage to persist address range
information into its footer via min_account_address and
max_account_address.

#### Test Plan
Updated tiered-storage test to verify persisted account address range.
2024-03-20 16:28:38 -05:00
Yueh-Hsuan Chiang e97d3590c7 [TieredStorage] Refactor TieredStorage::new_readonly() code path (#195)
#### Problem
The TieredStorage::new_readonly() function currently has the following
problems:

* It opens the file without checking the magic number before checking and loading the footer.
* It opens the file twice: first to load the footer, then open again by the reader.

#### Summary of Changes
This PR refactors TieredStorage::new_readonly() so that it first performs all
checks inside the constructor of TieredReadableFile.  The TieredReadableFile
instance is then passed to the proper reader (currently HotStorageReader)
when all checks are passed.

#### Test Plan
* Added a new test to check MagicNumberMismatch.
* Existing tiered-storage tests
2024-03-20 16:28:38 -05:00
Trent Nelson 81c8ed73bc rpc-sts: add config options for stake-weighted qos (#197)
* rpc-sts: plumb options for swqos config

* rpc-sts: send to specific tpu peers when configured
2024-03-20 16:28:38 -05:00
HaoranYi 304333405c Revert deprecate executable feature (#309)
* revert deprecate executable feature

* add native loader account transfer test

---------

Co-authored-by: HaoranYi <haoran.yi@solana.com>
2024-03-20 16:28:38 -05:00
Pankaj Garg 78f033d9e6 Move code to check_program_modification_slot out of SVM (#329)
* Move code to check_program_modification_slot out of SVM

* add documentation for the public function
2024-03-20 16:28:38 -05:00
Jon C 5871a0e7ed CI: Add windows clippy job and fix clippy errors (#330)
* CI: Run clippy on windows

* Update cargo-clippy-before-script.sh for Windows

* Pacify clippy
2024-03-20 16:28:38 -05:00
Tao Zhu 0119437764 qos service should also accumulate executed but errored units (#328)
qos service should also accumulated executed but errored units
2024-03-20 16:28:38 -05:00
Alessandro Decina dcdce7c9f1 accounts-db: unpack_archive: unpack accounts straight into their final destination (#289)
* accounts-db: unpack_archive: avoid extra iteration on each path

We used to do a iterator.clone().any(...) followed by
iterator.collect(). Merge the two and avoid an extra iteration and
re-parsing of the path.

* accounts-db: unpack_archive: unpack accounts straight into their final destination

We used to unpack accounts into account_path/accounts/<account> then
rename to account_path/<account>. We now unpack them into their final
destination directly and avoid the rename syscall.
2024-03-20 16:28:38 -05:00
Ashwin Sekar 7ad99f3b11 vote: reuse ff to gate tvc constant update from 8 -> 16 (#322) 2024-03-20 16:28:38 -05:00
Dmitri Makarov e2c1cbfd57 SVM: minor refactoring to improve code readability (#317) 2024-03-20 16:28:38 -05:00
Lucas Steuernagel 170df8328d SVM integration test (#307) 2024-03-20 16:28:37 -05:00
Jon C e39bd8d11d install: Fix check for windows build (#295) 2024-03-20 16:28:37 -05:00
Brennan 01e48239be fix polarity for concurrent replay (#297)
* fix polarity for concurrent replay
2024-03-20 16:28:37 -05:00
Greg Cusack f35bda5067 add in method for building a `TpuClient` for `LocalCluster` tests (#258)
* add in method for building a TpuClient for LocalCluster tests

* add cluster trait. leave dependency on solana_client::tpu_client
2024-03-20 16:28:37 -05:00
Justin Starry 21eff36754 cli: skip no-op program buffer writes (#277)
cli: skip no-op program deploy write txs
2024-03-20 16:28:37 -05:00
Yueh-Hsuan Chiang fee4d82a56 [TieredStorage] Use BufWriter in TieredWritableFile (#261)
#### Problem
TieredWritableFile currently uses File instead of BufWriter.
This will introduce more syscall when doing file writes.

#### Summary of Changes
This PR makes TieredWritableFile uses BufWriter to allow the
write-call to be more optimized to reduce the number of syscalls.

#### Test Plan
Existing tiered-storage test.
Will run experiments to verify its performance improvement.

#### Dependency
https://github.com/anza-xyz/agave/pull/260
2024-03-20 16:28:37 -05:00
Lijun Wang 9e2768ad5a Net script fix for expected shred version (#280)
Fix for --expected-shred-version when maybeWaitForSupermajority is on

Co-authored-by: Lijun Wang <lijun@anza.xyz>
2024-03-20 16:27:59 -05:00
Yueh-Hsuan Chiang 62c458e4de [TieredStorage] TieredStorageFile -> TieredReadonlyFile and TieredWritableFIle (#260)
#### Problem
TieredStorageFile struct currently offers new_readonly() and new_writable()
to allow both read and write work-load to share the same struct.  However,
as we need the writer to use BufWriter to improve performance as well as
enable Hasher on writes.  There is a need to refactor TieredStorageFile to
split its usage for read-only and writable.

#### Summary of Changes
Refactor TieredStorageFile to TieredReadonlyFIle and TieredWritableFile.

#### Test Plan
Existing tiered-storage tests.
2024-03-20 16:25:25 -05:00
ryleung-solana f6c22e9796 Make the quic server connection table use an async lock, reducing thrashing (#293)
Make the quic server connection table use an async lock, reducing lock contention
2024-03-20 16:25:25 -05:00
WGB5445 6846756926 [solana-install-init] Optimize error message for Windows user permission installation (#234)
* feat: check user's permissions in Windows

* feat: Remove check fun and check os_err

* fmt and optimize code
2024-03-20 16:25:25 -05:00
Jeff Washington (jwash) 928ede1d49 add stats for write cache flushing (#233)
* add stats for write cache flushing

* some renames
2024-03-20 16:25:25 -05:00
Pankaj Garg 403225f112 Remove public visibility of program cache from bank (#279) 2024-03-20 16:24:48 -05:00
Ashwin Sekar d3a7d99ef0 vote: deprecate unused legacy vote tx plumbing (#274) 2024-03-20 16:24:48 -05:00
听寒 cdb0d15283 fix typo (#264)
fix typo in comment of test_optimistic_confirmation_violation_without_tower
2024-03-15 22:25:14 -05:00
ripatel-fd 133bd7f652 Use static memory layout for all sysvars (#249)
Co-authored-by: Richard Patel <ripatel@jumptrading.com>
2024-03-15 22:25:14 -05:00
Andrew Fitzgerald 4550245379 congestion_multiplier as u64 (#267) 2024-03-15 22:25:14 -05:00
Tyera c85844a319 Clean up require_rent_exempt_split_destination feature (#253)
* Clean up require_rent_exempt_split_destination feature

* Clean up comment
2024-03-15 22:25:14 -05:00
HaoranYi 00a34d7d3a fix typos (#247)
Co-authored-by: HaoranYi <haoran.yi@solana.com>
2024-03-15 22:25:14 -05:00
Trent Nelson e80f8fa9e6 remove raptor coding experiments (#255) 2024-03-15 22:25:14 -05:00
Wen 9e394bd0e5 Wen_restart: check block full using blockstore (#250)
* Switch to blockstore.is_full() check because replay thread isn't active.

* Use make_chaining_slot_entries and add first_parent to the method.
Small style fixes.

* Switch to blockstore.is_full() check because replay thread isn't active.
2024-03-15 22:25:14 -05:00
Wen 5d7219675b wen_restart: replace get_aggregate_result() with more methods (#254)
* Replace AggregateResult with more methods.

* Rename slots_to_repair() to slots_to_repair_iter().
2024-03-15 22:25:14 -05:00
samkim-crypto 683e1071f7 [zk-token-sdk] Implement `FromStr` for `ElGamalPubkey`, `ElGamalCiphertext`, and `AeCiphertext` (#130)
* add `ParseError` in `zk-token-elgamal`

* implement `FromStr` for `ElGamalPubkey` and `ElGamalCiphertext`

* implement `FromStr` for `AeCiphertext`

* fix target

* cargo fmt

* use constants for byte length check

* make `FromStr` functions available on chain

* use macros for the `FromStr` implementations

* restrict `from_str` macro to `pub(crate)`

* decode directly into array

* cargo fmt

* Apply suggestions from code review

Co-authored-by: Jon C <me@jonc.dev>

* remove unnecessary imports

* remove the need for `ParseError` dependency

---------

Co-authored-by: Jon C <me@jonc.dev>
2024-03-15 22:25:14 -05:00
sakridge 1036abf250 Fix gossip contact trace (#241) 2024-03-15 22:25:14 -05:00
Greg Cusack 274382ff68 Add in metrics for detecting Redundant Pulls (#199) 2024-03-15 22:25:14 -05:00
Nagaprasad V R 50cc77c58f relax stake split destination check (#162)
* relax stake split destination check

* change stake_account error handling logic

* fmt

* modify logic

* change error message when account is neither owned by stake program or system program

* add a comment explaining nested ifs in stake_account error handling

* fix typos in comments

* update comment
2024-03-15 22:25:14 -05:00
dependabot[bot] 429a68cce1 build(deps): bump hidapi from 2.6.0 to 2.6.1 (#237)
Bumps [hidapi](https://github.com/ruabmbua/hidapi-rs) from 2.6.0 to 2.6.1.
- [Release notes](https://github.com/ruabmbua/hidapi-rs/releases)
- [Commits](https://github.com/ruabmbua/hidapi-rs/commits)

---
updated-dependencies:
- dependency-name: hidapi
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-03-15 22:25:14 -05:00
Justin Starry 0d1cb92ed9 Add `reserved_account_keys` module to sdk (#84) 2024-03-15 22:25:14 -05:00
Jeff Washington (jwash) de9358638b allow FlushStats to accumulate (#215) 2024-03-15 22:25:14 -05:00
Kirill Fomichev 219d7772b1 rpc: disable BigTable with header (#226)
* rpc: disable BigTable with header

* use is_some_and instead map+unwrap_or

Co-authored-by: Tyera <tyera@anza.xyz>

---------

Co-authored-by: Tyera <tyera@anza.xyz>
2024-03-15 22:25:14 -05:00
Greg Cusack 82fefb7de7 update changelog and remove deprecated label on `gossip_service::get_client()` (#227)
update changelog and remove deprecated label on get_client
2024-03-15 22:25:14 -05:00
Jon C d3c0adf1c4 deps: Update base64 to 0.22 (#225)
* deps: Update base64 to 0.22

* Add changelog entry
2024-03-15 22:25:14 -05:00
anwayde 6e29c801f3 bench-tps: allow option to not set account data size on every transaction (#209)
bench-tps: allow option to not set account data size
2024-03-15 22:25:14 -05:00
Yueh-Hsuan Chiang 871421023e [TieredStorage] Remove the general-purposed TieredStorageWriter (#196)
#### Problem
tiered_storage/writer.rs was added when we planned to support multiple
tiers in the tiered-storage (i.e., at least hot and cold).  However, as we
changed our plan to handle cold accounts as state-compressed accounts,
we don't need a general purposed tiered-storage writer at this moment.


#### Summary of Changes
Remove tiered_storage/writer.rs as we currently don't have plans to develop cold storage.

#### Test Plan
Existing tiered-storage tests.
2024-03-15 22:25:14 -05:00
Yueh-Hsuan Chiang 2a4eb962ca [TieredStorage] Repurpose TieredReadableAccount to HotAccount (#218)
#### Problem
As we further optimize the HotStorageMeta in #146, there is a need
for a HotAccount struct that contains all the hot account information.
Meanwhile, we currently don't have plans to develop a cold account
format at this moment.  As a result, this makes it desirable to repurpose
TieredReadableAccount to HotAccount.

#### Summary of Changes
Repurpose TieredReadableAccount to HotAccount.

#### Test Plan
Existing tiered-storage tests.
2024-03-15 22:25:14 -05:00
steviez 3cf4f1e38a ledger-tool: Allow compute-slot-cost to operate on dead slots (#213)
Make this command accept the --allow-dead-slots arg as well
2024-03-15 22:25:14 -05:00
Dmitri Makarov 9ec8755819 SVM: Eliminate filter_executable_program_accounts from pub interface (#214) 2024-03-15 22:25:14 -05:00
Brennan 5ab9808a9e Loaded program test robustness (#59)
* tests robust to prog entry count
2024-03-15 22:25:14 -05:00