Commit Graph

24871 Commits

Author SHA1 Message Date
Brooks 140c21f8a9
Removes ReadAccountMapEntry (#35351) 2024-02-28 16:08:00 -05:00
Brooks 6aaaf858c9
Adds more info to panic message in AccountsHashVerifier (#35353) 2024-02-28 15:55:05 -05:00
behzad nouri a7a41e7631
adds Merkle shred variant with retransmitter's signature (#35293)
Moving towards locking down Turbine propagation path, the commit
reserves a buffer within shred payload for retransmitter's signature.
2024-02-28 20:31:40 +00:00
Greg Cusack 98ec72e6ed
change default `bench-tps` client to `tpu-client` (#35335)
* change default bench-tps client to tpu-client

* remote client default to tpu-client

* add --use-tpu-client back in. hide --use-thin-client

* address nit, inform of future thinclient deprecation
2024-02-28 12:30:24 -08:00
steviez 140818221c
Rename SamplePerformanceService thread for consistency (#35332)
- Rename thread
- Add uniform service start/stop logs
- Misc cleanup with variables / constants / exit flag check
2024-02-28 13:47:27 -06:00
Brooks 7c48cbb7aa
Replaces InMemAccountsIndex::get() with AccountsIndex::get_cloned() (#35352) 2024-02-28 14:45:08 -05:00
Brooks 6402198902
Replaces ReadAccountMapEntry in calculate_accounts_hash_from_index() (#35349) 2024-02-28 14:38:19 -05:00
Lucas Steuernagel 8f3e960640
Add tests for private functions in SVM `account_loader.rs` (#35334) 2024-02-28 14:17:45 -03:00
Brooks f340c1c181
Replaces ReadAccountMapEntry in do_scan_secondary_index() (#35219) 2024-02-28 11:43:33 -05:00
dependabot[bot] 695fe1e7c8
build(deps): bump ahash from 0.8.9 to 0.8.10 (#35347)
* build(deps): bump ahash from 0.8.9 to 0.8.10

Bumps [ahash](https://github.com/tkaitchuck/ahash) from 0.8.9 to 0.8.10.
- [Release notes](https://github.com/tkaitchuck/ahash/releases)
- [Commits](https://github.com/tkaitchuck/ahash/compare/v0.8.9...v0.8.10)

---
updated-dependencies:
- dependency-name: ahash
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2024-02-28 23:38:18 +08:00
Tao Zhu 089cead024
add bench for precompiled programs (#35310)
* add bench for ed25519 instruction

* add bench for secp256k1 instruction

* Apply suggestions from code review

Co-authored-by: Andrew Fitzgerald <apfitzge@gmail.com>

* prepare unique txs for benching

* use iter::Cycle for endless loop

---------

Co-authored-by: Andrew Fitzgerald <apfitzge@gmail.com>
2024-02-28 09:06:55 -06:00
Alexander Meißner e6f8cdce01
Refactor - `LoadedPrograms::assign_program()` (#35233)
* Forbids all program replacements except for reloads and builtins.

* Adds test_assign_program_failure() and test_assign_program_success().

* Explicitly disallows LoadedProgramType::DelayVisibility to be inserted in the global cache.
2024-02-28 09:20:11 +01:00
dependabot[bot] 6ee3bb973c
build(deps): bump tempfile from 3.10.0 to 3.10.1 (#35328)
* build(deps): bump tempfile from 3.10.0 to 3.10.1

Bumps [tempfile](https://github.com/Stebalien/tempfile) from 3.10.0 to 3.10.1.
- [Changelog](https://github.com/Stebalien/tempfile/blob/master/CHANGELOG.md)
- [Commits](https://github.com/Stebalien/tempfile/compare/v3.10.0...v3.10.1)

---
updated-dependencies:
- dependency-name: tempfile
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2024-02-28 11:36:14 +08:00
Brooks a4e1a9ac98
Adds AccountsIndex::get_account_info_with_and_then() (#35336) 2024-02-27 20:01:29 -05:00
Brooks da088681ba
Adds safer alternatives to get_internal() (#35325) 2024-02-27 18:08:25 -05:00
steviez 94698b8dd0
Name PubSubService tokio threads (#35331)
Also add logs for service starting/stopping
2024-02-27 14:59:07 -06:00
Lucas Steuernagel 8be9930c98
Bump platform tools version (#35330)
Co-authored-by: Dmitri Makarov <dmakarov@users.noreply.github.com>
2024-02-27 17:58:26 -03:00
Kirill Fomichev 8ad125d0c0
rpc: optimize `getTokenLargestAccounts` (#35315)
* rpc: optimize `getTokenLargestAccounts`

* use tuple instead of struct

* untuple

Co-authored-by: Tyera <teulberg@gmail.com>

---------

Co-authored-by: Tyera <teulberg@gmail.com>
2024-02-27 00:08:29 -07:00
steviez 09925a11eb
Remove the Blockstore thread pool used for fetching Entries (#34768)
There are several cases for fetching entries from the Blockstore:
- Fetching entries for block replay
- Fetching entries for CompletedDataSetService
- Fetching entries to service RPC getBlock requests

All of these operations occur in a different calling thread. However,
the currently implementation utilizes a shared thread-pool within the
Blockstore function. There are several problems with this:
- The thread pool is shared between all of the listed cases, despite
  block replay being the most critical. These other services shouldn't
  be able to interfere with block replay
- The thread pool is overprovisioned for the average use; thread
  utilization on both regular validators and RPC nodes shows that many
  of the thread see very little activity. But, these thread existing
  introduce "accounting" overhead
- rocksdb exposes an API to fetch multiple items at once, potentially
  with some parallelization under the hood. Using parallelization in
  our API and the underlying rocksdb is overkill and we're doing more
  damage than good.

This change removes that threadpool completely, and instead fetches
all of the desired entries in a single call. This has been observed
to have a minor degradation on the time spent within the Blockstore
get_slot_entries_with_shred_info() function. Namely, some buffer
copying and deserialization that previously occurred in parallel now
occur serially.

However, the metric that tracks the amount of time spent replaying
blocks (inclusive of fetch) is unchanged. Thus, despite spending
marginally more time to fetch/copy/deserialize with only a single
thread, the gains from not thrashing everything else with the pool
keep us at parity.
2024-02-26 20:27:03 -06:00
Brooks bf2e8ee32f
AccountsIndex::get_cloned() *must* add entry to in-mem cache (#35322) 2024-02-26 18:20:21 -05:00
Brooks 8143fc3f4a
Replaces ReadAccountMapEntry in read_index_for_accessor_or_load_slow() (#35220) 2024-02-26 14:19:18 -05:00
Brooks 8235feefc4
Removes get_for_tests() (#35311) 2024-02-26 14:17:33 -05:00
behzad nouri 0ab425b43b
splits test_shred_variant_compat into separate test-cases (#35306) 2024-02-26 17:32:47 +00:00
behzad nouri c8ee4f59ad
uses struct instead of tuple for Merkle shreds variant (#35303)
Working towards adding a new Merkle shred variant with retransmitter's
signature, the commit uses struct instead of tuple to describe Merkle shred
variant.
2024-02-26 15:58:40 +00:00
dependabot[bot] 6c6e691c80
build(deps): bump socket2 from 0.5.5 to 0.5.6 (#35318)
* build(deps): bump socket2 from 0.5.5 to 0.5.6

Bumps [socket2](https://github.com/rust-lang/socket2) from 0.5.5 to 0.5.6.
- [Release notes](https://github.com/rust-lang/socket2/releases)
- [Changelog](https://github.com/rust-lang/socket2/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/socket2/compare/v0.5.5...v0.5.6)

---
updated-dependencies:
- dependency-name: socket2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2024-02-26 23:04:25 +08:00
dependabot[bot] 2fd2f34b35
build(deps): bump syn from 2.0.50 to 2.0.51 (#35317)
* build(deps): bump syn from 2.0.50 to 2.0.51

Bumps [syn](https://github.com/dtolnay/syn) from 2.0.50 to 2.0.51.
- [Release notes](https://github.com/dtolnay/syn/releases)
- [Commits](https://github.com/dtolnay/syn/compare/2.0.50...2.0.51)

---
updated-dependencies:
- dependency-name: syn
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2024-02-26 23:03:33 +08:00
Michal Rostecki e74d5ccca3
hash: Use `finalize().into()` instead of `try_into()` for hash results (#35300)
`sha2` and `sha3` crates already moved to `generic-array` 0.14.7,
which means that we can safely convert the hash result to a sized
array just by calling `finalize().into()`, which doesn't return
any errors.
2024-02-23 19:31:32 -07:00
Andrew Fitzgerald 9f581113bd
Scheduler: Leader-Slot metrics for Scheduler (#35087) 2024-02-23 17:06:22 -08:00
Brooks 54706a885b
Adds get_with_and_then() to AccountsIndex (#35307) 2024-02-23 18:48:36 -05:00
Brooks 1e47aacd0d
Removes get_account_read_entry() (#35309) 2024-02-23 22:25:30 +00:00
Brooks 58c1b7aefb
Replaces ReadAccountMapEntry in more tests (#35308) 2024-02-23 21:20:42 +00:00
Lucas Steuernagel ec63b813f5
Move account filter test to SVM (#35304) 2024-02-23 17:44:26 -03:00
enjoyoor c02f47a6fb
fix: cleanup (#35298) 2024-02-23 19:59:52 +00:00
Greg Cusack fe571bbab3
Plumb `CommitmentConfig` through `bench-tps` client types (#35282)
* use --commitment-config <commitment-level> for setting blockhash commitment level for sending transactions with rpc-client

* clarify default

* leave get_balance_with_commitment at processed()

* rm unused variable

* refactor commitment_config flag read in

* update cli and change send_batch's get_latest_blockhash() to get_latest_blockhash_with_client_commitment() and use client's internal commitment level

* change fix some nits based on PR comments

* rm unused import
2024-02-23 11:50:41 -08:00
Andrew Fitzgerald 72734a9539
move changelog note of default central-scheduler (#35217) 2024-02-23 11:13:23 -08:00
Alexander Meißner 74758d9fbf
Refactor - Move recompilation out of program loading (#35297)
Moves recompilation specifics out of load_program().
2024-02-23 17:25:32 +01:00
Brooks 7da8d82aa1
Adds snapshot_utils::purge_all_bank_snapshots() (#35291) 2024-02-23 11:15:10 -05:00
Brooks 923cac8d7f
Replaces ReadAccountMapEntry in exhaustively_verify_refcounts() (#35243) 2024-02-23 11:13:57 -05:00
Brooks b5ffc24a38
Replaces ReadAccountMapEntry in retry_to_get_account_accessor() (#35244) 2024-02-23 11:12:24 -05:00
Tao Zhu 139b9c8c25
Add fee_details to fee calculation (#35021)
* add fee_details to fee calculation

* fix - no need to round after summing u64

* feature gate on removing unwanted rounding
2024-02-23 08:58:48 -06:00
Alexander Meißner 2891ce886b
Fix - program loading with effective slot at epoch boundary (#35283)
* Always limit effective slot to the begin of the current epoch.

* Adds comments.

* Optimizes to avoid having two entries if there is no relevant feature activation.

* Adds test_feature_activation_loaded_programs_epoch_transition().
2024-02-23 15:15:28 +01:00
Andrew Fitzgerald 367f489f63
scheduler inner metrics (#35271) 2024-02-22 15:01:08 -08:00
Lucas Steuernagel 31a73ab731
Move `test_rent_state_list_len` to SVM (#35287) 2024-02-22 18:37:09 -03:00
steviez d9802027cb
genesis: Skip inserting genesis accounts for Development clusters (#35266)
solana-genesis currently includes a list of accounts that exist in
MainnetBeta genesis. These accounts are added for all cluster types,
including Development clusters.

There is no need for these accounts to get added to dev clusters so
skip adding them for ClusterType::Development case
2024-02-22 14:42:35 -06:00
Illia Bobyr c98ff7a6da
loaded-programs: Remove unnecessary `allow(unused_mut)` (#35284)
Only the `executable` might need to be mutable.  This
`allow(unused_mut)` is probably a leftover from a previous version.
2024-02-22 12:11:00 -08:00
dependabot[bot] 3284d34e04
build(deps): bump openssl from 0.10.63 to 0.10.64 (#35251)
* build(deps): bump openssl from 0.10.63 to 0.10.64

Bumps [openssl](https://github.com/sfackler/rust-openssl) from 0.10.63 to 0.10.64.
- [Release notes](https://github.com/sfackler/rust-openssl/releases)
- [Commits](https://github.com/sfackler/rust-openssl/compare/openssl-v0.10.63...openssl-v0.10.64)

---
updated-dependencies:
- dependency-name: openssl
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2024-02-22 20:12:59 +08:00
steviez bfcd4c8656
ledger-tool: Use error handling in blockstore command code (#35157)
There are lots of operations that could fail, including lots of the
Blockstore calls. The old code matched on Ok(_) or did unwrap()'s
which clutter the code and increase indentation.

This change wraps the entire command in a function that returns a
Result. The wrapper then does a single unwrap_or_else() and prints
any error message. Everywhere else is now free to use the ? operator
2024-02-22 00:19:28 -06:00
Ashwin Sekar 07955e79ad
replay: gracefully exit if tower load fails (#35269) 2024-02-21 18:51:30 -08:00
nathan haim 60ccdb3fb4
cli: program set-upgrade-authority sign-only (#35203)
* cli: program set-upgrade-authority
Add --sign-only flag

* Apply suggestions from code review

This fixes nits.
It modifies tests: it will sign offline with `newAuthority` keypair instead of fee payer. Then it reuses the signature in the next command.

Co-authored-by: Jon C <me@jonc.dev>

* cli/tests/program: fix tests

* cli/src/program: delete unsed import

* fix to pass ci cargo-clippy-nightly

---------

Co-authored-by: Jon C <me@jonc.dev>
2024-02-22 02:07:00 +01:00
Ryo Onodera 024d6ecc4f
Add --unified-scheduler-handler-threads (#35195)
* Add --unified-scheduler-handler-threads

* Adjust value name

* Warn if the flag was ignored

* Tweak message a bit
2024-02-22 09:05:17 +09:00