Commit Graph

24246 Commits

Author SHA1 Message Date
behzad nouri d5eee01950
adds feature gated code to drop legacy shreds (#34328) 2023-12-06 22:47:46 +00:00
samkim-crypto d21e7edd2b
[zk-token-sdk] Fix transfer with fee edge case error (#34314) 2023-12-07 06:46:57 +09:00
Alexander Meißner cf0c52c207
Cleanup - program runtime (#34318)
* Removes retain() from prune_by_deployment_slot() as that is covered in remove_programs_with_no_entries() already.

* Removes remove_programs_with_no_entries() from sort_and_unload().

* Fixes ix_usage_counter in LoadedProgram::to_unloaded().

* Fixes doc comment of LoadedProgram::ix_usage_counter.

* Removes unused num_total_programs from test_eviction().

* Replaces .as_ref() in lambda with std::option::Option::as_ref.

* Replaces .for_each() with a for-loop.

* Uses .retain() instead of iter().filter().cloned().collect().
2023-12-06 22:05:26 +01:00
behzad nouri b1c701e53b
patches slots_per_epoch vs last_slot ordering in shred_fetch_stage (#34338) 2023-12-06 20:47:53 +00:00
Brooks 61fa130175
Removes glob import of SnapshotKind (#34333) 2023-12-06 13:46:10 -05:00
Brooks b32903a534
Removes glob import of AccountsHashVerificationError (#34332) 2023-12-06 13:46:00 -05:00
Brooks 49ffe42661
Removes glob import of ApplyFeatureActivationsCaller (#34334) 2023-12-06 12:48:25 -05:00
Lucas Steuernagel 1877fdb273
Use BankForks on tests - Part 4 (#34271)
* Use BankForks on tests - Part 4

* Ensure the correct slot is set
2023-12-06 13:32:04 -03:00
Yihau Chen fa251768b3
ci: block auto-merge before SPL downstream tests are all green (#34326)
* ci: block auto-merge before all spl downstream tests are green

* follow Github Actions triggering rule
2023-12-06 13:39:28 +08:00
dependabot[bot] 821968d3d0
build(deps): bump openssl from 0.10.60 to 0.10.61 (#34315)
* build(deps): bump openssl from 0.10.60 to 0.10.61

Bumps [openssl](https://github.com/sfackler/rust-openssl) from 0.10.60 to 0.10.61.
- [Release notes](https://github.com/sfackler/rust-openssl/releases)
- [Commits](https://github.com/sfackler/rust-openssl/compare/openssl-v0.10.60...openssl-v0.10.61)

---
updated-dependencies:
- dependency-name: openssl
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2023-12-06 13:31:17 +08:00
Brooks 5c00a33252
Removes unnecessary #[allow(dead_code)] in tiered storage (#34320) 2023-12-06 00:23:01 +00:00
Alexander Meißner f70d5481bc
Fix - Uses `fetch_add()` to accumulate usage counters in `LoadedPrograms` (#34319)
Uses fetch_add() to accumulate usage counters.
2023-12-05 23:21:06 +01:00
Steven Luscher 70896b8999
Link to latest version of the off-chain message signing proposal in the docs (#34329) 2023-12-05 21:49:12 +00:00
Kirill Fomichev c82fc6c98e
geyser: allow custom name in config file (#33550)
Allow loading the plugin name from the json config file as opposed to use plugin.name which is called before config file is passed to it. Allowing different plugins using the same executable to use different names.
2023-12-05 12:07:02 -08:00
behzad nouri 03fbe083b8
exits send_datagram_task if the connection is closed (#33836)
Waiting on receiver.recv() can unnecessarily block while the connection is already closed.
The commit exits send_datagram_task if the connection is closed.
2023-12-05 18:29:36 +00:00
Yihau Chen bbb58c20d2
ci: block automerge until clippy (macos-latest) green (#34325) 2023-12-06 01:51:44 +08:00
Brooks 62681b1089
Removes unnecessary #[allow(dead_code)] in bucket map (#34323) 2023-12-05 16:41:32 +00:00
Brooks 20d33a5274
Removes unnecessary #[allow(dead_code)] in stake flags (#34322) 2023-12-05 16:12:22 +00:00
behzad nouri 2096626633
exits send_requests_task if the connection is closed (#33837)
receiver.recv() can unnecessarily block when the connection is already closed.
The commit exits send_requests_task if the connection is closed.
2023-12-05 16:08:06 +00:00
Brooks c971ad2944
Removes unnecessary #[allow(dead_code)] in partitioned rewards (#34321) 2023-12-05 15:58:11 +00:00
Jeff Washington (jwash) 091fc820ba
ancient append vec append creates big enough ancient append vecs (#34154)
* ancient append vec append creates big enough ancient append vecs

* assert we write the correct # of bytes

* fix tests

* u64 align accounts bytes written

* fix build

* add tests

* we don't need to align the store to u64.

When we call `remaining_bytes` to calculate, for the last element, u64_align!(len) could be greater than capacity.  However, because saturate_sub is used, we still get the correct answer of 0. Therefore, no need to align the storage to u64.

* add comments

* add shink test for min_size

* remove dead code

* fix build

* reviews

* add test for squash ancient vec overflow too much to fit into ideal av

* add test for get_bytes on AccountToStore

* clippy

* modify test to assert 0 size of opposite

---------

Co-authored-by: HaoranYi <haoran.yi@solana.com>
2023-12-05 09:25:11 -06:00
norwnd f41badfc19
cli: (program) move buffer fetching/validation into separate fn (#34317)
* cli: (program) move buffer fetching/validation into separate fn

* error gracefully (don't panic)

---------

Co-authored-by: norwnd <norwnd>
2023-12-05 16:11:29 +01:00
norwnd 8d7e63d420
cli: (program) enhance test_cli_program_set_buffer_authority (#34316)
* cli: (program) enhance test_cli_program_set_buffer_authority

* actually use buffer identity keys as buffer authority in tests(since it's a common real-world usage pattern)

---------

Co-authored-by: norwnd <norwnd>
2023-12-05 13:51:46 +01:00
Andrew Fitzgerald d451675d50
TransactionScheduler: check transaction age and already processed during receive (#34253) 2023-12-04 14:41:05 -08:00
Brooks 2c033e05e5
Switches order of tiered storage footer size and version (#34310) 2023-12-04 14:23:56 -05:00
Brooks 4bba59f063
Rebuilds skipped rewrites when loading from a snapshot (#34280) 2023-12-04 14:23:15 -05:00
Brooks efaec08f88
Clones skipped rewrites instead of taking (#34311) 2023-12-04 12:49:05 -05:00
Brooks 8fbe033eac
Moves val into .extend() instead of &mut and .into_iter() (#34285) 2023-12-01 14:38:01 -05:00
steviez c9cb727b1f
Add additional context to ReplayStage log message (#34296)
The pre-existing message was somewhat vague, so give more details about the caller
2023-12-01 13:04:39 -06:00
Brooks 4f65f7dc8d
Sanitizes tiered storage footer after reading from disk (#34200) 2023-12-01 13:58:10 -05:00
Yueh-Hsuan Chiang fa0930f254
[TieredStorage] Allow HotStorage to handle more account data (#34155)
#### Problem
After we have defined AccountOffset to be u32, it means the address space within
one HotAccountsFile is up to 4GB.  However, since the Accounts Blocks in a
HotAccountsFile is 8-byte aligned, it has the opportunity to store more data by
internally multiplying the AccountOffset by 8.

#### Summary of Changes
This PR allows a HotAccountsFile to store up to 32GB accounts data by using
8 x AccountOffset as its actual offset.

#### Test Plan
Updated existing unit-tests.
2023-12-01 10:39:15 -08:00
Lucas Steuernagel b97b3dd4ab
Use BankForks on tests - Part 3 (#34248)
* Add BankForks to core tests

* Refactor functions under DCOU
2023-12-01 13:47:22 -03:00
Andrew Fitzgerald 2294801954
Do not derive Copy for EpochSchedule and Rent (#32767) 2023-12-01 07:57:25 -08:00
Tyera 68928bb7d6
solana-cli: format strings (#34288)
* Format strings in solana-cli

* Restore some wallet.rs strings that were better before

* Fixup vote.rs string

* Fixup stake.rs string

* Fixup program.rs strings

* Fixup some address_lookup_table.rs strings

* Fixup clap_app.rs string

* Clarify validator_info help

* Fixup cluster_query.rs
2023-12-01 09:00:14 +00:00
steviez 7c4e7230b0
ledger-tool: Run rustfmt with format_strings = true (#34284)
Long string literals can cause rustfmt to fail, which results in rustfmt
failing to format entire functions. There are several instances of this
in ledger-tool, so format these files with wrapped strings so that
formatting will apply to functions again.

Note that this PR was created by adding format_strings = true to
rustfmt.toml; however, this change does NOT persist that rule as the
rule would format the entire repo.
2023-12-01 02:50:18 -06:00
Ashwin Sekar d84dcd37bc
blockstore: use u32 for fec_set_index in erasure set index store key (#34268)
* blockstore: use u32 for fec_set_index in erasure set index store key

* pr feedback u64::from
2023-11-30 19:17:49 -05:00
Andrew Fitzgerald 18309ba8da
TransactionScheduler: Schedule Filter (#34252) 2023-11-30 14:41:11 -08:00
steviez 479b7ee9f2
Bubble up errors in bank_fork_utils instead of exiting process (#34277)
There are operations in bank_fork_utils that may fail; we explicitly
call std::process::exit() on several of these. Granted we may end up
exiting the process higher up the callstack, bubbling the errors up
allow a caller that could handle the error to do so.
2023-11-30 16:35:59 -06:00
Tyera c5368a3d35
ledger-tool: read bigtable entries (#34266)
* Add helper to get a protobuf cell without bothering with bincode

* Add conversion back to EntrySummaries

* Add LedgerStorage::get_block_entries

* Add ledger-tool bigtable entries subcommand

* Move CliEntries into ledger_tool::output for now

* Fetch entries with read-only token
2023-11-30 21:33:04 +00:00
steviez 71c1782c74
Allow Blockstore to open unknown columns (#34174)
As we develop new features or modifications, we occassionally need to
introduce new columns to the Blockstore. Adding a new column introduces
a compatibility break given that opening the database in Primary mode
(R/W access) requires opening all columns. Reverting to an old software
version that is unaware of the new column is obviously problematic.

In the past, we have addressed by backporting minimal "stub" PR's to
older versions. This is annoying, and only allow compatibility for the
single version or two that we backport to.

This PR adds a change to automatically detect all columns, and create
default column descriptors for columns we were unaware of. As a result,
older software versions can open a Blockstore that was modified by a
newer software version, even if that new version added columns that the
old version is unaware of.
2023-11-30 13:24:56 -06:00
Tyera 834a3968ef
Fix patch comment typos and wrap lines (#34251)
* Comment nits

* Wrap to 80 chars

* Wrap some more comment lines
2023-11-30 12:18:54 -07:00
Yihau Chen 4832b4ecec
ci: add macos clippy test (#34272)
* ci: add clippy test for macos and windows

* ci: remove windows from clippy test

* ci: cancel cargo clippy on Github Actions when new commits comming

* ci: set -e for .github/scripts/cargo-clippy-before-script.sh
2023-12-01 00:59:45 +08:00
Tyera 5085ee9b1a
Only compile bench_program_alu for x86_64 (#34282)
Ignore bench if not x86_64
2023-11-30 07:48:08 -07:00
Tao Zhu c3323c0c36
Add test module to runtime-transaction (#34273)
* basic tests
2023-11-29 18:30:59 -06:00
steviez 935e06f8f1
Output BankHashDetails file when leader drops its' own block (#34256)
Currently, the file is generated when a node drops a block that was
produced by another node. However, it would also be beneficial to see
the account state when a node drops its' own block.

Output the file in this additional failure codepath
2023-11-29 17:20:27 -06:00
Andrew Fitzgerald e949ef9daa
Trailing _us for scheduler time metrics (#34263) 2023-11-29 13:18:56 -08:00
Lijun Wang f211c86e69
Support logging in geyser plugins (#34101)
The change creates a new interface in the GeyserPlugin interface to allow the runtime to pass the logging configuration to the plugin.
2023-11-29 11:31:11 -08:00
Ashwin Sekar 60c09d3d89
local-cluster: ignore test_duplicate_shreds_switch_failure (#34267) 2023-11-29 17:34:57 +00:00
Lucas Steuernagel aeb4a348fb
Use BankForks on tests - Part 2 (#34234)
* Use BankForks on tests - Part 2

Signed-off-by: Lucas Steuernagel <lucas.tnagel@gmail.com>
2023-11-29 14:28:31 -03:00
Ashwin Sekar e1165aaf00
blockstore: populate merkle root metas column (#34097) 2023-11-29 11:14:24 -05:00