Commit Graph

21481 Commits

Author SHA1 Message Date
Jeff Washington (jwash) 3fc4015a4d
do_shrink_slot_stores -> do_shrink_slot_store (#29552) 2023-01-06 09:23:32 -08:00
Jeff Washington (jwash) 39dc034c05
ShrinkCandidates only hold 1 append vec per slot (#29525)
* ShrinkCandidates only hold 1 append vec per slot

* Fixed select_candidates_by_total_usage unit tests, use distinct slots for stores
as each slot has one store now.

* rename

Co-authored-by: Lijun Wang <83639177+lijunwangs@users.noreply.github.com>
2023-01-06 06:54:34 -08:00
HaoranYi b3a3e03c6a
Refresh blockhash for retrials of failed get_fee calls (#29494)
* add log for error for failed rpc calls

* refresh blockhash when retry get_fee_for_message

* clippy
2023-01-06 08:53:38 -06:00
Jeff Washington (jwash) 272e5a05f2
remove AccountStorage::is_empty() (#29548) 2023-01-06 06:51:25 -08:00
Yihau Chen b030a96321
chore: increase stable-perf timeout_in_minutes to 35 (#29551) 2023-01-06 17:27:25 +08:00
Jeff Washington (jwash) 8e19909c7d
stop returning remaining # stores from mark_dirty_dead_stores (#29545) 2023-01-05 19:21:01 -08:00
Tyera 7689044521
Fix solana-stake-accounts version (#29542)
Fix clap inputs
2023-01-05 19:49:28 -07:00
Xiang Zhu a0f045d2c3
storage rebuilder regex cleanup (#29408)
* storage rebuilder regex cleanup

* Update runtime/src/snapshot_utils/snapshot_storage_rebuilder.rs

Co-authored-by: apfitzge <apfitzge@gmail.com>

Co-authored-by: apfitzge <apfitzge@gmail.com>
2023-01-05 16:16:24 -08:00
Lijun Wang a6b492b2c1
Removed assert on write_version ordering (#29530)
Removed assert on write_version ordering as snapshot created by earlier version
is not honoring that.
2023-01-05 15:14:55 -08:00
Xiang Zhu a6bf68cec8
Refine appendvec sanitize error message to include path (#29541) 2023-01-05 14:43:10 -08:00
Jeff Washington (jwash) 72a7cbd6a2
shrink fns take a single append vec (#29522) 2023-01-05 13:17:04 -08:00
behzad nouri b71cb9d9c7
simplifies sigverify copy_return_values (#29495) 2023-01-05 19:45:52 +00:00
Will Hickey 0581fc2def
Update CI pipeline to only run `checks` step on version bump PRs (#29243)
* Add logic to buildkite pipeline so version bump PRs don't run the full CI
2023-01-05 12:45:29 -06:00
behzad nouri 12da2da389
fixes errors from clippy::redundant_clone (#29536)
https://rust-lang.github.io/rust-clippy/master/index.html#redundant_clone
2023-01-05 18:42:19 +00:00
Jeff Washington (jwash) 3b7ebfe50e
migrate tests to use get_storage_for_slot (#29518) 2023-01-05 10:38:58 -08:00
Jeff Washington (jwash) d3f45c6901
shrink_slot_forced uses a single append vec (#29521) 2023-01-05 10:38:05 -08:00
behzad nouri d87128e02c
fixes errors from clippy::needless_borrow (#29535)
https://rust-lang.github.io/rust-clippy/master/index.html#needless_borrow
2023-01-05 18:21:56 +00:00
Brooks d611cc2e75
Fixes format string (#29533) 2023-01-05 13:06:00 -05:00
behzad nouri 5c9beef498
fixes errors from clippy::useless_conversion (#29534)
https://rust-lang.github.io/rust-clippy/master/index.html#useless_conversion
2023-01-05 18:05:32 +00:00
Lijun Wang 1e8a8e07b6
Stream the executed transaction count in the block notification (#29272)
Problem

The plugins need to know when all transactions for a block have been all notified to serve getBlock request correctly. As block and transaction notifications are sent asynchronously to each other it will be difficult.

Summary of Changes

Include the executed transaction count in block notification which can be used to check if all transactions have been notified.
2023-01-05 09:36:19 -08:00
Jeff Washington (jwash) dbac798ef6
process_storage_slot takes a single append vec (#29519) 2023-01-05 07:33:47 -08:00
Jeff Washington (jwash) a140f06a67
use get_storage_for_slot() in tests (#29517) 2023-01-05 07:32:00 -08:00
Jeff Washington (jwash) 8cdf5cdb50
get_storage_to_move_to_ancient_append_vec returns a single append vec (#29482)
get_storages_to_move_to_ancient_append_vec returns a single append vec
2023-01-05 07:31:26 -08:00
Jeff Washington (jwash) 221f547ba9
remove store_ids from a few shrink data structures (#29360) 2023-01-05 07:30:43 -08:00
Brooks 05d5aed4d8
Adds SnapshotError::IoWithSourceAndFile (#29527) 2023-01-05 09:08:15 -05:00
Jeff Biseda 832302485e
require repair request signature, ping/pong for Testnet, Development clusters (#29351) 2023-01-04 14:54:19 -08:00
Jeff Washington (jwash) fc8cd82309
migrate tests to get_slot_storage_entry (#29515) 2023-01-04 13:12:48 -08:00
Jeff Washington (jwash) 4e636ab153
cleanup ancient append vec tests (#29514) 2023-01-04 12:39:42 -08:00
Jeff Washington (jwash) 150ac80cfa
filter_storages ignores reclaims (#29512) 2023-01-04 12:38:30 -08:00
Jeff Washington (jwash) ef72140173
remove unnecessary to_vec() (#29516) 2023-01-04 12:38:16 -08:00
Jon Cinque aa593a31f8
logger: Update to env_logger 0.9.3 (#29510) 2023-01-04 21:35:37 +01:00
Jeff Washington (jwash) 98dc3db4d1
make rpc test tolerant of rent_epoch being set to max (#29508) 2023-01-04 12:15:46 -08:00
Tyera c479a6409a
Fix TransactionPrecompileVerification RPC error (#29490) 2023-01-04 12:46:13 -07:00
hana 023509ff1b
solana-install: check for fixed releases directly (#29365)
when initializing, if a specific release is requested, we only need to confirm it exists
this can be done with the download url itself, rather than pulling the list of releases
2023-01-04 11:28:05 -08:00
Michael Vines 6ae2eb3eb9
Rework method for reporting security problems (#29511) 2023-01-04 11:00:26 -08:00
Jeff Washington (jwash) 093b1a5e10
get_storages_for_slot uses get_slot_storage_entry (#29498) 2023-01-04 10:15:56 -08:00
Ikko Ashimine f505c2d233
Fix typo in blockstore_metrics.rs (#29503)
embeded -> embedded
2023-01-04 09:44:45 -07:00
Jeff Washington (jwash) 10135dbd10
should_move_to_ancient_append_vec works with a single storage (#29484) 2023-01-04 07:03:46 -08:00
behzad nouri 9db25655f7
recovers merkle roots from shreds binary in {verify,sign}_shreds_gpu (#29445)
{verify,sign}_shreds_gpu need to point to offsets within the packets for
the signed data. For merkle shreds this signed data is the merkle root
of the erasure batch and this would necessitate embedding the merkle
roots in the shreds payload.
However this is wasteful and reduces shreds capacity to store data
because the merkle root can already be recovered from the encoded merkle
proof.

Instead of pointing to offsets within the shreds payload, this commit
recovers merkle roots from the merkle proofs and stores them in an
allocated buffer. {verify,sign}_shreds_gpu would then point to offsets
within this new buffer for the respective signed data.

This would unblock us from removing merkle roots from shreds payload
which would save capacity to send more data with each shred.
2023-01-04 14:20:05 +00:00
Yihau Chen d078c041d1
chore: add missing members back to workspace.members (#29450)
* frozen-abi/macro

* program-runtime

* sdk/macro

* sdk/program

* storage-bigtable/build-proto

* fix sorting
2023-01-04 13:22:00 +08:00
Illia Bobyr d7bd1bf970
bank: Record non-vote transaction count (#29383)
A subsequent change to `SamplePerformanceService` introduces non-vote transaction counts, which `bank`s need to store.

Part of work on https://github.com/solana-labs/solana/issues/29159
2023-01-03 14:46:20 -08:00
Jeff Washington (jwash) 89deecb979
add AccountStorage.is_empty_entry for tests (#29489) 2023-01-03 12:36:31 -08:00
Jeff Washington (jwash) 0f2bfa2b02
cleanup get_snapshot_storages (#29488)
* cleanup get_snapshot_storages

* pr feedback
2023-01-03 12:22:28 -08:00
Jeff Washington (jwash) 95d2f002bd
add AccountStorage.get_slot_storage_entry (#29480) 2023-01-03 12:10:34 -08:00
behzad nouri e5323166b3
dedups gossip addresses, taking the one with highest weight (#29421)
dedups gossip addresses, keeping only the one with the highest weight

In order to avoid traffic congestion or sending duplicate packets, when
sampling gossip nodes if several nodes have the same gossip address
(because they are behind a relayer or whatever), they need to be
deduplicated into one.
2023-01-03 19:37:43 +00:00
HaoranYi d22c1d1ce4
Add retries for get_latest_blockhash for accounts cluster bench (#29456)
* retry get_latest_blockhash

* more retries for get_latest_blockhash

* retry for get_fee too

* clippy

* fmt

* sleep

* Update accounts-cluster-bench/src/main.rs

Co-authored-by: Tyera <teulberg@gmail.com>

* Update accounts-cluster-bench/src/main.rs

Co-authored-by: Tyera <teulberg@gmail.com>

* rename

Co-authored-by: Tyera <teulberg@gmail.com>
2023-01-03 13:35:12 -06:00
HaoranYi 4a8b980b81
update leger tool help for db verify refcounts (#29486) 2023-01-03 13:34:06 -06:00
HaoranYi 95ecd38af5
typo (#29485) 2023-01-03 13:32:45 -06:00
Jeff Washington (jwash) 9ee4555f53
remove should_retain from mark_dirty_dead_stores (#29358) 2023-01-03 09:53:43 -08:00
Jeff Washington (jwash) 102dc08378
get_snapshot_storages removes call to AccountStorage.get (#29466) 2023-01-03 09:15:07 -08:00