Commit Graph

784 Commits

Author SHA1 Message Date
Brooks 453f272698
Rename IncrementalSnapshotHashes to SnapshotHashes (#31136) 2023-04-11 10:30:29 -04:00
Brooks f3083ad2e0
Rename SnapshotHashes to LegacySnapshotHashes (#31086) 2023-04-10 17:52:20 -04:00
Brennan 861bca87f9
Reconfigure genesis hash download and check (#30949)
* Reconfigure genesis hash download and check

* Propagate get genesis errors
2023-04-10 11:46:56 -07:00
Brennan b3bb4357bb
Add snapshot download metrics (#31111)
* Add snapshot download metrics
2023-04-07 13:19:14 -07:00
behzad nouri ff9a42a354
uses Duration type instead of untyped ..._ms: u64 (#30971) 2023-03-31 15:42:49 +00:00
Brennan b35e83376a
Minimize lock on blacklist nodes (#30987) 2023-03-30 11:08:43 -07:00
Brennan 7804d9aa92
Eliminate in-out parameter for gossip shutdown function (#30986) 2023-03-30 09:25:25 -07:00
steviez cc8e531a5d
Enforce a minimum of 1 on full and incremental snapshot retention (#30968) 2023-03-30 10:16:36 -05:00
Illia Bobyr dcb675eea1
validator: Rename arg: skip-{=>startup-ledger-}verification (#30973)
`--skip-verification` was introduced as a replacement for
`--skip-poh-verify`, but the name does not reflect the important detail,
that this argument only disables verification during the validator
startup.

As the argument was introduced just a few days ago, it seems fine to
rename it "in-place".
2023-03-29 23:34:47 -07:00
Brennan 9f7e041829
Sub function getting vetted RPC nodes (#30946)
* Sub function getting vetted RPC nodes

* PR feedback
2023-03-29 17:22:40 +00:00
Brennan 0679db23ee
Abstract RPC bootstrap gossip exit into function (#30948) 2023-03-29 08:19:30 -07:00
Illia Bobyr 7fbd9e526a
validator: Refactor deprecated arguments logic (#30924)
It is better to have a single list of deprecated arguments.  It is also
very common that an argument is replaced by another single argument -
provide a standard way to describe that.
2023-03-27 22:58:05 -07:00
Illia Bobyr 301a944e51
validator: Move `skip_poh_verify` into the deprecated list (#30885)
Follow-up for

  commit 809041b
  Author: Illia Bobyr <illia.bobyr@solana.com>
  Date:   Wed Mar 22 11:03:30 2023 -0700

      poh_verify => run_verification: Rename to be more accurate (#30811)

      `poh_verify` actually disables transaction signature, tick count and
      built in program argument verifications as well.  It is somewhat
      confusing to call it `poh_verify`.
2023-03-27 16:12:18 -07:00
Ryo Onodera 6c444df9e0
Add --block-{verification,production}-method flags (noop atm) (#30746)
* Add --{replaying,banking}-backend flags (noop atm)

* Greatly simplify enums with strum

* Update programs/sbf/Cargo.lock...

* Rely on Display, removing Debug

* constify cli_names()

* Don't allow omitting bankend value

* Rename to --block-{verification,production}-method

* Use more specific name

* Actually support missing value....

* Remove strictly-unnecessary flags

* Use lazy_static! instead of abusing DefaultArgs...
2023-03-23 12:57:28 +09:00
Illia Bobyr 809041b151
poh_verify => run_verification: Rename to be more accurate (#30811)
`poh_verify` actually disables transaction signature, tick count and
built in program argument verifications as well.  It is somewhat
confusing to call it `poh_verify`.
2023-03-22 11:03:30 -07:00
Ryo Onodera 721719d776
Add solana_clap_utils::hidden_unless_forced() (#30843)
* Add solana_clap_utils::hidden_unless_forced()

* Use more descriptive env name

Co-authored-by: mvines <mvines@gmail.com>

---------

Co-authored-by: mvines <mvines@gmail.com>
2023-03-22 08:59:17 +00:00
Xiang Zhu 8e3a30c22c
Clean orphaned account snapshot dirs (#30645)
* Clean up orphaned account snapshot hardlink dirs

* fix compilation issues

* debugged, now working.  seeing the orphaned directories deleted

* change back to eprintln + exit for account_path error

* changed eprintln to panic for now

* add test_clean_orphaned_account_snapshot_dirs for codecov check

* address a few comments and nit isseus

* directly unzip, skipped the intermediate array of tuples

* let set_up_account_run_and_snapshot_paths return Result

* 'proper' typo, and comment on return

* use map_err

* use for loop in clean_orphaned_account_snapshot_dirs, removed panic

* add test_set_up_account_run_and_snapshot_paths

* minor, replace .for_each with .all

* rename set_up_account_run_and_snapshot_paths to create_all_accounts_run_and_snapshot_dirs

* remove unnecessary closure return type

* change to for loop

* change match to unwrap_or_else

* remove create_dir_all(&account_path) in create_all

* minor comment cleanup
2023-03-17 15:22:10 -07:00
cavemanloverboy 10f49d4e26
Geyser Runtime Reload (#30352)
Support dynamic geyser plugin load, unload, and listing through admin RPC.
2023-03-16 17:03:00 -07:00
Jeff Washington (jwash) 62fe6ea7a0
validators always skip clean/shrink on startup (#30710)
* validators always skip clean/shrink on startup

* move arg to get_deprecated_arguments
2023-03-14 11:41:15 -07:00
Brennan 76eef7ed0d
Require minimum of 2 ticks per slot for test val (#30646) 2023-03-09 08:18:00 -08:00
kirill lykov 5631685d30
replaced chunks with tuples in validator (#30499)
* replaced chunks with tuples in test validator

* update Cargo.lock

* replaced chunks with tuples in genesis

* Update Cargo.lock
2023-03-07 14:32:42 +01:00
Tyera 7b1d446001
Admin RPC Service: move post-init activation to before wait-for-supermajority (#30544)
* Move AdminRpcRequestMetadataPostInit to solana-core

* Move AdminRpcRequestMetadataPostInit write to just before wait_for_supermajority

* Pass AdminRpcRequestMetadataPostInit in TestValidatorGenesis

* Fixup local-cluster
2023-03-01 19:38:11 -07:00
HaoranYi 16db984cb5
improve supermajority waiting logging (#30479)
make logging for supermajority waiting and stake percent from gossip in sync
2023-03-01 08:57:42 -06:00
Tyera 08ca9cc17d
Default solana-test-validator --bpf-programs to upgradeable loader (#30485)
* Dedupe parsing

* Update --bpf-program param to use upgradeable loader, upgrade authority disabled

* Update add_program to use upgreadeable loader

* Deprecate non-upgradeable utils
2023-02-24 09:22:27 -07:00
Tyera fbd8ef5bab
Add solana-test-validator --upgradeable-program (#30412)
* Add TestValidator handling for upgradeable programs

* Plumb --upgradeable-program for solana-test-validator
2023-02-23 18:25:14 +00:00
Yihau Chen df3ef111f7
chore: workspace inheritance (#29893)
* introduce workspace.package

* introduce workspace.dependencies

* read version from root cargo.toml

* pass check when version = { workspace = true }

* don't bump version when version = { workspace = true }

* including workspace Cargo.toml when bump version

* programs/sbf use workspace inheritance

* fix increasing cargo version ignore program/sbf/Cargo.toml
2023-02-23 22:01:54 +08:00
Michael Vines 5136ed3448
Update homepage value for all crates (#30444) 2023-02-23 02:20:18 +00:00
Tyera 47777c47bf
Revert "change test validator from non upgradable bpf loader to upgra… (#30409)
Revert "change test validator from non upgradable bpf loader to upgradable bpf loader (#29051)"

This reverts commit b43eabaa47.
2023-02-20 16:23:20 -07:00
Trent Nelson 0487a4bd60
hide commonly misused `--no-port-check` validator arg (#30306) 2023-02-16 13:34:45 -07:00
skrrb 6717c07af1
test-validator: clone upgradeable programs (#30279)
* test-validator: clone upgradeable programs

* typo & drop bincode dep & clone bound
2023-02-16 16:28:01 +01:00
Brennan ba43883cce
Add timeout for getting bootstrap RPC peers (#30277)
* Add timeout for getting bootstrap RPC peers

* Get RPC return errors when no peers found

* Add config guidance
2023-02-15 14:24:42 -08:00
Trent Nelson 2ba7327027
validator: let `clap` double-dash prefix long args (#30303) 2023-02-13 22:19:13 -07:00
Brennan 5108350710
Remove dead code (#30263)
* Remove dead code
* Get RPC Peers return vector
2023-02-13 12:05:42 -08:00
Trent Nelson 8770b15bb2
remove recommendations to skip validator startup tests on failure (#30250) 2023-02-10 18:14:47 -07:00
Brennan 06617d719c
Replace magic numbers and update comments (#30256) 2023-02-10 23:56:13 +00:00
behzad nouri ded457cd73
embeds the new gossip ContactInfo in ClusterInfo (#30022)
Working towards replacing the legacy gossip contact-info with the new
one, the commit updates the respective field in gossip cluster-info.
2023-02-10 20:07:45 +00:00
Lijun Wang ada6136a6c
Refactor connection cache to support generic msgs (#29774)
tpu-client/tpu_connection_cache is refactored out the module and moved to connection-cache/connection_cache and the logic in client/connection_cache is consolidated to connection-cache/connection_cache as well. client/connection_cache only has a thin wrapper which forward calls to connection-cache/connection_cache and deal with constructions of quic/udp connection cache for clients using them both.2.

The TpuConnection is refactored to ClientConnection to make it generic and functions renamed to be proper for other workflows. eg. tpu_addr -> server_addr, send_transaction --> send_data and etc...

The enum dispatch is removed so that we can make the bulk of code of quic and udp agnostic of each other. The client is possible to load quic or udp only into its runtime.

The generic type parameter in the tpu-client/tpu_connection_cache is removed in order to create both quic and udp connection cache and use the object to send transactions with multiple branching when sending data. The generic type parameters and associated types are dropped in other types in order to make the trait "object safe" for this purpose.

I have annotated the code explaining the reasoning and the refactoring source -> destination.

There is no functional changes

bench-tps has been performed for rpc-client, thin-client and tpu-client. And it is found the performance number largely match the ones before the refactoring.
2023-02-01 18:10:06 -08:00
dependabot[bot] 232e252014
Bump serde from 1.0.144 to 1.0.152 (#29696)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
Co-authored-by: Tyera <tyera@solana.com>
2023-02-01 16:27:17 -07:00
Andrew Fitzgerald d944c657a2
Use RangeBounds for is_within_range (#29763) 2023-02-01 09:17:12 -08:00
Jeff Washington (jwash) d5a7b5715c
add cli arg for choosing ancient append vec format (#30025) 2023-02-01 07:21:28 -08:00
K-anon aa353e4b83
Add Admin RPC Front End for Top N Secondary Index Key Sizes Query. (#29352)
Co-authored-by: K-anon <IntokuSatori@users.noreply.github.com>
2023-01-31 22:46:55 -07:00
Will Hickey 04a6a631bc
Bump version to v1.16 (#30028) 2023-01-31 17:48:33 -06:00
joeaba a12bf8c003
Update maintainers references (#29997)
* update maintainers references

* chore: update maintainers reference
2023-01-31 08:07:13 -05:00
Xiang Zhu 856598969c
Account path add run parent with old path cleanup (#29942)
* Add run parent directory for accounts files

* fix test test_concurrent_snapshot_packaging

* review comments.  renamed the path setup function

* Addressed most of the review comments

* remove explict type def for map result

* handle create_accounts_run_and_snapshot_dirs error with expect

* update with more review comments

* minor fixes from review comments

* simplify account_filename option assignment

* handle error from create_accounts_run_and_snapshot_dirs

* use then instead of then_some for lazy evaluation

* Clean up files in the old account_path before trasitioning to the new run path

* try_exist and accounts_dir removing extra

* sync rmdir, is_dir check

* handle the account_path not deletable case
2023-01-30 10:26:43 -08:00
steviez 3478833ad8
chore: Cleanup repeat instances of solana_version!() (#29944) 2023-01-27 00:44:46 -06:00
Brooks 685c22ff41
Inlines variables into format strings (#29945) 2023-01-27 06:23:03 +00:00
Xiang Zhu 4ebcacb4a3
Revert "Add run parent directory for accounts files (#29794)" (#29899)
This PR is causing OOM on master.  Reverting it for now.

This reverts commit 74f89d1494.
2023-01-25 10:03:01 -08:00
Ryo Onodera 40bbf99c74
Add fully-reproducible online tracer for banking (#29196)
* Add fully-reproducible online tracer for banking

* Don't use eprintln!()...

* Update programs/sbf/Cargo.lock...

* Remove meaningless assert_eq

* Group test-only code under aptly named mod

* Remove needless overflow handling in receive_until

* Delay stat aggregation as it's possible now

* Use Cow to avoid needless heap allocs

* Properly consume metrics action as soon as hold

* Trace UnprocessedTransactionStorage::len() instead

* Loosen joining api over type safety for replaystage

* Introce hash event to override these when simulating

* Use serde_with/serde_as instead of hacky workaround

* Update another Cargo.lock...

* Add detailed comment for Packet::buffer serialize

* Rename sender_overhead_minimized_receiver_loop()

* Use type interference for TraceError

* Another minor rename

* Retire now useless ForEach to simplify code

* Use type alias as much as possible

* Properly translate and propagate tracing errors

* Clarify --enable-banking-trace with better naming

* Consider unclean (signal-based) node restarts..

* Tweak logging and cli

* Remove Bank events as it's not needed anymore

* Make tpu own banking tracer thread

* Reduce diff a bit..

* Use latest serde_with

* Finally use the published rolling-file crate

* Make test code change more consistent

* Revive dead and non-terminating test code path...

* Dispose batches early now that possible

* Split off thread handle very early at ::new()

* Tweak message for TooSmallDirByteLimitl

* Remove too much of indirection

* Remove needless pub from ::channel()

* Clarify test comments

* Avoid needless event creation if tracer is disabled

* Write tests around file rotation and spill-over

* Remove unneeded PathBuf::clone()s...

* Introduce inner struct instead of tuple...

* Remove unused enum BankStatus...

* Avoid .unwrap() for the case of disabled tracer...
2023-01-25 21:54:38 +09:00
Yihau Chen 9193b4221d
Revert "chore: workspace inheritance (#29509)" (#29892)
This reverts commit a67d239dde.
2023-01-25 15:50:41 +08:00
Yihau Chen a67d239dde
chore: workspace inheritance (#29509)
* introduce workspace.package

* introduce workspace.dependencies

* read version from root cargo.toml

* pass check when version = { workspace = true }

* don't bump version when version = { workspace = true }

* including workspace Cargo.toml when bump version

* programs/sbf use workspace inheritance

* fix increasing cargo version ignore program/sbf/Cargo.toml
2023-01-25 13:59:59 +08:00