Commit Graph

118 Commits

Author SHA1 Message Date
Pankaj Garg f4287d70bb
Move accounts-db code to its own crate (#32766) 2023-08-09 13:03:36 -07:00
carllin d5faa6e8aa
Local Cluster Duplicate Switch Test (#32614)
* Add test for broken behavior in same batch

* tests

* redo test

* Important fixes to not immediately duplicate confirm by adding extra node

* Fixup merge

* PR comments

* Redo stakes

* clippy

* fixes

* Resolve conflicts

* add thread logging

* Fixup merge

* Fixup bugs

* Revert "add thread logging"

This reverts commit 9dc22401054b8f91f2b2aa3033e482996913febb.

* Hide scope

* Fixes

* Cleanup test_faulty_node

* More fixes

* Fixes

* Error logging

* Fix duplicate confirmed

* done

* PR comments

* Revert "Error logging"

This reverts commit 18953c36a5e865ecdd38bbf49b8d0502448087d2.

* PR comments

* nit
2023-08-08 19:29:39 -04:00
Ashwin Sekar fa3506631a
stake: deprecate on chain warmup/cooldown rate and config (#32723)
* stake: deprecate on chain warmup/cooldown rate and config

* Pr feedback: Deprecate since 1.16.7

Co-authored-by: Jon Cinque <me@jonc.dev>

---------

Co-authored-by: Jon Cinque <me@jonc.dev>
2023-08-07 13:23:24 -07:00
Ryo Onodera f949a95a9e
Define ClusterConfig::new_with_equal_stakes() (#32330)
* Define ClusterConfig::new_with_equal_stakes()

* Remove needless variable assignment
2023-06-30 10:35:26 +09:00
Jeff Biseda 87c1b67d53
refactor core to create consensus module (#32282) 2023-06-27 17:25:08 -07:00
behzad nouri 9281ab7d97
separates out connection-cache metrics for different protocols (#31803) 2023-05-25 14:48:22 +00:00
Ashwin Sekar fb09cba8ed
local-cluster: use feature flag for size for vote accounts (#31723) 2023-05-18 19:39:28 -07:00
behzad nouri 6a4a0418a6
removes hard-coded QUIC_PORT_OFFSET from connection-cache (#31541)
New ContactInfo has api identifying QUIC vs UDP ports; no need to hard-code
port-offset deep in connection-cache.
2023-05-09 13:46:17 +00:00
Tyera 9af7009bb4
Add new CreateVoteAccount instruction-set builders (#31330)
* Add failing test

* Add config struct for vote-account creation

* Add create-vote-account ix builders that use CreateVoteAccountConfig

* Test now passes; add case to demonstrate failure after activation

* Deprecate existing ix builders

* Use new builders in solana-cli

* Query feature status to use correct VoteState size in solana-cli

* Fix tests and clippy warnings

* Improve ugly conditional block
2023-04-27 11:14:39 -06:00
Brooks 35437b8dad
Makes AccountsHashVerifier aware of Incremental Accounts Hash (#30820) 2023-03-22 10:20:16 -04:00
Brooks b64d0de771
Makes snapshot_utils aware of Incremental Accounts Hash (#30804) 2023-03-21 16:34:30 +00:00
cavemanloverboy 10f49d4e26
Geyser Runtime Reload (#30352)
Support dynamic geyser plugin load, unload, and listing through admin RPC.
2023-03-16 17:03:00 -07:00
Tyera 7b1d446001
Admin RPC Service: move post-init activation to before wait-for-supermajority (#30544)
* Move AdminRpcRequestMetadataPostInit to solana-core

* Move AdminRpcRequestMetadataPostInit write to just before wait_for_supermajority

* Pass AdminRpcRequestMetadataPostInit in TestValidatorGenesis

* Fixup local-cluster
2023-03-01 19:38:11 -07:00
behzad nouri ded457cd73
embeds the new gossip ContactInfo in ClusterInfo (#30022)
Working towards replacing the legacy gossip contact-info with the new
one, the commit updates the respective field in gossip cluster-info.
2023-02-10 20:07:45 +00:00
Lijun Wang ada6136a6c
Refactor connection cache to support generic msgs (#29774)
tpu-client/tpu_connection_cache is refactored out the module and moved to connection-cache/connection_cache and the logic in client/connection_cache is consolidated to connection-cache/connection_cache as well. client/connection_cache only has a thin wrapper which forward calls to connection-cache/connection_cache and deal with constructions of quic/udp connection cache for clients using them both.2.

The TpuConnection is refactored to ClientConnection to make it generic and functions renamed to be proper for other workflows. eg. tpu_addr -> server_addr, send_transaction --> send_data and etc...

The enum dispatch is removed so that we can make the bulk of code of quic and udp agnostic of each other. The client is possible to load quic or udp only into its runtime.

The generic type parameter in the tpu-client/tpu_connection_cache is removed in order to create both quic and udp connection cache and use the object to send transactions with multiple branching when sending data. The generic type parameters and associated types are dropped in other types in order to make the trait "object safe" for this purpose.

I have annotated the code explaining the reasoning and the refactoring source -> destination.

There is no functional changes

bench-tps has been performed for rpc-client, thin-client and tpu-client. And it is found the performance number largely match the ones before the refactoring.
2023-02-01 18:10:06 -08:00
Xiang Zhu 856598969c
Account path add run parent with old path cleanup (#29942)
* Add run parent directory for accounts files

* fix test test_concurrent_snapshot_packaging

* review comments.  renamed the path setup function

* Addressed most of the review comments

* remove explict type def for map result

* handle create_accounts_run_and_snapshot_dirs error with expect

* update with more review comments

* minor fixes from review comments

* simplify account_filename option assignment

* handle error from create_accounts_run_and_snapshot_dirs

* use then instead of then_some for lazy evaluation

* Clean up files in the old account_path before trasitioning to the new run path

* try_exist and accounts_dir removing extra

* sync rmdir, is_dir check

* handle the account_path not deletable case
2023-01-30 10:26:43 -08:00
Xiang Zhu 4ebcacb4a3
Revert "Add run parent directory for accounts files (#29794)" (#29899)
This PR is causing OOM on master.  Reverting it for now.

This reverts commit 74f89d1494.
2023-01-25 10:03:01 -08:00
Xiang Zhu 74f89d1494
Add run parent directory for accounts files (#29794)
* Add run parent directory for accounts files

* fix test test_concurrent_snapshot_packaging

* review comments.  renamed the path setup function

* Addressed most of the review comments

* remove explict type def for map result

* handle create_accounts_run_and_snapshot_dirs error with expect

* update with more review comments

* minor fixes from review comments

* simplify account_filename option assignment

* handle error from create_accounts_run_and_snapshot_dirs

* use then instead of then_some for lazy evaluation
2023-01-24 16:44:35 -08:00
behzad nouri 8c212f59ad
renames ContactInfo to LegacyContactInfo (#29566)
Working towards adding a new ContactInfo where new sockets can be
added in a backward compatible way.
2023-01-08 16:00:55 +00:00
steviez aeb6b53502
Remove unused Option<> around ValidatorConfig's SnapshotConfig (#29090)
Remove Option<> around ValidatorConfig's SnapshotConfig

The SnapshotConfig is required and is currently hard-coded to be a
Some().
2022-12-06 22:47:55 +00:00
Tyera c32377b5af
Split out quic- and udp-client definitions (#28762)
* Move ConnectionCache back to solana-client, and duplicate ThinClient, TpuClient there

* Dedupe thin_client modules

* Dedupe tpu_client modules

* Move TpuClient to TpuConnectionCache

* Move ThinClient to TpuConnectionCache

* Move TpuConnection and quic/udp trait implementations back to solana-client

* Remove enum_dispatch from solana-tpu-client

* Move udp-client to its own crate

* Move quic-client to its own crate
2022-11-18 12:21:45 -07:00
Brooks Prumo a8c6a9e5fc
Bank::freeze() waits for EAH calculation to complete (#28170) 2022-10-05 17:44:35 -04:00
Brooks Prumo 2f8f6c6a31
Send Epoch Accounts Hash requests from set_root() (#27764) 2022-09-30 14:59:41 -04:00
Christian Kamm 90b8a3a44d
Remove KeypairInsecureClone trait and add insecure_clone() instead (#27396)
See discussion in #26248
2022-09-12 14:59:41 +00:00
Lijun Wang 7f223dc582
Added option to turn on UDP for TPU transaction and make UDP based TPU off by default (#27462)
--tpu-enable-udp is introduced. And when this is on, the transaction receive and transaction forward is enabled using udp.

Except for a few tests which was hard-coded sending transactions using udp, most tests are being done with udp based tpu disabled.
2022-09-07 13:19:14 -07:00
Brooks Prumo 93a4f80a2c
Handling snapshot requests is now required (#27537) 2022-09-07 10:08:42 -04:00
Brooks Prumo 6684c62280
Add SnapshotUsage to SnapshotConfig (#27508) 2022-09-02 08:56:23 -04:00
Tyera Eulberg b8b3d723da
Use new client crates (#27360)
* Update ancillary cli crates

* Update cli

* Update command-line tools

* Update rpc, etc

* Update client-test

* Update core, validator

* Update local-cluster
2022-08-24 10:47:02 -06:00
Brennan Watt e4a7d01e10
Rust v1.63 (#27303)
* Upgrade to Rust v1.63.0

* Add nightly_clippy_allows

* Resolve some new clippy nightly lints

* Increase QUIC packets completion timeout

* Update quinn-udp crate

Co-authored-by: Michael Vines <mvines@gmail.com>
2022-08-22 18:01:03 -07:00
Brennan Watt 7573000d87
Revert "Rust v1.63.0 (#27148)" (#27245)
This reverts commit a2e7bdf50a.
2022-08-19 09:19:44 +01:00
Brennan Watt a2e7bdf50a
Rust v1.63.0 (#27148)
* Upgrade to Rust v1.63.0

* Add nightly_clippy_allows

* Resolve some new clippy nightly lints

* Increase QUIC packets completion timeout

Co-authored-by: Michael Vines <mvines@gmail.com>
2022-08-17 15:48:33 -07:00
Tyera Eulberg 1aa8e06fd5
Fix local-cluster for QUIC more (#27096) 2022-08-12 06:20:04 -07:00
Jeff Biseda e50013acdf
Handle JsonRpcService startup failure (#27075) 2022-08-11 23:25:20 -07:00
Michael Vines ccfbc54195 Move vote program state and instructions to solana-program 2022-08-09 20:52:47 -07:00
Christian Kamm cf58640937
Keypair: implement clone() (#26248)
* Keypair: implement clone()

This was not implemented upstream in ed25519-dalek to force everyone to
think twice before creating another copy of a potentially sensitive
private key in memory.

See https://github.com/dalek-cryptography/ed25519-dalek/issues/76

However, there are now 9 instances of
  Keypair::from_bytes(&keypair.to_bytes())
in the solana codebase and it would be preferable to have a function.

In particular since this also comes up when writing programs and can
cause users to either start messing with lifetimes or discover the
from_bytes() workaround themselves.

This patch opts to not implement the Clone trait. This avoids automatic
use in order to preserve some of the original "let developers think
twice about this" intention.

* Use Keypair::clone
2022-08-06 11:54:38 -06:00
Nick Rempel 7e4a5de99c
Refactor ConnectionCache::use_quic (#26235)
* Remove UseQuic type

Move to storing the UdpSocket on ConnectionCache and accepting a bool

* Remove use_quic from ConnectionCache constructor

Replace with separate with_udp constructor to force callers to choose
2022-07-05 10:49:42 -07:00
Ryo Onodera cd2878acf9
Avoid to miss to root for local slots before the hard fork (#19912)
* Make sure to root local slots even with hard fork

* Address review comments

* Cleanup a bit

* Further clean up

* Further clean up a bit

* Add comment

* Tweak hard fork reconciliation code placement
2022-06-26 15:14:17 +09:00
Pankaj Garg 43ff65ece9
Use single send socket in UdpTpuConnection (#26105) 2022-06-21 14:56:21 -07:00
Lijun Wang 29b597cea5
Connection pool support in connection cache and QUIC connection reliability improvement (#25793)
* Connection pool in connection cache and handle connection errors

1. The connection not has a pool of connections per address, configurable, default 4
2. The connections per address share a lazy initialized endpoint
3. Handle connection issues better, avoid race conditions
4. Various log improvement for help debug connection issues
2022-06-10 09:25:24 -07:00
Jon Cinque 79a8ecd0ac
client: Remove static connection cache, plumb it instead (#25667)
* client: Remove static connection cache, plumb it instead

* Add TpuClient::new_with_connection_cache to not break downstream

* Refactor get_connection and RwLock into ConnectionCache

* Fix merge conflicts from new async TpuClient

* Remove `ConnectionCache::set_use_quic`

* Move DEFAULT_TPU_USE_QUIC to client, use ConnectionCache::default()
2022-06-08 13:57:12 +02:00
Brooks Prumo 195bb8bd36
Update local-cluster tests to work with higher minimum stake delegation (#25023) 2022-05-05 22:00:33 -05:00
Pankaj Garg 88c16c0176
Check if quic is enabled before warming up quic connections (#24821)
* Check if quic is enabled before warming up quic connections

* fix after rebase

* don't start warmup service if quic not enabled

* fix test
2022-05-01 03:52:38 +00:00
BG Zhu 22224127e0
Refactor thin_client::create_client (#24067)
Refactor the thin_client::create_client to take addresses separately instead of as a tuple

Co-authored-by: Bijie Zhu <bijiezhu@Bijies-MBP.cable.rcn.com>
2022-04-06 11:03:38 -04:00
ryleung-solana 8b72200afb
Thin client quic (#23973)
Change thin-client to use connection-cache
2022-03-31 15:47:00 -04:00
ryleung-solana 17b00ad3a4
Add quic-client module (#23166)
* Add quic-client module to send transactions via quic, abstracted behind the TpuConnection trait (along with a legacy UDP implementation of TpuConnection) and change thin-client to use TpuConnection
2022-03-09 21:33:05 -05:00
Trent Nelson eac4a6df68 rpc: use minimal mode by default 2022-02-01 19:00:06 -07:00
Ashwin Sekar eeec1ce2ad
Add local cluster test to repro slot hash expiry bug (#21873) 2022-01-10 00:58:21 -05:00
Justin Starry 0e1afcbb26
Split up local cluster tests into separate CI steps (#22295)
* Split up local cluster tests into separate CI steps

* Update buildkite-pipeline.sh
2022-01-05 14:44:15 +00:00
carllin f493a88258
Fixup flaky tests (#21617)
* Fixup flaky tests

* Fixup listeners
2021-12-06 17:14:38 -05:00
Michael Vines b8837c04ec Reformat imports to a consistent style for imports
rustfmt.toml configuration:
  imports_granularity = "One"
  group_imports = "One"
2021-12-03 09:19:13 -08:00