Commit Graph

374 Commits

Author SHA1 Message Date
DimAn 2fa9bc3e70
Add options to store full and/or incremental snapshots in separate locations (#24247) 2022-05-10 16:37:41 -04:00
Pankaj Garg 88c16c0176
Check if quic is enabled before warming up quic connections (#24821)
* Check if quic is enabled before warming up quic connections

* fix after rebase

* don't start warmup service if quic not enabled

* fix test
2022-05-01 03:52:38 +00:00
Yueh-Hsuan Chiang 5245eb4229
(LedgerStore) Hidden validator argument for RocksDB perf samples (#24684)
#### Summary of Changes
This PR replaces the use of thread_rng in RocksDB perf metric samples by
AtomicU32 with Ordering::Relaxed to improve the performance of determining
whether to sample the current RocksDB's read/write perf metric.
2022-04-29 17:55:34 -07:00
Yueh-Hsuan Chiang b56c091b37
(LedgerStore) Hidden validator argument for RocksDB perf samples (#24682)
#### Problem
Currently, the number of RocksDB perf samples is controlled by an env arg
which is later handled using a lazy_static variable.  However, there is a known
performance overhead of using lazy_static as mentioned in
https://github.com/solana-labs/solana/pull/6472.

#### Summary of Changes
Instead, this PR uses a hidden validator argument, --rocksdb-perf-sample-interval,
for controlling how often RocksDB read/write performance sample is collected.
2022-04-29 15:28:50 -07:00
Code Monkey aee17f3b50
fix typo on validator help (#24844)
Co-authored-by: Austin <support@thetaproxies.com>
2022-04-29 13:47:04 -06:00
Michael Vines aee48406bc Increase default --maximum-local-snapshot-age to 2500 2022-04-21 16:06:11 -07:00
Lijun Wang 7c61e438fc
Batch tpu calls in send-transaction-service (#24083)
Introduced flag --tpu-do-batch2.
Introduced flag to control the batch size-- by default 100
The default batch timeout is 200ms -- configurable. If either it time out or the batch size is filled, a new batch is sent
The batch honor the retry rate on the transaction already sent before.
Introduced two threads in STS: one for receiving new transactions and doing batch send and one for retrying old transactions and doing batch.6.
Fixes #
2022-04-21 12:43:08 -07:00
Jeff Washington (jwash) cfe2177e16
validator option: accounts-db-skip-rewrites (#24504) 2022-04-20 09:41:00 -05:00
Michael Vines 9e4999ef6a Remove halt_at_slot from RuntimeConfig, it's not a runtime concern 2022-04-19 19:23:58 -07:00
Tyera Eulberg 26899359d1
Support quic in bench-tps (#24295)
* Update comment

* Use connection_cache in tpu_client

* Add --tpu-use-quic to bench-tps

* Use connection_cache async send
2022-04-13 12:17:10 -06:00
Jeff Washington (jwash) 1bc49d219d
IndexLimitMb option adds 'Unspecified' state (#24249) 2022-04-12 09:38:09 -05:00
Jon Cinque 9b8850f99e
test-validator: Add `--max-compute-units` flag (#24130)
* test-validator: Add `--max-compute-units` flag

* Add `RuntimeConfig` for tweaking runtime behavior

* Actually add the file

* Move RuntimeConfig to runtime
2022-04-12 02:28:10 +02:00
Giorgio Gambino 60b2155bd3
Add accounts-filler-size command line option (#23896) 2022-04-11 13:10:09 -05:00
Tyera Eulberg fb67ff14de
Remove replica-node crates (#24152) 2022-04-06 16:52:19 -06:00
ryleung-solana 8b72200afb
Thin client quic (#23973)
Change thin-client to use connection-cache
2022-03-31 15:47:00 -04:00
Lijun Wang 98525ddea9
Make tpu_use_quic a flag only without argument (#24018) 2022-03-31 10:04:24 -07:00
Jeff Washington (jwash) c24de17278
remove index hash calculation as an option (#23928) 2022-03-25 15:32:53 -05:00
Jon Cinque 7af48465fa
transaction-status: Add return data to meta (#23688)
* transaction-status: Add return data to meta

* Add return data to simulation results

* Use pretty-hex for printing return data

* Update arg name, make TransactionRecord struct

* Rename TransactionRecord -> ExecutionRecord
2022-03-22 23:17:05 +01:00
Yueh-Hsuan Chiang ae75b1a25f
(LedgerStore) Add compression type (#23578)
This PR adds `--rocksdb-ledger-compression` as a hidden argument to the validator
for specifying the compression algorithm for TransactionStatus.  Available compression
algorithms include `lz4`, `snappy`, `zlib`. The default value is `none`.

Experimental results show that with lz4 compression, we can achieve ~37% size-reduction
on the TransactionStatus column family, or ~8% size-reduction of the ledger store size.
2022-03-22 02:27:09 -07:00
Trent Nelson 9b32b72990 bigtable: allow custom instance names 2022-03-21 11:26:49 -06:00
Tyera Eulberg 63ee00e647 Refactor validator bigtable config 2022-03-21 11:26:49 -06:00
Lijun Wang ae76fe2bd7
Made connection cache configurable. (#23783)
Added command-line argument tpu-use-quic argument.
Changed connection cache to return different connections based on the config.
2022-03-21 09:31:37 -07:00
Yueh-Hsuan Chiang f999eef452
(LedgerStore) Rename BlockstoreAdvancedOptions to LedgerColumnOptions (#23764)
This PR renames BlockstoreAdvancedOptions to LedgerColumnOptions, as we will
pass-down this struct to LedgerColumn to allow it to perform metric reporting.
2022-03-18 11:13:35 -07:00
Trent Nelson ce2e82cfb6 validator: `--only-known-rpc` requires a `--known-validator ...` 2022-03-18 07:02:16 +00:00
Michael Vines 2da4e3eb6c Add --no-os-memory-stats-reporting 2022-03-15 17:07:40 -07:00
Tyera Eulberg 102dd68a03
Rename AccountsDb plugins to Geyser plugins (#23604) 2022-03-14 19:18:46 -06:00
Yueh-Hsuan Chiang 1e20bd8f9a
(LedgerStore) Include storage type as a tag in RocksDB metric reporting (#23523)
#### Summary of Changes
This PR further enables group by operation on storage type in blockstore_rocksdb_cfs metrics.
Such group-by allows us to further compare the performance metrics between rocks-level and
rocks-fifo.

To make things extensible, this PR introduces BlockstoreAdvancedOptions and move shred_storage_type. 
All fields in BlockstoreAdvancedOptions will support group-by operation in blockstore_rocksdb_cfs.

Dependency: #23580
2022-03-11 15:17:34 -08:00
behzad nouri 043086081f
checks if --entrypoint is consistent with --allow-private-addr (#23559)
If entrypoints are excluded by SocketAddrSpace, the node cannot start.
Private --entrypoint addresses require --allow-private-addr.
2022-03-11 14:15:05 +00:00
HaoranYi 463cd564cf
fix typos (#23495)
* fix typos
2022-03-05 20:46:46 -06:00
Yueh-Hsuan Chiang 62d2a4cd88
Make ShredStorageType::RocksLevel public (#23272)
#### Summary of Changes
This PR adds two hidden arguments to the validator that allow users to use RocksDB's FIFO compaction for storing shreds.

        --shred-storage <SHRED_STORAGE>
            EXPERIMENTAL: Controls how RocksDB compacts shreds.  *WARNING*: You will lose your ledger data
            when you switch between options. Possible values are: 'level': stores shreds using RocksDB's default (level)
            compaction. 'fifo': stores shreds under RocksDB's FIFO compaction. This option is more efficient on
            disk-write-bytes of the ledger store. [default: level]  [possible values: level, fifo]

        --shred-storage-size <SHRED_STORAGE_SIZE_BYTES>
            The shred storage size in bytes. The suggested value is 50% of your ledger storage size in bytes. [default:
            268435456000]
2022-03-03 12:43:58 -08:00
sakridge a4f4ac5279
add plumbing to allow for arbitrary tpu address in gossip (#22703)
* add plumbing to allow for arbitrary tpu address in gossip

* make clippy happy

* Review comments

Co-authored-by: CherryWorm <nico.gruendel@web.de>
2022-03-02 09:42:14 +01:00
Jeff Washington (jwash) 4bc440666a
add arg --disable_accounts_disk_index (#23308) 2022-02-23 18:07:24 -06:00
Lijun Wang 084fb79ad8
Unhide AccountsDb plugin parameters (#23284)
* Unhide AccountsDb plugin parameters

* addressed feedback from Trent
2022-02-23 10:01:14 -08:00
Jeff Washington (jwash) 7ebf398ed7
AcctIdx: env var "SOLANA_TEST_ACCOUNTS_INDEX_MEMORY_LIMIT_MB" (#23194)
* AcctIdx: env var "SOLANA_TEST_ACCOUNTS_INDEX_MEMORY_LIMIT_MB"

* ignore env var when starting as validator

* Update runtime/src/bucket_map_holder.rs

Co-authored-by: Trent Nelson <trent.a.b.nelson@gmail.com>

Co-authored-by: Trent Nelson <trent.a.b.nelson@gmail.com>
2022-02-22 09:40:12 -06:00
Trent Nelson de30699fa5 validator: invert vote account sanity check arg 2022-02-16 10:05:46 -07:00
Michael Vines a6d736572c `solana-validator set-identity` now supports the `--require-tower` flag 2022-02-15 19:45:00 -08:00
Michael Vines 527f62c744 Add --skip-new-snapshot-check to exit and wait-for-restart-window subcommands 2022-02-15 16:22:31 -08:00
Michael Vines 98707baec2 `solana-validator wait-for-restart-window --min-idle-time X` now works 2022-02-15 08:55:47 -08:00
Jeff Biseda d15d495f17
require --no-os-network-limits-test to bypass failed network limit checks (#22617) 2022-02-14 07:48:30 -08:00
behzad nouri 78089941ff
adds validator version to set_panic_hook (#23082) 2022-02-11 18:04:10 +00:00
Brooks Prumo 67f6787f7a
Enable incremental snapshots by default (#22938)
Co-authored-by: Michael Vines <mvines@gmail.com>
2022-02-09 13:26:35 -06:00
Trent Nelson eac4a6df68 rpc: use minimal mode by default 2022-02-01 19:00:06 -07:00
Michael Vines 6d5bbca630 Pacify clippy 2022-01-21 19:12:57 -08:00
Jeff Biseda e7777281d6
regularly report network limits (#22563) 2022-01-20 12:38:42 -08:00
segfaultdoctor 7171b3a3ac
Bugfix/block subscribe (#22516)
* use correct operation name

* require enable_rpc_transaction_history flag when enabling block_subscription

Co-authored-by: Zano <segfaultdoctor@protonmail.com>
2022-01-14 19:53:34 -07:00
Trent Nelson b93ab5d295 validator: add contact-info query to admin port 2021-12-23 20:50:21 +00:00
segfaultdoctor 76098dd42a
RPC Block Subscription (#21787)
* add stuff

* compiling

* add notify block

* wip

* feat: add blockSubscribe pubsub method

* address PR comments

Co-authored-by: Lucas B <buffalu@jito.network>
Co-authored-by: Zano <segfaultdoctor@protonmail.com>
2021-12-17 16:03:09 -07:00
Justin Starry 1430b58a6d
Remove deprecated slow epoch boundary methods (#21568) 2021-12-03 17:59:10 +00:00
Michael Vines b8837c04ec Reformat imports to a consistent style for imports
rustfmt.toml configuration:
  imports_granularity = "One"
  group_imports = "One"
2021-12-03 09:19:13 -08:00
Jeff Washington (jwash) 308d7d40d0
aggressively flush cache based on cache size (#21332)
* aggressively flush cache based on cache size

* find_older_frozen_slots -> cached_frozen_slots

* remove 'WRITE_CACHE_LIMIT_BYTES_DEFAULT'

* tweaks to stats

* fix tests
2021-12-01 13:10:48 -06:00