Commit Graph

357 Commits

Author SHA1 Message Date
Jon Cinque 7af48465fa
transaction-status: Add return data to meta (#23688)
* transaction-status: Add return data to meta

* Add return data to simulation results

* Use pretty-hex for printing return data

* Update arg name, make TransactionRecord struct

* Rename TransactionRecord -> ExecutionRecord
2022-03-22 23:17:05 +01:00
Yueh-Hsuan Chiang ae75b1a25f
(LedgerStore) Add compression type (#23578)
This PR adds `--rocksdb-ledger-compression` as a hidden argument to the validator
for specifying the compression algorithm for TransactionStatus.  Available compression
algorithms include `lz4`, `snappy`, `zlib`. The default value is `none`.

Experimental results show that with lz4 compression, we can achieve ~37% size-reduction
on the TransactionStatus column family, or ~8% size-reduction of the ledger store size.
2022-03-22 02:27:09 -07:00
Trent Nelson 9b32b72990 bigtable: allow custom instance names 2022-03-21 11:26:49 -06:00
Tyera Eulberg 63ee00e647 Refactor validator bigtable config 2022-03-21 11:26:49 -06:00
Lijun Wang ae76fe2bd7
Made connection cache configurable. (#23783)
Added command-line argument tpu-use-quic argument.
Changed connection cache to return different connections based on the config.
2022-03-21 09:31:37 -07:00
Yueh-Hsuan Chiang f999eef452
(LedgerStore) Rename BlockstoreAdvancedOptions to LedgerColumnOptions (#23764)
This PR renames BlockstoreAdvancedOptions to LedgerColumnOptions, as we will
pass-down this struct to LedgerColumn to allow it to perform metric reporting.
2022-03-18 11:13:35 -07:00
Trent Nelson ce2e82cfb6 validator: `--only-known-rpc` requires a `--known-validator ...` 2022-03-18 07:02:16 +00:00
Michael Vines 2da4e3eb6c Add --no-os-memory-stats-reporting 2022-03-15 17:07:40 -07:00
Tyera Eulberg 102dd68a03
Rename AccountsDb plugins to Geyser plugins (#23604) 2022-03-14 19:18:46 -06:00
Yueh-Hsuan Chiang 1e20bd8f9a
(LedgerStore) Include storage type as a tag in RocksDB metric reporting (#23523)
#### Summary of Changes
This PR further enables group by operation on storage type in blockstore_rocksdb_cfs metrics.
Such group-by allows us to further compare the performance metrics between rocks-level and
rocks-fifo.

To make things extensible, this PR introduces BlockstoreAdvancedOptions and move shred_storage_type. 
All fields in BlockstoreAdvancedOptions will support group-by operation in blockstore_rocksdb_cfs.

Dependency: #23580
2022-03-11 15:17:34 -08:00
behzad nouri 043086081f
checks if --entrypoint is consistent with --allow-private-addr (#23559)
If entrypoints are excluded by SocketAddrSpace, the node cannot start.
Private --entrypoint addresses require --allow-private-addr.
2022-03-11 14:15:05 +00:00
HaoranYi 463cd564cf
fix typos (#23495)
* fix typos
2022-03-05 20:46:46 -06:00
Yueh-Hsuan Chiang 62d2a4cd88
Make ShredStorageType::RocksLevel public (#23272)
#### Summary of Changes
This PR adds two hidden arguments to the validator that allow users to use RocksDB's FIFO compaction for storing shreds.

        --shred-storage <SHRED_STORAGE>
            EXPERIMENTAL: Controls how RocksDB compacts shreds.  *WARNING*: You will lose your ledger data
            when you switch between options. Possible values are: 'level': stores shreds using RocksDB's default (level)
            compaction. 'fifo': stores shreds under RocksDB's FIFO compaction. This option is more efficient on
            disk-write-bytes of the ledger store. [default: level]  [possible values: level, fifo]

        --shred-storage-size <SHRED_STORAGE_SIZE_BYTES>
            The shred storage size in bytes. The suggested value is 50% of your ledger storage size in bytes. [default:
            268435456000]
2022-03-03 12:43:58 -08:00
sakridge a4f4ac5279
add plumbing to allow for arbitrary tpu address in gossip (#22703)
* add plumbing to allow for arbitrary tpu address in gossip

* make clippy happy

* Review comments

Co-authored-by: CherryWorm <nico.gruendel@web.de>
2022-03-02 09:42:14 +01:00
Jeff Washington (jwash) 4bc440666a
add arg --disable_accounts_disk_index (#23308) 2022-02-23 18:07:24 -06:00
Lijun Wang 084fb79ad8
Unhide AccountsDb plugin parameters (#23284)
* Unhide AccountsDb plugin parameters

* addressed feedback from Trent
2022-02-23 10:01:14 -08:00
Jeff Washington (jwash) 7ebf398ed7
AcctIdx: env var "SOLANA_TEST_ACCOUNTS_INDEX_MEMORY_LIMIT_MB" (#23194)
* AcctIdx: env var "SOLANA_TEST_ACCOUNTS_INDEX_MEMORY_LIMIT_MB"

* ignore env var when starting as validator

* Update runtime/src/bucket_map_holder.rs

Co-authored-by: Trent Nelson <trent.a.b.nelson@gmail.com>

Co-authored-by: Trent Nelson <trent.a.b.nelson@gmail.com>
2022-02-22 09:40:12 -06:00
Trent Nelson de30699fa5 validator: invert vote account sanity check arg 2022-02-16 10:05:46 -07:00
Michael Vines a6d736572c `solana-validator set-identity` now supports the `--require-tower` flag 2022-02-15 19:45:00 -08:00
Michael Vines 527f62c744 Add --skip-new-snapshot-check to exit and wait-for-restart-window subcommands 2022-02-15 16:22:31 -08:00
Michael Vines 98707baec2 `solana-validator wait-for-restart-window --min-idle-time X` now works 2022-02-15 08:55:47 -08:00
Jeff Biseda d15d495f17
require --no-os-network-limits-test to bypass failed network limit checks (#22617) 2022-02-14 07:48:30 -08:00
behzad nouri 78089941ff
adds validator version to set_panic_hook (#23082) 2022-02-11 18:04:10 +00:00
Brooks Prumo 67f6787f7a
Enable incremental snapshots by default (#22938)
Co-authored-by: Michael Vines <mvines@gmail.com>
2022-02-09 13:26:35 -06:00
Trent Nelson eac4a6df68 rpc: use minimal mode by default 2022-02-01 19:00:06 -07:00
Michael Vines 6d5bbca630 Pacify clippy 2022-01-21 19:12:57 -08:00
Jeff Biseda e7777281d6
regularly report network limits (#22563) 2022-01-20 12:38:42 -08:00
segfaultdoctor 7171b3a3ac
Bugfix/block subscribe (#22516)
* use correct operation name

* require enable_rpc_transaction_history flag when enabling block_subscription

Co-authored-by: Zano <segfaultdoctor@protonmail.com>
2022-01-14 19:53:34 -07:00
Trent Nelson b93ab5d295 validator: add contact-info query to admin port 2021-12-23 20:50:21 +00:00
segfaultdoctor 76098dd42a
RPC Block Subscription (#21787)
* add stuff

* compiling

* add notify block

* wip

* feat: add blockSubscribe pubsub method

* address PR comments

Co-authored-by: Lucas B <buffalu@jito.network>
Co-authored-by: Zano <segfaultdoctor@protonmail.com>
2021-12-17 16:03:09 -07:00
Justin Starry 1430b58a6d
Remove deprecated slow epoch boundary methods (#21568) 2021-12-03 17:59:10 +00:00
Michael Vines b8837c04ec Reformat imports to a consistent style for imports
rustfmt.toml configuration:
  imports_granularity = "One"
  group_imports = "One"
2021-12-03 09:19:13 -08:00
Jeff Washington (jwash) 308d7d40d0
aggressively flush cache based on cache size (#21332)
* aggressively flush cache based on cache size

* find_older_frozen_slots -> cached_frozen_slots

* remove 'WRITE_CACHE_LIMIT_BYTES_DEFAULT'

* tweaks to stats

* fix tests
2021-12-01 13:10:48 -06:00
Michael Vines ba9dfa0d22 Remove frozen account support 2021-11-29 08:38:11 -08:00
Jeff Washington (jwash) 79d21d6805
add --accounts-index-scan-results-limit-mb to allow scans to abort (#21327)
* ScanConfig -> &ScanConfig

* add --accounts-index-scan-results-limit-mb to allow scans to abort

* feedback
2021-11-19 09:00:19 -06:00
Jeff Biseda d5de0c8e12
add --no-os-network-stats-reporting option (#21296) 2021-11-16 10:26:03 -08:00
Tyera Eulberg ae3777cadb
Update various terms to "known" (#21257)
* Update snapshot bootstrap terminology

* Update only known rpc terminology

* Update replica-node
2021-11-12 20:13:59 -07:00
Michael Keleti b0ca335463
Rename "trusted" to "known" in `validators/` (#21197)
* Replaced trusted with known validator

* Format Convention
2021-11-12 11:57:55 -07:00
Ivan Mironov 0e751ef7df Add validator option to change niceness of RPC server threads
Fixes https://github.com/solana-labs/solana/issues/14556
2021-11-04 17:16:46 -06:00
Ivan Mironov c78f474373 Add validator option to change niceness of snapshot packager thread 2021-11-04 17:16:46 -06:00
Brooks Prumo 7495cbd1c8
Remove --no-incremental-snapshot-fetch CLI arg (#21172)
Now that bootstrap with incremental snapshots enabled has a fallback
mechanism in place, this no-incremental-snapshot-fetch flag is no longer
required.

Fixes #21127
2021-11-04 11:54:25 -05:00
Pavel Strakhov 9fabff5129
Parallel notifications in RPC PubSub (#20543)
* generate rpc notifications in parallel

* Use multithreaded runtime for pubsub

* add metric for time since creation of rpc notification to queue

* measure notification entry processing

* fix: add n_threads config argument

* configure rayon thread pool for rpc notifications

* add config option for pubsub notification threads

* rename metric to created_to_queue_time_us

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>

* rename metric to notification_entry_processing_time_us

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>

* use value_of for rpc_pubsub_notification_threads parsing

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>

* rename threads to sol-sub-notif-N

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>

* fix rpc tests for TimestampedNotificationEntry

* rustfmt

* use rayon thread limit for rpc

Co-authored-by: Pavel Strakhov <p.strakhov@iconic.vc>
Co-authored-by: Alexander Polakov <a.polakov@zubr.io>
Co-authored-by: Nikita Podoliako <bananaelecitrus@gmail.com>
Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
2021-11-01 00:17:24 -06:00
Jeff Biseda 6470560dd1
check OS network limits when starting validator (#20874) 2021-10-25 18:25:55 -07:00
Jeff Washington (jwash) 43ea579f63
add cli for --accounts-hash-num-passes (#20827) 2021-10-25 09:45:46 -05:00
Brooks Prumo 4821b0a1d9
Download incremental snapshots during bootstrap (#20696) 2021-10-21 09:14:51 -05:00
Trent Nelson 2744a2128c rpc-send-tx-svc: server-side retry knobs 2021-10-20 13:43:27 -06:00
Trent Nelson fe098b5ddc rpc-send-tx-svc: add with_config constructor 2021-10-20 13:43:27 -06:00
Brooks Prumo 72e1efb847
Refactor port check in bootstrap (#20793) 2021-10-19 21:35:42 +00:00
Brooks Prumo 7a36bf5095
Add CLI args for no-incremental-snapshot-fetch (#20787) 2021-10-19 13:15:30 -05:00
Jeff Washington (jwash) a8e000a2a6
add filler accounts to bloat validator and predict failure (#20491)
* add filler accounts to bloat validator and predict failure

* assert no accounts match filler

* cleanup magic numbers

* panic if can't load from snapshot with filler accounts specified

* some renames

* renames

* into_par_iter

* clean filler accts, too
2021-10-11 12:46:27 -05:00