Commit Graph

738 Commits

Author SHA1 Message Date
Xiang Zhu 4ebcacb4a3
Revert "Add run parent directory for accounts files (#29794)" (#29899)
This PR is causing OOM on master.  Reverting it for now.

This reverts commit 74f89d1494.
2023-01-25 10:03:01 -08:00
Ryo Onodera 40bbf99c74
Add fully-reproducible online tracer for banking (#29196)
* Add fully-reproducible online tracer for banking

* Don't use eprintln!()...

* Update programs/sbf/Cargo.lock...

* Remove meaningless assert_eq

* Group test-only code under aptly named mod

* Remove needless overflow handling in receive_until

* Delay stat aggregation as it's possible now

* Use Cow to avoid needless heap allocs

* Properly consume metrics action as soon as hold

* Trace UnprocessedTransactionStorage::len() instead

* Loosen joining api over type safety for replaystage

* Introce hash event to override these when simulating

* Use serde_with/serde_as instead of hacky workaround

* Update another Cargo.lock...

* Add detailed comment for Packet::buffer serialize

* Rename sender_overhead_minimized_receiver_loop()

* Use type interference for TraceError

* Another minor rename

* Retire now useless ForEach to simplify code

* Use type alias as much as possible

* Properly translate and propagate tracing errors

* Clarify --enable-banking-trace with better naming

* Consider unclean (signal-based) node restarts..

* Tweak logging and cli

* Remove Bank events as it's not needed anymore

* Make tpu own banking tracer thread

* Reduce diff a bit..

* Use latest serde_with

* Finally use the published rolling-file crate

* Make test code change more consistent

* Revive dead and non-terminating test code path...

* Dispose batches early now that possible

* Split off thread handle very early at ::new()

* Tweak message for TooSmallDirByteLimitl

* Remove too much of indirection

* Remove needless pub from ::channel()

* Clarify test comments

* Avoid needless event creation if tracer is disabled

* Write tests around file rotation and spill-over

* Remove unneeded PathBuf::clone()s...

* Introduce inner struct instead of tuple...

* Remove unused enum BankStatus...

* Avoid .unwrap() for the case of disabled tracer...
2023-01-25 21:54:38 +09:00
Yihau Chen 9193b4221d
Revert "chore: workspace inheritance (#29509)" (#29892)
This reverts commit a67d239dde.
2023-01-25 15:50:41 +08:00
Yihau Chen a67d239dde
chore: workspace inheritance (#29509)
* introduce workspace.package

* introduce workspace.dependencies

* read version from root cargo.toml

* pass check when version = { workspace = true }

* don't bump version when version = { workspace = true }

* including workspace Cargo.toml when bump version

* programs/sbf use workspace inheritance

* fix increasing cargo version ignore program/sbf/Cargo.toml
2023-01-25 13:59:59 +08:00
Xiang Zhu 74f89d1494
Add run parent directory for accounts files (#29794)
* Add run parent directory for accounts files

* fix test test_concurrent_snapshot_packaging

* review comments.  renamed the path setup function

* Addressed most of the review comments

* remove explict type def for map result

* handle create_accounts_run_and_snapshot_dirs error with expect

* update with more review comments

* minor fixes from review comments

* simplify account_filename option assignment

* handle error from create_accounts_run_and_snapshot_dirs

* use then instead of then_some for lazy evaluation
2023-01-24 16:44:35 -08:00
behzad nouri 1c7662a37f
asserts that cluster-info keypair is consistent with contact-info id (#29818) 2023-01-24 16:57:55 +00:00
Kevin Ji dd92f225bb
Use Ipv4Addr::{LOCALHOST, UNSPECIFIED} constants (#29813) 2023-01-23 16:49:51 -06:00
Kevin Ji f4339bc0f2
faucet: Expose time-slice, per-time-cap, and per-request-cap in test validator (#29812) 2023-01-21 08:18:49 -08:00
Trent Nelson 9a5aca36e6
validator: remove `--no-duplicate-instance-check` (#29785) 2023-01-19 14:51:40 -07:00
Brooks d0aa93de21
Refactors accounts hash cache (#29625) 2023-01-12 10:43:50 -05:00
Jeff Washington (jwash) c9fe21e3c3
ancient append vecs enabled at -10k by default (#29565) 2023-01-10 05:33:43 -08:00
Pires 8db1f53fe7
Add pubsub address to RpcContactInfo (#29165)
* rpc: add pubsub address to RpcContactInfo

* cli: add pubsub endpoint to gossip node info

* dashboard: add pubsub endpoint

* add missing pubsub info test_rpc_get_cluster_nodes
2023-01-08 22:25:56 -07:00
behzad nouri 8c212f59ad
renames ContactInfo to LegacyContactInfo (#29566)
Working towards adding a new ContactInfo where new sockets can be
added in a backward compatible way.
2023-01-08 16:00:55 +00:00
Jeff Washington (jwash) 0f2bfa2b02
cleanup get_snapshot_storages (#29488)
* cleanup get_snapshot_storages

* pr feedback
2023-01-03 12:22:28 -08:00
Jeff Washington (jwash) 4cc1890f00
skip_rewrites will only be feature driven (#29468) 2023-01-02 20:15:42 -08:00
Tyera 067facfbef
Remove println (#29342) 2022-12-20 19:31:40 +00:00
Brooks 053775ad77
Elides unnecessary lifetimes (#29299) 2022-12-20 12:44:17 -05:00
Tyera c8dd5bbd48
Move getSecondaryIndexKeySize RPC to admin_rpc_service (#29003)
* Move getSecondaryIndexKeySize to admin_rpc_service

* Add AdminRpc test framework

* Move test to admin_rpc_service

* Update test

* Remove full-api infra
2022-12-19 19:54:05 -07:00
Brooks Prumo 4530d161b7
Fixes format strings (#29327) 2022-12-19 17:26:38 -05:00
Jeff Biseda a44ea779bd
add support for a repair protocol whitelist (#29161) 2022-12-15 19:24:23 -08:00
Jeff Washington (jwash) 356372b175
remove TestValidatorGenesis::caching_enabled (#29229) 2022-12-13 07:20:44 -08:00
Jeff Washington (jwash) 560143a267
remove ValidatorConfig.caching_enabled (#29172) 2022-12-09 11:31:55 -08:00
HaoranYi 33b15240ac
Revert #28945 (#29127)
revert #28945
2022-12-06 17:08:56 -06:00
steviez aeb6b53502
Remove unused Option<> around ValidatorConfig's SnapshotConfig (#29090)
Remove Option<> around ValidatorConfig's SnapshotConfig

The SnapshotConfig is required and is currently hard-coded to be a
Some().
2022-12-06 22:47:55 +00:00
behzad nouri 9524c9dbff patches errors from clippy::uninlined_format_args
https://rust-lang.github.io/rust-clippy/master/index.html#uninlined_format_args
2022-12-06 19:32:15 +00:00
HaoranYi b090bb4f56 Update validator/src/cli.rs
Co-authored-by: Michael Vines <mvines@gmail.com>
2022-12-06 13:31:50 -06:00
Haoran Yi e1ba5a2a63 add monitoring for open file descriptors stat 2022-12-06 13:31:50 -06:00
Luca b43eabaa47
change test validator from non upgradable bpf loader to upgradable bpf loader (#29051)
change for test validator from non upgradable to upgradable
2022-12-05 11:49:40 -08:00
Jeff Washington (jwash) e42649a8ba
remove cli arg to specify # accounts hash scan passes (#28976) 2022-11-29 16:47:51 -08:00
Jeff Washington (jwash) f0ef316b19
remove cli arg "--no-accounts-db-caching" (#28912) 2022-11-21 12:38:35 -08:00
eloylp 3d91c3f5cf
Refactor, move solana-validator CLI related configs to its own module (#28548)
* Refactor, move validator CLI related stuff to its own module(cli.rs)

The main function of the validator is getting big. Improve readability by moving away the CLI setup stuff to its own module.

* Restore help cli line

* Refactor,  move test validator CLI config to cli.rs module

* Fix  imports (cargo fmt)
2022-11-18 21:39:20 -08:00
Tyera c32377b5af
Split out quic- and udp-client definitions (#28762)
* Move ConnectionCache back to solana-client, and duplicate ThinClient, TpuClient there

* Dedupe thin_client modules

* Dedupe tpu_client modules

* Move TpuClient to TpuConnectionCache

* Move ThinClient to TpuConnectionCache

* Move TpuConnection and quic/udp trait implementations back to solana-client

* Remove enum_dispatch from solana-tpu-client

* Move udp-client to its own crate

* Move quic-client to its own crate
2022-11-18 12:21:45 -07:00
K-anon e8c8235474
Get Index Key Size RPC Support (#28383)
Co-authored-by: K-anon <IntokuSatori@users.noreply.github.com>
2022-11-16 20:39:11 -07:00
Jeff Washington (jwash) f69e847137
allow ancient append vec threshold to be set on cli (#28707)
* allow ancient append vec threshold to be set on cli

* pr responses
2022-11-11 11:30:05 -08:00
Brooks Prumo d1ba42180d
clippy for rust 1.65.0 (#28765) 2022-11-09 19:39:38 +00:00
Brooks Prumo 59bf1809fe
Uses SnapshotHash type in snapshot archive fields (#28681) 2022-10-31 14:28:35 -04:00
Dmitri Makarov 34865d032c chore: update Solana docs and code comments that specify "BPF" to "SBF" 2022-10-31 14:14:25 -04:00
steviez 2272fd807e
Remove Blockstore manual compaction code (#28409)
The manual Blockstore compaction that was being initiated from
LedgerCleanupService has been disabled for quite some time in favor of
several optimizations.

Co-authored-by: Ryo Onodera <ryoqun@gmail.com>
2022-10-28 10:39:00 +02:00
Brooks Prumo 510cd933a1
Fixes typo in bootstrap log (#28483) 2022-10-19 17:58:03 -04:00
Brooks Prumo 1e79053dd1
Requires known validators to have all snapshot hashes at bootstrap (#28284) 2022-10-19 10:08:02 -04:00
Tyera Eulberg 235da8c538
chore: bump serde_yaml from 0.8.26 to 0.9.13 (#28388)
* chore: bump serde_yaml from 0.8.26 to 0.9.13 (#27970)

* chore: bump serde_yaml from 0.8.26 to 0.9.13

Bumps [serde_yaml](https://github.com/dtolnay/serde-yaml) from 0.8.26 to 0.9.13.
- [Release notes](https://github.com/dtolnay/serde-yaml/releases)
- [Commits](https://github.com/dtolnay/serde-yaml/compare/0.8.26...0.9.13)

---
updated-dependencies:
- dependency-name: serde_yaml
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>

* Prepend ---

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2022-10-14 22:02:39 -06:00
Yueh-Hsuan Chiang 1224c545d5
Make FIFO related validator arguments public (#28282)
#### Summary of Changes
This PR makes two FIFO-related validator arguments public:
--rocksdb_shred_compaction and --rocksdb_fifo_shred_storage_size.

#### Test Plan
* There're already ~26 validators running FIFO in mainnet-beta for more than 30 days,
   no issues were reported so far.
* Ran a validator with FIFO for days and observed it's able to consistently catch up
  and create new roots.
2022-10-09 16:42:59 -07:00
Brooks Prumo 00a18a962e
formats logs in bootstrap (#28291) 2022-10-07 18:42:57 +00:00
steviez c802b12e10
Separate deprecated arguments out to reduce clutter (#28250) 2022-10-06 14:27:32 -05:00
steviez 7fef7d569a
Make RocksFifo storage size API expose an Option<u64> (#28192)
A fifo rocksdb instance must be opened with max size parameter on the
fifo columns. To support this, we previously plumbed a constant up to
callers that provided a default if unbounded growth desired.

This change attempts to be more rusty by exposing an option for this
value, and converting the option to a constant at the lowest level
possible.
2022-10-04 12:38:40 -05:00
Yueh-Hsuan Chiang 599677f965
Derive rocksdb_fifo_shred_storage_size based on limit-ledger-size (#27459)
### Problem
When FIFO compaction is used while --rocksdb_fifo_shred_storage_size
is unspecified, the FIFO shred storage size is set to a const default based
on the default `--limit-ledger-size`.

### Summary of the Change
When --rocksdb_fifo_shred_storage_size is unspecified, it is now
derived from `--limit-ledger-size` by reserving 1500 bytes for each
shred.
2022-09-28 00:32:27 -07:00
Tyera Eulberg 060ecf9cdb
Revert "chore: bump serde_yaml from 0.8.26 to 0.9.13 (#27970)" (#28055)
This reverts commit da9206fb87.
2022-09-25 04:49:11 +00:00
dependabot[bot] da9206fb87
chore: bump serde_yaml from 0.8.26 to 0.9.13 (#27970)
* chore: bump serde_yaml from 0.8.26 to 0.9.13

Bumps [serde_yaml](https://github.com/dtolnay/serde-yaml) from 0.8.26 to 0.9.13.
- [Release notes](https://github.com/dtolnay/serde-yaml/releases)
- [Commits](https://github.com/dtolnay/serde-yaml/compare/0.8.26...0.9.13)

---
updated-dependencies:
- dependency-name: serde_yaml
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2022-09-22 16:25:20 -06:00
behzad nouri 9a57c64f21
patches clippy errors from new rust nightly release (#27996) 2022-09-22 22:23:03 +00:00
dependabot[bot] e1a49f7766
chore: bump indicatif from 0.17.0 to 0.17.1 (#27968)
* chore: bump indicatif from 0.17.0 to 0.17.1

Bumps [indicatif](https://github.com/console-rs/indicatif) from 0.17.0 to 0.17.1.
- [Release notes](https://github.com/console-rs/indicatif/releases)
- [Commits](https://github.com/console-rs/indicatif/compare/0.17.0...0.17.1)

---
updated-dependencies:
- dependency-name: indicatif
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2022-09-21 23:32:11 +00:00