3865d9e80d
Do not check for full snapshot interval during bootstrap ( #21300 )
2021-11-16 13:38:23 -06:00
d5de0c8e12
add --no-os-network-stats-reporting option ( #21296 )
2021-11-16 10:26:03 -08:00
ae3777cadb
Update various terms to "known" ( #21257 )
...
* Update snapshot bootstrap terminology
* Update only known rpc terminology
* Update replica-node
2021-11-12 20:13:59 -07:00
b0ca335463
Rename "trusted" to "known" in `validators/` ( #21197 )
...
* Replaced trusted with known validator
* Format Convention
2021-11-12 11:57:55 -07:00
b4d09388aa
Disable JIT compiler on unsupported platforms ( #21215 )
2021-11-09 11:31:21 +00:00
c2754b8a51
chore: bump libc from 0.2.106 to 0.2.107 ( #21202 )
...
* chore: bump libc from 0.2.106 to 0.2.107
Bumps [libc](https://github.com/rust-lang/libc ) from 0.2.106 to 0.2.107.
- [Release notes](https://github.com/rust-lang/libc/releases )
- [Commits](https://github.com/rust-lang/libc/compare/0.2.106...0.2.107 )
---
updated-dependencies:
- dependency-name: libc
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2021-11-08 12:14:12 -07:00
0e751ef7df
Add validator option to change niceness of RPC server threads
...
Fixes https://github.com/solana-labs/solana/issues/14556
2021-11-04 17:16:46 -06:00
c78f474373
Add validator option to change niceness of snapshot packager thread
2021-11-04 17:16:46 -06:00
7495cbd1c8
Remove --no-incremental-snapshot-fetch CLI arg ( #21172 )
...
Now that bootstrap with incremental snapshots enabled has a fallback
mechanism in place, this no-incremental-snapshot-fetch flag is no longer
required.
Fixes #21127
2021-11-04 11:54:25 -05:00
d2974fcefc
Cleanup typos/comments in Bootstrap ( #21171 )
2021-11-04 08:21:33 -05:00
7508c36209
Bootstrap falls back to getting highest full snapshot ( #21124 )
2021-11-03 19:23:30 -05:00
fc3b1bc62f
chore: bump sysctl from 0.4.2 to 0.4.3 ( #21117 )
...
Bumps [sysctl](https://github.com/johalun/sysctl-rs ) from 0.4.2 to 0.4.3.
- [Release notes](https://github.com/johalun/sysctl-rs/releases )
- [Changelog](https://github.com/johalun/sysctl-rs/blob/master/CHANGELOG.md )
- [Commits](https://github.com/johalun/sysctl-rs/commits )
---
updated-dependencies:
- dependency-name: sysctl
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-11-02 00:27:22 +00:00
ca51ea3e04
chore: bump libc from 0.2.105 to 0.2.106 ( #21093 )
...
* chore: bump libc from 0.2.105 to 0.2.106
Bumps [libc](https://github.com/rust-lang/libc ) from 0.2.105 to 0.2.106.
- [Release notes](https://github.com/rust-lang/libc/releases )
- [Commits](https://github.com/rust-lang/libc/compare/0.2.105...0.2.106 )
---
updated-dependencies:
- dependency-name: libc
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2021-11-01 14:17:45 -06:00
9fabff5129
Parallel notifications in RPC PubSub ( #20543 )
...
* generate rpc notifications in parallel
* Use multithreaded runtime for pubsub
* add metric for time since creation of rpc notification to queue
* measure notification entry processing
* fix: add n_threads config argument
* configure rayon thread pool for rpc notifications
* add config option for pubsub notification threads
* rename metric to created_to_queue_time_us
Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
* rename metric to notification_entry_processing_time_us
Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
* use value_of for rpc_pubsub_notification_threads parsing
Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
* rename threads to sol-sub-notif-N
Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
* fix rpc tests for TimestampedNotificationEntry
* rustfmt
* use rayon thread limit for rpc
Co-authored-by: Pavel Strakhov <p.strakhov@iconic.vc>
Co-authored-by: Alexander Polakov <a.polakov@zubr.io>
Co-authored-by: Nikita Podoliako <bananaelecitrus@gmail.com>
Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
2021-11-01 00:17:24 -06:00
855b3ccdc0
chore: bump signal-hook from 0.2.3 to 0.3.10 ( #21081 )
...
* chore: bump signal-hook from 0.2.3 to 0.3.10
Bumps [signal-hook](https://github.com/vorner/signal-hook ) from 0.2.3 to 0.3.10.
- [Release notes](https://github.com/vorner/signal-hook/releases )
- [Changelog](https://github.com/vorner/signal-hook/blob/master/CHANGELOG.md )
- [Commits](https://github.com/vorner/signal-hook/compare/v0.2.3...v0.3.10 )
---
updated-dependencies:
- dependency-name: signal-hook
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* Update apis
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
2021-10-30 00:21:54 -06:00
9d07746642
Retain trusted peer snapshot hashes bugfix ( #21077 )
...
When retaining trusted peer snapshot hashes, and the peer's full
snapshot hashes match a trusted snapshot hash, but the peer doesn't have
incremental snapshot hashes, that's fine; the peer's hashes should be
retained, not discarded.
2021-10-29 20:21:33 -05:00
1453fce6c6
Check and unwrap() Option instead of .iter().for_each() ( #21072 )
2021-10-29 14:17:03 -05:00
a8d78e89d3
Move test-validator to own module to reduce core dependencies ( #20658 )
...
* Move test-validator to own module to reduce core dependencies
* Fix a few TestValidator paths
* Use solana_test_validator crate for solana_test_validator bin
* Move client int tests to separate crate
Co-authored-by: Tyera Eulberg <tyera@solana.com>
2021-10-29 01:27:07 +00:00
cb1947bea2
chore: bump libc from 0.2.104 to 0.2.105 ( #20972 )
...
* chore: bump libc from 0.2.104 to 0.2.105
Bumps [libc](https://github.com/rust-lang/libc ) from 0.2.104 to 0.2.105.
- [Release notes](https://github.com/rust-lang/libc/releases )
- [Commits](https://github.com/rust-lang/libc/compare/0.2.104...0.2.105 )
---
updated-dependencies:
- dependency-name: libc
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2021-10-26 00:14:58 -06:00
6470560dd1
check OS network limits when starting validator ( #20874 )
2021-10-25 18:25:55 -07:00
58aa2b964b
[solana-test-validator] add support for keypair file parsing for --bpf-program address argument
2021-10-25 16:02:11 -07:00
43ea579f63
add cli for --accounts-hash-num-passes ( #20827 )
2021-10-25 09:45:46 -05:00
350bb561eb
Clippy
2021-10-23 08:21:20 +00:00
86bf071d77
`solana-test-validator --log` now includes version/argument information
2021-10-22 13:01:56 -07:00
4821b0a1d9
Download incremental snapshots during bootstrap ( #20696 )
2021-10-21 09:14:51 -05:00
d1d4d687af
chore: bump console from 0.14.1 to 0.15.0 ( #20817 )
...
* chore: bump console from 0.14.1 to 0.15.0
Bumps [console](https://github.com/mitsuhiko/console ) from 0.14.1 to 0.15.0.
- [Release notes](https://github.com/mitsuhiko/console/releases )
- [Changelog](https://github.com/mitsuhiko/console/blob/master/CHANGELOG.md )
- [Commits](https://github.com/mitsuhiko/console/compare/0.14.1...v0.15.0 )
---
updated-dependencies:
- dependency-name: console
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2021-10-20 21:43:46 +00:00
2744a2128c
rpc-send-tx-svc: server-side retry knobs
2021-10-20 13:43:27 -06:00
fe098b5ddc
rpc-send-tx-svc: add with_config constructor
2021-10-20 13:43:27 -06:00
66e431dad5
chore: bump libc from 0.2.103 to 0.2.104 ( #20764 )
...
* chore: bump libc from 0.2.103 to 0.2.104
Bumps [libc](https://github.com/rust-lang/libc ) from 0.2.103 to 0.2.104.
- [Release notes](https://github.com/rust-lang/libc/releases )
- [Commits](https://github.com/rust-lang/libc/compare/0.2.103...0.2.104 )
---
updated-dependencies:
- dependency-name: libc
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2021-10-19 22:38:02 -06:00
72e1efb847
Refactor port check in bootstrap ( #20793 )
2021-10-19 21:35:42 +00:00
7a36bf5095
Add CLI args for no-incremental-snapshot-fetch ( #20787 )
2021-10-19 13:15:30 -05:00
3f559cc2c9
Move bootstrap code into inner module ( #20730 )
2021-10-19 09:50:15 -05:00
33f4e79589
Refactor out get_rpc_peers() ( #20744 )
2021-10-18 14:01:52 -05:00
a8e000a2a6
add filler accounts to bloat validator and predict failure ( #20491 )
...
* add filler accounts to bloat validator and predict failure
* assert no accounts match filler
* cleanup magic numbers
* panic if can't load from snapshot with filler accounts specified
* some renames
* renames
* into_par_iter
* clean filler accts, too
2021-10-11 12:46:27 -05:00
d621994fee
Accountsdb stream plugin improvement ( #20419 )
...
Support using connection pooling and use multiple threads to do Postgres db operations. The performance is improved from 1500 RPS to 40,000 RPS measured during validator start.
Support multiple plugins at the same time.
2021-10-08 20:06:58 -07:00
177a375479
Tpu vote 1.7 ( #20187 ) ( #20494 )
...
* Add separate vote processing tpu port
* Add feature to send to tpu vote port
* Add vote rejecting sigverify mode
* use packet.meta.is_simple_vote_tx in place of deserialization
* consolidate code that identifies vote tx atcommon path for cpu and gpu
* new key for feature set
* banking forward tpu vote
* add tpu vote port to dockerfile and other review changes
* Simplify thread id compare
* fix a test; updated cluster_info ABI change
Co-authored-by: Tao Zhu <tao@solana.com>
Co-authored-by: sakridge <sakridge@gmail.com>
2021-10-07 09:38:23 +00:00
767f740305
Bump version to 1.9.0
2021-10-06 17:57:41 -07:00
129716f3f0
Optimize stakes cache and rewards at epoch boundaries ( #20432 )
...
* Optimize stakes cache and rewards at epoch boundaries
* Fetch from accounts db
* Add cli flag for disabling epoch boundary optimization
2021-10-06 00:53:26 -04:00
221343e849
Default --rpc-bind-address to 127.0.0.1 when --private-rpc is provided and --bind-address is not
2021-10-04 16:32:18 -07:00
852b25e167
Move bootstrap code into its own module ( #20372 )
2021-10-03 13:13:16 -05:00
fc5dd7f3bc
Ignore delinquent stake on exit ( #20367 )
...
* add --ignore-delinquency flag to validator exit and wait-for-restart-window sub commands
* Fix a merge issue
* Add missing variable declaration
* Remove empty line to help CI checks pass
* run rustfmt
* Change argument wording for clarity and verbosity
* Change --ignore-delinquent-stake to --max-delinquent-stake
* cargo fmtgit add validator/src/main.rsgit add validator/src/main.rs
* Adjust per mvines
* Formatting
* Improve input validation
* Please automate cargo fmt somehow
2021-10-01 19:52:28 +00:00
fe97cb2ddf
AccountsDb plugin framework ( #20047 )
...
Summary of Changes
Create a plugin mechanism in the accounts update path so that accounts data can be streamed out to external data stores (be it Kafka or Postgres). The plugin mechanism allows
Data stores of connection strings/credentials to be configured,
Accounts with patterns to be streamed
PostgreSQL implementation of the streaming for different destination stores to be plugged in.
The code comprises 4 major parts:
accountsdb-plugin-intf: defines the plugin interface which concrete plugin should implement.
accountsdb-plugin-manager: manages the load/unload of plugins and provide interfaces which the validator can notify of accounts update to plugins.
accountsdb-plugin-postgres: the concrete plugin implementation for PostgreSQL
The validator integrations: updated streamed right after snapshot restore and after account update from transaction processing or other real updates.
The plugin is optionally loaded on demand by new validator CLI argument -- there is no impact if the plugin is not loaded.
2021-09-30 14:26:17 -07:00
8ae88632cb
Properly enable unprefixed_malloc_on_supported_platforms in tikv-jemallocator ( #20351 )
...
Trivial typo fix.
Fixes: 4bf6d0c4d7#20317 )")
2021-09-30 12:42:15 -06:00
4bf6d0c4d7
adds unprefixed_malloc_on_supported_platforms to jemalloc ( #20317 )
...
Without this feature jemalloc is used only for Rust code but not for
bundled C/C++ libraries (like rocksdb).
https://github.com/solana-labs/solana/issues/14366#issuecomment-930404992
2021-09-29 20:41:09 +00:00
5810568c65
add --accounts-index-path to be used by disk accounts index ( #19710 )
...
* add --accounts-index-paths to be used by disk accounts index
* add multiple and default to help
2021-09-28 11:07:47 -05:00
9393654af0
chore: bump libc from 0.2.102 to 0.2.103 ( #20280 )
...
* chore: bump libc from 0.2.102 to 0.2.103
Bumps [libc](https://github.com/rust-lang/libc ) from 0.2.102 to 0.2.103.
- [Release notes](https://github.com/rust-lang/libc/releases )
- [Commits](https://github.com/rust-lang/libc/compare/0.2.102...0.2.103 )
---
updated-dependencies:
- dependency-name: libc
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <anatoly+githubjenkins@solana.io>
2021-09-28 07:50:46 +00:00
49e85afdcd
Make rpc_bootstrap() aware of local incremental snapshots ( #20171 )
2021-09-27 13:05:32 -05:00
2cf081d863
uses tikv_jemallocator::Jemalloc as the global allocator ( #20149 )
...
https://github.com/solana-labs/solana/pull/16346
switched default allocator from jemalloc to system allocator, but that
has shown regressions in form of higher ram usage causing nodes go OOM:
https://discord.com/channels/428295358100013066/439194979856809985/890413193858539580
This commit sets jemalloc as the default allocator.
2021-09-24 14:26:25 +00:00
567f30aa1a
windows: Make solana-test-validator work ( #20099 )
...
* windows: Make solana-test-validator work
The important changes to get this going on Windows:
* ledger lock needs to be done on a file instead of the directory
* IPC service needs to use the Windows pipe naming scheme
* always disable the JIT
* file logging not possible yet because we can't redirect stderr,
but this will change once env_logger fixes the pipe output target!
* Integrate review feedback
2021-09-22 23:10:35 +02:00
c1d181add5
add --accounts-index-memory-limit-mb ( #19269 )
2021-09-19 18:00:15 -05:00
Brooks Prumo
Jeff Biseda
Tyera Eulberg
Michael Keleti
Kirill Fomichev
dependabot[bot]
Ivan Mironov
Pavel Strakhov
sakridge
Paul Schaaf
Jeff Washington (jwash)
Michael Vines
Trent Nelson
Lijun Wang
Tao Zhu
Justin Starry
Michael
behzad nouri
Jon Cinque