Commit Graph

260 Commits

Author SHA1 Message Date
Pavel Strakhov 65227f44dc
Optimize RPC pubsub for multiple clients with the same subscription (#18943)
* reimplement rpc pubsub with a broadcast queue

* update tests for new pubsub implementation

* fix: fix review suggestions

* chore(rpc): add additional pubsub metrics

* integrate max subscriptions check into SubscriptionTracker to reduce locking

* separate subscription control from tracker

* limit memory usage of items in pubsub broadcast queue, improve error handling

* add more pubsub metrics

* add final count metrics to pubsub

* add metric for total number of subscriptions

* fix small review suggestions

* remove by_params from SubscriptionTracker and add node_progress_watchers map instead

* add subscription tracker tests

* add metrics for number of pubsub notifications as a counter

* ignore clippy lint in TokenCounter

* fix underflow in token counter

* reduce queue capacity in pubsub tests

* fix(rpc): fix test timeouts

* fix race in account subscription test

* Add RpcSubscriptions::new_for_tests

Co-authored-by: Pavel Strakhov <p.strakhov@iconic.vc>
Co-authored-by: Nikita Podoliako <n.podoliako@zubr.io>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
2021-09-17 13:40:14 -06:00
dependabot[bot] c19fce1b19
chore: bump env_logger from 0.8.4 to 0.9.0 (#18690)
* chore: bump env_logger from 0.8.4 to 0.9.0

Bumps [env_logger](https://github.com/env-logger-rs/env_logger) from 0.8.4 to 0.9.0.
- [Release notes](https://github.com/env-logger-rs/env_logger/releases)
- [Changelog](https://github.com/env-logger-rs/env_logger/blob/main/CHANGELOG.md)
- [Commits](https://github.com/env-logger-rs/env_logger/compare/v0.8.4...v0.9.0)

---
updated-dependencies:
- dependency-name: env_logger
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* Update all Cargo.lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
2021-07-15 17:49:59 +00:00
dependabot[bot] 05924423c2
chore: bump reqwest from 0.11.2 to 0.11.4 (#18362)
* chore: bump reqwest from 0.11.2 to 0.11.4

Bumps [reqwest](https://github.com/seanmonstar/reqwest) from 0.11.2 to 0.11.4.
- [Release notes](https://github.com/seanmonstar/reqwest/releases)
- [Changelog](https://github.com/seanmonstar/reqwest/blob/master/CHANGELOG.md)
- [Commits](https://github.com/seanmonstar/reqwest/compare/v0.11.2...v0.11.4)

---
updated-dependencies:
- dependency-name: reqwest
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <you@example.com>
2021-07-01 09:59:30 -06:00
dependabot[bot] 78968d132f
chore: bump log from 0.4.11 to 0.4.14 (#18323)
* chore: bump log from 0.4.11 to 0.4.14

Bumps [log](https://github.com/rust-lang/log) from 0.4.11 to 0.4.14.
- [Release notes](https://github.com/rust-lang/log/releases)
- [Changelog](https://github.com/rust-lang/log/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/log/compare/0.4.11...0.4.14)

---
updated-dependencies:
- dependency-name: log
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* Make version consistent

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
2021-06-30 18:32:01 +00:00
dependabot[bot] 9b30f094f1
chore: bump env_logger from 0.8.3 to 0.8.4 (#18224)
* chore: bump env_logger from 0.8.3 to 0.8.4

Bumps [env_logger](https://github.com/env-logger-rs/env_logger) from 0.8.3 to 0.8.4.
- [Release notes](https://github.com/env-logger-rs/env_logger/releases)
- [Changelog](https://github.com/env-logger-rs/env_logger/blob/main/CHANGELOG.md)
- [Commits](https://github.com/env-logger-rs/env_logger/compare/v0.8.3...v0.8.4)

---
updated-dependencies:
- dependency-name: env_logger
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2021-06-25 11:28:09 -06:00
Alexander Meißner 6514096a67 chore: cargo +nightly clippy --fix -Z unstable-options 2021-06-18 10:42:46 -07:00
Trent Nelson 5cc073420a metrics: Don't unwrap client instantiation errors 2021-06-16 18:32:17 +00:00
dependabot[bot] 3670435db4
chore: bump serial_test from 0.4.0 to 0.5.1 (#17705)
Bumps [serial_test](https://github.com/palfrey/serial_test) from 0.4.0 to 0.5.1.
- [Release notes](https://github.com/palfrey/serial_test/releases)
- [Commits](https://github.com/palfrey/serial_test/compare/v0.4.0...v0.5.1)

---
updated-dependencies:
- dependency-name: serial_test
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-06-03 10:07:56 -06:00
Michael Vines 8eab0e8602
Bump version to v1.8.0 (#17541) 2021-05-27 08:51:53 -07:00
Ruud van Asseldonk 3f92abedd5
Make metrics tests independent of RUST_LOG env var (#16710)
Previously, running the tests with RUST_LOG=none would fail, because the
env logger would set its filter level to reject all log messages, and
incrementing a counter only happens if the global logger has at least
the specified log level. Having the tests behave differently when
RUST_LOG is set is surprising, they should be self-contained.

Nix' buildRustPackage sets RUST_LOG="" to make the build logs less
verbose. I have trouble packaging Solana for Nix because of this, and I
believe making the tests independent of the environment is a good
solution for this.
2021-04-21 18:14:49 -06:00
Justin Starry 85eb37fab0
Merge pull request from GHSA-8v47-8c53-wwrc
* Track transaction check time separately from account loads

* banking packet process metrics

* Remove signature clone in status cache lookup

* Reduce allocations when converting packets to transactions

* Add blake3 hash of transaction messages in status cache

* Bug fixes

* fix tests and run fmt

* Address feedback

* fix simd tx entry verification

* Fix rebase

* Feedback

* clean up

* Add tests

* Remove feature switch and fall back to signature check

* Bump programs/bpf Cargo.lock

* clippy

* nudge benches

* Bump `BankSlotDelta` frozen ABI hash`

* Add blake3 to sdk/programs/Cargo.lock

* nudge bpf tests

* short circuit status cache checks

Co-authored-by: Trent Nelson <trent@solana.com>
2021-04-13 00:28:08 -06:00
sakridge ff2242d0e3
Update dashboard with new poh metrics (#15991) 2021-03-22 11:38:57 -07:00
Michael Vines 02b81dd05d Update to reqwest 0.11.2 2021-03-18 11:59:41 -07:00
Michael Vines a2eb655322 =1.7.0 2021-03-16 07:51:07 +00:00
Michael Vines 0c9ca5522c Bump version to v1.7.0 2021-03-13 09:01:21 +00:00
Trent Nelson 24d18b3cf2 docs: add docs links for crates published to crates.io 2021-03-11 08:38:18 +00:00
dependabot[bot] 4eab51fe06 chore: bump env_logger from 0.8.2 to 0.8.3
Bumps [env_logger](https://github.com/env-logger-rs/env_logger) from 0.8.2 to 0.8.3.
- [Release notes](https://github.com/env-logger-rs/env_logger/releases)
- [Changelog](https://github.com/env-logger-rs/env_logger/blob/master/CHANGELOG.md)
- [Commits](https://github.com/env-logger-rs/env_logger/compare/v0.8.2...v0.8.3)

Signed-off-by: dependabot[bot] <support@github.com>
2021-03-05 09:06:36 -08:00
Trent Nelson 7f7370c306 Re-allow clippy::integer_arithmetic at crate-level 2021-02-17 13:55:08 -07:00
Tom Parker-Shemilt 01230a0105
Remove serial_test_derive dependency (#14891) 2021-01-28 22:35:31 -07:00
Michael Vines cbffab7850 Upgrade to Rust v1.49.0 2021-01-23 19:16:36 -08:00
sakridge 391a1ea760
Add panel to show slot production (#14615) 2021-01-17 08:35:01 -08:00
Michael Vines 1e977ac025 Bump version to v1.6.0 2020-12-15 18:28:04 +00:00
dependabot[bot] 88d950e5a8
chore: bump log from 0.4.8 to 0.4.11 (#13691)
* chore: bump log from 0.4.8 to 0.4.11

Bumps [log](https://github.com/rust-lang/log) from 0.4.8 to 0.4.11.
- [Release notes](https://github.com/rust-lang/log/releases)
- [Changelog](https://github.com/rust-lang/log/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/log/compare/0.4.8...0.4.11)

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2020-12-13 13:48:23 +09:00
dependabot[bot] 549a3107cb
chore: bump env_logger from 0.7.1 to 0.8.2 (#14093)
* chore: bump env_logger from 0.7.1 to 0.8.2

Bumps [env_logger](https://github.com/env-logger-rs/env_logger) from 0.7.1 to 0.8.2.
- [Release notes](https://github.com/env-logger-rs/env_logger/releases)
- [Changelog](https://github.com/env-logger-rs/env_logger/blob/master/CHANGELOG.md)
- [Commits](https://github.com/env-logger-rs/env_logger/compare/v0.7.1...v0.8.2)

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2020-12-13 05:59:54 +09:00
sakridge f0e9341450
Add dropped hash to dashboard (#13972) 2020-12-05 18:02:01 -08:00
Michael Vines 62f20bc170 Remove unsupported metrics tarball from release artifacts 2020-10-20 13:10:33 -07:00
Michael Vines 8a119c1483 Bump reqwest/rayon to get past `cargo audit` 2020-10-11 18:51:09 +00:00
Michael Vines 11df2e2236 Bump version to v1.5.0 2020-10-08 04:51:36 +00:00
Tyera Eulberg be7c30df35
Update Current Leader query (#12465) 2020-09-24 20:42:30 -06:00
sakridge 4bd0ced87d
Back out dashboard (#11791) 2020-08-21 21:48:37 -07:00
sakridge b10f874f49
Reorganize dashboard (#11639)
Split out sections for gossip, repair, leader.
2020-08-14 14:42:33 -07:00
sakridge bb8ccbb022
Add retransmit stats (#11588) 2020-08-12 12:26:14 -07:00
Trent Nelson e12ab9d0dd Bump version to 1.4.0 2020-08-05 12:04:15 -06:00
carllin a7ea340f22
Track votes from gossip for optimistic confirmation (#11209)
* Add check in cluster_info_vote_listenere to see if optimstic conf was achieved
Add OptimisticConfirmationVerifier

* More fixes

* Fix merge conflicts

* Remove gossip notificatin

* Add dashboards

* Fix rebase

* Count switch votes as well toward optimistic conf

* rename

Co-authored-by: Carl <carl@solana.com>
2020-07-28 09:33:27 +00:00
carllin 7a14e359d7
Switch to using weighted repair in RepairService (#10735)
* Plumb votes into repair service

* Remove refactoring

* Fix tests

* Switch to using RepairWeight for generating repairs

* Revert "Weight repair slots based on vote stake (#10741)"

This reverts commit cabd0a09c3.

* Update logging

Co-authored-by: Carl <carl@solana.com>
2020-07-10 05:52:54 +00:00
sakridge ea708b0d84
Add blockstore insert metrics to dashboard (#10787) 2020-06-24 18:21:26 -07:00
sakridge 4e04282564
Add correct erasure metrics to dashboard (#10781) 2020-06-24 13:16:56 -07:00
sakridge b172b3369e
Add new split poh/transaction verify times to dashboard (#10706) 2020-06-18 22:21:11 -07:00
R. M. Shea 48febf153b
Change documentation introduction to indicate Foundation IP transfer (#10440) 2020-06-13 14:41:05 -07:00
Michael Vines 769f456a22 Add RPC Send Transaction Service graphs 2020-06-10 18:14:06 -07:00
sakridge d59fd7d512
Add gossip/poh to dashboard (#10449) 2020-06-09 10:25:10 -07:00
Kristofer Peterson e23340d89e
Clippy cleanup for all targets and nighly rust (also support 1.44.0) (#10445)
* address warnings from 'rustup run beta cargo clippy --workspace'

minor refactoring in:
- cli/src/cli.rs
- cli/src/offline/blockhash_query.rs
- logger/src/lib.rs
- runtime/src/accounts_db.rs

expect some performance improvement AccountsDB::clean_accounts()

* address warnings from 'rustup run beta cargo clippy --workspace --tests'

* address warnings from 'rustup run nightly cargo clippy --workspace --all-targets'

* rustfmt

* fix warning stragglers

* properly fix clippy warnings test_vote_subscribe()
replace ref-to-arc with ref parameters where arc not cloned

* Remove lock around JsonRpcRequestProcessor (#10417)

automerge

* make ancestors parameter optional to avoid forcing construction of empty hash maps

Co-authored-by: Greg Fitzgerald <greg@solana.com>
2020-06-09 09:38:14 +09:00
dependabot-preview[bot] 6cf30c38a4 Bump reqwest from 0.10.5 to 0.10.6
Bumps [reqwest](https://github.com/seanmonstar/reqwest) from 0.10.5 to 0.10.6.
- [Release notes](https://github.com/seanmonstar/reqwest/releases)
- [Changelog](https://github.com/seanmonstar/reqwest/blob/master/CHANGELOG.md)
- [Commits](https://github.com/seanmonstar/reqwest/commits)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-06-03 09:01:36 -07:00
Michael Vines 9eefb2c929 Bump version to 1.3.0 2020-05-28 15:01:13 -07:00
Michael Vines 156387aba4
LedgerCleanupService no longer causes an OOM and actually purges (#10199)
* cleanup_ledger() now services new_root_receiver while purging
* purge_slots() now fully deletes before compacting
* Add ledger pruning grafana graph
2020-05-24 21:41:54 -07:00
sakridge 9222bc2b35
Add 30s option to metrics. (#10081)
10s too short, 1m too long, 30s just right.
2020-05-16 10:46:17 -07:00
Kristofer Peterson 58ef02f02b
9951 clippy errors in the test suite (#10030)
automerge
2020-05-15 09:35:43 -07:00
carllin 01ab1d1369
Add metrics for logging time taken in replaystage steps (#9933)
automerge
2020-05-08 03:46:29 -07:00
Stephen Akridge eb24f3df84 Update dalek version 2020-04-28 21:02:47 -06:00
Michael Vines 3acfe42622
Exit cleanly on panic! so the process don't limp along in a half-dead state (#9690) 2020-04-23 12:05:13 -07:00