Commit Graph

3258 Commits

Author SHA1 Message Date
behzad nouri 9a57c64f21
patches clippy errors from new rust nightly release (#27996) 2022-09-22 22:23:03 +00:00
Brooks Prumo ff71df4695
Remove unnecessary call to `set_startup_verification_complete()` (#27986) 2022-09-22 16:54:17 -04:00
Brooks Prumo 1ee595ca9c
remove AccountsDb::initial_blockstore_processing_complete (#27974) 2022-09-22 13:52:04 -04:00
dependabot[bot] c4fa849844
chore: bump itertools from 0.10.3 to 0.10.5 (#27962)
* chore: bump itertools from 0.10.3 to 0.10.5

Bumps [itertools](https://github.com/rust-itertools/itertools) from 0.10.3 to 0.10.5.
- [Release notes](https://github.com/rust-itertools/itertools/releases)
- [Changelog](https://github.com/rust-itertools/itertools/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-itertools/itertools/commits)

---
updated-dependencies:
- dependency-name: itertools
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2022-09-21 15:28:49 -06:00
dependabot[bot] 11b7c45bff
chore: bump systemstat from 0.1.11 to 0.2.0 (#27967)
Bumps [systemstat](https://github.com/unrelentingtech/systemstat) from 0.1.11 to 0.2.0.
- [Release notes](https://github.com/unrelentingtech/systemstat/releases)
- [Commits](https://github.com/unrelentingtech/systemstat/compare/v0.1.11...v0.2.0)

---
updated-dependencies:
- dependency-name: systemstat
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-09-21 15:28:31 -06:00
Jeff Washington (jwash) f2d6a7ecea
bank.initial_blockstore_processing_complete to avoid concurrent hash calculations (#27776)
* bank.initial_blockstore_processing_complete to avoid concurrent hash calculations

* Update runtime/src/bank.rs

Co-authored-by: Brooks Prumo <brooks@prumo.org>

* Update runtime/src/bank.rs

Co-authored-by: Brooks Prumo <brooks@prumo.org>

* Rename TestValidator::set_startup_verification_complete()

* Initialize with `AtomicBool::new(false)` instead of `default()`

* snapshot tests: move where `initial_blockstore_processing_completed()` is called

* fixup bank_forks.rs calling `is_initial_blockstore_processing_complete()`

* only call initial_blockstore_processing_completed() in blockstore_processor

Co-authored-by: Brooks Prumo <brooks@prumo.org>
Co-authored-by: Brooks Prumo <brooks@solana.com>
2022-09-19 13:00:21 -07:00
behzad nouri abfb996135
tracks number of staked/stale/dead nodes in turbine cluster-nodes (#27915) 2022-09-19 18:16:04 +00:00
Ashwin Sekar 9119dc13ec
Add structure to house unprocessed transactions in banking_stage (#27777)
Separate storage for voting and transaction threads:
- Voting threads utilize a shared reference in order to dedup extraneous
  votes
- Transactions have thread local storage like before
2022-09-14 10:40:44 -07:00
Ashwin Sekar c74df830b1
Add structure to collect and coalesce vote packets (#27558)
* Add structure to collect and coalesce vote packets

Will be used in banking stage to throw out extraneous vote packets
before processing

* pr comments

* Update inner lock to arc to improve performance
2022-09-14 00:44:26 -07:00
Will Hickey c0e4379f43
Whickey/version v1.15 (#27739)
* Bump version to v1.13.0
* Bump version to v1.14.0
* Bump version to v1.15.0
2022-09-13 09:06:15 -05:00
apfitzge 079bf561b0
Clean_up/upb_push_comment (#27707) 2022-09-12 18:59:41 -05:00
Jeff Washington (jwash) 765c628546
use exit signal for acct idx bg threads (#27483) 2022-09-12 11:51:12 -07:00
behzad nouri 4f22ee8f9b uses varint encoding for vote-state lockout offsets
The commit removes CompactVoteStateUpdate and instead reduces serialized
size of VoteStateUpdate using varint encoding for vote-state lockout
offsets.
2022-09-12 16:31:20 +00:00
Christian Kamm 90b8a3a44d
Remove KeypairInsecureClone trait and add insecure_clone() instead (#27396)
See discussion in #26248
2022-09-12 14:59:41 +00:00
Michael Vines 83d4d128c2 Add --process-ledger-before-service flag to solana-validator 2022-09-11 07:58:42 -07:00
Jeff Washington (jwash) abd01553d5
tests: Keypair::new().pubkey() -> pubkey::new_rand (#27705) 2022-09-10 13:56:45 -07:00
Jeff Washington (jwash) 1f00b468e5
add enable_rehashing to AccountsPackage (#27644) 2022-09-08 09:25:25 -07:00
apfitzge a9c5adbf88
UnprocessedPacketBatches pop_max fn are only used in tests (#27645) 2022-09-08 11:01:14 -05:00
Maximilian Schneider cc58968b76
add new leader slot metric to track account contention throttling (#27654) 2022-09-08 09:22:58 -05:00
dependabot[bot] f338aa62ba
chore: bump serde from 1.0.143 to 1.0.144 (#27511)
* chore: bump serde from 1.0.143 to 1.0.144

Bumps [serde](https://github.com/serde-rs/serde) from 1.0.143 to 1.0.144.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.143...v1.0.144)

---
updated-dependencies:
- dependency-name: serde
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2022-09-07 16:54:44 -06:00
Xiang Zhu 4308c300b4
In ledger-tool delete the account files in the async way (#27622)
* In ledger-tool delete the account files in the async way

* format changes by ./cargo nightly fmt --all
2022-09-07 14:35:06 -07:00
Brooks Prumo 6a322de845
Make Accounts Background Services aware of Epoch Accounts Hash (#27626) 2022-09-07 20:41:40 +00:00
Lijun Wang 7f223dc582
Added option to turn on UDP for TPU transaction and make UDP based TPU off by default (#27462)
--tpu-enable-udp is introduced. And when this is on, the transaction receive and transaction forward is enabled using udp.

Except for a few tests which was hard-coded sending transactions using udp, most tests are being done with udp based tpu disabled.
2022-09-07 13:19:14 -07:00
apfitzge c04747dd66
cluster_slot_state_verifier: clippy nightly fixes (#27521)
clippy nightly fixes
2022-09-07 15:04:56 -05:00
apfitzge 1465ec947d
replay_stage: clippy nightly fixes (#27520)
clippy nightly fixes
2022-09-07 15:04:46 -05:00
apfitzge 452866dbcf
shredder: clippy nightly fixes (#27522)
clippy nightly fixes
2022-09-07 15:04:32 -05:00
apfitzge d6a1e7498f
Add tests for deserialize_and_collect_packets (#27623) 2022-09-07 12:52:18 -05:00
Jeff Washington (jwash) 22007a3c96
allow accounts hash calc to specify enable_rehashing (#27615) 2022-09-07 10:16:52 -07:00
Jeff Washington (jwash) a31d4a597d
serialize epoch_accounts_hash (#27516) 2022-09-07 10:07:00 -07:00
Brooks Prumo 93a4f80a2c
Handling snapshot requests is now required (#27537) 2022-09-07 10:08:42 -04:00
Jeff Biseda 269eb519dd
track time to coalesce entries in recv_slot_entries (#27525) 2022-09-06 16:07:17 -07:00
apfitzge a67d56f462
refactor: add function for deserializing and collecting packets - separate from channel receive (#27548) 2022-09-06 15:54:31 -05:00
Brooks Prumo 6684c62280
Add SnapshotUsage to SnapshotConfig (#27508) 2022-09-02 08:56:23 -04:00
Brennan Watt 242c9cb442
RPC Notifier Signal when Setup Complete (#27481)
* RPC notifier signal when ready
2022-09-01 16:39:55 -07:00
Tyera Eulberg 9b8bed86f9
Add getRecentPrioritizationFees RPC endpoint (#27278)
* Plumb priority_fee_cache into rpc

* Add PrioritizationFeeCache api

* Add getRecentPrioritizationFees rpc endpoint

* Use MAX_TX_ACCOUNT_LOCKS to limit input keys

* Remove unused cache apis

* Map fee data by slot, and make rpc account inputs optional

* Add priority_fee_cache to rpc test framework, and add test

* Add endpoint to jsonrpc docs

* Update docs/src/developing/clients/jsonrpc-api.md

* Update docs/src/developing/clients/jsonrpc-api.md
2022-09-01 23:12:12 +00:00
apfitzge 3bdc5b3f2b
separate packet_deserializer inside banking_stage (#27120)
* separate packet_deserializer inside banking_stage

* Make ReceivePacketResults into a struct with named fields
2022-09-01 10:00:48 -05:00
dependabot[bot] 66717ff87d
chore: bump chrono from 0.4.21 to 0.4.22 (#27509)
* chore: bump chrono from 0.4.21 to 0.4.22

Bumps [chrono](https://github.com/chronotope/chrono) from 0.4.21 to 0.4.22.
- [Release notes](https://github.com/chronotope/chrono/releases)
- [Changelog](https://github.com/chronotope/chrono/blob/v0.4.22/CHANGELOG.md)
- [Commits](https://github.com/chronotope/chrono/compare/v0.4.21...v0.4.22)

---
updated-dependencies:
- dependency-name: chrono
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2022-08-31 22:39:12 +00:00
Tao Zhu 8bb039d08d
collect min prioritization fees when replaying sanitized transactions (#26709)
* Collect blocks' minimum prioritization fees when replaying sanitized transactions

* Limits block min-fee metrics reporting to top 10 writable accounts

* Add service thread to asynchronously update and finalize prioritization fee cache

* Add bench test for prioritization_fee_cache

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
2022-08-31 08:00:55 -05:00
Tyera Eulberg 7bd08ad3ae
Featurize spinner in rpc-client and tpu-client (#27381)
* Add spinner features to rpc-client and tpu-client, and disable where unneeded

* Add doc comment
2022-08-30 09:56:56 -06:00
Haoran Yi 5b64107626 make pruned_bank channel unbonded.
In kin-sim, we found that bounded channel causes halt for account
background services. As the number of accounts grows, the time for
pruning and cleaning increases, which would leads to longer intervals
between the pruning of deaded bank slots. With 1.7B accounts, we will
exceed the 10K bounded channel threshold that causes halt of account
back ground services. Without pruning, the node will eventually run out
of memory.
2022-08-29 19:06:30 -05:00
Brooks Prumo 3c7cd62030
Move pruned_banks_receiver into PrunedBanksRequestHandler (#27445) 2022-08-29 13:30:06 -04:00
Brennan Watt 46a48760db
Switch concurrent replay from feature to param (#27401)
* Switch concurrent replay from feature to param
2022-08-26 12:36:02 -07:00
Will Hickey 5eefc256d6
Fix startup panic if removing accounts directory fails (#27386)
* Remove contents of accounts directory if deleting the directory fails.
2022-08-25 20:35:12 -05:00
Trent Nelson b1cff5d740 make fatal log message sound fatal 2022-08-25 21:49:12 +00:00
Jeff Biseda d1522fc790
coalesce entries in recv_slot_entries to target byte count (#27321) 2022-08-25 13:51:55 -07:00
Jeff Washington (jwash) 2da93bd45a
add text to assert (#27377) 2022-08-24 14:11:53 -05:00
Tyera Eulberg b8b3d723da
Use new client crates (#27360)
* Update ancillary cli crates

* Update cli

* Update command-line tools

* Update rpc, etc

* Update client-test

* Update core, validator

* Update local-cluster
2022-08-24 10:47:02 -06:00
Ashwin Sekar efa6201eda
Check overflow on vote tx compaction boundary (#27185)
* Check overflow on vote tx compaction boundary

Check for overflow during the conversion between VoteStateUpdate and
CompactVoteStateUpdate.

* Try removing clippy supress
2022-08-23 22:29:03 -07:00
Xiang Zhu 827d8e4bc0
Fallback to synchronous rm_dir call if path moving fails (#27306)
Remove some log lines, as suggested in PR #26910
2022-08-22 22:47:39 -07:00
Brennan Watt e4a7d01e10
Rust v1.63 (#27303)
* Upgrade to Rust v1.63.0

* Add nightly_clippy_allows

* Resolve some new clippy nightly lints

* Increase QUIC packets completion timeout

* Update quinn-udp crate

Co-authored-by: Michael Vines <mvines@gmail.com>
2022-08-22 18:01:03 -07:00