blockworks-foundation
/
solana
mirror of https://github.com/blockworks-foundation/solana.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
20,778 Commits 180 Branches 498 Tags 557 MiB
Commit Graph

3337 Commits

Author SHA1 Message Date
Jason Davis c899ededfc Minor refactoring and cleaning of cpuid code 2022-10-05 11:43:27 -05:00
Jason Davis 3b2ab313de Use num-enum crate to make everything typesafe 2022-10-05 11:43:27 -05:00
Jason Davis 1e1455688d Convert magic numbers to named constants 2022-10-05 11:43:27 -05:00
Jason Davis fac772ff90 Update naming, style after PR review comments 2022-10-05 11:43:27 -05:00
Jason Davis 13b095b4ab Fix a fmt problem, I think. Shows up in the git check, but not when I run here 2022-10-05 11:43:27 -05:00
Jason Davis c8584b0cdd Cargo fmt applied 2022-10-05 11:43:27 -05:00
Jason Davis d841286c21 Add cpuid calls and metric reporting; change cpu info sampling interval from 1s to 10s 2022-10-05 11:43:27 -05:00
Jeff Biseda e3e888c0e0
stats for staked/unstaked repair requests (#28215) 2022-10-04 17:37:24 -07:00
behzad nouri 9e7a0e7420
rolls out merkle shreds to ~5% of testnet (#28199) 2022-10-04 19:36:16 +00:00
carllin 14a415ccf3
Consensus Logging (#28176) 2022-10-03 20:45:55 -05:00
haoran c4aab3f178 typo 2022-10-03 09:41:15 -05:00
Justin Starry c2bb2b8e60
Allow validators to reset to the slot which matches their last voted slot (#28172) 
* Add failing test

* Allow resetting to duplicate was confirmed

* feedback

* feedback

* bump

* simplify change

* Revert "simplify change"

This reverts commit 72e5de3e5bdac595f71dc7fc01650ca3bc7da98e.

* update comment

* Update core/src/replay_stage.rs
 2022-10-03 16:49:47 +08:00
Yueh-Hsuan Chiang 6b17bee5a8
Remove the const default for RocksFifo (#27965) 
#### Summary of Changes
Removes the constant default for ShredStorageType::RocksFifo
as the shred storage size is either user-specified or derived
from --limit-ledger-size in #27459.
 2022-10-01 15:10:54 -07:00
Brooks Prumo 8877ac2aa9
Fix call to calculate_accounts_hash() (#28169) 2022-09-30 15:29:18 -04:00
Brooks Prumo 2f8f6c6a31
Send Epoch Accounts Hash requests from set_root() (#27764) 2022-09-30 14:59:41 -04:00
Jeff Washington (jwash) cfc124c825
acct idx can no longer use write cache (#28150) 2022-09-30 10:55:27 -07:00
apfitzge 82558226f7
ImmutableDeserializedPacket rc to arc (#28145) 2022-09-30 12:07:48 -05:00
Tao Zhu 82e65593ee
Batch filtering invalid transactions before forwarding (#26798) 
- Batch filtering invalid transactions (fail to sanitize, too old or already processed) before forwarding
- Combine packet filtering and forwarding to share sanitized transactions
- `iter_desc` is no longer needed, remove it;
- Add a method to share the logic of removing packets from buffer after they were removed from MinMaxHeap
- Add test coverage for forward_packet_batches_by_accounts
- rebase, resolve conflicts
 2022-09-29 16:33:40 -05:00
Ashwin Sekar 84acef007c
Add bench test for voting threads (#28031) 2022-09-27 12:12:22 -07:00
Jeff Biseda 8b0f9b4917
make ping cache rate limit delay configurable (#27955) 2022-09-26 14:16:56 -07:00
behzad nouri f49beb0cbc
caches reed-solomon encoder/decoder instance (#27510) 
ReedSolomon::new(...) initializes a matrix and a data-decode-matrix cache:
https://github.com/rust-rse/reed-solomon-erasure/blob/273ebbced/src/core.rs#L460-L466

In order to cache this computation, this commit caches the reed-solomon
encoder/decoder instance for each (data_shards, parity_shards) pair.
 2022-09-25 18:09:47 +00:00
Jeff Biseda 9816c94d7e
metrics to distinguish why repair packets are dropped (#27960) 2022-09-24 23:20:05 -07:00
Jeff Biseda 8b43215ddd
count unsigned repair requests (#27953) 2022-09-24 12:56:02 -07:00
Tao Zhu e51cf46d6b
Remove priority from vote transactions (#28030) 
vote transactions have same priority fee
 2022-09-24 00:31:50 +00:00
behzad nouri 9ee53e594d
patches clippy errors from new rust nightly release (#28028) 2022-09-23 20:57:27 +00:00
Brooks Prumo d9b31fd6b0
ahv: Add debug logging for EAH (#27998) 2022-09-23 14:04:48 -04:00
Jeff Biseda 206cc9407b
allow unsigned repair requests (#27910) 2022-09-23 10:11:08 -07:00
behzad nouri 97c9af4c6b plumbs through flag to generate merkle variant of shreds 2022-09-23 16:45:18 +00:00
steviez e4affb9fea
Add Blockstore::highest_slot() method (#27981) 2022-09-23 04:53:43 -05:00
behzad nouri 9a57c64f21
patches clippy errors from new rust nightly release (#27996) 2022-09-22 22:23:03 +00:00
Brooks Prumo ff71df4695
Remove unnecessary call to `set_startup_verification_complete()` (#27986) 2022-09-22 16:54:17 -04:00
Brooks Prumo 1ee595ca9c
remove AccountsDb::initial_blockstore_processing_complete (#27974) 2022-09-22 13:52:04 -04:00
dependabot[bot] c4fa849844
chore: bump itertools from 0.10.3 to 0.10.5 (#27962) 
* chore: bump itertools from 0.10.3 to 0.10.5

Bumps [itertools](https://github.com/rust-itertools/itertools) from 0.10.3 to 0.10.5.
- [Release notes](https://github.com/rust-itertools/itertools/releases)
- [Changelog](https://github.com/rust-itertools/itertools/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-itertools/itertools/commits)

---
updated-dependencies:
- dependency-name: itertools
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
 2022-09-21 15:28:49 -06:00
dependabot[bot] 11b7c45bff
chore: bump systemstat from 0.1.11 to 0.2.0 (#27967) 
Bumps [systemstat](https://github.com/unrelentingtech/systemstat) from 0.1.11 to 0.2.0.
- [Release notes](https://github.com/unrelentingtech/systemstat/releases)
- [Commits](https://github.com/unrelentingtech/systemstat/compare/v0.1.11...v0.2.0)

---
updated-dependencies:
- dependency-name: systemstat
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-09-21 15:28:31 -06:00
Jeff Washington (jwash) f2d6a7ecea
bank.initial_blockstore_processing_complete to avoid concurrent hash calculations (#27776) 
* bank.initial_blockstore_processing_complete to avoid concurrent hash calculations

* Update runtime/src/bank.rs

Co-authored-by: Brooks Prumo <brooks@prumo.org>

* Update runtime/src/bank.rs

Co-authored-by: Brooks Prumo <brooks@prumo.org>

* Rename TestValidator::set_startup_verification_complete()

* Initialize with `AtomicBool::new(false)` instead of `default()`

* snapshot tests: move where `initial_blockstore_processing_completed()` is called

* fixup bank_forks.rs calling `is_initial_blockstore_processing_complete()`

* only call initial_blockstore_processing_completed() in blockstore_processor

Co-authored-by: Brooks Prumo <brooks@prumo.org>
Co-authored-by: Brooks Prumo <brooks@solana.com>
 2022-09-19 13:00:21 -07:00
behzad nouri abfb996135
tracks number of staked/stale/dead nodes in turbine cluster-nodes (#27915) 2022-09-19 18:16:04 +00:00
Ashwin Sekar 9119dc13ec
Add structure to house unprocessed transactions in banking_stage (#27777) 
Separate storage for voting and transaction threads:
- Voting threads utilize a shared reference in order to dedup extraneous
  votes
- Transactions have thread local storage like before
 2022-09-14 10:40:44 -07:00
Ashwin Sekar c74df830b1
Add structure to collect and coalesce vote packets (#27558) 
* Add structure to collect and coalesce vote packets

Will be used in banking stage to throw out extraneous vote packets
before processing

* pr comments

* Update inner lock to arc to improve performance
 2022-09-14 00:44:26 -07:00
Will Hickey c0e4379f43
Whickey/version v1.15 (#27739) 
* Bump version to v1.13.0
* Bump version to v1.14.0
* Bump version to v1.15.0
 2022-09-13 09:06:15 -05:00
apfitzge 079bf561b0
Clean_up/upb_push_comment (#27707) 2022-09-12 18:59:41 -05:00
Jeff Washington (jwash) 765c628546
use exit signal for acct idx bg threads (#27483) 2022-09-12 11:51:12 -07:00
behzad nouri 4f22ee8f9b uses varint encoding for vote-state lockout offsets 
The commit removes CompactVoteStateUpdate and instead reduces serialized
size of VoteStateUpdate using varint encoding for vote-state lockout
offsets.
 2022-09-12 16:31:20 +00:00
Christian Kamm 90b8a3a44d
Remove KeypairInsecureClone trait and add insecure_clone() instead (#27396) 
See discussion in #26248
 2022-09-12 14:59:41 +00:00
Michael Vines 83d4d128c2 Add --process-ledger-before-service flag to solana-validator 2022-09-11 07:58:42 -07:00
Jeff Washington (jwash) abd01553d5
tests: Keypair::new().pubkey() -> pubkey::new_rand (#27705) 2022-09-10 13:56:45 -07:00
Jeff Washington (jwash) 1f00b468e5
add enable_rehashing to AccountsPackage (#27644) 2022-09-08 09:25:25 -07:00
apfitzge a9c5adbf88
UnprocessedPacketBatches pop_max fn are only used in tests (#27645) 2022-09-08 11:01:14 -05:00
Maximilian Schneider cc58968b76
add new leader slot metric to track account contention throttling (#27654) 2022-09-08 09:22:58 -05:00
dependabot[bot] f338aa62ba
chore: bump serde from 1.0.143 to 1.0.144 (#27511) 
* chore: bump serde from 1.0.143 to 1.0.144

Bumps [serde](https://github.com/serde-rs/serde) from 1.0.143 to 1.0.144.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.143...v1.0.144)

---
updated-dependencies:
- dependency-name: serde
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
 2022-09-07 16:54:44 -06:00
Xiang Zhu 4308c300b4
In ledger-tool delete the account files in the async way (#27622) 
* In ledger-tool delete the account files in the async way

* format changes by ./cargo nightly fmt --all
 2022-09-07 14:35:06 -07:00
Brooks Prumo 6a322de845
Make Accounts Background Services aware of Epoch Accounts Hash (#27626) 2022-09-07 20:41:40 +00:00
Lijun Wang 7f223dc582
Added option to turn on UDP for TPU transaction and make UDP based TPU off by default (#27462) 
--tpu-enable-udp is introduced. And when this is on, the transaction receive and transaction forward is enabled using udp.

Except for a few tests which was hard-coded sending transactions using udp, most tests are being done with udp based tpu disabled.
 2022-09-07 13:19:14 -07:00
apfitzge c04747dd66
cluster_slot_state_verifier: clippy nightly fixes (#27521) 
clippy nightly fixes
 2022-09-07 15:04:56 -05:00
apfitzge 1465ec947d
replay_stage: clippy nightly fixes (#27520) 
clippy nightly fixes
 2022-09-07 15:04:46 -05:00
apfitzge 452866dbcf
shredder: clippy nightly fixes (#27522) 
clippy nightly fixes
 2022-09-07 15:04:32 -05:00
apfitzge d6a1e7498f
Add tests for deserialize_and_collect_packets (#27623) 2022-09-07 12:52:18 -05:00
Jeff Washington (jwash) 22007a3c96
allow accounts hash calc to specify enable_rehashing (#27615) 2022-09-07 10:16:52 -07:00
Jeff Washington (jwash) a31d4a597d
serialize epoch_accounts_hash (#27516) 2022-09-07 10:07:00 -07:00
Brooks Prumo 93a4f80a2c
Handling snapshot requests is now required (#27537) 2022-09-07 10:08:42 -04:00
Jeff Biseda 269eb519dd
track time to coalesce entries in recv_slot_entries (#27525) 2022-09-06 16:07:17 -07:00
apfitzge a67d56f462
refactor: add function for deserializing and collecting packets - separate from channel receive (#27548) 2022-09-06 15:54:31 -05:00
Brooks Prumo 6684c62280
Add SnapshotUsage to SnapshotConfig (#27508) 2022-09-02 08:56:23 -04:00
Brennan Watt 242c9cb442
RPC Notifier Signal when Setup Complete (#27481) 
* RPC notifier signal when ready
 2022-09-01 16:39:55 -07:00
Tyera Eulberg 9b8bed86f9
Add getRecentPrioritizationFees RPC endpoint (#27278) 
* Plumb priority_fee_cache into rpc

* Add PrioritizationFeeCache api

* Add getRecentPrioritizationFees rpc endpoint

* Use MAX_TX_ACCOUNT_LOCKS to limit input keys

* Remove unused cache apis

* Map fee data by slot, and make rpc account inputs optional

* Add priority_fee_cache to rpc test framework, and add test

* Add endpoint to jsonrpc docs

* Update docs/src/developing/clients/jsonrpc-api.md

* Update docs/src/developing/clients/jsonrpc-api.md
 2022-09-01 23:12:12 +00:00
apfitzge 3bdc5b3f2b
separate packet_deserializer inside banking_stage (#27120) 
* separate packet_deserializer inside banking_stage

* Make ReceivePacketResults into a struct with named fields
 2022-09-01 10:00:48 -05:00
dependabot[bot] 66717ff87d
chore: bump chrono from 0.4.21 to 0.4.22 (#27509) 
* chore: bump chrono from 0.4.21 to 0.4.22

Bumps [chrono](https://github.com/chronotope/chrono) from 0.4.21 to 0.4.22.
- [Release notes](https://github.com/chronotope/chrono/releases)
- [Changelog](https://github.com/chronotope/chrono/blob/v0.4.22/CHANGELOG.md)
- [Commits](https://github.com/chronotope/chrono/compare/v0.4.21...v0.4.22)

---
updated-dependencies:
- dependency-name: chrono
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
 2022-08-31 22:39:12 +00:00
Tao Zhu 8bb039d08d
collect min prioritization fees when replaying sanitized transactions (#26709) 
* Collect blocks' minimum prioritization fees when replaying sanitized transactions

* Limits block min-fee metrics reporting to top 10 writable accounts

* Add service thread to asynchronously update and finalize prioritization fee cache

* Add bench test for prioritization_fee_cache

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
 2022-08-31 08:00:55 -05:00
Tyera Eulberg 7bd08ad3ae
Featurize spinner in rpc-client and tpu-client (#27381) 
* Add spinner features to rpc-client and tpu-client, and disable where unneeded

* Add doc comment
 2022-08-30 09:56:56 -06:00
Haoran Yi 5b64107626 make pruned_bank channel unbonded. 
In kin-sim, we found that bounded channel causes halt for account
background services. As the number of accounts grows, the time for
pruning and cleaning increases, which would leads to longer intervals
between the pruning of deaded bank slots. With 1.7B accounts, we will
exceed the 10K bounded channel threshold that causes halt of account
back ground services. Without pruning, the node will eventually run out
of memory.
 2022-08-29 19:06:30 -05:00
Brooks Prumo 3c7cd62030
Move pruned_banks_receiver into PrunedBanksRequestHandler (#27445) 2022-08-29 13:30:06 -04:00
Brennan Watt 46a48760db
Switch concurrent replay from feature to param (#27401) 
* Switch concurrent replay from feature to param
 2022-08-26 12:36:02 -07:00
Will Hickey 5eefc256d6
Fix startup panic if removing accounts directory fails (#27386) 
* Remove contents of accounts directory if deleting the directory fails.
 2022-08-25 20:35:12 -05:00
Trent Nelson b1cff5d740 make fatal log message sound fatal 2022-08-25 21:49:12 +00:00
Jeff Biseda d1522fc790
coalesce entries in recv_slot_entries to target byte count (#27321) 2022-08-25 13:51:55 -07:00
Jeff Washington (jwash) 2da93bd45a
add text to assert (#27377) 2022-08-24 14:11:53 -05:00
Tyera Eulberg b8b3d723da
Use new client crates (#27360) 
* Update ancillary cli crates

* Update cli

* Update command-line tools

* Update rpc, etc

* Update client-test

* Update core, validator

* Update local-cluster
 2022-08-24 10:47:02 -06:00
Ashwin Sekar efa6201eda
Check overflow on vote tx compaction boundary (#27185) 
* Check overflow on vote tx compaction boundary

Check for overflow during the conversion between VoteStateUpdate and
CompactVoteStateUpdate.

* Try removing clippy supress
 2022-08-23 22:29:03 -07:00
Xiang Zhu 827d8e4bc0
Fallback to synchronous rm_dir call if path moving fails (#27306) 
Remove some log lines, as suggested in PR #26910
 2022-08-22 22:47:39 -07:00
Brennan Watt e4a7d01e10
Rust v1.63 (#27303) 
* Upgrade to Rust v1.63.0

* Add nightly_clippy_allows

* Resolve some new clippy nightly lints

* Increase QUIC packets completion timeout

* Update quinn-udp crate

Co-authored-by: Michael Vines <mvines@gmail.com>
 2022-08-22 18:01:03 -07:00
Michael Vines 7bdeea10ad Assign custom names to the Rayon global thread pool 2022-08-22 17:56:55 +00:00
Michael Vines 7c01c1ecc6
Update delete_path thread name 2022-08-22 09:01:26 -07:00
Jeff Washington (jwash) fc1a4dd11a
run hash calc with index on failure (#27279) 2022-08-22 10:58:04 -05:00
Michael Vines 3f4731b37f Standardize thread names 
Tenets:
1. Limit thread names to 15 characters
2. Prefix all Solana-controlled threads with "sol"
3. Use Camel case. It's more character dense than Snake or Kebab case
 2022-08-20 07:49:39 -07:00
Xiang Zhu c54824e4f5
Account files remove (#26910) 
* Create a new function cleanup_accounts_paths, a trivial change

* Remove account files asynchronously

* Update and simplify the implementation after the validator test runs.

* Fixes after testing on the dev device

* Discard tokio.  Use thread instead

* Fix comments format

* Fix config type to pass the github test

* Fix failed tests.  Handle the case of non-existing path

* Final cleanup, addressing the review comments
Avoided OsString.
Made the function more generic with "impl AsRef<Path>"

Co-authored-by: Jeff Washington <jeff.washington@solana.com>
 2022-08-19 23:56:52 -07:00
apfitzge eb06bb61e8
banking stage: actually aggregate tracer packet stats (#27118) 
* aggregated_tracer_packet_stats_option was alwasys None

* Actually accumulate tracer packet stats
 2022-08-19 15:16:56 -05:00
Will Hickey dba2fd5a16
Enable QUIC client by default. Add arg to disable QUIC client. (Forward port #26927) (#27194) 
Enable QUIC client by default. Add arg to disable QUIC client.

* Enable QUIC client by default. Add arg to disable QUIC client.
* Deprecate --disable-quic-servers arg
* Add #[ignore] annotation to failing tests
 2022-08-19 09:15:15 -05:00
Brennan Watt 7573000d87
Revert "Rust v1.63.0 (#27148)" (#27245) 
This reverts commit a2e7bdf50a.
 2022-08-19 09:19:44 +01:00
behzad nouri 6928b2a5af
adds hash domain to ping-pong protocol (#27193) 
In order to maintain backward compatibility, for now the responding node
will hash the token both with and without domain so that the other node
will accept the response regardless of its upgrade status.
Once the cluster has upgraded to the new code, we will remove the legacy
domain = false case.
 2022-08-18 22:39:31 +00:00
Brennan Watt a2e7bdf50a
Rust v1.63.0 (#27148) 
* Upgrade to Rust v1.63.0

* Add nightly_clippy_allows

* Resolve some new clippy nightly lints

* Increase QUIC packets completion timeout

Co-authored-by: Michael Vines <mvines@gmail.com>
 2022-08-17 15:48:33 -07:00
behzad nouri fea66c8b63
derives Error trait for ClusterInfoError and core::result::Error (#27208) 2022-08-17 22:01:51 +00:00
Jeff Washington (jwash) 225cddcffb
serialize incremental_snapshot_hash (#26839) 
* serialize incremental_snapshot_hash

* pr feedback
 2022-08-17 15:14:31 -05:00
behzad nouri 3b87aa9227
reverts wide fanout in broadcast when the root node is down (#26359) 
A change included in
https://github.com/solana-labs/solana/pull/20480
was that when the root node in turbine broadcast tree is down, the
leader will broadcast the shred to all nodes in the first layer.
The intention was to mitigate the impact of dead nodes on shreds
propagation, because if the root node is down, then the entire cluster
will miss out the shred.
On the other hand, if x% of stake is down, this will cause 200*x% + 1
packets/shreds ratio at the broadcast stage which might contribute to
line-rate saturation and packet drop.
To avoid this bandwidth saturation issue, this commit reverts that logic
and always broadcasts shreds from the leader only to the root node.
As before we rely on erasure codes to recover shreds lost due to staked
nodes being offline.
 2022-08-16 19:40:06 +00:00
dependabot[bot] a0d1f4ef88
chore: bump serial_test from 0.8.0 to 0.9.0 (#27097) 
Bumps [serial_test](https://github.com/palfrey/serial_test) from 0.8.0 to 0.9.0.
- [Release notes](https://github.com/palfrey/serial_test/releases)
- [Commits](https://github.com/palfrey/serial_test/compare/v0.8.0...v0.9.0)

---
updated-dependencies:
- dependency-name: serial_test
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-08-15 15:53:12 -06:00
Justin Starry bdce208fe5
clean feature: `request_units_deprecated` (#27102) 
clean feature: request_units_deprecated
 2022-08-13 13:12:35 +01:00
Jeff Biseda e50013acdf
Handle JsonRpcService startup failure (#27075) 2022-08-11 23:25:20 -07:00
janlegner fc6cee9c06
allow staked nodes weight override (#26870) 
* Allowed staked nodes weight override (#26407)

* Allowed staked nodes weight override, passing only HashMap over to core module

Co-authored-by: Ondra Chaloupka <chalda@chainkeepers.io>
 2022-08-11 14:34:04 -07:00
dependabot[bot] f641d3bad6
chore: bump chrono from 0.4.19 to 0.4.21 (#27076) 
* chore: bump chrono from 0.4.19 to 0.4.21

Bumps [chrono](https://github.com/chronotope/chrono) from 0.4.19 to 0.4.21.
- [Release notes](https://github.com/chronotope/chrono/releases)
- [Changelog](https://github.com/chronotope/chrono/blob/main/CHANGELOG.md)
- [Commits](https://github.com/chronotope/chrono/compare/v0.4.19...v0.4.21)

---
updated-dependencies:
- dependency-name: chrono
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <you@example.com>
 2022-08-11 09:28:50 -06:00
behzad nouri ac91cdab74
removes buffering when generating coding shreds in broadcast (#25807) 
Given the 32:32 erasure recovery schema, current implementation requires
exactly 32 data shreds to generate coding shreds for the batch (except
for the final erasure batch in each slot).
As a result, when serializing ledger entries to data shreds, if the
number of data shreds is not a multiple of 32, the coding shreds for the
last batch cannot be generated until there are more data shreds to
complete the batch to 32 data shreds. This adds latency in generating
and broadcasting coding shreds.

In addition, with Merkle variants for shreds, data shreds cannot be
signed and broadcasted until coding shreds are also generated. As a
result *both* code and data shreds will be delayed before broadcast if
we still require exactly 32 data shreds for each batch.

This commit instead always generates and broadcast coding shreds as soon
as there any number of data shreds available. When serializing entries
to shreds:
* if the number of resulting data shreds is less than 32, then more
  coding shreds will be generated so that the resulting erasure batch
  has the same recovery probabilities as a 32:32 batch.
* if the number of data shreds is more than 32, then the data shreds are
  split uniformly into erasure batches with _at least_ 32 data shreds in
  each batch. Each erasure batch will have the same number of code and
  data shreds.

For example:
* If there are 19 data shreds, 27 coding shreds are generated. The
  resulting 19(data):27(code) erasure batch has the same recovery
  probabilities as a 32:32 batch.
* If there are 107 data shreds, they are split into 3 batches of 36:36,
  36:36 and 35:35 data:code shreds each.

A consequence of this change is that code and data shreds indices will
no longer align as there will be more coding shreds than data shreds
(not only in the last batch in each slot but also in the intermediate
ones);
 2022-08-11 12:44:27 +00:00
Michael Vines 4e79d78629 `solana-validator monitor` how displays slot and gossip stake % while waiting for supermajority 2022-08-10 11:13:25 -07:00
dependabot[bot] e3a8d2ecdd
chore: bump serde_json from 1.0.81 to 1.0.83 (#27036) 
* chore: bump serde_json from 1.0.81 to 1.0.83

Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.81 to 1.0.83.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.81...v1.0.83)

---
updated-dependencies:
- dependency-name: serde_json
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
 2022-08-10 09:45:42 -06:00