blockworks-foundation
/
solana
mirror of https://github.com/blockworks-foundation/solana.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
19,442 Commits 174 Branches 497 Tags 541 MiB
Commit Graph

3116 Commits

Author SHA1 Message Date
behzad nouri 5dbf7d8f91
removes raw indexing into packet data (#25554) 
Packets are at the boundary of the system where, vast majority of the
time, they are received from an untrusted source. Raw indexing into the
data buffer can open attack vectors if the offsets are invalid.
Validating offsets beforehand is verbose and error prone.

The commit updates Packet::data() api to take a SliceIndex and always to
return an Option. The call-sites are so forced to explicitly handle the
case where the offsets are invalid.
 2022-06-03 01:05:06 +00:00
behzad nouri 81231a89b9 adds support for different variants of ShredCode and ShredData 
The commit implements two new types:
    pub enum ShredCode {
        Legacy(legacy::ShredCode),
    }
    pub enum ShredData {
        Legacy(legacy::ShredData),
    }

Following commits will extend these types by adding merkle variants:
    pub enum ShredCode {
        Legacy(legacy::ShredCode),
        Merkle(merkle::ShredCode),
    }
    pub enum ShredData {
        Legacy(legacy::ShredData),
        Merkle(merkle::ShredData),
    }
 2022-06-02 18:55:50 +00:00
Pankaj Garg 1c2ae470c5
Fix forwarding of transactions over QUIC (#25674) 
* Spawn QUIC server to receive forwarded txs

* Update validator port range

* forward votes using UDP

* no forwarding from unstaked nodes

* forwarding stats in banking stage

* fix test builds

* fix lifetime of forward sender
 2022-06-02 11:14:58 -07:00
HaoranYi d3ac4e941b
Bench: preshrink + sigverify (#25480) 
* double shrinking

* add bench

* rename

* aggregate timing

* remove pre/post shrink time

* update api after merge
 2022-06-02 09:19:01 -05:00
Tao Zhu 51ac599915
Add user requested CU (eg. compute_budget.compute_unit_limit) to immutable_deserialized_packet, to be used in cost model and prioritized forwarding (#25695) 2022-06-01 22:43:48 +00:00
Ryo Onodera aedcb05dc8
Record solana-validator ver to metrics at startup (#25635) 
* Record solana-validator ver to metrics at startup

* Update Cargo.lock
 2022-06-01 13:37:50 +09:00
Christian Kamm 02b26ddd82
SigVerify: Fix num_valid_packets metric (#25643) 
It used to report the number of packets with successful signature
validations but was accidentally changed to count packets passed into
the verifier by e4409a87fe.

This restores the previous meaning.
 2022-05-31 18:51:20 +10:00
carllin 90a3315b69
Detect tracer key in sigverify (#25579) 
* Mark the tracer transaction

* simplify tracer check
 2022-05-30 18:41:54 -05:00
Justin Starry e4409a87fe
Add pre shrink pass before sigverify batch (#25136) 2022-05-28 01:51:55 +10:00
Yueh-Hsuan Chiang 5b67960c76
(Refactor) Move blocktore options related stuff to blockstore_options.rs (#25509) 
#### Problem
blockstore_db.rs has a mutual dependency between blockstore_metrics.rs.

#### Summary of Changes
This PR removes the mutual dependency by moving the option-related stuff
out from blockstore_db.rs to its new home --- blockstore_options.rs.

By doing this, we address the mutual dependency and also make the code cleaner.
 2022-05-26 16:59:26 -07:00
dependabot[bot] 7f4128947b
chore: bump lru from 0.7.5 to 0.7.6 (#25572) 
* chore: bump lru from 0.7.5 to 0.7.6

Bumps [lru](https://github.com/jeromefroe/lru-rs) from 0.7.5 to 0.7.6.
- [Release notes](https://github.com/jeromefroe/lru-rs/releases)
- [Changelog](https://github.com/jeromefroe/lru-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/jeromefroe/lru-rs/compare/0.7.5...0.7.6)

---
updated-dependencies:
- dependency-name: lru
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
 2022-05-26 19:05:02 +00:00
ryleung-solana 1ca5c3a7bd
Switch to using enum-dispatch to switch between UDP and Quic (#24713) 2022-05-26 11:21:16 -04:00
behzad nouri de612c25b3
removes shred wire layout specs from sigverify (#25520) 
sigverify_shreds relies on wire layout specs of shreds:
https://github.com/solana-labs/solana/blob/0376ab41a/ledger/src/sigverify_shreds.rs#L39-L46
https://github.com/solana-labs/solana/blob/0376ab41a/ledger/src/sigverify_shreds.rs#L298-L305

In preparation of
https://github.com/solana-labs/solana/pull/25237
which adds a new shred variant with different layout and signed message,
this commit removes shred layout specification from sigverify and
instead encapsulate that in shred module.
 2022-05-26 13:06:27 +00:00
Christian Kamm 0efb7478cd
FindPacketSenderStake: Remove parallelism to improve performance (#25562) 
* FindPacketSenderStake: Remove parallelism to improve performance

The work unit sizes were so small that using the thread pool
slowed down this stage significantly.

* fix checks

Co-authored-by: Justin Starry <justin@solana.com>
 2022-05-26 21:17:52 +10:00
behzad nouri cafa85bfbb
includes shred-type when computing turbine broadcast seed (#25556) 
Indices for code and data shreds of the same slot overlap; and so they
will have the same random number generator seed when shuffling cluster
nodes for turbine broadcast.

This results in the same propagation path for code and data shreds of
the same index and effectively smaller sample size for re-transmitter
nodes. For example a 32:32 batch (32 code + 32 data shreds), is
retransmitted through _at most_ 32 unique nodes, whereas ideally we want
~64 unique re-transmitters.

This commit adds shred-type to seed function so that code and data
sherds of the same (slot, index) will (most likely) have different
propagation paths.
 2022-05-25 20:31:53 +00:00
behzad nouri 880684565c
limits read access into Packet data to Packet.meta.size (#25484) 
Bytes past Packet.meta.size are not valid to read from.

The commit makes the buffer field private and instead provides two
methods:
* Packet::data() which returns an immutable reference to the underlying
  buffer up to Packet.meta.size. The rest of the buffer is not valid to
  read from.
* Packet::buffer_mut() which returns a mutable reference to the entirety
  of the underlying buffer to write into. The caller is responsible to
  update Packet.meta.size after writing to the buffer.
 2022-05-25 16:52:54 +00:00
carllin 9651cdad99
Refactor Sigverify trait (#25359) 2022-05-24 16:01:41 -05:00
Jeff Biseda 61c5a471e8
preserve optimistic_slot in blockstore (#25311) 2022-05-24 12:03:28 -07:00
Justin Starry e66ea7cb6a Clean up Bank::commit_transactions parameters 2022-05-24 20:24:42 +08:00
Justin Starry cad1c41ce2 Add Packet::deserialize_slice convenience method 2022-05-24 17:31:14 +08:00
Tyera Eulberg 514f73f4b1
Remove retain_mut dep (#25494) 2022-05-23 21:45:49 +00:00
steviez ec7ca411dd
Make PacketBatch packets vector non-public (#25413) 
Upcoming changes to PacketBatch to support variable sized packets will
modify the internals of PacketBatch. So, this change removes usage of
the internal packet struct and instead uses accessors (which are
currently just wrappers of Vector functions but will change down the
road).
 2022-05-23 15:30:15 -05:00
dependabot[bot] e75569e85a
chore: bump systemstat from 0.1.10 to 0.1.11 (#25471) 
Bumps [systemstat](https://github.com/unrelentingtech/systemstat) from 0.1.10 to 0.1.11.
- [Release notes](https://github.com/unrelentingtech/systemstat/releases)
- [Commits](https://github.com/unrelentingtech/systemstat/compare/v0.1.10...v0.1.11)

---
updated-dependencies:
- dependency-name: systemstat
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-05-23 20:08:47 +00:00
Christian Kamm 6429aff13b
findpacketsenderstake: add discard after receive (#25458) 
This mimics a similar change in sigverify, see #25388
 2022-05-23 21:27:20 +02:00
behzad nouri c248fb3f51
renames Packet Meta::{,set_}addr methods to {,set_}socket_addr (#25478) 
In order to distinguish between Meta.addr field which is an IpAddr and
the methods which refer to a SocketAddr.
 2022-05-23 15:48:59 +00:00
Michael Vines b05c7d91ed Fix derive_partial_eq_without_eq clippy lint 2022-05-22 22:22:21 -07:00
sakridge e22be02d3a
sigverify: add discard before dedup (#25388) 2022-05-23 03:40:33 +02:00
Brooks Prumo f8842032c6
clippy: fix "this let-binding has unit value" warnings (#25429) 2022-05-22 12:17:59 -04:00
Pankaj Garg 7fb0ef1fa5
Use async send for forwarding transactions (#25435) 2022-05-20 21:20:47 -07:00
Jeff Biseda e263be2000
handle start_http failure in rpc_service (#25400) 2022-05-20 17:59:23 -07:00
Brennan Watt e025376719
Fix packet accounting after dedup (#25357) 
* Fix packet accounting after dedup
* Rename function to better represent intent
 2022-05-20 17:00:13 -07:00
Brennan Watt 2fdc850176
Use Shared IP to Stake Map (#25377) 
* Find packet sender stake stage use shared IP to stake map
 2022-05-20 12:51:07 -07:00
Michael Vines c54e06355f
voteSubscribe pubsub notification now includes the vote transaction signature (#25291) 2022-05-19 18:28:46 -07:00
Michael Vines 97efbdc303
Defer tower saving until push_vote(), there's no need to do it sooner (#25374) 2022-05-19 18:27:58 -07:00
buffalu 971748b335
fix banking stage starvation (#25245) 2022-05-18 22:37:47 +02:00
dependabot[bot] 7402878628
chore: bump raptorq from 1.6.5 to 1.7.0 (#25330) 
Bumps [raptorq](https://github.com/cberner/raptorq) from 1.6.5 to 1.7.0.
- [Release notes](https://github.com/cberner/raptorq/releases)
- [Commits](https://github.com/cberner/raptorq/compare/v1.6.5...v1.7.0)

---
updated-dependencies:
- dependency-name: raptorq
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-05-18 17:56:38 +00:00
dependabot[bot] 542bd0ec3c
chore: bump rayon from 1.5.2 to 1.5.3 (#25242) 
* chore: bump rayon from 1.5.2 to 1.5.3

Bumps [rayon](https://github.com/rayon-rs/rayon) from 1.5.2 to 1.5.3.
- [Release notes](https://github.com/rayon-rs/rayon/releases)
- [Changelog](https://github.com/rayon-rs/rayon/blob/master/RELEASES.md)
- [Commits](https://github.com/rayon-rs/rayon/compare/v1.5.2...v1.5.3)

---
updated-dependencies:
- dependency-name: rayon
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
 2022-05-18 09:39:57 -06:00
Justin Starry 5548baf4dd
Don't drop transactions which use a request heap size ix (#25315) 2022-05-18 17:47:24 +08:00
steviez b27125815a
Simplify logic around MAX_ORPHAN_REPAIR_RESPONSES constant (#25032) 2022-05-17 19:45:45 -06:00
Tao Zhu b1b3702e6d
Prioritize transactions in banking stage by their compute unit price (#25178) 
* - get prioritization fee from compute_budget instruction;
- update compute_budget::process_instruction function to take instruction iter to support sanitized versioned message;
- updated runtime.md

* update transaction fee calculation for prioritization fee rate as lamports per 10K CUs

* review changes

* fix test

* fix a bpf test

* fix bpf test

* patch feedback

* fix clippy

* fix bpf test

* feedback

* rename prioritization fee rate to compute unit price

* feedback

Co-authored-by: Justin Starry <justin@solana.com>
 2022-05-16 12:06:33 +08:00
sakridge 52db2e19bc
Lower default batch size to 64 and add 2 banking threads (#25226) 2022-05-15 16:52:47 +02:00
sakridge 3d96a1ab76
Block packets in vote-only mode (#24906) 2022-05-14 17:53:37 +02:00
Pankaj Garg 71dd95e842
Tune banking_stage receive loop timing (#25172) 2022-05-13 03:42:08 +00:00
Jeff Washington (jwash) 896729f25e
keep track of oldest slot used by last hash calculation (#25152) 2022-05-12 11:18:08 -05:00
Jeff Washington (jwash) 3a4f0d3397
println -> info (#25163) 2022-05-12 11:07:13 -05:00
Pankaj Garg bcf4d54235
Update test_banking_stage_entryfication to be more deterministic (#25146) 
* Update test_banking_stage_entryfication to be more deterministic

* revert to original test with updated checks
 2022-05-12 15:36:19 +00:00
HaoranYi 41d34d45e0
pass exit by ref (#25120) 2022-05-11 09:17:21 -05:00
DimAn 2fa9bc3e70
Add options to store full and/or incremental snapshots in separate locations (#24247) 2022-05-10 16:37:41 -04:00
Justin Starry e3bdc38f0a
Add sanitized types for use in banking stage (#25067) 2022-05-11 00:30:48 +08:00
HaoranYi de96663cc4
fix typo (#25083) 2022-05-09 12:42:58 -05:00
Pankaj Garg 362b0526cd
Greedy receive in banking stage (#25060) 
* Greedy receive in banking stage

* add upperbound to batch size and batching time

* update test_banking_stage_entryfication test
 2022-05-08 10:47:55 -07:00
HaoranYi 8e37e364b1
fix typo in measure name (#25058) 2022-05-07 10:04:42 -05:00
Justin Starry c920d411f7
Clean up logging and make variables consistent (#25049) 2022-05-07 03:52:45 +08:00
Justin Starry 082502d4f3
Fail tx sanitization when ix program id uses lookup table (#25035) 
* Fail tx sanitization when ix program id uses lookup table

* feedback
 2022-05-07 03:19:50 +08:00
Christian Kamm cb6cd5d60f
FindPacketSenderStake: Improve metrics (#24971) 
- separate names for vote and non-vote thread
- time unit postfixes (one is in ns!)
 2022-05-06 21:16:13 +02:00
behzad nouri 492f89a170
checks account owner when initializing a vote-account (#25018) 
A VoteAccount may only wrap an account if the account owner is
solana_vote_program:id or equivalently this check returns true:
solana_vote_program::check_id(account.owner())
 2022-05-06 16:22:49 +00:00
Tyera Eulberg c4aa082efc
Rollback tokio to LTS release v1.14 (#25028) 2022-05-06 06:34:49 +00:00
behzad nouri a01291069a
initializes thread-pools with lazy_static instead of thread_local (#24853) 
In addition to thread_local -> lazy_static change, a number of thread-pools are
initialized with get_max_thread_count to achieve parity with the older code in
terms of number of validator threads.
 2022-05-05 20:00:50 +00:00
Justin Starry 7100f1c94b
Collect stats in streamer receiver and report fetch stage metrics (#25010) 2022-05-06 02:56:18 +08:00
carllin 870ac80b79
Prioritize BankingStage packets individually in min-max heap (#24187) 2022-05-04 21:50:56 -05:00
dependabot[bot] bece7f32c8
chore: bump log from 0.4.16 to 0.4.17 (#24987) 
* chore: bump log from 0.4.16 to 0.4.17

Bumps [log](https://github.com/rust-lang/log) from 0.4.16 to 0.4.17.
- [Release notes](https://github.com/rust-lang/log/releases)
- [Changelog](https://github.com/rust-lang/log/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/log/commits/0.4.17)

---
updated-dependencies:
- dependency-name: log
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
 2022-05-05 01:49:18 +00:00
dependabot[bot] 61a9faae17
chore: bump serde_json from 1.0.80 to 1.0.81 (#24960) 
* chore: bump serde_json from 1.0.80 to 1.0.81

Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.80 to 1.0.81.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.80...v1.0.81)

---
updated-dependencies:
- dependency-name: serde_json
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <you@example.com>
 2022-05-04 14:57:38 -06:00
dependabot[bot] 9258d81ba3
chore: bump serde from 1.0.136 to 1.0.137 (#24957) 
* chore: bump serde from 1.0.136 to 1.0.137

Bumps [serde](https://github.com/serde-rs/serde) from 1.0.136 to 1.0.137.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.136...v1.0.137)

---
updated-dependencies:
- dependency-name: serde
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
 2022-05-03 22:14:59 -06:00
dependabot[bot] 2c9d2a2140
chore: bump serde_json from 1.0.79 to 1.0.80 (#24943) 
* chore: bump serde_json from 1.0.79 to 1.0.80

Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.79 to 1.0.80.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.79...v1.0.80)

---
updated-dependencies:
- dependency-name: serde_json
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
 2022-05-03 13:46:45 -06:00
Christian Kamm 503d0baf6d
SigVerify: Add total time metrics for dedup/discard/verify (#24768) 
* SigVerify: Add total time metrics for dedup/discard/verify

Previously it was impossible to determine the total time the stage spent
on these activities within a measurement window.

* SigVerify: Add _us postfix to time metrics
 2022-05-03 14:59:25 +02:00
behzad nouri eff59193db
enforces that LAST_SHRED_IN_SLOT is also DATA_COMPLETE_SHRED (#24892) 
A data shred cannot be LAST_SHRED_IN_SLOT if not also DATA_COMPLETE_SHRED.
So LAST_SHRED_IN_SLOT should also imply DATA_COMPLETE_SHRED:
https://github.com/solana-labs/solana/blob/74b586ae7/ledger/src/shredder.rs#L116-L117
https://github.com/solana-labs/solana/blob/74b586ae7/core/src/broadcast_stage/standard_broadcast_run.rs#L80-L81

However current shred constructs allow specifying a shred which is
LAST_SHRED_IN_SLOT but not DATA_COMPLETE_SHRED:
https://github.com/solana-labs/solana/blob/74b586ae7/ledger/src/shred.rs#L117-L118
https://github.com/solana-labs/solana/blob/74b586ae7/ledger/src/shred.rs#L272-L273

The commit updates ShredFlags so that if a shred is not
DATA_COMPLETE_SHRED it cannot be LAST_SHRED_IN_SLOT either.
 2022-05-02 23:33:53 +00:00
apfitzge 112a0b475a
Revert "Refactor to use EpochSchedule from within RentCollector struct" (#24893) 
* Revert "Ran cargo fmt"

This reverts commit 9052e41b32.

* Revert "Fix build error introduced by my editor setup, part 2"

This reverts commit 4dfeab3b38.

* Revert "Fix build error introduced by my editor setup"

This reverts commit 87fb78dc56.

* Revert "Remove redundant epoch_schedule from AccountsPackage"

This reverts commit c2f7f2fff8.

* Revert "Fix a test"

This reverts commit 36c0bdaa78.

* Revert "Fixes to initial code"

This reverts commit ed7813e698.

* Revert "Removing redundant EpochSchedule param from fns"

This reverts commit 5472d2e605.
 2022-05-02 13:46:17 -05:00
behzad nouri e812430e28
defines shred flags using bitflags crate (#24874) 
Shred flags uses raw bit-masking ops which lacks type-safety:
https://github.com/solana-labs/solana/blob/a829ddc92/ledger/src/shred.rs#L112-L114

This commit instead uses bitflags crate to define shred flags.
 2022-05-01 19:25:15 +00:00
Pankaj Garg 88c16c0176
Check if quic is enabled before warming up quic connections (#24821) 
* Check if quic is enabled before warming up quic connections

* fix after rebase

* don't start warmup service if quic not enabled

* fix test
 2022-05-01 03:52:38 +00:00
Justin Starry a61652104b
Avoid holding lock guards in match expressions (#24805) 
* Avoid holding bank forks read lock for RPC requests

* Avoid using lock guards in temporaries

* revert fetch stage change
 2022-04-29 16:32:46 +08:00
Justin Starry 4e58b3870c
Update all BankForks methods to return owned values (#24801) 2022-04-28 18:51:00 +00:00
dependabot[bot] b22a14ca68
chore: bump etcd-client from 0.9.0 to 0.9.1 (#24774) 
* chore: bump etcd-client from 0.9.0 to 0.9.1

Bumps [etcd-client](https://github.com/etcdv3/etcd-client) from 0.9.0 to 0.9.1.
- [Release notes](https://github.com/etcdv3/etcd-client/releases)
- [Commits](https://github.com/etcdv3/etcd-client/compare/v0.9.0...v0.9.1)

---
updated-dependencies:
- dependency-name: etcd-client
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
 2022-04-28 12:30:37 -06:00
sakridge 5a430c15e2
Separate sigverify metrics for each verifier (#24744) 2022-04-28 01:16:17 -07:00
behzad nouri 0f60665100
replaces Shred::new_empty_coding with Shred::new_from_parity_shard (#24749) 
Removing implementation details of shreds and payload offsets from
shredder, so that shredder does not need to mutate payload:
https://github.com/solana-labs/solana/blob/71ad12128/ledger/src/shred.rs#L968-L977

Also, Shred::new_from_data can simply obtain a slice as opposed to
Option<&[u8]>:
https://github.com/solana-labs/solana/blob/71ad12128/ledger/src/shred.rs#L268-L278
 2022-04-27 18:04:10 +00:00
behzad nouri 081c844d6e
removes Shred::new_empty_data_shred (#24714) 
Shred::new_empty_data_shred returns an invalid shred (i.e.
shred.sanitize() returns error). The method is only used in tests and
can be easily replaced with Shred::new_from_data. To keep the shred api
surface small, this commit removes this method.
 2022-04-26 23:13:12 +00:00
behzad nouri 12ae8d3be5
returns Error when Shred::sanitize fails (#24653) 
Including the error in the output allows to debug when Shred::sanitize
fails.
 2022-04-25 23:19:37 +00:00
behzad nouri 895f76a93c
hides implementation details of shred from its public interface (#24563) 
Working towards embedding versioning into shreds binary, so that a new
variant of shred struct can include merkle tree hashes of the erasure
set.
 2022-04-25 12:43:22 +00:00
carllin 8a062273de
Move error counters to be reported by leader only at end of slot (#24581) 
* Add error counters to leader metrics only

* Add dependencies
 2022-04-23 18:10:47 -05:00
Michael Vines d0a8a16a57 ReplayStage no longer relies on Validator to reset the poh recorder at start 2022-04-22 21:17:49 -07:00
Michael Vines 84e3342612 Process blockstore after starting the TVU 2022-04-22 21:17:49 -07:00
Michael Vines 83e041299a Run real snapshot packager while processing blockstore at validator startup 2022-04-22 21:17:49 -07:00
HaoranYi 2d4defa477
fix typo (#24576) 2022-04-22 08:43:57 -05:00
Jon Cinque 0d51596224
sim: Override slot hashes account on simulation bank (#24543) 
* sim: Override slot hashes during simulation

* Add simulation test program

* Address feedback

* Add AccountOverrides explicit type

* Cargo fmt
 2022-04-22 12:32:31 +02:00
Justin Starry c544742091
Local cluster test cleanup and refactoring (#24559) 
* remove FixedSchedule.start_epoch

* use duration for timing

* Rename to partition bool to turbine_disabled

* simplify partition config
 2022-04-22 12:14:07 +08:00
behzad nouri 454ef38e43 caches StakeAccount instead of Delegation in Stakes 
The commit makes values in stake_delegations map in Stakes struct
generic. Stakes<Delegation> is equivalent to the old code and is used
for backward compatibility in BankFieldsTo{Serialize,Deserialize}.

But banks cache Stakes<StakeAccount> which includes the entire stake
account and StakeState deserialized from account. Doing so, will remove
the need to load stake account from accounts-db when working with
stake-delegations.
 2022-04-21 15:28:41 +00:00
Justin Starry 02bfb85c16
Refactor transaction processing in banking stage (#24336) 
* Refactor transaction processing in banking stage

* feedback

* more feedback
 2022-04-21 21:06:26 +08:00
Justin Starry d5127abf46
Only add hashes for completed blocks to recent blockhashes (#24389) 
* Only add hashes for completed blocks to recent blockhashes

* feedback
 2022-04-21 21:05:29 +08:00
Tao Zhu a21fc3f303
Apply transaction actual execution units to cost_tracker (#24311) 
* Pass the sum of consumed compute units to cost_tracker

* cost model tracks builtins and bpf programs separately, enabling adjust block cost by actual bpf programs execution costs

* Copied nightly-only experimental `checked_add_(un)signed` implementation to sdk

* Add function to update cost tracker with execution cost adjustment

* Review suggestion - using enum instead of struct for CommitTransactionDetails
Co-authored-by: Justin Starry <justin.m.starry@gmail.com>

* review - rename variable to distinguish accumulated_consumed_units from individual compute_units_consumed

* not to use signed integer operations

* Review - using saturating_add_assign!(), and checked_*().unwrap_or()

* Review - using Ordering enum to cmp

* replace checked_ with saturating_

* review - remove unnecessary Option<>

* Review - add function to report number of non-zero units account to metrics
 2022-04-21 07:38:07 +00:00
Justin Starry 79923c3b58
Refactor: Rename BlockhashQueue fields and methods for clarity (#24426) 2022-04-21 11:57:17 +08:00
HaoranYi d0761d0ca4
demote receive_window_num_slot_shreds to debug logging (#24505) 2022-04-20 08:51:46 -05:00
dependabot[bot] dd15193c69
chore: bump rayon from 1.5.1 to 1.5.2 (#24470) 
* chore: bump rayon from 1.5.1 to 1.5.2

Bumps [rayon](https://github.com/rayon-rs/rayon) from 1.5.1 to 1.5.2.
- [Release notes](https://github.com/rayon-rs/rayon/releases)
- [Changelog](https://github.com/rayon-rs/rayon/blob/master/RELEASES.md)
- [Commits](https://github.com/rayon-rs/rayon/commits)

---
updated-dependencies:
- dependency-name: rayon
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
 2022-04-19 23:17:52 -06:00
Michael Vines 05f32f287c solana-validator monitor now reports slot-level progress while loading blockstore 2022-04-19 22:09:48 -07:00
Michael Vines 9e4999ef6a Remove halt_at_slot from RuntimeConfig, it's not a runtime concern 2022-04-19 19:23:58 -07:00
Michael Vines 988210908c Move verify_udp_stats_access out of the way 2022-04-19 19:23:58 -07:00
Michael Vines c6f3da4879 blockstore_processor now accepts an Arc<Rwlock<BankForks>> 2022-04-19 19:23:58 -07:00
Michael Vines 0e2e0c8b7d Extract most storage-related services from the Tvu abstraction 2022-04-19 19:23:58 -07:00
Michael Vines 268a2109de Relocate hard forks info log 2022-04-19 19:23:58 -07:00
Michael Vines dd766042df Remove LedgerMetricReportService from TVU 2022-04-19 19:23:58 -07:00
behzad nouri 705ea53353
moves sign_shred and new_coding_shred_header out of Shredder (#24487) 2022-04-19 20:00:05 +00:00
Tao Zhu 94b0186a96
Cost model tracks builtins and bpf programs separately (#24468) 
* Cost model tracks builtins and bpf programs separatele (enables adjusting block cost by actual bpf programs execution costs)

* Address reviews: expand test; add metrics stat
 2022-04-19 13:25:47 -05:00