blockworks-foundation
/
solana
mirror of https://github.com/blockworks-foundation/solana.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
22,038 Commits 174 Branches 497 Tags 541 MiB
Commit Graph

52 Commits

Author SHA1 Message Date
behzad nouri c6e7aaf96c
removes lazy-static thread-pool from sigverify-shreds (#30787) 
Instead the thread-pool is passed explicitly from higher in the call
stack so that
https://github.com/solana-labs/solana/pull/30786
can use the same thread-pool for shred deduplication.
 2023-03-20 20:33:22 +00:00
behzad nouri cf0a149add
removes the merkle root from shreds binary (#29427) 
https://github.com/solana-labs/solana/pull/29445
makes it unnecessary to embed merkle roots into shreds binary. This
commit removes the merkle root from shreds binary.

This adds 20 bytes to shreds capacity to store more data.
Additionally since we no longer need to truncate the merkle root, the
signature would be on the full 32 bytes of hash as opposed to the
truncated one.

Also signature verification would now effectively verify merkle proof as
well, so we no longer need to verify merkle proof in the sanitize
implementation.
 2023-02-15 21:17:24 +00:00
steviez d3dab24bbe
chore: Use `i` over `ix` variable name when naming worker threads (#30206) 2023-02-09 01:24:57 +00:00
behzad nouri b71cb9d9c7
simplifies sigverify copy_return_values (#29495) 2023-01-05 19:45:52 +00:00
behzad nouri 9db25655f7
recovers merkle roots from shreds binary in {verify,sign}_shreds_gpu (#29445) 
{verify,sign}_shreds_gpu need to point to offsets within the packets for
the signed data. For merkle shreds this signed data is the merkle root
of the erasure batch and this would necessitate embedding the merkle
roots in the shreds payload.
However this is wasteful and reduces shreds capacity to store data
because the merkle root can already be recovered from the encoded merkle
proof.

Instead of pointing to offsets within the shreds payload, this commit
recovers merkle roots from the merkle proofs and stores them in an
allocated buffer. {verify,sign}_shreds_gpu would then point to offsets
within this new buffer for the respective signed data.

This would unblock us from removing merkle roots from shreds payload
which would save capacity to send more data with each shred.
 2023-01-04 14:20:05 +00:00
behzad nouri 50afb80f52
adds shred::layout::get_signed_data (#29438) 
Working towards removing merkle root from shreds payload, the commit
implements api to obtain signed data from shreds binary.
 2022-12-30 18:52:10 +00:00
behzad nouri 1de161c13a
simplifies shreds sigverify (#29436) 
Simplifying this code in preparation of removing merkle roots from
shreds binary.
 2022-12-29 20:51:57 +00:00
behzad nouri fba4a57060
expands test coverage for sign_shreds_gpu (#29429) 2022-12-29 00:33:51 +00:00
behzad nouri d172ab342f
expands test coverage for merkle/legacy shreds sigverify (#29424) 2022-12-28 17:42:24 +00:00
behzad nouri 92ca725c6b
patches shred::merkle::make_shreds_from_data when data is empty (#29295) 
If data is empty, make_shreds_from_data will now return one data shred
with empty data. This preserves invariants verified in tests regardless
of data size.
 2022-12-16 18:56:21 +00:00
behzad nouri 9524c9dbff patches errors from clippy::uninlined_format_args 
https://rust-lang.github.io/rust-clippy/master/index.html#uninlined_format_args
 2022-12-06 19:32:15 +00:00
Jon Cinque 504091f9be
perf: Change underlying Elems to *const u8 (#29089) 
* perf: Change underlying Elems to *const u8

* Remove clippy allows
 2022-12-06 18:06:27 +01:00
Jon Cinque b1340d77a2
sdk: Make Packet::meta private, use accessor functions (#29092) 
sdk: Make packet meta private
 2022-12-06 12:54:49 +01:00
behzad nouri 9a57c64f21
patches clippy errors from new rust nightly release (#27996) 2022-09-22 22:23:03 +00:00
Michael Vines 3f4731b37f Standardize thread names 
Tenets:
1. Limit thread names to 15 characters
2. Prefix all Solana-controlled threads with "sol"
3. Use Camel case. It's more character dense than Snake or Kebab case
 2022-08-20 07:49:39 -07:00
behzad nouri 5dbf7d8f91
removes raw indexing into packet data (#25554) 
Packets are at the boundary of the system where, vast majority of the
time, they are received from an untrusted source. Raw indexing into the
data buffer can open attack vectors if the offsets are invalid.
Validating offsets beforehand is verbose and error prone.

The commit updates Packet::data() api to take a SliceIndex and always to
return an Option. The call-sites are so forced to explicitly handle the
case where the offsets are invalid.
 2022-06-03 01:05:06 +00:00
behzad nouri 81231a89b9 adds support for different variants of ShredCode and ShredData 
The commit implements two new types:
    pub enum ShredCode {
        Legacy(legacy::ShredCode),
    }
    pub enum ShredData {
        Legacy(legacy::ShredData),
    }

Following commits will extend these types by adding merkle variants:
    pub enum ShredCode {
        Legacy(legacy::ShredCode),
        Merkle(merkle::ShredCode),
    }
    pub enum ShredData {
        Legacy(legacy::ShredData),
        Merkle(merkle::ShredData),
    }
 2022-06-02 18:55:50 +00:00
behzad nouri de612c25b3
removes shred wire layout specs from sigverify (#25520) 
sigverify_shreds relies on wire layout specs of shreds:
https://github.com/solana-labs/solana/blob/0376ab41a/ledger/src/sigverify_shreds.rs#L39-L46
https://github.com/solana-labs/solana/blob/0376ab41a/ledger/src/sigverify_shreds.rs#L298-L305

In preparation of
https://github.com/solana-labs/solana/pull/25237
which adds a new shred variant with different layout and signed message,
this commit removes shred layout specification from sigverify and
instead encapsulate that in shred module.
 2022-05-26 13:06:27 +00:00
behzad nouri 880684565c
limits read access into Packet data to Packet.meta.size (#25484) 
Bytes past Packet.meta.size are not valid to read from.

The commit makes the buffer field private and instead provides two
methods:
* Packet::data() which returns an immutable reference to the underlying
  buffer up to Packet.meta.size. The rest of the buffer is not valid to
  read from.
* Packet::buffer_mut() which returns a mutable reference to the entirety
  of the underlying buffer to write into. The caller is responsible to
  update Packet.meta.size after writing to the buffer.
 2022-05-25 16:52:54 +00:00
Justin Starry cad1c41ce2 Add Packet::deserialize_slice convenience method 2022-05-24 17:31:14 +08:00
steviez ec7ca411dd
Make PacketBatch packets vector non-public (#25413) 
Upcoming changes to PacketBatch to support variable sized packets will
modify the internals of PacketBatch. So, this change removes usage of
the internal packet struct and instead uses accessors (which are
currently just wrappers of Vector functions but will change down the
road).
 2022-05-23 15:30:15 -05:00
behzad nouri eff59193db
enforces that LAST_SHRED_IN_SLOT is also DATA_COMPLETE_SHRED (#24892) 
A data shred cannot be LAST_SHRED_IN_SLOT if not also DATA_COMPLETE_SHRED.
So LAST_SHRED_IN_SLOT should also imply DATA_COMPLETE_SHRED:
https://github.com/solana-labs/solana/blob/74b586ae7/ledger/src/shredder.rs#L116-L117
https://github.com/solana-labs/solana/blob/74b586ae7/core/src/broadcast_stage/standard_broadcast_run.rs#L80-L81

However current shred constructs allow specifying a shred which is
LAST_SHRED_IN_SLOT but not DATA_COMPLETE_SHRED:
https://github.com/solana-labs/solana/blob/74b586ae7/ledger/src/shred.rs#L117-L118
https://github.com/solana-labs/solana/blob/74b586ae7/ledger/src/shred.rs#L272-L273

The commit updates ShredFlags so that if a shred is not
DATA_COMPLETE_SHRED it cannot be LAST_SHRED_IN_SLOT either.
 2022-05-02 23:33:53 +00:00
behzad nouri 0f60665100
replaces Shred::new_empty_coding with Shred::new_from_parity_shard (#24749) 
Removing implementation details of shreds and payload offsets from
shredder, so that shredder does not need to mutate payload:
https://github.com/solana-labs/solana/blob/71ad12128/ledger/src/shred.rs#L968-L977

Also, Shred::new_from_data can simply obtain a slice as opposed to
Option<&[u8]>:
https://github.com/solana-labs/solana/blob/71ad12128/ledger/src/shred.rs#L268-L278
 2022-04-27 18:04:10 +00:00
behzad nouri 895f76a93c
hides implementation details of shred from its public interface (#24563) 
Working towards embedding versioning into shreds binary, so that a new
variant of shred struct can include merkle tree hashes of the erasure
set.
 2022-04-25 12:43:22 +00:00
behzad nouri 705ea53353
moves sign_shred and new_coding_shred_header out of Shredder (#24487) 2022-04-19 20:00:05 +00:00
behzad nouri 01a096adc8 adds bitflags to Packet.Meta 
Instead of a separate bool type for each flag, all the flags can be
encoded in a type-safe bitflags encoded in a single u8:
https://github.com/solana-labs/solana/blob/d6ec103be/sdk/src/packet.rs#L19-L31
 2022-01-04 13:53:40 +00:00
Justin Starry 254ef3e7b6
Rename Packets to PacketBatch (#21794) 2021-12-11 09:44:15 -05:00
Michael Vines b8837c04ec Reformat imports to a consistent style for imports 
rustfmt.toml configuration:
  imports_granularity = "One"
  group_imports = "One"
 2021-12-03 09:19:13 -08:00
Michael Vines 7027d56064 Resolve nightly-2021-10-05 clippy complaints 2021-10-06 10:37:58 -07:00
Alexander Meißner 6514096a67 chore: cargo +nightly clippy --fix -Z unstable-options 2021-06-18 10:42:46 -07:00
Brooks Prumo b074e86ad2
Fix clippy (#17214) 
Newer clippy from +nightly have new errors about where annotations can and
cannot go.  This commit fixes two of them.
 2021-05-13 17:51:36 -05:00
behzad nouri e405747409 Revert "Add limit and shrink policy for recycler (#15320)" 
This reverts commit c2e8814dce.
 2021-04-18 19:29:24 +00:00
carllin c2e8814dce
Add limit and shrink policy for recycler (#15320) 2021-02-24 00:15:58 -08:00
sakridge 191193289f
Upgrade sha2 to 0.9.3 (#14746) 2021-01-22 22:25:22 -08:00
Kristofer Peterson e23340d89e
Clippy cleanup for all targets and nighly rust (also support 1.44.0) (#10445) 
* address warnings from 'rustup run beta cargo clippy --workspace'

minor refactoring in:
- cli/src/cli.rs
- cli/src/offline/blockhash_query.rs
- logger/src/lib.rs
- runtime/src/accounts_db.rs

expect some performance improvement AccountsDB::clean_accounts()

* address warnings from 'rustup run beta cargo clippy --workspace --tests'

* address warnings from 'rustup run nightly cargo clippy --workspace --all-targets'

* rustfmt

* fix warning stragglers

* properly fix clippy warnings test_vote_subscribe()
replace ref-to-arc with ref parameters where arc not cloned

* Remove lock around JsonRpcRequestProcessor (#10417)

automerge

* make ancestors parameter optional to avoid forcing construction of empty hash maps

Co-authored-by: Greg Fitzgerald <greg@solana.com>
 2020-06-09 09:38:14 +09:00
carllin 97f2bcff69
master: Add nonce to shreds repairs, add shred data size to header (#10109) 
* Add nonce to shreds/repairs

* Add data shred size to header

Co-authored-by: Carl <carl@solana.com>
 2020-05-19 12:38:18 -07:00
Kristofer Peterson 58ef02f02b
9951 clippy errors in the test suite (#10030) 
automerge
 2020-05-15 09:35:43 -07:00
Tyera Eulberg ab361a8073
Rename KeypairUtil to Signer (#8360) 
automerge
 2020-02-20 13:28:55 -08:00
Greg Fitzgerald 127553ce4b
Wrap ed25519_dalek::Keypair (#8247) 2020-02-12 14:15:12 -07:00
Ryo Onodera 46b6cedff4
Fix nightly clippy warnings (#8199) 
automerge
 2020-02-10 22:48:50 -08:00
Sagar Dhawan 2dd8ab197d
Remove redundant threadpools in sigverify (#7888) 
* Limit the number of thread pools sigverify creates

* Name local threadpools
 2020-01-20 20:08:19 -08:00
Pankaj Garg 87b2525e03
Limit maximum number of shreds in a slot to 32K (#7584) 
* Limit maximum number of shreds in a slot to 32K

* mark dead slot replay as fatal error
 2019-12-30 07:42:09 -08:00
Pankaj Garg d94041e98d
Allow coding shred index to be different than data shred index (#7438) 
* Allow coding shred index to be different than data shred index

* move fec_set_index to shred's common header

* fix bench
 2019-12-12 16:50:29 -08:00
anatoly yakovenko e082418e4a Cleanup the sign shreds interface (#7456) 
automerge
 2019-12-12 13:27:33 -08:00
anatoly yakovenko bec5835289 Automatically call `.recycle` in Drop (#7429) 
automerge
 2019-12-11 11:58:40 -08:00
Sagar Dhawan 6bfe0fca1f
Add a version field to shreds (#7023) 
* Add a version field to shreds

* Clippy

* Fix Chacha Golden

* Fix shredder bench compile

* Fix blocktree bench compile
 2019-11-18 18:05:02 -08:00
anatoly yakovenko 0ca943f49b RecyclerCache for shred_sigverify (#6986) 
automerge
 2019-11-15 12:16:56 -08:00
sakridge 8e81bc1b49
Fix pinning (#6604) 
Remove Deref implementations and add more pass-throughs to the PinnedVec
wrapper.
Warm recyclers
set_pinnable
 2019-11-07 19:48:33 -08:00
Rob Walker b64b54f48f
unfork dalek ed25519 (#6776) 2019-11-07 17:08:10 -08:00
Pankaj Garg 0ace79939b
Add reference tick to data shreds (#6772) 
* Add reference tick to data shreds

* fix tests
 2019-11-06 13:27:58 -08:00