Commit Graph

141 Commits

Author SHA1 Message Date
Ryo Onodera 95810d876a
Enable frozen_abi on banking trace file (#33501)
* Enable frozen_abi on banking trace file

* Fix ci with really correct bugfix...

* Remove tracker_callers

* Fix typo...

* Fix AbiExample for Arc/Rc's Weaks

* Added comment for AbiExample impl of SystemTime

* Simplify and document EvenAsOpaque with new usage

* Minor clean-ups

* Simplify SystemTime::example() with UNIX_EPOCH...

* Add comment for AbiExample subtleties
2023-10-07 13:15:38 +09:00
behzad nouri 528a03f32a
removes outdated matches crate from dependencies (#33172)
removes outdated matches crate from the dependencies

std::matches has been stable since rust 1.42.0.
Other use-cases are covered by assert_matches crate.
2023-09-07 12:52:57 +00:00
Alexander Meißner 9e703f85de
Upgrades Rust to 1.72.0 & nightly-2023-08-25 (#32961)
* allow pedantic invalid cast lint

* allow lint with false-positive triggered by `test-case` crate

* nightly `fmt` correction

* adapt to rust layout changes

* remove dubious test

* Use transmute instead of pointer cast and de/ref when check_aligned is false.

* Renames clippy::integer_arithmetic to clippy::arithmetic_side_effects.

* bump rust nightly to 2023-08-25

* Upgrades Rust to 1.72.0

---------

Co-authored-by: Trent Nelson <trent@solana.com>
2023-09-01 07:26:13 +00:00
Alexander Meißner 150a798d32
Fix - Upcoming `arithmetic_side_effects` lints (#33000)
* dereplicode address alignment check

* Uses `checked_div` and `checked_rem` in built-in loaders.

* Uses `checked_div` and `checked_rem`.

* sdk: replace sub() with saturating_sub()

* eliminate `String` "arithmetic"

* allow arithmetic side-effects in tests and benches and on types we don't control

---------

Co-authored-by: Trent Nelson <trent@solana.com>
2023-08-29 20:58:53 +02:00
Jon Cinque 0fe902ced7
Bump rand to 0.8, rand_chacha to 0.3, getrandom to 0.2 (#32871)
* sdk: Add concurrent support for rand 0.7 and 0.8

* Update rand, rand_chacha, and getrandom versions

* Run command to replace `gen_range`

Run `git grep -l gen_range | xargs sed -i'' -e 's/gen_range(\(\S*\), /gen_range(\1../'

* sdk: Fix users of older `gen_range`

* Replace `hash::new_rand` with `hash::new_with_thread_rng`

Run:
```
git grep -l hash::new_rand | xargs sed -i'' -e 's/hash::new_rand([^)]*/hash::new_with_thread_rng(/'
```

* perf: Use `Keypair::new()` instead of `generate`

* Use older rand version in zk-token-sdk

* program-runtime: Inline random key generation

* bloom: Fix clippy warnings in tests

* streamer: Scope rng usage correctly

* perf: Fix clippy warning

* accounts-db: Map to char to generate a random string

* Remove `from_secret_key_bytes`, it's just `keypair_from_seed`

* ledger: Generate keypairs by hand

* ed25519-tests: Use new rand

* runtime: Use new rand in all tests

* gossip: Clean up clippy and inline keypair generators

* core: Inline keypair generation for tests

* Push sbf lockfile change

* sdk: Sort dependencies correctly

* Remove `hash::new_with_thread_rng`, use `Hash::new_unique()`

* Use Keypair::new where chacha isn't used

* sdk: Fix build by marking rand 0.7 optional

* Hardcode secret key length, add static assertion

* Unify `getrandom` crate usage to fix linking errors

* bloom: Fix tests that require a random hash

* Remove some dependencies, try to unify others

* Remove unnecessary uses of rand and rand_core

* Update lockfiles

* Add back some dependencies to reduce rebuilds

* Increase max rebuilds from 14 to 15

* frozen-abi: Remove `getrandom`

* Bump rebuilds to 17

* Remove getrandom from zk-token-proof
2023-08-21 19:11:21 +02:00
behzad nouri 0b52b8a46e
switches from dlopen to dlopen2 (#32640)
dlopen is unmaintained:
https://github.com/szymonwieloch/rust-dlopen/issues/47
2023-07-27 16:30:22 +00:00
behzad nouri cfb028819a
deprecates Signature::new in favor of Signature::{try_,}from (#32481) 2023-07-14 22:51:12 +00:00
Brooks 3b0e4c8452
Removes unnecessary default() (#32477) 2023-07-13 13:59:57 -04:00
behzad nouri d54b6204be
removes instances of clippy::manual_let_else (#32417) 2023-07-09 21:41:36 +00:00
behzad nouri 0da01270ef
removes redundant recycler clones (#32401) 2023-07-06 18:25:20 +00:00
steviez c2e45773af
Fixup is_niceness_adjustment_valid unit test for non-linux case (#32363)
The test needs to be aware of is_niceness_adjustment_valid from super
scope; however, the use directive in test module only runs on Linux
systems.
2023-07-05 14:27:03 -04:00
sakridge 7ff5e463e9
Move is_niceness logic out of clap utils to reduce dependencies (#32331)
Move is_niceness logic out of clap utils to reduce dependencies..

..for solana-keygen
2023-06-29 14:03:36 +02:00
Tao Zhu 9d6c921b5f
check simple vote could have 1 or 2 signatures when creating sanitized transaction (#31807)
* check simple vote could have 1 or 2 signatures when creating sanitized transaction
2023-05-25 15:45:15 -05:00
Brennan e79b84ea70
Rework tx sig verify batching (#31355)
* Rework tx sig verify batching to eliminate special casing and increase the packet limit for sigverify from 2k to 5k
2023-04-28 09:21:12 -07:00
Ryo Onodera a30830d7a9
ci: treewide: deny used_underscore_binding (#31319)
* Enforce used_underscore_binding

* Fix all

* Work around for cfg()-ed code...

* ci....

* Make clipply fixes more pleasant

* Clone exit signal while intentionally shadowing

* Use more verbose code to avoid any #[allow(...)]s
2023-04-27 10:10:16 +09:00
Andrew Fitzgerald 65d9238e31
remove pub from to_packet_batches_for_tests (#31056) 2023-04-05 08:57:17 -07:00
behzad nouri 359d9c7b9f
upgrades ahash to 0.8.3 (#30935) 2023-03-28 18:23:33 +00:00
Jeff Biseda 04f0311aa1
check data budget before accessing blockstore (#30809) 2023-03-22 15:56:06 -07:00
behzad nouri 9ad77485ce
generalizes deduper to work with any hashable type (#30753)
generalizes Deduper to work with any hashable type

Current Deduper is hard-coded only for Packet type. In order to use
Deduper in retransmit-stage, we need to dedup types other than Packet.
The commit generalizes Deduper to any hashable type.
2023-03-20 18:04:46 +00:00
behzad nouri 46614c0e9f
removes false_positive_rate field from Deduper (#30788)
removes the false_positive_rate field from the Deduper

Deduper.false_positive_rate field is misleading because it is not
enforced until maybe_reset is called. But then maybe_reset can be
invoked with an explicit argument.
2023-03-20 13:16:52 +00:00
behzad nouri 93f696dac7
increases shred-fetch-stage deduper capacity and reset-cycle (#30690) 2023-03-17 00:05:29 +00:00
behzad nouri 7a7b020580
dedups packets using an atomic bloom filter (#30726)
Current Deduper implementation uses many bits per entry:
https://github.com/solana-labs/solana/blob/65cd55261/perf/src/deduper.rs#L70-L73
and may be saturated quickly. It also lacks api to specify desired false
positive rate.

The commit instead uses an atomic bloom filter with K hash functions.
The false positive rate is obtained by tracking popcount of bits.
2023-03-16 16:45:42 +00:00
Tao Zhu ee81679804
Not to tag versioned transaction as simple_vote_tx (#30517)
* not to tag versioned transaction as simple_vote_tx
* better error code
2023-02-24 23:48:15 -06:00
steviez 2ba76e4235
chore: Use Self in Packet/PacketBatch impl blocks (#30417) 2023-02-22 10:02:50 -05:00
steviez d3dab24bbe
chore: Use `i` over `ix` variable name when naming worker threads (#30206) 2023-02-09 01:24:57 +00:00
behzad nouri 4cc07a176e
reduces number of gossip pull requests/responses (#29974) 2023-01-30 17:59:56 +00:00
steviez fa39bfef6b
Move Deduper into a separate file (#29891) 2023-01-25 15:34:53 -06:00
Ryo Onodera 40bbf99c74
Add fully-reproducible online tracer for banking (#29196)
* Add fully-reproducible online tracer for banking

* Don't use eprintln!()...

* Update programs/sbf/Cargo.lock...

* Remove meaningless assert_eq

* Group test-only code under aptly named mod

* Remove needless overflow handling in receive_until

* Delay stat aggregation as it's possible now

* Use Cow to avoid needless heap allocs

* Properly consume metrics action as soon as hold

* Trace UnprocessedTransactionStorage::len() instead

* Loosen joining api over type safety for replaystage

* Introce hash event to override these when simulating

* Use serde_with/serde_as instead of hacky workaround

* Update another Cargo.lock...

* Add detailed comment for Packet::buffer serialize

* Rename sender_overhead_minimized_receiver_loop()

* Use type interference for TraceError

* Another minor rename

* Retire now useless ForEach to simplify code

* Use type alias as much as possible

* Properly translate and propagate tracing errors

* Clarify --enable-banking-trace with better naming

* Consider unclean (signal-based) node restarts..

* Tweak logging and cli

* Remove Bank events as it's not needed anymore

* Make tpu own banking tracer thread

* Reduce diff a bit..

* Use latest serde_with

* Finally use the published rolling-file crate

* Make test code change more consistent

* Revive dead and non-terminating test code path...

* Dispose batches early now that possible

* Split off thread handle very early at ::new()

* Tweak message for TooSmallDirByteLimitl

* Remove too much of indirection

* Remove needless pub from ::channel()

* Clarify test comments

* Avoid needless event creation if tracer is disabled

* Write tests around file rotation and spill-over

* Remove unneeded PathBuf::clone()s...

* Introduce inner struct instead of tuple...

* Remove unused enum BankStatus...

* Avoid .unwrap() for the case of disabled tracer...
2023-01-25 21:54:38 +09:00
behzad nouri b71cb9d9c7
simplifies sigverify copy_return_values (#29495) 2023-01-05 19:45:52 +00:00
behzad nouri d172ab342f
expands test coverage for merkle/legacy shreds sigverify (#29424) 2022-12-28 17:42:24 +00:00
behzad nouri 9524c9dbff patches errors from clippy::uninlined_format_args
https://rust-lang.github.io/rust-clippy/master/index.html#uninlined_format_args
2022-12-06 19:32:15 +00:00
Jon Cinque 504091f9be
perf: Change underlying Elems to *const u8 (#29089)
* perf: Change underlying Elems to *const u8

* Remove clippy allows
2022-12-06 18:06:27 +01:00
Jon Cinque b1340d77a2
sdk: Make Packet::meta private, use accessor functions (#29092)
sdk: Make packet meta private
2022-12-06 12:54:49 +01:00
Brooks Prumo d1ba42180d
clippy for rust 1.65.0 (#28765) 2022-11-09 19:39:38 +00:00
Brooks Prumo 12df0f234d
Upgrade to Rust 1.64.0 (#28034) 2022-09-29 09:32:24 -04:00
behzad nouri 9a57c64f21
patches clippy errors from new rust nightly release (#27996) 2022-09-22 22:23:03 +00:00
Brennan Watt e4a7d01e10
Rust v1.63 (#27303)
* Upgrade to Rust v1.63.0

* Add nightly_clippy_allows

* Resolve some new clippy nightly lints

* Increase QUIC packets completion timeout

* Update quinn-udp crate

Co-authored-by: Michael Vines <mvines@gmail.com>
2022-08-22 18:01:03 -07:00
Michael Vines 3f4731b37f Standardize thread names
Tenets:
1. Limit thread names to 15 characters
2. Prefix all Solana-controlled threads with "sol"
3. Use Camel case. It's more character dense than Snake or Kebab case
2022-08-20 07:49:39 -07:00
Brennan Watt 7573000d87
Revert "Rust v1.63.0 (#27148)" (#27245)
This reverts commit a2e7bdf50a.
2022-08-19 09:19:44 +01:00
Brennan Watt a2e7bdf50a
Rust v1.63.0 (#27148)
* Upgrade to Rust v1.63.0

* Add nightly_clippy_allows

* Resolve some new clippy nightly lints

* Increase QUIC packets completion timeout

Co-authored-by: Michael Vines <mvines@gmail.com>
2022-08-17 15:48:33 -07:00
Brian Anderson db9004bd0f
Fix doc warnings (#25953) 2022-06-14 21:55:08 -06:00
Brennan Watt fc166daa17
Add minimum iterator packet count for rayon verification threads (#25883)
* Add minimum iterator packet count of 128 for rayon sig verification threads
* Add bench performance and unit verification tests
2022-06-13 15:48:13 -07:00
carllin bf8faa8a30
Report banking stage tracer metrics (#25620) 2022-06-09 00:25:37 -05:00
Jack May 7c93504fa8
less noisy tests in CI (#25799) 2022-06-06 23:13:14 +00:00
behzad nouri 5dbf7d8f91
removes raw indexing into packet data (#25554)
Packets are at the boundary of the system where, vast majority of the
time, they are received from an untrusted source. Raw indexing into the
data buffer can open attack vectors if the offsets are invalid.
Validating offsets beforehand is verbose and error prone.

The commit updates Packet::data() api to take a SliceIndex and always to
return an Option. The call-sites are so forced to explicitly handle the
case where the offsets are invalid.
2022-06-03 01:05:06 +00:00
Trent Nelson ad82fe6372 perf: correct tracer packet detection bounds check 2022-06-01 22:57:22 -06:00
carllin 90a3315b69
Detect tracer key in sigverify (#25579)
* Mark the tracer transaction

* simplify tracer check
2022-05-30 18:41:54 -05:00
Justin Starry e4409a87fe
Add pre shrink pass before sigverify batch (#25136) 2022-05-28 01:51:55 +10:00
HaoranYi ae9513f00a
Opt: optimize dedup hash filter reset (#25555)
* refactor compute_hash

* optimize dedup filter reset

* clippy

* clippy2

* Revert "refactor compute_hash"

This reverts commit 4d09ad07f71c8cd634d405876ae8139a5e7a121a.
2022-05-26 14:06:53 +00:00
HaoranYi b09faf4cee
refactor: extract compute_hash fn (#25553)
* refactor compute_hash

* fix build
2022-05-26 08:48:34 -05:00