ed25519_verify_cpu: remove allocation in hot path (#3941)
The old code was a remnant from a time when we tried to "balance" the
amount of work across threads by re-batching and therefore allocating.
This is a hot path. Allocations are bad. Let work stealing work its
magic.
(cherry picked from commit a53a87657c945efd0fc2c6d7edcc2213cff44495)
Co-authored-by: Alessandro Decina <alessandro.d@gmail.com>
* replace cfg(RUSTC_WITH_SPECIALIZATION) with cfg(feature = "frozen-abi")
* remove the build scripts for the two frozen-abi crates
* remove all rustc_version deps
* remove a rustc_version dep that I missed
* fix duplicate lines in Cargo.toml files
* remove build.rs from instruction crate
* remove rustc_version from instruction crate
* remove no-longer-needed check-cfg entries
* update lock file after rebase
These counters should be removed for several reasons:
- Counters in general are bad (multiple atomics for a single value)
- The counters are at debug level, so effectively unused
- The reported values can be determined from SigVerifierStats
The only thing lost is whether verification was done by cpu/gpu/disabled
verifier. But, most validators are not running gpu's, so ripping this
out now and it can be re-added more properly if more validators shift to
using gpu's.
* bump rust to 1.80
* bump nightly version to 2024-07-21
* bump rust stable to 1.80.1, nightly to 2024-08-08
* clippy: macro_metavars_in_unsafe
* fix unexpected tag
* run anchor downstream test with their master
* add no-entrypoint into workspace level lint
* use correct llvm path for coverage test
* add cfg(feature = "frozen-abi") to build.rs
* only depend on rustc_version when frozen-abi feature is activated
* remove extraneous dirs that snuck in from another branch
* update perf/build.rs as it's different from the standard build script
* use symlink for svm/build.rs
* remove unused build dep rustc_version from wen-restart
* fmt Cargo.toml
* move short_vec.rs to its own crate
* update short_vec dependents to use new crate
* update lock file
* reexport short_vec with deprecation notice
* re-export in sdk too
* fmt
* remove duplicate frozen-abi stuff
* fix frozen-abi usage and activation
* update lock file
* fmt
* update expected digests after confirming not a real abi change
* update another digest
* add missing activation of frozen-abi feature
* update digests in sdk and program
* update VoteTransaction digest
* fmt
* Bump curve25519-dalek from 3.2.1 to 4.1.3
* hack spl test
* ignore some spl downstream tests
* use into_option
* hack spl test 2
---------
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* put most AbiExample derivations behind a cfg_attr
* feature gate all `extern crate solana_frozen_abi_macro;`
* use cfg_attr wherever we were deriving both AbiExample and AbiEnumVisitor
* fix cases where AbiEnumVisitor was still being derived unconditionally
* fix a case where AbiExample was derived unconditionally
* fix more cases where both AbiEnumVisitor and AbiExample were derived unconditionally
* two more cases where AbiExample and AbiEnumVisitor were unconditionally derived
* fix remaining unconditional derivations of AbiEnumVisitor
* fix cases where AbiExample is the first thing derived
* fix most remaining unconditional derivations of AbiExample
* move all `frozen_abi(digest =` behind cfg_attr
* replace incorrect cfg with cfg_attr
* fix one more unconditionally derived AbiExample
* feature gate AbiExample impls
* add frozen-abi feature to required Cargo.toml files
* make frozen-abi features activate recursively
* fmt
* add missing feature gating
* fix accidentally changed digest
* activate frozen-abi in relevant test scripts
* don't activate solana-program's frozen-abi in sdk dev-dependencies
* update to handle AbiExample derivation on new AppendVecFileBacking enum
* revert toml formatting
* remove unused frozen-abi entries from address-lookup-table Cargo.toml
* remove toml references to solana-address-lookup-table-program/frozen-abi
* update lock file
* remove no-longer-used generic param
* Enable frozen_abi on banking trace file
* Fix ci with really correct bugfix...
* Remove tracker_callers
* Fix typo...
* Fix AbiExample for Arc/Rc's Weaks
* Added comment for AbiExample impl of SystemTime
* Simplify and document EvenAsOpaque with new usage
* Minor clean-ups
* Simplify SystemTime::example() with UNIX_EPOCH...
* Add comment for AbiExample subtleties
removes outdated matches crate from the dependencies
std::matches has been stable since rust 1.42.0.
Other use-cases are covered by assert_matches crate.
* allow pedantic invalid cast lint
* allow lint with false-positive triggered by `test-case` crate
* nightly `fmt` correction
* adapt to rust layout changes
* remove dubious test
* Use transmute instead of pointer cast and de/ref when check_aligned is false.
* Renames clippy::integer_arithmetic to clippy::arithmetic_side_effects.
* bump rust nightly to 2023-08-25
* Upgrades Rust to 1.72.0
---------
Co-authored-by: Trent Nelson <trent@solana.com>
* dereplicode address alignment check
* Uses `checked_div` and `checked_rem` in built-in loaders.
* Uses `checked_div` and `checked_rem`.
* sdk: replace sub() with saturating_sub()
* eliminate `String` "arithmetic"
* allow arithmetic side-effects in tests and benches and on types we don't control
---------
Co-authored-by: Trent Nelson <trent@solana.com>
* sdk: Add concurrent support for rand 0.7 and 0.8
* Update rand, rand_chacha, and getrandom versions
* Run command to replace `gen_range`
Run `git grep -l gen_range | xargs sed -i'' -e 's/gen_range(\(\S*\), /gen_range(\1../'
* sdk: Fix users of older `gen_range`
* Replace `hash::new_rand` with `hash::new_with_thread_rng`
Run:
```
git grep -l hash::new_rand | xargs sed -i'' -e 's/hash::new_rand([^)]*/hash::new_with_thread_rng(/'
```
* perf: Use `Keypair::new()` instead of `generate`
* Use older rand version in zk-token-sdk
* program-runtime: Inline random key generation
* bloom: Fix clippy warnings in tests
* streamer: Scope rng usage correctly
* perf: Fix clippy warning
* accounts-db: Map to char to generate a random string
* Remove `from_secret_key_bytes`, it's just `keypair_from_seed`
* ledger: Generate keypairs by hand
* ed25519-tests: Use new rand
* runtime: Use new rand in all tests
* gossip: Clean up clippy and inline keypair generators
* core: Inline keypair generation for tests
* Push sbf lockfile change
* sdk: Sort dependencies correctly
* Remove `hash::new_with_thread_rng`, use `Hash::new_unique()`
* Use Keypair::new where chacha isn't used
* sdk: Fix build by marking rand 0.7 optional
* Hardcode secret key length, add static assertion
* Unify `getrandom` crate usage to fix linking errors
* bloom: Fix tests that require a random hash
* Remove some dependencies, try to unify others
* Remove unnecessary uses of rand and rand_core
* Update lockfiles
* Add back some dependencies to reduce rebuilds
* Increase max rebuilds from 14 to 15
* frozen-abi: Remove `getrandom`
* Bump rebuilds to 17
* Remove getrandom from zk-token-proof
* Enforce used_underscore_binding
* Fix all
* Work around for cfg()-ed code...
* ci....
* Make clipply fixes more pleasant
* Clone exit signal while intentionally shadowing
* Use more verbose code to avoid any #[allow(...)]s
generalizes Deduper to work with any hashable type
Current Deduper is hard-coded only for Packet type. In order to use
Deduper in retransmit-stage, we need to dedup types other than Packet.
The commit generalizes Deduper to any hashable type.
removes the false_positive_rate field from the Deduper
Deduper.false_positive_rate field is misleading because it is not
enforced until maybe_reset is called. But then maybe_reset can be
invoked with an explicit argument.
Current Deduper implementation uses many bits per entry:
https://github.com/solana-labs/solana/blob/65cd55261/perf/src/deduper.rs#L70-L73
and may be saturated quickly. It also lacks api to specify desired false
positive rate.
The commit instead uses an atomic bloom filter with K hash functions.
The false positive rate is obtained by tracking popcount of bits.
* introduce workspace.package
* introduce workspace.dependencies
* read version from root cargo.toml
* pass check when version = { workspace = true }
* don't bump version when version = { workspace = true }
* including workspace Cargo.toml when bump version
* programs/sbf use workspace inheritance
* fix increasing cargo version ignore program/sbf/Cargo.toml
* Add fully-reproducible online tracer for banking
* Don't use eprintln!()...
* Update programs/sbf/Cargo.lock...
* Remove meaningless assert_eq
* Group test-only code under aptly named mod
* Remove needless overflow handling in receive_until
* Delay stat aggregation as it's possible now
* Use Cow to avoid needless heap allocs
* Properly consume metrics action as soon as hold
* Trace UnprocessedTransactionStorage::len() instead
* Loosen joining api over type safety for replaystage
* Introce hash event to override these when simulating
* Use serde_with/serde_as instead of hacky workaround
* Update another Cargo.lock...
* Add detailed comment for Packet::buffer serialize
* Rename sender_overhead_minimized_receiver_loop()
* Use type interference for TraceError
* Another minor rename
* Retire now useless ForEach to simplify code
* Use type alias as much as possible
* Properly translate and propagate tracing errors
* Clarify --enable-banking-trace with better naming
* Consider unclean (signal-based) node restarts..
* Tweak logging and cli
* Remove Bank events as it's not needed anymore
* Make tpu own banking tracer thread
* Reduce diff a bit..
* Use latest serde_with
* Finally use the published rolling-file crate
* Make test code change more consistent
* Revive dead and non-terminating test code path...
* Dispose batches early now that possible
* Split off thread handle very early at ::new()
* Tweak message for TooSmallDirByteLimitl
* Remove too much of indirection
* Remove needless pub from ::channel()
* Clarify test comments
* Avoid needless event creation if tracer is disabled
* Write tests around file rotation and spill-over
* Remove unneeded PathBuf::clone()s...
* Introduce inner struct instead of tuple...
* Remove unused enum BankStatus...
* Avoid .unwrap() for the case of disabled tracer...
mergify[bot]
dependabot[bot]
Kevin Heavey
Yihau Chen
steviez
Ashwin Sekar
Yihau Chen
Jon C
GoodDaisy
Ryo Onodera
behzad nouri
Alexander Meißner
Brooks
steviez
sakridge
Tao Zhu
Brennan
Andrew Fitzgerald
Jeff Biseda
Michael Vines
Will Hickey
joeaba