Push message propagation has improved in recent versions of the gossip
code and we don't rely on pull requests as much as before. Handling pull
requests is also inefficient and expensive.
The commit reduces number of outgoing pull requests by down sampling.
* sdk: Add concurrent support for rand 0.7 and 0.8
* Update rand, rand_chacha, and getrandom versions
* Run command to replace `gen_range`
Run `git grep -l gen_range | xargs sed -i'' -e 's/gen_range(\(\S*\), /gen_range(\1../'
* sdk: Fix users of older `gen_range`
* Replace `hash::new_rand` with `hash::new_with_thread_rng`
Run:
```
git grep -l hash::new_rand | xargs sed -i'' -e 's/hash::new_rand([^)]*/hash::new_with_thread_rng(/'
```
* perf: Use `Keypair::new()` instead of `generate`
* Use older rand version in zk-token-sdk
* program-runtime: Inline random key generation
* bloom: Fix clippy warnings in tests
* streamer: Scope rng usage correctly
* perf: Fix clippy warning
* accounts-db: Map to char to generate a random string
* Remove `from_secret_key_bytes`, it's just `keypair_from_seed`
* ledger: Generate keypairs by hand
* ed25519-tests: Use new rand
* runtime: Use new rand in all tests
* gossip: Clean up clippy and inline keypair generators
* core: Inline keypair generation for tests
* Push sbf lockfile change
* sdk: Sort dependencies correctly
* Remove `hash::new_with_thread_rng`, use `Hash::new_unique()`
* Use Keypair::new where chacha isn't used
* sdk: Fix build by marking rand 0.7 optional
* Hardcode secret key length, add static assertion
* Unify `getrandom` crate usage to fix linking errors
* bloom: Fix tests that require a random hash
* Remove some dependencies, try to unify others
* Remove unnecessary uses of rand and rand_core
* Update lockfiles
* Add back some dependencies to reduce rebuilds
* Increase max rebuilds from 14 to 15
* frozen-abi: Remove `getrandom`
* Bump rebuilds to 17
* Remove getrandom from zk-token-proof
This commit adds CrdsEntry trait which allows generic lookups into crds
table. For example to get ContactInfo or LowestSlot associated with a
Pubkey, the lookup code would be respectively:
crds.get::<&ContactInfo>(pubkey)
crds.get::<&LowestSlot>(pubkey)
Current implementation of weighted_shuffle:
https://github.com/solana-labs/solana/blob/b08f8bd1b/gossip/src/weighted_shuffle.rs#L11-L37
uses a heuristic which results in biased samples.
For example, if the weights are [1, 10, 100], then the 3rd index should
come first 100 times more often than the 1st index. However,
weighted_shuffle is picking the 3rd index 200+ times more often than the
1st index, showing a disproportional bias in favor of higher weights.
This commit implements weighted shuffle using binary indexed tree to
maintain cumulative sum of weights while sampling. The resulting samples
are demonstrably unbiased and precisely proportional to the weights.
Additionally the iterator interface allows to skip computations when
not all indices are processed.
Of the use cases of weighted_shuffle, changing turbine code requires
feature-gating to keep the cluster in sync. That is not updated in
this commit, but can be done together with future updates to turbine.
* Move gossip modules to solana-gossip
* Update Protocol abi digest due to move
* Move gossip benches and hook up CI
* Remove unneeded Result entries
* Single use statements
behzad nouri
Jon Cinque
carllin
Tyera Eulberg