blockworks-foundation
/
solana
mirror of https://github.com/blockworks-foundation/solana.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
11,576 Commits 168 Branches 497 Tags 539 MiB
Commit Graph

227 Commits

Author SHA1 Message Date
behzad nouri 537bbde22e
builds crds filters in parallel (#12360) 
Based on run-time profiles, the majority time of new_pull_requests is
spent building bloom filters, in hashing and bit-vec ops.

This commit builds crds filters in parallel using rayon constructs. The
added benchmark shows ~5x speedup (4-core machine, 8 threads).
 2020-09-29 23:06:02 +00:00
behzad nouri 0d5258b6d3
separates out ClusterInfo::{gossip,listen} thread-pools (#12535) 
https://github.com/solana-labs/solana/pull/12402
moved gossip-work threads:
https://github.com/solana-labs/solana/blob/afd9bfc45/core/src/cluster_info.rs#L2330-L2334
to ClusterInfo::new as a new field in the ClusterInfo struct:
https://github.com/solana-labs/solana/blob/35208c5ee/core/src/cluster_info.rs#L249
So that they can be shared between listen and gossip threads:
https://github.com/solana-labs/solana/blob/afd9bfc45/core/src/gossip_service.rs#L54-L67

However, in testing https://github.com/solana-labs/solana/pull/12360
it turned out this will cause breakage:
https://buildkite.com/solana-labs/solana/builds/31646
https://buildkite.com/solana-labs/solana/builds/31651
https://buildkite.com/solana-labs/solana/builds/31655
Whereas with separate thread pools all is good. It might be the case
that one thread is slowing down the other by exhausting the thread-pool
whereas with separate thread-pools we get fair scheduling guarantees
from the os.

This commit reverts https://github.com/solana-labs/solana/pull/12402
and instead adds separate thread-pools for listen and gossip threads:
https://github.com/solana-labs/solana/blob/afd9bfc45/core/src/gossip_service.rs#L54-L67
 2020-09-29 09:05:31 +00:00
Michael Vines 35f5f9fc7b Add feature set identifier to gossiped version information 2020-09-25 11:40:36 -07:00
behzad nouri 42f1ef8acb
moves gossip-work thread pool cons to ClusterInfo::new (#12402) 2020-09-24 18:36:31 +00:00
Michael Vines daae638781 Add --gossip-validator argument 2020-09-14 20:18:27 -07:00
carllin 0f0a2ddafe
Filter push/pulls from spies (#11620) 
* Filter push/pulls from spies

* Don't pull from peers with shred version == 0, don't push to people with shred_version == 0

Co-authored-by: Carl <carl@solana.com>
 2020-08-18 18:52:45 -07:00
Michael Vines d15173ad9d Address latest nightly clippy lints, but globally disable stable_sort_primitive 2020-08-17 22:36:10 -07:00
sakridge 54137e3446
Add incoming pull response counter (#11591) 2020-08-12 14:07:05 -07:00
carllin 1b238dd63e
Gossip log (#11555) 
Co-authored-by: Carl <carl@solana.com>
 2020-08-11 21:03:54 +00:00
anatoly yakovenko 713851b68d
filter out old gossip pull requests (#11448) 
* init

* builds

* stats

* revert

* tests

* clippy

* add some jitter

* shorter jitter timer

* update

* fixup! update

* use saturating_sub

* fix filters
 2020-08-11 06:26:42 -07:00
Greg Fitzgerald bad486823c
Add a client for BankForks (#10728) 
Also:
* Use BanksClient in solana-tokens
 2020-08-07 08:45:17 -06:00
Michael Vines eefcf484cb clippy 2020-08-03 18:35:15 +00:00
Ryo Onodera 39b3ac6a8d
Introduce automatic ABI maintenance mechanism (2/2; rollout) (#8012) 
* Introduce automatic ABI maintenance mechanism (2/2; rollout)

* Fix stable clippy

* Change to symlink

* Freeze abi of Tower

* fmt...

* Improve dev-experience!

* Update BankSlotDelta

$ diff -u /tmp/abi8/*7dg6BreYxTuxiVz6aLvk3p2Z7GQk2cJqfGvC9h4FAoSj* /tmp/abi8/*9chBcbXVJ4fK7uGgydQzam5aHipaAKFw6V4LDFpjbE4w*
--- /tmp/abi8/bank__BankSlotDelta_frozen_abi__test_abi_digest_7dg6BreYxTuxiVz6aLvk3p2Z7GQk2cJqfGvC9h4FAoSj      2020-06-18 18:01:22.831228087 +0900
+++ /tmp/abi8/bank__BankSlotDelta_frozen_abi__test_abi_digest_9chBcbXVJ4fK7uGgydQzam5aHipaAKFw6V4LDFpjbE4w      2020-07-03 15:59:58.430695244 +0900
@@ -140,7 +140,7 @@
                                                         field u8
                                                             primitive u8
                                                         field solana_sdk::instruction::InstructionError
-                                                            enum InstructionError (variants = 34)
+                                                            enum InstructionError (variants = 35)
                                                                 variant(0) GenericError (unit)
                                                                 variant(1) InvalidArgument (unit)
                                                                 variant(2) InvalidInstructionData (unit)
@@ -176,6 +176,7 @@
                                                                 variant(31) CallDepth (unit)
                                                                 variant(32) MissingAccount (unit)
                                                                 variant(33) ReentrancyNotAllowed (unit)
+                                                                variant(34) MaxSeedLengthExceeded (unit)
                                                     variant(9) CallChainTooDeep (unit)
                                                     variant(10) MissingSignatureForFee (unit)
                                                     variant(11) InvalidAccountIndex (unit)

* Fix some merge conflicts...
 2020-07-06 20:22:23 +09:00
sakridge d9b389f510
Reduce logging lines (#10835) 2020-06-29 15:57:28 -07:00
Greg Fitzgerald 6ee222363e
Move BankForks to solana_runtime (#10637) 
* Move BankForks to solana_runtime

* Update imports
 2020-06-17 15:27:03 +00:00
sakridge 1eca9b19ab
Entry verify cleanup and gossip counters (#10632) 
* Add prune message counter

* Switch to us verification time to match other counters

* Add separate transaction/poh verify timing
 2020-06-16 14:00:29 -07:00
anatoly yakovenko ba83e4ca50
Fix fannout gossip bench (#10509) 
* Gossip benchmark

* Rayon tweaking

* push pulls

* fanout to max nodes

* fixup! fanout to max nodes

* fixup! fixup! fanout to max nodes

* update

* multi vote test

* fixup prune

* fast propagation

* fixups

* compute up to 95%

* test for specific tx

* stats

* stats

* fixed tests

* rename

* track a lagging view of which nodes have the local node in their active set in the local received_cache

* test fixups

* dups are old now

* dont prune your own origin

* send vote to tpu

* tests

* fixed tests

* fixed test

* update

* ignore scale

* lint

* fixup

* fixup

* fixup

* cleanup

Co-authored-by: Stephen Akridge <sakridge@gmail.com>
 2020-06-13 22:03:38 -07:00
sakridge 4c140acb3b
ClusterInfo cleanup (#10504) 
automerge
 2020-06-10 17:00:17 -07:00
sakridge 6eb5ef6ac7
Add back missing pull_response success counter (#10491) 2020-06-10 09:17:57 -07:00
sakridge ecb6959720
Optimize process pull responses (#10460) 
* Batch process pull responses

* Generate pull requests at 1/2 rate

* Do filtering work of process_pull_response in read lock

Only take write lock to insert if needed.
 2020-06-09 17:08:13 -07:00
anatoly yakovenko 832d324a23
Revert "Gossip PullRequests tend to return a lot of duplicates. (#10326)" (#10455) 
This reverts commit 31e20eff82.
 2020-06-09 07:27:00 -07:00
Kristofer Peterson e23340d89e
Clippy cleanup for all targets and nighly rust (also support 1.44.0) (#10445) 
* address warnings from 'rustup run beta cargo clippy --workspace'

minor refactoring in:
- cli/src/cli.rs
- cli/src/offline/blockhash_query.rs
- logger/src/lib.rs
- runtime/src/accounts_db.rs

expect some performance improvement AccountsDB::clean_accounts()

* address warnings from 'rustup run beta cargo clippy --workspace --tests'

* address warnings from 'rustup run nightly cargo clippy --workspace --all-targets'

* rustfmt

* fix warning stragglers

* properly fix clippy warnings test_vote_subscribe()
replace ref-to-arc with ref parameters where arc not cloned

* Remove lock around JsonRpcRequestProcessor (#10417)

automerge

* make ancestors parameter optional to avoid forcing construction of empty hash maps

Co-authored-by: Greg Fitzgerald <greg@solana.com>
 2020-06-09 09:38:14 +09:00
Greg Fitzgerald af8c21c559
Remove lock around JsonRpcRequestProcessor (#10417) 
automerge
 2020-06-07 20:54:03 -07:00
sakridge 0645a0c96d
Gossip cleanup remove duplicate gossip metrics and name worker threads (#10435) 
Refactor into functions
 2020-06-06 15:05:45 -07:00
sakridge 3d2230f1a9
Add pull request count metrics (#10421) 2020-06-05 09:36:31 -07:00
anatoly yakovenko 31e20eff82
Gossip PullRequests tend to return a lot of duplicates. (#10326) 
* filter messages that are likely to be pushed from the response

* tests

* tests

* wait to start filtering responses, and push stats to influx

* wait to start filtering responses, and push stats to influx

* reduce the timers to match the publish self timeout

* fmt

* fmt
 2020-06-05 08:01:45 -07:00
sakridge ef37b82ffa
More cluster stats and add epoch stakes cache in retransmit stage (#10345) 
* More cluster info metrics for push request/response counts

* Cache staked peers for the epoch
 2020-06-01 08:37:54 -07:00
Kristofer Peterson fb4d8e1f62
cleanup clippy tests (#10172) 
automerge
 2020-05-29 00:26:06 -07:00
sakridge 3f508b37fd
Skip gossip requests with different shred version and split lock (#10240) 2020-05-28 11:38:13 -07:00
sakridge 7ebd8ee531
Cluster info metrics (#10215) 2020-05-25 15:03:34 -07:00
carllin 3aae98c8be
Add switching vote instruction (#10197) 
* Add switching vote

* Make sure vote size stays under gossip limit

Co-authored-by: Carl <carl@solana.com>
 2020-05-24 15:38:35 -07:00
Kristofer Peterson 58ef02f02b
9951 clippy errors in the test suite (#10030) 
automerge
 2020-05-15 09:35:43 -07:00
Jack May eb1acaf927
Remove archiver and storage program (#9992) 
automerge
 2020-05-14 18:22:47 -07:00
Michael Vines 4e4a21f9b7
`solana-gossip spy` can now specify a shred version (#10040) 2020-05-13 19:37:40 -07:00
Michael Vines 2521f75c18
Advertise node software version in gossip (#9981) 
* Advertise node version in gossip

* Remove solana_clap_utils::version! macro
 2020-05-11 15:02:01 -07:00
Michael Vines 16ddd001f6
Gossip no longer pushes/pulls from nodes with a different shred version (#9868) 2020-05-05 20:15:19 -07:00
anatoly yakovenko a0514eb2ae
thiserror, docs, remove general Failure case (#9741) 
automerge
 2020-04-29 18:12:51 -07:00
anatoly yakovenko 8ef097bf6f
Input values are not sanitized after they are deserialized, making it far too easy for Leo to earn SOL (#9706) 
* sanitize gossip protocol messages
* sanitize transactions
* crds protocol sanitize
 2020-04-27 11:06:00 -07:00
carllin bab3502260
Push down cluster_info lock (#9594) 
* Push down cluster_info lock

* Rework budget decrement

Co-authored-by: Carl <carl@solana.com>
 2020-04-21 12:54:45 -07:00
sakridge 65a9658b13
Budget for gossip traffic (#9550) 2020-04-18 22:11:17 -07:00
Michael Vines 7b5cdf6adf Reduce ReceiveUpdates log spam 2020-04-17 13:32:19 -07:00
Michael Vines 83a96c557d Ensure --dynamic-port-range is wide enough 2020-04-14 12:21:05 -07:00
Michael Vines 8cda974552
Fix flaky new_archiver_external_ip_test (#9457) 
automerge
 2020-04-12 11:49:34 -07:00
sakridge ae5a6419d4
ReceiveUpdates spams the log, adjust the threshold higher (#9429) 2020-04-10 10:21:46 -07:00
Michael Vines 85feca305b Avoid port conflict in new_with_external_ip_test_gossip 2020-04-09 20:27:54 -07:00
sakridge 4677cdb4c2
Optimize broadcast cluster_info critical section (#9327) 2020-04-06 17:36:22 -07:00
carllin 7b68628e6c
Remove write lock (#9311) 
* Remove write lock

Co-authored-by: Carl <carl@solana.com>
 2020-04-05 15:18:45 -07:00
anatoly yakovenko 9cedeb0a8d
Pull streamer out into its own module. (#8917) 
automerge
 2020-03-17 23:30:23 -07:00
sakridge dc347dd3d7
Add Accounts hash consistency halting (#8772) 
* Accounts hash consistency halting

* Add option to inject account hash faults for testing.

Enable option in local cluster test to see that node halts.
 2020-03-16 08:37:31 -07:00
carllin 9afc5da2e1
Fix vote polling (#8829) 
Co-authored-by: Carl <carl@solana.com>
 2020-03-15 20:31:05 -07:00