blockworks-foundation
/
solana
mirror of https://github.com/blockworks-foundation/solana.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
11,576 Commits 168 Branches 497 Tags 539 MiB
Commit Graph

227 Commits

Author SHA1 Message Date
Michael Vines 5481d1a039 Validators now run a full gossip node while looking for a snapshot 2020-03-15 09:31:55 -07:00
anatoly yakovenko f64ab49307
Cluster has no way to know which slots are available (#8732) 
automerge
 2020-03-11 21:31:50 -07:00
Michael Vines 448b957a13
Add --bind-address and --rpc-bind-address validator arguments (#8628) 2020-03-04 22:46:43 -07:00
Michael Vines 2a5605db24 Reduce max snapshot hashes to stay under MTU 2020-02-29 09:21:52 -07:00
Pankaj Garg aa80f69171
Promote some datapoints to `info` to fix dashboard (#8381) 
automerge
 2020-02-21 13:41:49 -08:00
Michael Vines cc7c6c960e Search for the validator with the highest snapshot 2020-02-20 17:04:48 -07:00
Tyera Eulberg ab361a8073
Rename KeypairUtil to Signer (#8360) 
automerge
 2020-02-20 13:28:55 -08:00
sakridge 1720fe6a46
Snapshot hash gossip changes (#8358) 2020-02-20 11:46:13 -08:00
Pankaj Garg e50bc0d34b
Do not compress small incomplete slot list (#8355) 
automerge
 2020-02-20 09:48:39 -08:00
Pankaj Garg ea8d9d1aea
Bitwise compress incomplete epoch slots (#8341) 2020-02-19 20:24:09 -08:00
Sagar Dhawan 221866f74e
Process Gossip in parallel and add an upper limit (#8328) 2020-02-19 21:31:55 -06:00
Michael Vines 83a8e82626 Remove dead code 2020-02-18 21:08:43 -07:00
Pankaj Garg ac1d075d73
Drop packet if destination is unspecified (0.0.0.0/0) (#8321) 2020-02-18 16:14:20 -08:00
Pankaj Garg 0d5c1239c6
Update epoch slots to include all missing slots (#8276) 
* Update epoch slots to include all missing slots

* new test for compress/decompress

* address review comments

* limit cache based on size, instead of comparing roots
 2020-02-17 12:39:30 -08:00
anatoly yakovenko 17fb8258e5
Datapoints overwhelm the metrics queue and blow up ram usage. (#8272) 
automerge
 2020-02-14 11:11:55 -08:00
Greg Fitzgerald 127553ce4b
Wrap ed25519_dalek::Keypair (#8247) 2020-02-12 14:15:12 -07:00
Michael Vines ecb055a252
Expel ContactInfo::new() (#8245) 
automerge
 2020-02-12 12:58:51 -08:00
Michael Vines d0a4686990 Avoid assigning the serve repair port to the storage port 2020-02-12 12:00:11 -07:00
carllin d3712dd26d Factor repair from gossip (#8044) 2020-02-11 13:11:48 -07:00
Sagar Dhawan fa00803fbf
Filter old CrdsValues received via Pull Responses in Gossip (#8150) 
* Add CrdsValue timeout checks on Pull Responses

* Allow older values to enter Crds as long as a ContactInfo exists

* Allow staked contact infos to be inserted into crds if they haven't expired

* Try and handle oveflows

* Fix test

* Some comments

* Fix compile

* fix test deadlock

* Add a test for processing timed out values received via pull response
 2020-02-07 12:38:24 -08:00
carllin fe590da3b6
Revert "Factor repair from gossip (#8044)" (#8143) 
This reverts commit e61257695f.
 2020-02-06 11:44:20 -08:00
François Garillot f016c9a669
Maintenance : simplify a few patterns, remove unneeded dependencies (#8137) 
* Simplify a few pattern matches

* Removing unneeded dependencies, upgrading internal version #s

 net-shaper: Removing log, semver, serde_derive
 bench-tps: Removing serde, serde_derive
 banking-bench: Removing solana
 ledger-tool: Removing bincode, serde, serde_derive
 librapay: Removing solana, language_e2e_tests
 log-analyzer: Removing log, semver, serde_derive
 exchange: Removing solana
 core: Removing crc, memmap, symlink, untrusted
 perf: Removing serde_derive
 genesis: Removing hex, serde_derive
 sdk-c: Removing sha2
 sys-tuner: Removing semver
 bench-exchange: Removing bincode, bs58, env_logger, serde, serde_derive, untrusted, ws
 btc_spv_bin: Removing serde_json
 btc_spv: Removing chrono
 bpf_loader: Removing serde
 ledger: Removing dlopen, dlopen_derive, serde_derive
 move_loader: Removing byteorder, libc, language_e2e_tests
 ownable: Removing serde, serde_derive
 client: Removing rand
 archiver-utils: Removing rand_chacha
 validator: Removing serde_json, tempfile
 param_passing_dep: Removing solana
 failure: Removing log
 vest: Removing log
 vote-signer: Removing bs58, serde
 local-cluster: Removing symlink
 keygen: Removing rpassword
 install: Removing bs58, log
 upload-perf: Removing log
 runtime: Removing serde_json
 stake: Removing rand

* Add modified Cargo.lock

* fixup! Simplify a few pattern matches

* fixup! Simplify a few pattern matches
 2020-02-06 10:02:38 -07:00
carllin e61257695f
Factor repair from gossip (#8044) 2020-01-31 14:23:50 -08:00
Michael Vines 775fa0c968
Minor --expected-shred fix, clean up shred-related gossip log messages (#8041) 
automerge
 2020-01-30 13:22:05 -08:00
Michael Vines 0c55b37976 Add different shred test to test_tvu_peers_and_stakes 2020-01-30 10:30:32 -07:00
Sagar Dhawan 64c42e28dc
Add shred version filters to Crds Accessors (#8027) 
* Add shred version filters to Crds Accessors

* Adopt entrypoint shred_version if one isn't provided
 2020-01-30 00:15:37 -08:00
Michael Vines bea9cd9684 Add --expected-shred-version option 2020-01-29 20:08:42 -07:00
Ryo Onodera a8d33c9950
Spy just for RPC to avoid premature supermajority (#7856) 
* Spy just for RPC to avoid premature supermajority

* Make gossip_content_info private

Co-Authored-By: Michael Vines <mvines@gmail.com>

* Fix misindent...

Co-authored-by: Michael Vines <mvines@gmail.com>
 2020-01-20 10:50:31 +09:00
Michael Vines e6af4511a8 Include shred version in gossip 2020-01-14 14:32:40 -07:00
Greg Fitzgerald b5dba77056 Rename blocktree to blockstore (#7757) 
automerge
 2020-01-13 13:13:52 -08:00
Michael Vines a0fb9de515 Move thread_mem_usage module into measure/ 2020-01-03 13:25:37 -07:00
Rob Walker a7040896f0
Update to rust 1.40.0 (#7572) 
* Update to rust 1.40.0

* fixups
 2019-12-19 23:27:54 -08:00
anatoly yakovenko 97589f77f8 Pipeline broadcast socket transmit and blocktree record (#7481) 
automerge
 2019-12-16 17:11:18 -08:00
Sagar Dhawan a95d37ea25
Fix repair slowness when most peers are unable to serve requests (#7287) 
* Fix repair when most peers are incapable of serving requests

* Add a test for getting the lowest slot in blocktree

* Replace some more u64s with Slot
 2019-12-05 11:25:13 -08:00
Michael Vines 0bd41f98ed Avoid jemalloc in windows build (#7089) 
automerge
 2019-11-21 18:39:29 -08:00
Sagar Dhawan 100b9dd12a
Fix num nodes metrics (#7068) 
* Fix num nodes metric

* Fix node count metrics
 2019-11-20 17:00:31 -08:00
anatoly yakovenko b150da837a Use epoch as the gossip purge timeout for staked nodes. (#7005) 
automerge
 2019-11-20 11:25:18 -08:00
Jack May d2ed921bc6
Cleanup nightly warnings (#7055) 2019-11-19 20:15:37 -08:00
Sagar Dhawan f2badf2c5d
Fix a bug where gossip loops forever while splitting messages (#7032) 
* Fix a bug where gossip loops forever while splitting messages

* Get rid of while loop

* Minor clean up and rename
 2019-11-19 11:51:51 -08:00
Michael Vines 006c39380a
Display 'none' instead of 0.0.0.0 (#6973) 2019-11-14 20:24:35 -07:00
Sagar Dhawan 79d7090867
Remove obsolete references to Blob (#6957) 
* Remove the name "blob" from archivers

* Remove the name "blob" from broadcast

* Remove the name "blob" from Cluset Info

* Remove the name "blob" from Repair

* Remove the name "blob" from a bunch more places

* Remove the name "blob" from tests and book
 2019-11-14 11:49:31 -08:00
Michael Vines e7f63cd336
Upgrade to rust 1.39.0 (#6939) 
* Upgrade to rust 1.39.0

* 1.39.0 clippy
 2019-11-14 12:27:01 -07:00
Sagar Dhawan f108f483b7
Remove Blobs and switch to Packets (#6937) 
* Remove Blobs and switch to Packets

* Fix some gossip messages not respecting MTU size

* Failure to serialize is not fatal

* Add log macros

* Remove unused extern

* Apparently macro use is required

* Explicitly scope macro

* Fix test compile
 2019-11-14 10:24:53 -08:00
carllin 43e2301e2c
Fix roots overrunning broadcast (#6884) 
* Add trusted pathway for insert_shreds to avoid checks
 2019-11-14 00:32:07 -08:00
Greg Fitzgerald 30a08f4282 Cleanup ledger macros (#6916) 
automerge
 2019-11-13 07:14:09 -08:00
Ryo Onodera 3faeb7fa79 Rename solana-netutil to solana-net-utils for consistency (#6895) 
* sed -i -e 's/netutil/net_utils/g' $(git grep --files-with-matches netutil :**.rs)

* sed -i -e 's/netutil/net-utils/g' $(git grep --files-with-matches netutil)

* git mv netutil/ net-utils

* Tweak a bit

* Fix rustfmt & clippy
 2019-11-12 13:37:13 -07:00
Michael Vines 1ddf90ed08
Compress contact_info_trace() output to improve CI log rendering (#6852) 2019-11-09 01:12:18 -07:00
Sagar Dhawan 67d1e2903c
Upgrade Repair be more intelligent and agressive (#6789) 
* Upgrade Repair be more intelligent and agressive

* Fix u64 casts

* Fix missing bracket

* Add 1 second delay to test to allow repair to kick in
 2019-11-07 11:08:09 -08:00
Jack May 65de227520
Don't panic on packet data (#6769) 2019-11-06 14:32:37 -08:00
anatoly yakovenko 67f636545a Refactor sigverify to stage for signing shreds on the GPU (#6635) 
automerge
 2019-11-06 10:52:30 -08:00
Jack May 9614d17024
Limit deserialization of data coming off the wire (#6751) 
* Limit deserialization of data coming off the wire

* Feedback and cleanup
 2019-11-06 00:07:57 -08:00
anatoly yakovenko b825d04597 Pull perf into a separate module. (#6718) 
automerge
 2019-11-04 20:13:43 -08:00
anatoly yakovenko f54cfcdb8f
Store and persists full stack of tower votes in gossip (#6695) 
* vote array

wip

wip

wip

update

gossip index should match tower index

tests build

clippy

test index after expired vote

test

bank specific last vote sync time

* verify

* we are likely to see many more warnings about old votes now
 2019-11-04 16:19:54 -08:00
Pankaj Garg c138d692b1
Show all ports for nodes in gossip table (#6717) 
* Show all ports for nodes in gossip table

* review comments
 2019-11-04 15:05:08 -08:00
Sagar Dhawan 568475e2db
Fix incorrectly signed CrdsValues (#6696) 2019-11-03 10:07:51 -08:00
Michael Vines 50a17fc00b Use Slot and Epoch type aliases instead of raw u64 (#6693) 
automerge
 2019-11-02 00:38:30 -07:00
Sagar Dhawan a57f6b70da Fix swapped repair and forwards addrs (#6691) 
automerge
 2019-11-01 16:01:42 -07:00
Sagar Dhawan 2d67962c2f
Send repairman shreds to the repair socket (#6671) 2019-10-31 18:23:50 -07:00
Sagar Dhawan 801337a422 Refactor Weighted Shuffle (#6614) 
automerge
 2019-10-29 21:02:11 -07:00
Michael Vines 397ea05aa7
spy nodes are now gossip entrypoints (#6532) 2019-10-24 15:35:33 -07:00
Greg Fitzgerald 9232057e95
Rename replicator to archiver (#6464) 
* Rename replicator to archiver

* cargo fmt

* Fix grammar
 2019-10-21 11:29:37 -06:00
Greg Fitzgerald 2636418659
Move blocktree_processor to solana_ledger (#6460) 
* Drop core::result dependency in bank_forks

* Move blocktree_processor into solana_ledger
 2019-10-20 09:54:38 -06:00
Pankaj Garg badeb4d31a
Rework shred headers to fix position of signature (#6451) 
* Rework shred headers to fix position of signature

* fix clippy
 2019-10-18 22:55:59 -07:00
Greg Fitzgerald 5468be2ef9 Add solana-ledger crate (#6415) 
automerge
 2019-10-18 09:28:51 -07:00
Pankaj Garg 33052c1dd2 Cleanup shred header structures (#6378) 
automerge
 2019-10-15 20:48:45 -07:00
sakridge 1afc527919
Lower cluster_info-num_nodes datapoint (#6368) 2019-10-15 14:42:19 -07:00
Pankaj Garg 364781366a
Use sendmmsg for broadcasting shreds (#6325) 
* Replace packet with slice of data in sendmmsg

* fixes

* fix bench
 2019-10-10 19:38:48 -07:00
Greg Fitzgerald c6e4641781
Remove many uses of legacy term 'fullnode' (#6324) 2019-10-10 17:33:00 -06:00
Pankaj Garg 753bd77b41
Use multicast to send retransmit packets (#6319) 2019-10-10 15:02:36 -07:00
sakridge 1b775044f7
Use multiple retransmit stage threads/sockets (#6279) 2019-10-10 13:24:03 -07:00
carllin dd66d16fdb
Broadcast final shred for slots that are interrupted (#6269) 
* Broadcast final shred for slots that are interrupted
 2019-10-09 16:07:18 -07:00
Sagar Dhawan 32312f3c16
Do not retransmit Repair responses (#6284) 
* Do not retransmit Repair responses

* Add a test

* Refactor neighboring functionality
 2019-10-09 13:11:19 -07:00
Sagar Dhawan 723f9a9b81
Remove unnecessary locking in retransmit stage (#6276) 
* Add more detailed metrics to retransmit

* Remove unnecessary locking and add more metrics
 2019-10-08 14:41:16 -07:00
carllin ac2374e9a1
Shred entries in parallel (#6180) 
* Make shredding more parallel

* Fix erasure tests

* Fix replicator test

* Remove UnfinishedSlotInfo
 2019-10-08 00:42:51 -07:00
sakridge 5ef012b2c1
Tweak debug to remove unreadable datapoints (#6060) 2019-10-04 16:25:22 -07:00
Michael Vines 7f53737000
Periodically pull from the entrypoint if it's no longer in Crdt (#6240) 2019-10-04 14:18:07 -07:00
Sagar Dhawan 23ea8ae56b
Optimize retransmit stage (#6231) 
* Optimize retransmit stage

* Remove comment

* Fix test

* Skip iteration to fixup 0 stakes
 2019-10-04 11:52:02 -07:00
Michael Vines f9f5bc2eb5
More clippy 2019-10-02 21:21:07 -07:00
sakridge ae7700296d
broadcast_shreds opt (#6175) 
* Don't clone/copy/sort ContactInfo vec
 2019-10-01 09:38:29 -07:00
Pankaj Garg 57e90948a8
Remove dead code from cluster_info (#6051) 2019-09-24 10:27:59 -07:00
Sagar Dhawan d379786c90
Fix bind errors (#5986) 
* Add ability to bind to a common tcp/udp port

* Extend port range for local-net sanity and fix validator executable
 2019-09-19 17:16:22 -07:00
Pankaj Garg 783e8672e7
Removed Shred enum (#5963) 
* Remove shred enum and it's references

* rename ShredInfo to Shred

* clippy
 2019-09-18 16:24:30 -07:00
Pankaj Garg 6c4e656795
Remove obsoleted code from shred (#5954) 
* Remove obsoleted code from shred

* fix broken test
 2019-09-18 13:56:44 -07:00
Pankaj Garg ff608992ee
Replace Shred usage with ShredInfo (#5939) 
* Replace Shred usage with ShredInfo

* Fix tests

* fix clippy
 2019-09-17 18:22:46 -07:00
Sagar Dhawan c44e7ce184
Leaders should not broadcast to replicators (#5917) 2019-09-16 17:56:34 -07:00
Pankaj Garg 5f54573613
More shred related cleanup (#5909) 
* More shred related cleanup

* fix uncle
 2019-09-16 10:28:28 -07:00
Jack May e8d88f3237
Split SDK's timing.rs (#5823) 2019-09-06 14:30:56 -07:00
Pankaj Garg 3d3b03a123
Verify signature of recovered shred before adding them to blocktree (#5811) 
* Verify signature of recovered shred before adding them to blocktree

* fix failing tests, and review comments
 2019-09-05 18:20:30 -07:00
Pankaj Garg f78b865cba
Cleanup shreds to remove FirstShred data structure (#5789) 
* Cleanup shreds to remove FirstShred data structure

* Also reduce size used by parent slot information in shred header

* clippy

* fixes

* fix chacha test
 2019-09-04 21:06:47 -07:00
Pankaj Garg 02ee2a601c
Further cleanup of blocktree after Blob deprecation (#5780) 2019-09-04 12:47:09 -07:00
Pankaj Garg 3b0d48e3b8
Remove blocktree blob references (#5691) 
* Remove blocktree blob references

* fixes and cleanup

* replace uninitialized() call with MaybeUninit

* fix bench
 2019-09-03 21:32:51 -07:00
Michael Vines a29f0484dc
Add newline before cluster info log (#5671) 2019-08-27 08:33:48 -07:00
Michael Vines db768b4c3a
Log contact info every 10 seconds (#5663) 2019-08-26 18:31:14 -07:00
Pankaj Garg 4ac1213c9c
Integrate coding shreds and recovery (#5625) 
* Integrate coding shreds and recovery

* More tests for shreds and some fixes

* address review comments

* fixes to code shred generation

* unignore tests

* fixes to recovery
 2019-08-26 18:27:45 -07:00
Pankaj Garg 4798e7fa73
Integrate data shreds (#5541) 
* Insert data shreds in blocktree and database

* Integrate data shreds with rest of the code base

* address review comments, and some clippy fixes

* Fixes to some tests

* more test fixes

* ignore some local cluster tests

* ignore replicator local cluster tests
 2019-08-20 17:16:06 -07:00
Sagar Dhawan bd193535c9
Cap CrdsFilter sizes such that PullRequest no longer exceeds MTU (#5561) 2019-08-19 18:14:10 -07:00
Sagar Dhawan e73cbdda61 Reduce log level for known issue (#5536) 
automerge
 2019-08-15 19:42:27 -07:00
Sagar Dhawan 4ee212ae4c
Coalesce gossip pull requests and serve them in batches (#5501) 
* Coalesce gossip pull requests and serve them in batches

* batch all filters and immediately respond to messages in gossip

* Fix tests

* make download_from_replicator perform a greedy recv
 2019-08-15 17:04:45 -07:00
Sagar Dhawan 1d0608200c
Restore blob size fix (#5516) 
* Revert "Revert "Fix gossip messages growing beyond blob size  (#5460)" (#5512)"

This reverts commit 97d57d168b.

* Fix Crds filters
 2019-08-13 18:04:14 -07:00
Sagar Dhawan 97d57d168b Revert "Fix gossip messages growing beyond blob size (#5460)" (#5512) 
This reverts commit a8eb0409b7.
 2019-08-13 10:29:26 -07:00