Greg Fitzgerald
a19df7a36c
Add type annotations for external crates ( #4125 )
2019-05-06 10:11:50 -06:00
Tyera Eulberg
71f9b44687
Add Confirmations parameter to RPC Subscriptions ( #4154 )
...
* Add optional depth parameter to pubsub, and store in subscriptions
* Pass bank_forks into rpc_subscription; add method to check depth before notify and impl for account subscriptions
* Impl check-depth for signature subscriptions
* Impl check-depth for program subscriptions
* Plumb fork id through accounts
* Use fork id and root to prevent repeated account notifications; also s/Depth/Confirmations
* Write tests in terms of bank_forks
* Fixup accounts tests
* Add pubsub-confirmations tests
* Update pubsub documentation
2019-05-06 08:31:50 -06:00
carllin
297328ff9a
Fix improper tick sleeping time in test ( #4155 )
...
automerge
2019-05-03 20:15:10 -07:00
Pankaj Garg
3ef961fe37
Record poh ticks even when the node is not in leader schedule ( #4148 )
...
* remove obsolete test
2019-05-03 16:41:19 -07:00
Sagar Dhawan
a7b695c27a
Change replicators to slot-based ( #4118 )
2019-05-03 16:27:53 -07:00
Mark E. Sinclair
ed48d8323c
Reduce locking in Blocktree ( #4075 )
...
* Reduce lock contention in blocktree
* Store root slot in separate column
2019-05-03 16:46:02 -05:00
Michael Vines
7fe3c75c6b
Add a node-specific ip echo service to remove dependency on ifconfig.co ( #4137 )
2019-05-03 11:01:35 -07:00
Pankaj Garg
441e76ebeb
Index buffered transactions at the correct offset ( #4126 )
...
* tests
2019-05-02 19:05:53 -07:00
carllin
c2dfb9900e
Revert "Change forwarded metrics to be in line with fetch stage metrics ( #4068 )" ( #4135 )
...
automerge
2019-05-02 17:36:19 -07:00
Mark E. Sinclair
916458e132
Change erasure set size to 8:8 ( #4129 )
...
* Change erasure set size to 8:8
* Change tests to be agnostic to exact set size and ratio
* Add convenience methods for setting presence
2019-05-02 19:04:40 -05:00
Rob Walker
7f75cc8906
update nightly to 2019-05-01 ( #4111 )
...
* update nightly to 2019-05-01
* cargo fmt
* cargo fmt
* increase bench timeout
2019-05-01 20:08:42 -07:00
sakridge
1ab5098576
Move get_clients into gossip_service ( #4109 )
2019-05-01 17:14:01 -07:00
Pankaj Garg
3eec3cfac2
Cleanup banking stage in lieu of recent transaction forwarding changes ( #4101 )
2019-05-01 15:13:10 -07:00
Pankaj Garg
4f18fc836f
Forward transactions to the next slot leader ( #4092 )
...
- this ensures that transactions will reach in time for the next node to process them
2019-05-01 11:37:29 -07:00
Rob Walker
cb528af4e2
fix accounts_db storage.reset() ( #4094 )
...
* fix accounts_db storage.reset()
* fix compilation errors, remove unused, fix test_accounts_grow() failure
2019-05-01 09:27:13 -07:00
Sagar Dhawan
9add8d0afc
Add alternative to Spy Nodes that can fully participate in Gossip ( #4087 )
...
automerge
2019-04-30 16:42:56 -07:00
sakridge
408bdbce7a
Add non_default_stream parameter to cuda_verify ( #4079 )
2019-04-30 13:34:46 -07:00
carllin
1a259d4a3f
Prevent Requests/Caching of leader schedules for epochs beyond confirmed roots ( #4033 )
...
automerge
2019-04-30 13:23:21 -07:00
carllin
73f250f03a
Make minimum warmup period 32 slots long ( #4031 )
...
* Make minimum warmup period 32 slots long
* PR fixes
2019-04-29 15:26:52 -07:00
carllin
3b2adbc9df
Change forwarded metrics to be in line with fetch stage metrics ( #4068 )
...
automerge
2019-04-29 13:50:14 -07:00
Sagar Dhawan
4e41c81bc7
Fix the output from Gossip Discovery ( #4067 )
...
automerge
2019-04-29 13:19:24 -07:00
carllin
fabba82173
ignore non-descendants of roots in blocktree ( #4032 )
2019-04-29 12:29:14 -07:00
anatoly yakovenko
3e14af5033
bast bank ancestor check ( #4050 )
2019-04-28 10:27:09 -07:00
Michael Vines
656e2649a7
v0.15.0
2019-04-27 07:38:46 -07:00
Pankaj Garg
87f1bd58b9
Reduce submit frequency of counters that are very frequent and killing metrics DB ( #4027 )
2019-04-26 20:37:40 -07:00
sakridge
a056c1f18f
Don't send packets when msgs len is 0 ( #4030 )
...
And don't send metrics every iteration.
2019-04-26 17:27:31 -07:00
Jack May
b912ee7fdf
Nit: Spelling ( #4025 )
2019-04-26 16:02:37 -07:00
Mark E. Sinclair
8b1724bb70
Serialize blocktree writes by locking the database ( #4008 )
...
Move several private methods to free functions
2019-04-26 10:52:10 -05:00
Michael Vines
e71ab55288
Rename in-tree program_ids to be base-58 human readable ( #4001 )
2019-04-25 17:58:49 -07:00
Pankaj Garg
15aa07f2a0
Fix node count metrics ( #3997 )
...
- the count was truncated to avalanche neighborhood size
2019-04-25 13:47:38 -07:00
carllin
e4536621df
Log all uncommittable errors on validator ( #3999 )
2019-04-25 13:37:30 -07:00
Sagar Dhawan
a3c302c36a
Add signature polling to SyncClient ( #3996 )
...
automerge
2019-04-25 12:46:40 -07:00
carllin
c4d9dff590
Cherry-pick #3934 ( #3982 )
...
* Fix inserting bogus is_last blobs into blocktree
* Check for pre-existing blob before insert
* Ignore test that performs concurrent writes on blocktree as that is not supported
2019-04-25 00:04:49 -07:00
Sagar Dhawan
cf91ff8694
Remove Thin Client from storage stage ( #3976 )
...
automerge
2019-04-24 22:34:10 -07:00
Mark E. Sinclair
9a40ad76bd
Fix race in erasure metadata tracking ( #3962 )
...
* Fix erasure metadata race condition
* make erasure return the underlying error without wrapping it in the `solana::Error` type
* Add metric for erasure failures
* add tests to `ErasureMeta` indexing logic
* Add test to ensure erasure recovery failures don't cause panics
2019-04-24 17:53:01 -05:00
Pankaj Garg
9c7ccc0e2b
More metrics to track memory usage ( #3966 )
...
automerge
2019-04-24 11:11:30 -07:00
Sagar Dhawan
0cbac26591
Add genesis blockhash to blobs ( #3953 )
2019-04-23 16:24:44 -07:00
carllin
ab11327e34
Fix mismatch between leader/validator bank votability ( #3942 )
...
* Fix mismatch between leader/validator bank votability
2019-04-23 15:32:19 -07:00
Michael Vines
c309cd80aa
Add getClusterNodes/getSlotLeader JSON RPC API ( #3940 )
...
* Minor cleanup
* Include _this_ node in the contact info trace
* Add getClusterNodes/getSlotLeader RPC API
2019-04-23 14:46:41 -07:00
Pankaj Garg
d22a1c9b1f
Use smaller batch size in sigverify stage if CUDA is not available ( #3951 )
2019-04-23 12:41:50 -07:00
Pankaj Garg
7372ec9e1a
Use poh would_be_leader check in banking stage to hold or forward txs ( #3948 )
2019-04-23 11:56:30 -07:00
Pankaj Garg
de6af95061
Process forwarded packets only when the node is about to be the leader ( #3935 )
...
* Tests and metrics
2019-04-22 19:49:32 -07:00
Sagar Dhawan
43f7cd8149
Fix Retransmit slamming the leader with its own blobs ( #3938 )
2019-04-22 18:41:01 -07:00
Rob Walker
4b04c37c36
verify that blobs match a known leader for the slot ( #3927 )
...
* validate that blobs match a known leader for the slot
* clippy
2019-04-22 15:21:10 -07:00
sakridge
95a16426f3
Cleanup bench-exchange ( #3919 )
...
* bench-exchange changes
Generate new keypair for each bench-exchange
Add metrics
Tweak network sleep parameters for better reliability.
* Bench-exchange contract metrics
2019-04-22 13:16:28 -07:00
Michael Vines
af9ebf1d1a
Add status logging while processing a ledger ( #3916 )
2019-04-20 20:17:57 -07:00
Sagar Dhawan
349e8a9462
Ensure forwarded Blobs don't break Erasure ( #3907 )
2019-04-20 16:44:06 -07:00
Sagar Dhawan
c0bffb56df
Update Avalanche to use Epoch Stakes ( #3900 )
2019-04-19 22:31:40 -07:00
Sagar Dhawan
970cc32e65
Fix Data Plane computation when stakes are equal ( #3913 )
2019-04-19 21:07:21 -07:00
Rob Walker
3ab492ccf8
save erasure set size, initialize coding blob erasures with that size ( #3910 )
...
* save erasure set size, initialize coding blob erasures with that size
* fixup
* fixup
2019-04-19 20:22:51 -07:00
Rob Walker
efbb573316
add an env::var driven localcluster test ( #3906 )
2019-04-19 15:47:03 -07:00
Rob Walker
c3155a6e39
silence sigverify disabled, make disabling sigverify easier ( #3901 )
...
* silence sigverify disabled, make disabling sigverify easier
* fixup
2019-04-19 14:18:19 -07:00
Rob Walker
320bd66c84
handle zero num_will_fit ( #3892 )
...
* handle zero num_will_fit
* clippy
2019-04-19 12:02:33 -07:00
carllin
512bfc93cb
Add a cache for leader schedules ( #3841 )
...
* Add a cache for leader schedules
2019-04-19 02:39:44 -07:00
Rob Walker
6eac5951ed
Revert "Revert "revert-revert-erasure and erasure fixes ( #3833 )" ( #3855 )" ( #3889 )
...
This reverts commit 596f611ede
.
2019-04-18 21:56:43 -07:00
Sagar Dhawan
475a74d37f
Reduce packet recv batches ( #3894 )
...
* Reduce packet recv batches
* Fix type
2019-04-18 19:24:37 -07:00
Sagar Dhawan
7e409a13cd
Ack on empty Gossip Pull Responses and keep Entrypoint around ( #3881 )
...
* Ack on empty Gossip Pull Responses and keep Entrypoint around
* Address comments and fix test
* Update core/src/cluster_info.rs
Co-Authored-By: sagar-solana <sagar@solana.com>
* Update core/src/cluster_info.rs
Co-Authored-By: sagar-solana <sagar@solana.com>
2019-04-18 15:12:17 -07:00
Rob Walker
67b8ad6a0f
fix entries.to_blobs() ( #3882 )
...
* * rename Entry::serialized_size() to Entry::to_blob_size() to better
reduce confusion with bincode, et al. and to better reflect its
real meaning
* fix implementation of to_blob_size() to actually return what happens
when we do entries.to_blobs() (i.e. we serialize Vec<Entry>, not Entry)
* update tests to be more rigorous
* clippy
2019-04-18 14:45:41 -07:00
Tyera Eulberg
a9cfae486c
Revert-revert migrate to ed25519-dalek crate ( #3877 )
...
* Revert "Revert "Migrate from ring to ed25519-dalek, take 2 (#3844 )" (#3868 )"
This reverts commit 6a878602f2
.
* Fix Signature::verify method
2019-04-18 14:37:20 -06:00
Pankaj Garg
f2187780d2
Do not forward vote transactions ( #3871 )
2019-04-18 11:18:49 -07:00
Tyera Eulberg
6a878602f2
Revert "Migrate from ring to ed25519-dalek, take 2 ( #3844 )" ( #3868 )
...
This reverts commit e9b82bacda
.
2019-04-18 11:47:34 -06:00
Michael Vines
f8543a268f
solana-gossip now displays other spy nodes and contact info age ( #3867 )
2019-04-18 09:48:21 -07:00
Tyera Eulberg
e9b82bacda
Migrate from ring to ed25519-dalek, take 2 ( #3844 )
...
* Migrate from ring to ed25519-dalek
* Move gen_keypair_file test to a more appropriate location
* Fixup bench-exchange and add helper fn for single deterministic keypair
* Update golden
2019-04-18 10:38:32 -06:00
Pankaj Garg
2f4a3ed190
Use a separate channel to process votes in banking stage ( #3861 )
...
- This will help expedite the vote processing on peer nodes
2019-04-17 21:07:45 -07:00
Mark E. Sinclair
aa21f5343a
Remove broken read().unwrap() call to a non-shared packet
2019-04-17 21:24:40 -05:00
Sagar Dhawan
9c2809db21
Delete SharedPackets ( #3843 )
...
* Delete SharedPackets
* Fix bench and sigverify
2019-04-17 18:15:50 -07:00
Pankaj Garg
596f611ede
Revert "revert-revert-erasure and erasure fixes ( #3833 )" ( #3855 )
...
This reverts commit 6bef16a6a1
.
2019-04-17 18:04:30 -07:00
Sagar Dhawan
2b3218b5f2
Fix flaky tests by waiting for test node to boot ( #3845 )
2019-04-17 17:50:34 -07:00
Pankaj Garg
a2c8e3952f
Fixes to TPS calculation and reporting ( #3836 )
...
Fixes to TPS calculations and reporting
2019-04-17 15:37:01 -07:00
Rob Walker
6bef16a6a1
revert-revert-erasure and erasure fixes ( #3833 )
...
* fix erasure, more tests for full blobs, more metrics
* Revert "Revert "Use Rust erasure library and turn on erasure (#3768 )" (#3827 )"
This reverts commit 4b8cb72977
.
2019-04-17 15:13:54 -07:00
Rob Walker
01657ddfe7
packet.rs optimizations ( #3818 )
...
* packet.rs optimizations
* remove redundant and aggressive metric submission
* remove metrics submit(), get compiling again, honor log level in inc()
2019-04-17 14:14:57 -07:00
Greg Fitzgerald
51a2988bb2
Revert "Rename programs to instruction_processors ( #3789 )" ( #3824 )
...
This reverts commit 34344982a9
.
2019-04-17 15:05:49 -06:00
sakridge
4b8cb72977
Revert "Use Rust erasure library and turn on erasure ( #3768 )" ( #3827 )
...
This reverts commit b9bb5af4a5
.
2019-04-17 12:52:12 -07:00
Mark E. Sinclair
b9bb5af4a5
Use Rust erasure library and turn on erasure ( #3768 )
...
* split out erasure into new crate; add implementation using rust reed-solomon-library
* Track erasures with a &[bool] instead of indexes
* fix bug that reported the number of erasures incorrectly
* Introduce erasure `Session` for consistent config
* Increase test coverage; fix bugs
* Add ability to remove blobs from erasure meta tracking. test added
* Track deletion of coding blobs in blocktree via ErasureMeta. Added to
test
* Remove unused functions in blocktree
* add randomness to recovery thread to exercise recovery due to either new
data or coding blobs
* Add unit test for ErasureMeta index handling
* Re-enable test in broadcast stage
2019-04-16 23:00:24 -05:00
Rob Walker
e94f268346
reduce sigverify verbosity ( #3813 )
2019-04-16 18:25:53 -07:00
Amr Ali
34344982a9
Rename programs to instruction_processors ( #3789 )
...
* Rename programs to instruction_processors
* Updates around the code base to support instruction_processors rename
* Kabab instruction_processors
* Update Cargo.toml files and scripts to use instruction-processors
* Update Cargo.toml to use instruction-processors
* Update CI scripts to use instruction-processors
2019-04-16 22:39:00 +02:00
Michael Vines
63d66ece57
net/ testnet nodes now stake more lamports ( #3812 )
...
* Add --bootstrap-leader-lamports
* Generalize --no-stake into --stake NUM
* Use a large stake for net/ fullnodes
* Setup vote account before starting fullnode to avoid mixed log output
2019-04-16 13:03:01 -07:00
Michael Vines
d77359914f
Log the spy node id by default for better debug ( #3796 )
2019-04-15 20:58:37 -07:00
Dan Albert
8638b3bb19
Update Cargo.toml files for version = 0.14.0
2019-04-16 02:50:20 +00:00
anatoly yakovenko
68fc303b9b
Rework Accounts for fast squash, hashing state and checkpoint recovery. ( #3613 )
...
* accounts rewrite
* ignore grow tests
* skip duplicate roots
* allow for a root race
* logger
* accounts_index tests
* tests
* tests
2019-04-15 17:15:50 -07:00
Sagar Dhawan
3fcf03ff3e
Refactor LocalCluster and add support for listener nodes ( #3790 )
2019-04-15 15:27:45 -07:00
Greg Fitzgerald
80f3568062
Upgrade to Rust 1.34.0 ( #3781 )
...
* Upgrade to Rust 1.34.0
* Remove redundant closures
Thanks Clippy!
2019-04-15 15:56:08 -06:00
Michael Vines
d026ebb83a
Use tvu_peers() since validators no longer run an RPC port by default ( #3784 )
2019-04-15 13:25:09 -07:00
Rob Walker
64c6f05da2
persist set_root() and use it in blocktree_processor to limit squashes ( #3782 )
...
* rename locktower's slot to epoch
* persist set_root() and use it in blocktree_processor to limit squashes
2019-04-15 13:12:28 -07:00
Rob Walker
5c4689a326
rename locktower's slot to epoch ( #3776 )
2019-04-15 10:46:14 -07:00
Rob Walker
6859907df9
more rigorous erasure constants, comments ( #3766 )
...
* more rigorous erasure constants, comments
* new header size means new golden
2019-04-14 21:10:09 -07:00
Rob Walker
de52747950
remove max_tick_height replicode ( #3765 )
2019-04-14 19:15:31 -07:00
Rob Walker
bd1db51e07
delete db_window.rs, move contents to window_service, clean up process_blobs ( #3746 )
2019-04-14 18:52:05 -07:00
Rob Walker
dd005fb50e
fix broadcast to *always* call erasure generation, simplify generator, test slot reset better ( #3764 )
2019-04-14 18:12:37 -07:00
Rob Walker
542bafeb71
groom packet.rs, add blob.data alignment ( #3763 )
2019-04-14 17:30:08 -07:00
Rob Walker
e57a0ab05d
test some bits ( #3762 )
2019-04-14 17:10:30 -07:00
Pankaj Garg
2c745ce108
Shorten recv wait when there are buffered packets in banking stage ( #3757 )
...
- packets are buffered on leader rotation, when the next leader is
unknown
- shortening the wait allows the banking stage to poll for next
leader more frequently
2019-04-14 12:34:07 -07:00
Michael Vines
f6aa90e193
Add fullnode --dynamic-port-range option
2019-04-14 07:08:29 -07:00
Michael Vines
2277a39dd2
Default solana-gossip log-level to 'info'
2019-04-14 07:07:15 -07:00
Pankaj Garg
ee35ed5250
Refactored buffered packet forwarding code ( #3750 )
...
- Added unit tests
- Don't consume packets if bank is not known
2019-04-13 23:19:54 -07:00
Stephen Akridge
92b5e131fe
Name sigverify threads
2019-04-13 11:24:36 -07:00
Sagar Dhawan
1e8f83a74a
Use a better name for new api
2019-04-12 14:58:22 -07:00
Sagar Dhawan
1db80d79fc
Update get recent blockhashes to return confirmed blockhashes only
2019-04-12 14:58:22 -07:00
Stephen Akridge
1dac4c33b8
Change sigverify counter from entries to packets
...
batch or entries kind of useless since it can have some
variable number of packets
2019-04-12 13:19:46 -07:00
Pankaj Garg
44ebfa736a
Don't forward buffered packet to the same node ( #3712 )
...
- instead, process the packets
2019-04-11 17:23:45 -07:00