c95f6c4b83
Remove spammy invalid rpc log ( #13100 )
2020-10-23 07:05:29 +00:00
8b0242a5d8
Allow nodes to advertise a different rpc address over gossip ( #13053 )
...
* Allow nodes to advertise a different rpc address over gossip
* Feedback
2020-10-22 03:31:48 +00:00
959880db60
Remove unused pubkey::Pubkey imports
2020-10-21 19:08:13 -07:00
17c391121a
Run `codemod --extensions rs Hash::new_rand solana_sdk: #️⃣ :new_rand`
2020-10-21 19:08:13 -07:00
7bc073defe
Run `codemod --extensions rs Pubkey::new_rand solana_sdk::pubkey::new_rand`
2020-10-21 19:08:13 -07:00
0776fa05c7
Add ledger-tool dead-slots and improve purge a lot ( #13065 )
...
* Add ledger-tool dead-slots and improve purge a lot
* Reduce batch size...
* Add --dead-slots-only and fixed purge ordering
2020-10-21 17:45:21 +00:00
efdb560e97
Various clean-ups before assert adjustment ( #13006 )
...
* Various clean-ups before assert adjustment
* oops
2020-10-21 10:26:20 +09:00
6858950f76
Remove frozen ABI modules from solana-sdk
2020-10-20 16:11:30 -07:00
c0675968b1
Support Debug Bank ( #13017 )
2020-10-21 01:05:45 +09:00
3b3f7341fa
validator: Activate RPC before halting on slot
2020-10-20 02:09:07 +00:00
75d62ca095
improves threads' utilization in processing gossip packets ( #12962 )
...
ClusterInfo::process_packets handles incoming packets in a thread_pool:
https://github.com/solana-labs/solana/blob/87311cce7/core/src/cluster_info.rs#L2118-L2134
However, profiling runtime shows that threads are not well utilized and
a lot of the processing is done sequentially.
This commit redistributes the work done in parallel. Testing on a gce
cluster shows 20%+ improvement in processing gossip packets with much
smaller variations.
2020-10-19 19:03:38 +00:00
54517ea454
Follow up to persistent tower with tests and API cleaning ( #12350 )
...
* Follow up to persistent tower
* Ignore for now...
* Hard-code validator identities for easy reasoning
* Add a test for opt. conf violation without tower
* Fix compile with rust < 1.47
* Remove unused method
* More move of assert tweak to the asser pr
* Add comments
* Clean up
* Clean the test addressing various review comments
* Clean up a bit
2020-10-19 16:37:03 +09:00
fd8ec27fe8
Another some tower logging improvements ( #12940 )
2020-10-16 14:44:07 +09:00
48283161c3
passes through feature-set to gossip requests handling ( #12878 )
...
* passes through feature-set to down to gossip requests handling
* takes the feature-set from root_bank instead of working_bank
2020-10-15 20:54:21 +00:00
d008dfb7ad
Bump spl-memo and spl-token versions ( #12917 )
2020-10-15 18:23:41 +00:00
05cf15a382
implements DataBudget using atomics ( #12856 )
2020-10-15 11:33:58 +00:00
a44e4d386f
Better tower logs for SwitchForkDecision and etc ( #12875 )
...
* Better tower logs for SwitchForkDecision and etc
* nits
* Update comment
2020-10-15 18:30:33 +09:00
c26512255d
RPC: Add metrics for TX encoding
2020-10-14 12:24:11 -06:00
1f1eb9f26e
Add separate push queue to reduce push lock contention ( #12713 )
2020-10-13 18:10:25 -07:00
247228ee61
Implementation-defined RPC server errors are now accessible to client/ users
2020-10-13 10:05:44 -07:00
649fe6d3b6
get_vote_accounts: access HashMap directly instead of turning it into an iterator
2020-10-13 04:12:10 +00:00
8a119c1483
Bump reqwest/rayon to get past `cargo audit`
2020-10-11 18:51:09 +00:00
c5c8da1ac0
Expose all rewards (fees, rent, voting and staking) in RPC getConfirmedBlock and the cli
2020-10-09 21:54:13 -07:00
1f4bcf70b0
Fix various ledger-tool error due to no builtins ( #12759 )
...
* Fix various ledger-tool error due to no builtins
* Add missing file...
2020-10-09 12:19:36 -06:00
c879e7c1ad
Fix fee mismatch on snapshot deserialize ( #12697 )
...
Co-authored-by: Carl Lin <carl@solana.com>
2020-10-08 23:44:41 -07:00
2c5f83c264
Add new internal accounts ( #12740 )
...
Co-authored-by: publish-docs.sh <maintainers@solana.com>
2020-10-09 00:48:32 +00:00
8f5431551e
Store program logs in blockstore / bigtable (TransactionWithStatusMeta) ( #12678 )
...
* introduce store program logs in blockstore / bigtable
* fix test, transaction logs created for successful transactions
* fix test for legacy bincode implementation around log_messages
* only api nodes should record logs
* truncate transaction logs to 100KB
* refactor log truncate for improved coverage
2020-10-08 12:06:15 -07:00
11df2e2236
Bump version to v1.5.0
2020-10-08 04:51:36 +00:00
b5faa11f73
removes invalid/outdated pending push messages early ( #12555 )
...
In CrdsGossipPush::new_push_messages:
https://github.com/solana-labs/solana/blob/972619edb/core/src/crds_gossip_push.rs#L211-L228
we already have paid the cost of looking-up the label in crds table and
checking the hash value and wallclock only to find out that in some
cases the value is invalid or is outdated. So might as well remove the
value here rather than wait for the next call to
purge_old_pending_push_messages:
https://github.com/solana-labs/solana/blob/972619edb/core/src/crds_gossip_push.rs#L372
2020-10-07 18:29:20 +00:00
41ad3dd8f0
Remove skip-no-mangle entirely ( #12696 )
2020-10-07 09:13:48 -07:00
e35889542b
RPC: Support base64 encoded transactions
...
Defaults to base58
2020-10-06 22:41:06 -06:00
7f67d36777
RPC: Check encoded transaction size before decoding
2020-10-06 22:41:06 -06:00
a5c6a78f6d
filters out inactive nodes from push options ( #12674 )
...
* filters out inactive nodes from push options
https://github.com/solana-labs/solana/pull/12620
patched the DDOS issue with nodes which go offline:
https://github.com/solana-labs/solana/issues/12409
However, offline nodes still see (much lesser) traffic spike, likely
because no origins are pruned from their bloom filter in active set:
https://github.com/solana-labs/solana/blob/aaf3790d8/core/src/crds_gossip_push.rs#L276-L286
and so multiple nodes push redundant duplicate messages to them
simultaneously:
https://github.com/solana-labs/solana/blob/aaf3790d8/core/src/crds_gossip_push.rs#L254-L255
This commit will filter out inactive peers from potential push targets
entirely. To mitigate eclipse attacks, staked nodes are retried
periodically.
* uses current timestamp in test/crds_gossip
2020-10-06 13:48:32 +00:00
026e7de819
Show commit in `--version` and ledger-tool's log ( #12636 )
...
* Show commit in `--version` and ledger-tool's log
* Another handy hidden env var
* Fix test
* Rename to semver!
* Fix syntax error...
2020-10-04 00:30:26 +09:00
71c469c72b
Weight push peers by how long we haven't pushed to them ( #12620 )
2020-10-02 13:57:26 -07:00
2c669f65f1
limits number of threads in core/tests/crds_gossip.rs ( #12615 )
...
crds_gossip tests start large networks, which with large thread-pools
will exhaust system resources, causing failures in ci tests:
https://buildkite.com/solana-labs/solana/builds/31953
The commit limits size of thread-pools in the test.
2020-10-02 16:55:44 +00:00
75b621160e
Add GetConfirmedBlocksWithLimit RPC method
2020-10-01 22:56:17 -07:00
f41a73d76a
Expose validator cli arguments for pubsub buffer tuning
2020-10-01 20:30:40 -07:00
1866521df6
retains hash value of outdated responses received from pull requests ( #12513 )
...
pull_response_fail_inserts has been increasing:
https://cdn.discordapp.com/attachments/478692221441409024/759096187587657778/pull_response_fail_insert.png
but for outdated values which fail to insert:
https://github.com/solana-labs/solana/blob/a5c3fc14b3/core/src/crds_gossip_pull.rs#L332-L344
https://github.com/solana-labs/solana/blob/a5c3fc14b3/core/src/crds.rs#L104-L108
are not recorded anywhere, and so the next pull request may obtain the
same redundant payload again, unnecessary taking bandwidth.
This commit holds on to the hashes of failed-inserts for a while, similar
to purged_values:
https://github.com/solana-labs/solana/blob/a5c3fc14b3/core/src/crds_gossip_pull.rs#L380
and filter them out for the next pull request:
https://github.com/solana-labs/solana/blob/a5c3fc14b3/core/src/crds_gossip_pull.rs#L204
2020-10-01 00:39:22 +00:00
c31a34fbcb
Include post balance information for rewards ( #12598 )
...
* Include post balance information for rewards
* Add post-balance to stored Reward struct
* Handle extended Reward in bigtable
Co-authored-by: Michael Vines <mvines@gmail.com>
2020-09-30 17:57:06 -06:00
3c7b9c2938
Move remaining nonce utils from runtime to SDK
2020-09-30 05:45:42 +00:00
537bbde22e
builds crds filters in parallel ( #12360 )
...
Based on run-time profiles, the majority time of new_pull_requests is
spent building bloom filters, in hashing and bit-vec ops.
This commit builds crds filters in parallel using rayon constructs. The
added benchmark shows ~5x speedup (4-core machine, 8 threads).
2020-09-29 23:06:02 +00:00
96a7d4dbd8
Query BigTable for block time if does not exist in blockstore ( #12560 )
2020-09-29 21:39:36 +00:00
ce98088457
Track inserted repair shreds ( #12455 )
2020-09-29 14:13:21 -07:00
36d55c0667
Increase rpc pubsub max payload to unblock large account notifications ( #12548 )
2020-09-30 00:09:39 +08:00
0d5258b6d3
separates out ClusterInfo::{gossip,listen} thread-pools ( #12535 )
...
https://github.com/solana-labs/solana/pull/12402
moved gossip-work threads:
https://github.com/solana-labs/solana/blob/afd9bfc45/core/src/cluster_info.rs#L2330-L2334
to ClusterInfo::new as a new field in the ClusterInfo struct:
https://github.com/solana-labs/solana/blob/35208c5ee/core/src/cluster_info.rs#L249
So that they can be shared between listen and gossip threads:
https://github.com/solana-labs/solana/blob/afd9bfc45/core/src/gossip_service.rs#L54-L67
However, in testing https://github.com/solana-labs/solana/pull/12360
it turned out this will cause breakage:
https://buildkite.com/solana-labs/solana/builds/31646
https://buildkite.com/solana-labs/solana/builds/31651
https://buildkite.com/solana-labs/solana/builds/31655
Whereas with separate thread pools all is good. It might be the case
that one thread is slowing down the other by exhausting the thread-pool
whereas with separate thread-pools we get fair scheduling guarantees
from the os.
This commit reverts https://github.com/solana-labs/solana/pull/12402
and instead adds separate thread-pools for listen and gossip threads:
https://github.com/solana-labs/solana/blob/afd9bfc45/core/src/gossip_service.rs#L54-L67
2020-09-29 09:05:31 +00:00
57ed4e4657
patches bug in Crds::find_old_labels with pubkey specific timeout ( #12528 )
...
Current code only returns values which are expired based on the default
timeout. Example from the added unit test:
- value inserted at time 0
- pubkey specific timeout = 1
- default timeout = 3
Then at now = 2, the value is expired, but the function fails to return
the value because it compares with the default timeout.
2020-09-29 09:04:40 +00:00
89621adca7
Rpc -> proper optimistic confirmation ( #12514 )
...
* Add service to track the most recent optimistically confirmed bank
* Plumb service into ClusterInfoVoteListener and ReplayStage
* Clean up test
* Use OptimisticallyConfirmedBank in RPC
* Remove superfluous notifications from RpcSubscriptions
* Use crossbeam to avoid mpsc recv_timeout panic
* Review comments
* Remove superfluous last_checked_slots, but pass in OptimisticallyConfirmedBank for complete correctness
2020-09-28 20:43:05 -06:00
06f84c65f1
Fix rooted accounts cleanup, simplify locking ( #12194 )
...
Co-authored-by: Carl Lin <carl@solana.com>
2020-09-28 16:04:46 -07:00
c94fe9236f
purges old pending push messages more efficiently ( #12522 )
2020-09-28 21:59:59 +00:00
Justin Starry
Michael Vines
Ryo Onodera
Trent Nelson
behzad nouri
Tyera Eulberg
sakridge
carllin
Dan Albert
Josh
Jack May
