Commit Graph

2196 Commits

Author SHA1 Message Date
behzad nouri b041b55028
makes test_pull_request_time_pruning smaller (#16128) 2021-03-25 22:44:43 +00:00
sakridge 9b94741290
Fix test_replay_commitment_cache (#16131) 2021-03-25 21:16:39 +00:00
Justin Starry e817a6db00
Add timeout for local cluster partition tests (#16123)
* Add timeout for local cluster partition tests

* fix optimistic conf test logs

* Bump instruction count assertions
2021-03-25 13:27:07 -06:00
carllin 52703badfa
Setup ReplayStage confirmation scaffolding for duplicate slots (#9698) 2021-03-24 23:41:52 -07:00
Tyera Eulberg a8ef29df27
Support getBlockTime for unfinalized blocks (#16103) 2021-03-24 20:52:08 -06:00
sakridge 96ccc40f0a
Set ticks_per_slot higher for banking stage tests (#16094)
Tests are timing out because the bank hit the MaxTickHeight and
will not process the transactions.
2021-03-24 14:05:43 -07:00
Jeff Washington (jwash) f68860a643
poh record metrics (#16092) 2021-03-24 14:48:32 -05:00
behzad nouri a6c23648cb
limits CrdsGossipPull::pull_request_time size (#15793)
There is no pruning logic on CrdsGossipPull::pull_request_time
https://github.com/solana-labs/solana/blob/79ac1997d/core/src/crds_gossip_pull.rs#L172-L174
potentially allowing this to take too much memory.

Additionally, CrdsGossipPush::last_pushed_to is pruning recent push
timestamps:
https://github.com/solana-labs/solana/blob/79ac1997d/core/src/crds_gossip_push.rs#L275-L279
instead of the older ones.

Co-authored-by: Nathan Hawkins <utsl@utsl.org>
2021-03-24 18:33:56 +00:00
behzad nouri 570fd3f810
makes turbine peer computation consistent between broadcast and retransmit (#14910)
get_broadcast_peers is using tvu_peers:
https://github.com/solana-labs/solana/blob/84e52b606/core/src/broadcast_stage.rs#L362-L370
which is potentially inconsistent with retransmit_peers:
https://github.com/solana-labs/solana/blob/84e52b606/core/src/cluster_info.rs#L1332-L1345

Also, the leader does not include its own contact-info when broadcasting
shreds:
https://github.com/solana-labs/solana/blob/84e52b606/core/src/cluster_info.rs#L1324
but on the retransmit side, slot leader is removed only _after_ neighbors and
children are computed:
https://github.com/solana-labs/solana/blob/84e52b606/core/src/retransmit_stage.rs#L383-L384
So the turbine broadcast tree is different between the two stages.

This commit:
* Removes retransmit_peers. Broadcast and retransmit stages will use tvu_peers
  consistently.
* Retransmit stage removes slot leader _before_ computing children and
  neighbors.
2021-03-24 13:34:48 +00:00
Justin Starry e7fd7d46cf
rpc: add getSlotLeaders method (#16057) 2021-03-23 17:48:54 +00:00
behzad nouri 4f82b897bc
buffers data shreds to make larger erasure coded sets (#15849)
Broadcast stage batches up to 8 entries:
https://github.com/solana-labs/solana/blob/79280b304/core/src/broadcast_stage/broadcast_utils.rs#L26-L29
which will be serialized into some number of shreds and chunked into FEC
sets of at most 32 shreds each:
https://github.com/solana-labs/solana/blob/79280b304/ledger/src/shred.rs#L576-L597
So depending on the size of entries, FEC sets can be small, which may
aggravate loss rate.
For example 16 FEC sets of 2:2 data/code shreds each have higher loss
rate than one 32:32 set.

This commit broadcasts data shreds immediately, but also buffers them
until it has a batch of 32 data shreds, at which point 32 coding shreds
are generated and broadcasted.
2021-03-23 14:52:38 +00:00
Jeff Washington (jwash) 57ba86c821
eliminate lock on record (#15929)
* eliminate lock on record

* use same error as MaxHeightReached

* clippy

* review feedback

* refactor should_tick code

* pr feedback
2021-03-23 09:10:04 -05:00
Jeff Washington (jwash) 2fc609a294
add metric for ticks from poh_recorder.record (#16047) 2021-03-22 15:35:06 -05:00
Tyera Eulberg 2ec24d438f
Make getStakeActivation response consistent for undelegated accounts (#16038) 2021-03-19 14:54:56 -06:00
Jeff Washington (jwash) ddc758439e
metrics for poh_recorder.record (#15998) 2021-03-19 09:48:55 -05:00
Michael Vines 02b81dd05d Update to reqwest 0.11.2 2021-03-18 11:59:41 -07:00
Tyera Eulberg aa54c468ea
rpc: Add config options limiting getConfirmedBlock response data (#15970)
* Add new confirmed block struct

* Add RpcConfirmedBlockConfig options

* Configure block response based on new options

* Add client api, use in cli fetch_epoch_rewards

* Update docs

* Apply review suggestions
2021-03-18 17:58:20 +00:00
Michael Vines 04c99cf7ea Add --slots-per-epoch argument 2021-03-17 22:56:41 +00:00
carllin f548a04fae
Allow unbounded wallclock processing time in tests (#15961) 2021-03-17 15:48:50 -07:00
Michael Vines 59c19d9fbf Notice the user when the --mint, --bpf-program, or --clone arguments are ignored 2021-03-17 20:04:53 +00:00
Michael Vines 8a9b51952e Ignore flaky test_banking_stage_entries_only and test_banking_stage_entryfication 2021-03-17 11:28:56 -07:00
Jeff Washington (jwash) 40997d0aef
add metrics for tick producer and poh_recorder (#15931) 2021-03-17 10:38:26 -05:00
Jeff Washington (jwash) 5460fb10a2
drop poh lock after record (#15930) 2021-03-17 10:37:20 -05:00
Michael Vines a2eb655322 =1.7.0 2021-03-16 07:51:07 +00:00
Jeff Washington (jwash) efee8b62d7
a few missed set_data calls (#15846)
* a few missed set_data calls

* another set data call
2021-03-15 21:57:23 -05:00
Jeff Washington (jwash) c09ea2c314
More AccountSharedData construction (#15844)
* one more AccountSharedData construction

* one more construct
2021-03-15 19:27:17 -05:00
carllin c1ba265dd9
Wallclock BankingStage Throttle (#15731) 2021-03-15 17:11:15 -07:00
Michael Vines 0c9ca5522c Bump version to v1.7.0 2021-03-13 09:01:21 +00:00
Tyera Eulberg 5b2da19c93
Rpc: support extended config for getConfirmedBlock (#15827)
* Add rpc confirmed-block config wrapper to support struct of extended config

* Update docs

* Make config wrapper generic and use in getConfirmedTransaction as well

* Update/clean confirmed-tx docs
2021-03-12 22:19:45 +00:00
behzad nouri f2865dfd63
requires stakes for propagating crds values through gossip (#15561) 2021-03-12 15:50:14 +00:00
Justin Starry 918d04e3f0
Add more slot update notifications (#15734)
* Add more slot update notifications

* fix merge

* Address feedback and add integration test

* switch to datapoint

* remove unused shred method

* fix clippy

* new thread for rpc completed slots

* remove extra constant

* fixes

* rely on channel closing

* fix check
2021-03-12 21:44:06 +08:00
Ryo Onodera 4bbeb9c033
Remove old feature: simple_capitalization (#15763)
* Remove old feature: simple_capitalization

* Fix another failing test in core

* Finish up test cleanup

* Further clean up a bit
2021-03-12 11:12:40 +09:00
Jeff Washington (jwash) 952c3bcbb7
AccountSharedData construction (#15790) 2021-03-11 18:09:04 -06:00
Trent Nelson 24d18b3cf2 docs: add docs links for crates published to crates.io 2021-03-11 08:38:18 +00:00
Jeff Washington (jwash) 1135ffd595
mut data refs as slice (#15782) 2021-03-10 15:28:03 -06:00
behzad nouri 56923c91bf
limits number of unique pubkeys in the crds table (#15539) 2021-03-10 20:46:05 +00:00
Jeff Washington (jwash) 52e54e1100
account.data -> data() (#15778) 2021-03-09 22:31:33 +00:00
Jeff Washington (jwash) 8a3135d17b
Account->AccountSharedData (#15691) 2021-03-09 15:06:07 -06:00
carllin 2bee9435f3
Add tracer key for tracing transaction path through the network (#15732) 2021-03-08 19:31:00 -08:00
carllin 331c45decf
Report datapoint on number of retransmit shreds (#15694) 2021-03-08 17:54:53 -08:00
sakridge d09112fa6d
PoH batch size calibration (#15717) 2021-03-05 16:01:21 -08:00
Michael Vines 4a3ab77baf Remove unused id field 2021-03-05 19:07:59 +00:00
Michael Vines 66b781eec3 Add 'unknown' health check state 2021-03-05 17:46:50 +00:00
Tyera Eulberg 7e65289729
Convert blockstore TransactionStatus column family to protobufs (#15733)
* Prevent panic if TransactionStatus can't be deserialized

* Convert Blockstore TransactionStatus column to protobuf

* Add compatability test
2021-03-05 09:05:35 -07:00
Michael Vines bd13262b42 Add validator startup process reporting before RPC is available 2021-03-05 08:03:36 -08:00
Michael Vines 24ab84936e Break up RPC API into three categories: minimal, full and admin 2021-03-04 16:39:44 -08:00
Jeff Washington (jwash) 34bebb7d09
report execution details in replay time (#15693) 2021-03-04 11:38:12 -06:00
Jeff Washington (jwash) be35c1c1b7
add execute detail timings (#15638) 2021-03-03 17:07:45 -06:00
behzad nouri 658951e680
sends only the latest vote of each validator to the banking stage (#15629) 2021-03-03 19:07:16 +00:00
carllin aacb28c453
Only report metrics every second (#15652) 2021-03-03 10:58:47 -08:00