Commit Graph

2232 Commits

Author SHA1 Message Date
Justin Starry f641429056
Remove blake3 from bpf program dependencies (#16506) 2021-04-13 10:06:05 +00:00
Tyera Eulberg 37afa00ffb
Rpc: deprecate getConfirmed endpoints (#16502)
* Deprecate getConfirmed methods in rpc

* Add new methods to docs

* Move deprecated rpc methods to separate docs section

* Add note to docs about removal timing
2021-04-13 01:50:15 -06:00
Justin Starry 85eb37fab0
Merge pull request from GHSA-8v47-8c53-wwrc
* Track transaction check time separately from account loads

* banking packet process metrics

* Remove signature clone in status cache lookup

* Reduce allocations when converting packets to transactions

* Add blake3 hash of transaction messages in status cache

* Bug fixes

* fix tests and run fmt

* Address feedback

* fix simd tx entry verification

* Fix rebase

* Feedback

* clean up

* Add tests

* Remove feature switch and fall back to signature check

* Bump programs/bpf Cargo.lock

* clippy

* nudge benches

* Bump `BankSlotDelta` frozen ABI hash`

* Add blake3 to sdk/programs/Cargo.lock

* nudge bpf tests

* short circuit status cache checks

Co-authored-by: Trent Nelson <trent@solana.com>
2021-04-13 00:28:08 -06:00
Tyera Eulberg 70f3f7e679
Move obsolete rpc endpoints to separate api for removal (#16500)
* Move obsolete rpc methods to separate api for removal

* Remove obsolete method from docs

* Fix test using obs method
2021-04-12 20:33:40 -06:00
Michael Vines 2229b70c4e Add authorized-voter add/remove-all commands 2021-04-12 15:55:28 -07:00
Michael Vines 17a173ebb5 Clean up build warning 2021-04-12 15:55:28 -07:00
carllin dc7030ffaa
Allow fork choice to support multiple versions of a slot (#16266) 2021-04-12 01:00:59 -07:00
carllin 99b3aab703
Track gossip vote updates per hash for replay stage (#16421)
* Track gossip vote updates per hash for replay stage
2021-04-10 17:34:45 -07:00
Christian Drappi 54a04bac3d
Apple M1 compatibility (#16346)
Co-authored-by: Christian Drappi <christiandrappi@Christians-MacBook-Pro.local>
2021-04-09 17:21:01 -07:00
Tyera Eulberg 8bc0bdd40b
Fill in not-yet-finalized block-time if possible (#16460) 2021-04-09 20:25:47 +00:00
behzad nouri 22a18a68e3
stops consuming pinned vectors with a recycler (#16441)
If the vector is pinned and has a recycler, From<PinnedVec>
implementation of Vec should clone (instead of consuming) the underlying
vector so that the next allocation of a PinnedVec will recycle an
already pinned one.
2021-04-09 16:55:24 +00:00
François Garillot b08cff9e77
Simplify some pattern-matches (#16402)
When those match an exact combinator on Option / Result.

Tool-aided by [comby-rust](https://github.com/huitseeker/comby-rust).
2021-04-08 12:40:37 -06:00
Tyera Eulberg bb9d2fd07a
Cli: use get_inflation_rewards and limit epochs queried (#16408)
* Fix block-with-limit when not finalized blocks found

* Enable confirmed commitment in getInflationReward

* Use get_inflation_rewards in cli

* Line up rewards output

* Add range validator

* Change cli epoch arg -> num epochs

* Add solana inflation rewards subcommand

* Consolidate epoch rewards meta
2021-04-08 10:57:33 -06:00
Josh e501fa5f0b
Rpc: introduce get_inflation_reward rpc call (#16278)
* feat: introduce get_inflation_reward rpc call

* fix: style suggestions

* fix: more style changes and match how other rpc functions are defined

* feat: get reward for a single epoch

* feat: default to the most recent epoch

* fix: don't factor out get_confirmed_block

* style: introduce from impl for RpcEncodingConfigWrapper

* style: bring commitment into variable

* feat: support multiple pubkeys for get_inflation_reward

* feat: add get_inflation_reward to rpc client

* feat: return rewards in order

* fix: rename pubkeys to addresses

* docs: introduce jsonrpc docs for get_inflation_reward

* style: early return in map (not sure which is more idiomatic)

* fix: call the rpc client function args addresses as well

* fix: style

* fix: filter out only addresses we care about

* style: make this more idiomatic

* fix: change rpc client epoch to optional and include some docs edits

* feat: filter out rent rewards in get_inflation_reward

* feat: add option epoch config param to get_inflation_reward

* feat: rpc client get_inflation_reward takes epoch instead of config and some filter staking and voting rewards
2021-04-06 18:10:53 -07:00
carllin 1219842a96
No wallclock throttle tests (#16396) 2021-04-05 19:40:16 -07:00
Trent Nelson b71875df61 cluster-info: Get rid of some integer math while we're here 2021-04-06 00:09:37 +00:00
Trent Nelson b6b08706b9 cluster-info: Don't subtract non-shred spies from node count 2021-04-06 00:09:37 +00:00
Trent Nelson 7a2a39093d validator: Use a const for wait for supermajority threshold 2021-04-05 17:29:37 -06:00
Michael Vines 43feef7362 Adjust tokio version to just "1" 2021-04-05 17:13:20 +00:00
Michael Vines b242f82696 Reduce test-validator ledger size 2021-04-05 08:37:29 -07:00
behzad nouri 701fc93343
patches bug in banking stage where buffered packets are never retained (#16276)
banking_stage::handle_forwarding is retaining buffered packets with
empty index, so nothing is held:
https://github.com/solana-labs/solana/blob/6f3926b64/core/src/banking_stage.rs#L520
2021-04-05 12:46:21 +00:00
sakridge 3429785d9b
Wait for 90 percent of stake before starting (#16340) 2021-04-03 14:21:20 -07:00
carllin 4e5ef6bce2
Add cluster state verifier logging (#16330)
* Add cluster state verifier logging

* Add duplicate-slots iterator to ledger tool
2021-04-02 21:48:44 -07:00
Tyera Eulberg da27acabcc
Rpc: enable getConfirmedSignaturesForAddress2 to return confirmed (not yet finalized) data (#16281)
* Update blockstore method to allow return of unfinalized signature

* Support confirmed sigs in getConfirmedSignaturesForAddress2

* Add deprecated comments

* Update docs

* Enable confirmed transaction-history in cli

* Return real confirmation_status; fill in not-yet-finalized block time if possible
2021-04-01 04:35:57 +00:00
Tyera Eulberg 18bd47dbe1
Rpc: fix getConfirmedTransaction slot (#16288)
* Fix transaction blockstore apis

* Update blockstore apis in rpc
2021-03-31 21:04:00 -06:00
behzad nouri 3f63ed9a72
removes OrderedIterator and transaction batch iteration order (#16153)
In TransactionBatch,
https://github.com/solana-labs/solana/blob/e50f59844/runtime/src/transaction_batch.rs#L4-L11
lock_results[i] is aligned with transactions[iteration_order[i]]:
https://github.com/solana-labs/solana/blob/e50f59844/runtime/src/bank.rs#L2414-L2424
https://github.com/solana-labs/solana/blob/e50f59844/runtime/src/accounts.rs#L788-L817

However load_and_execute_transactions is iterating over
  lock_results[iteration_order[i]]
https://github.com/solana-labs/solana/blob/e50f59844/runtime/src/bank.rs#L2878-L2889
and then returning i as for the index of the retryable transaction.

If iteratorion_order is [1, 2, 0], and i is 0, then:
  lock_results[iteration_order[i]] = lock_results[1]
which corresponds to
  transactions[iteration_order[1]] = transactions[2]
so neither i = 0, nor iteration_order[i] = 1 gives the correct index for the
corresponding transaction (which is 2).

This commit removes OrderedIterator and transaction batch iteration order
entirely. There is only one place in blockstore processor which the
iteration order is not ordinal:
https://github.com/solana-labs/solana/blob/e50f59844/ledger/src/blockstore_processor.rs#L269-L271
It seems like, instead of using an iteration order, that can shuffle entry
transactions in-place.
2021-03-31 23:59:19 +00:00
sakridge 54c68ea83f
Drop write lock on sysvars (#15497)
* Drop write lock on sysvars

* adds env var for demoting sysvar write lock demotion

* moves demote logic to is_writable

* feature gates sysvar write lock demotion

* adds builtins to write lock demotion

* adds system program id to builtins

* adds Feature111...

* adds an abi-freeze test

* mvines set of builtin program keys

Co-authored-by: Michael Vines <mvines@gmail.com>

* update tests

* adds bpf loader keys

* Add test sysvar

* Plumb demote_sysvar to is_writable

* more plumbing of demote_sysvar_write_locks to is_writable

* patches test_program_bpf_instruction_introspection

* hard codes demote_sysvar_write_locks to false for serialization/encoding methods

* Revert "hard codes demote_sysvar_write_locks to false for serialization/encoding methods"

This reverts commit ae3e2d2e777437bddd753933097a210dcbc1b1fc.

* change the hardcoded ones to demote_sysvar_write_locks=true

* Use data_as_mut_slice

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
Co-authored-by: Michael Vines <mvines@gmail.com>
2021-03-30 10:05:09 -07:00
Jeff Washington (jwash) 414c7070cb
poll checking for new record in poh service after every batch of hashes instead of busy waiting (#16167)
* poll waiting in poh service after every batch of hashes

* clippy
2021-03-30 10:34:21 -05:00
Jeff Washington (jwash) 5eff23db0c
TransactionRecorder uses unique channel so we can use Recv instead of RecvTimeout (#16195)
* time

* new channel each call

* new channel every time
2021-03-30 00:51:35 -05:00
Tyera Eulberg 3977ed5c82
Future-aware enum name 2021-03-29 14:58:05 -06:00
Tyera Eulberg 60ed8e2892
Rpc: enable getConfirmedBlocks and getConfirmedBlocksWithLimit to return confirmed (not yet finalized) data (#16161)
* Add commitment config capabilities

* Use rpc limit if no end_slot provided

* Limit to actually finalized blocks

* Support confirmed blocks in getConfirmedBlocks and getConfirmedBlocksWithLimit

* Update docs

* Add client plumbing

* Rename config enum
2021-03-29 12:41:31 -06:00
sakridge 60b4771fc6
Only print skipped leader slot message when the node is actually leader (#16156)
Also, check vote signature after the vote is signed
2021-03-26 17:45:53 -07:00
Tyera Eulberg 433f1ead1c
Rpc: enable getConfirmedBlock and getConfirmedTransaction to return confirmed (not yet finalized) data (#16142)
* Add Blockstore block and tx apis that allow unrooted responses

* Add TransactionStatusMessage, and send on bank freeze; also refactor TransactionStatusSender

* Track highest slot with tx-status writes complete

* Rename and unpub fn

* Add commitment to GetConfirmed input configs

* Support confirmed blocks in getConfirmedBlock

* Support confirmed txs in getConfirmedTransaction

* Update sigs-for-addr2 comment

* Enable confirmed block in cli

* Enable confirmed transaction in cli

* Review comments

* Rename blockstore method
2021-03-26 16:47:35 -06:00
Jeff Washington (jwash) 4f4cffbd03
Throttle PoH ticks by cumulative slot time (#16139)
* Throttle PoH ticks by cumulative slot time

* respond to pr feedback

* saturating sub

* updated comment
2021-03-26 18:54:16 +00:00
Jeff Washington (jwash) 06ac0fe9a3
increase timeout in TransactionRecorder.record (#16133) 2021-03-25 21:31:07 -05:00
sakridge b99ae8f334
Skip leader slots until a vote lands (#15607) 2021-03-25 18:54:51 -07:00
behzad nouri b041b55028
makes test_pull_request_time_pruning smaller (#16128) 2021-03-25 22:44:43 +00:00
sakridge 9b94741290
Fix test_replay_commitment_cache (#16131) 2021-03-25 21:16:39 +00:00
Justin Starry e817a6db00
Add timeout for local cluster partition tests (#16123)
* Add timeout for local cluster partition tests

* fix optimistic conf test logs

* Bump instruction count assertions
2021-03-25 13:27:07 -06:00
carllin 52703badfa
Setup ReplayStage confirmation scaffolding for duplicate slots (#9698) 2021-03-24 23:41:52 -07:00
Tyera Eulberg a8ef29df27
Support getBlockTime for unfinalized blocks (#16103) 2021-03-24 20:52:08 -06:00
sakridge 96ccc40f0a
Set ticks_per_slot higher for banking stage tests (#16094)
Tests are timing out because the bank hit the MaxTickHeight and
will not process the transactions.
2021-03-24 14:05:43 -07:00
Jeff Washington (jwash) f68860a643
poh record metrics (#16092) 2021-03-24 14:48:32 -05:00
behzad nouri a6c23648cb
limits CrdsGossipPull::pull_request_time size (#15793)
There is no pruning logic on CrdsGossipPull::pull_request_time
https://github.com/solana-labs/solana/blob/79ac1997d/core/src/crds_gossip_pull.rs#L172-L174
potentially allowing this to take too much memory.

Additionally, CrdsGossipPush::last_pushed_to is pruning recent push
timestamps:
https://github.com/solana-labs/solana/blob/79ac1997d/core/src/crds_gossip_push.rs#L275-L279
instead of the older ones.

Co-authored-by: Nathan Hawkins <utsl@utsl.org>
2021-03-24 18:33:56 +00:00
behzad nouri 570fd3f810
makes turbine peer computation consistent between broadcast and retransmit (#14910)
get_broadcast_peers is using tvu_peers:
https://github.com/solana-labs/solana/blob/84e52b606/core/src/broadcast_stage.rs#L362-L370
which is potentially inconsistent with retransmit_peers:
https://github.com/solana-labs/solana/blob/84e52b606/core/src/cluster_info.rs#L1332-L1345

Also, the leader does not include its own contact-info when broadcasting
shreds:
https://github.com/solana-labs/solana/blob/84e52b606/core/src/cluster_info.rs#L1324
but on the retransmit side, slot leader is removed only _after_ neighbors and
children are computed:
https://github.com/solana-labs/solana/blob/84e52b606/core/src/retransmit_stage.rs#L383-L384
So the turbine broadcast tree is different between the two stages.

This commit:
* Removes retransmit_peers. Broadcast and retransmit stages will use tvu_peers
  consistently.
* Retransmit stage removes slot leader _before_ computing children and
  neighbors.
2021-03-24 13:34:48 +00:00
Justin Starry e7fd7d46cf
rpc: add getSlotLeaders method (#16057) 2021-03-23 17:48:54 +00:00
behzad nouri 4f82b897bc
buffers data shreds to make larger erasure coded sets (#15849)
Broadcast stage batches up to 8 entries:
https://github.com/solana-labs/solana/blob/79280b304/core/src/broadcast_stage/broadcast_utils.rs#L26-L29
which will be serialized into some number of shreds and chunked into FEC
sets of at most 32 shreds each:
https://github.com/solana-labs/solana/blob/79280b304/ledger/src/shred.rs#L576-L597
So depending on the size of entries, FEC sets can be small, which may
aggravate loss rate.
For example 16 FEC sets of 2:2 data/code shreds each have higher loss
rate than one 32:32 set.

This commit broadcasts data shreds immediately, but also buffers them
until it has a batch of 32 data shreds, at which point 32 coding shreds
are generated and broadcasted.
2021-03-23 14:52:38 +00:00
Jeff Washington (jwash) 57ba86c821
eliminate lock on record (#15929)
* eliminate lock on record

* use same error as MaxHeightReached

* clippy

* review feedback

* refactor should_tick code

* pr feedback
2021-03-23 09:10:04 -05:00
Jeff Washington (jwash) 2fc609a294
add metric for ticks from poh_recorder.record (#16047) 2021-03-22 15:35:06 -05:00
Tyera Eulberg 2ec24d438f
Make getStakeActivation response consistent for undelegated accounts (#16038) 2021-03-19 14:54:56 -06:00