behzad nouri
d1df9da7d3
fixes test_filter_current flakiness ( #14816 )
2021-01-25 15:57:46 +00:00
Ryo Onodera
bd0433c373
Clean up VerifiedVotePackets ( #14822 )
2021-01-25 09:01:47 +00:00
behzad nouri
e1021d9f83
removes redundant epoch stakes cache in retransmit ( #14781 )
...
Following d6d76219b
, staked nodes computed from vote accounts are
already cached in runtime::Stakes, so the caching in retransmit_stage is
redundant.
2021-01-24 21:15:09 +00:00
behzad nouri
491b059755
broadcasts duplicate shreds through gossip ( #14699 )
2021-01-24 15:47:43 +00:00
Michael Vines
cbffab7850
Upgrade to Rust v1.49.0
2021-01-23 19:16:36 -08:00
Michael Vines
bf1943e489
Add solana-test-validator --warp-slot argument
2021-01-22 21:17:02 -08:00
Michael Vines
cbb9ac19b9
Add ability to clone accounts from an RPC endpoint
2021-01-22 13:29:36 -08:00
Tyera Eulberg
71e9958e06
Rpc: Add custom error for BigTable data not found ( #14762 )
...
* Expose not-found bigtable error
* Add custom rpc error for bigtable data not found
* Return custom rpc error when bigtable block is not found
* Generalize long-term storage
2021-01-22 04:40:47 +00:00
behzad nouri
e4da6761a7
fixes test_filter_current flakiness ( #14749 )
2021-01-21 21:53:10 +00:00
behzad nouri
8e581601d6
patches crds vote-index assignment bug ( #14438 )
...
If tower is full, old votes are evicted from the front of the deque:
https://github.com/solana-labs/solana/blob/2074e407c/programs/vote/src/vote_state/mod.rs#L367-L373
whereas recent votes if expire are evicted from the back:
https://github.com/solana-labs/solana/blob/2074e407c/programs/vote/src/vote_state/mod.rs#L529-L537
As a result, from a single tower_index scalar, we cannot infer which crds-vote
should be overwritten:
https://github.com/solana-labs/solana/blob/2074e407c/core/src/crds_value.rs#L576
In addition there is an off by one bug in the existing code. tower_index is
bounded by MAX_LOCKOUT_HISTORY - 1:
https://github.com/solana-labs/solana/blob/2074e407c/core/src/consensus.rs#L382
So, it is at most 30, whereas MAX_VOTES is 32:
https://github.com/solana-labs/solana/blob/2074e407c/core/src/crds_value.rs#L29
Which means that this branch is never taken:
https://github.com/solana-labs/solana/blob/2074e407c/core/src/crds_value.rs#L590-L593
so crds table alwasys keeps 29 **oldest** votes by wallclock, and then
only overrides the 30st one each time. (i.e a tally of only two most
recent votes).
2021-01-21 13:08:07 +00:00
Michael Vines
04ce33a04e
Ensure sanitary transactions
2021-01-20 23:59:32 -08:00
Ryo Onodera
dcaa025822
Configure Bigtable's timeout, enabling by default ( #14657 )
...
* Configure bigtable's timeout when read-only
* Review comments
* Apply nits (thanks!)
Co-authored-by: Michael Vines <mvines@gmail.com>
* Timeout in the streamed decoding as well
Co-authored-by: Michael Vines <mvines@gmail.com>
2021-01-19 13:57:16 +00:00
behzad nouri
b5fd0ed859
rewrites turbine retransmit peers computation ( #14584 )
2021-01-19 04:18:47 +00:00
behzad nouri
c6ae0667e6
feature gates turbine retransmit peers patch ( #14631 )
2021-01-19 04:16:19 +00:00
Michael Vines
5d9dc609b1
Rename RpcNodeUnhealthy error to NodeUnhealthy, generalize `getHealth` RPC error object for the future
2021-01-18 19:57:25 -08:00
Tyera Eulberg
cbf8ef7480
Make Bigtable::get_confirmed_blocks inclusive of requested start_slot and end_slot ( #14651 )
...
* Fix off-by-one error
* Filter out blocks greater than end slot
2021-01-18 19:14:10 -07:00
carllin
5f14f45282
More generic accounts purge functions ( #14595 )
...
Co-authored-by: Carl Lin <carl@solana.com>
2021-01-17 20:31:03 -08:00
Ryo Onodera
8d4ab1bab1
Clean up and add comment
2021-01-17 19:59:59 -08:00
Ryo Onodera
5cf9094bb9
WIP fix the occasional stuck RPC request
2021-01-17 19:59:59 -08:00
Michael Vines
4003f86f04
Add `getSnapshotSlot` RPC method
2021-01-16 19:31:21 +00:00
Michael Vines
dacb95083d
Add `getHealth` RPC method
2021-01-16 10:10:48 -08:00
Tyera Eulberg
9a89689ad3
Use optimistic confirmation in getSignatureStatuses, and various downstream client methods ( #14430 )
...
* Add optimistically_confirmed field to TransactionStatus
* Update docs
* Convert new field to confirmation_status
* Update docs to confirmationStatus
* Update variants
* Update docs
* Just Confirmed
2021-01-15 16:05:05 +00:00
dependabot[bot]
4ecc0f4fa6
chore: bump rayon from 1.4.1 to 1.5.0 ( #14177 )
...
* chore: bump rayon from 1.4.1 to 1.5.0
Bumps [rayon](https://github.com/rayon-rs/rayon ) from 1.4.1 to 1.5.0.
- [Release notes](https://github.com/rayon-rs/rayon/releases )
- [Changelog](https://github.com/rayon-rs/rayon/blob/master/RELEASES.md )
- [Commits](https://github.com/rayon-rs/rayon/compare/rayon-core-v1.4.1...rayon-core-v1.5.0 )
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2021-01-15 18:07:35 +09:00
Michael Vines
1c2ae15b1d
Improve solana-test-validator output
2021-01-14 23:07:24 -08:00
sakridge
907f518f6d
Add load/execute/store timings ( #14561 )
2021-01-14 14:14:16 -08:00
behzad nouri
cfcca1cd3c
patches bug in turbine's neighbors computation ( #14565 )
...
Removing local node's index early from the set here:
https://github.com/solana-labs/solana/blob/e1b59ded4/core/src/retransmit_stage.rs#L346
distorts the order of nodes depending on which node is computing the
turbine fan-out tree, and results in incorrect neighbors computation.
2021-01-13 22:25:29 +00:00
Michael Vines
11daaadc93
Add --rpc-threads argument
2021-01-13 13:34:46 -08:00
Tyera Eulberg
e1b59ded4b
Use leader_forward_count for tx retries too ( #14547 )
2021-01-13 11:14:22 -07:00
Jeff Washington (jwash)
935dfdf0f6
fill in timing gaps in replay_stage ( #14550 )
...
* fill in timing gaps in replay_stage
* add replay_stage bank_count metric
* formatting
* handle another gap
* cleanup wait_receive_time to be more straightforward
2021-01-13 10:08:53 -06:00
carllin
6dfad0652f
Cache account stores, flush from AccountsBackgroundService ( #13140 )
2021-01-11 17:00:23 -08:00
Tyera Eulberg
b0e6e29527
Update timestamp max allowable drift to 50% of PoH ( #14531 )
...
* Repurpose warp-timestamp feature for general bump
* Change max_allowable_drift to 50%
* Fill in PR#
* Fix rpc test setup
2021-01-11 23:27:30 +00:00
Michael Vines
a95675a7ce
Avoid tmp snapshot backlog in SnapshotPackagerService under high load ( #14516 )
2021-01-11 10:21:15 -08:00
Michael Vines
7be6770808
Rename CompressionType to ArchiveFormat
2021-01-09 09:07:49 -08:00
Michael Vines
141e6706e6
Rename AccountsPackage::root to AccountsPackage::slot
2021-01-09 09:07:49 -08:00
behzad nouri
766195dded
limits number of crds values associated with a pubkey ( #14467 )
2021-01-08 18:54:40 +00:00
Jeff Washington (jwash)
773b21b34e
consolidate DEFAULT_HASHES_PER_TICK ( #14463 )
2021-01-07 09:49:42 -06:00
sakridge
c282586753
Add fixed tick rate adjustment ( #14447 )
2021-01-05 19:25:44 -08:00
Tyera Eulberg
ce1766d798
Enable program-id account index for supply calculations ( #14444 )
...
* Enable program-id account index for supply calculations
* Fixup comments
2021-01-06 02:42:33 +00:00
carllin
5affd8aa72
Add secondary indexes ( #14212 )
2020-12-31 18:06:03 -08:00
Michael Vines
4a3d217839
Only initialize BigTable upload service when requested
2020-12-31 17:40:51 -08:00
Michael Vines
0b23abd479
Bind RPC and faucet to 0.0.0.0
2020-12-31 07:55:26 +00:00
Michael Vines
3d077fb656
Revert "Upgrade in-tree tokio 0.2 usage to tokio 0.3"
...
This reverts commit 444ed768dc
.
2020-12-31 06:26:53 +00:00
Michael Vines
fb6c660cfd
Port ip-echo-server to tokio 0.3
2020-12-30 09:29:16 -08:00
Trent Nelson
fe667db910
validator: Add experimental flag to select PoH pinned core
2020-12-29 19:15:44 -07:00
Trent Nelson
82f61c0c4a
core: Update stale error message
2020-12-29 19:15:44 -07:00
sakridge
f8a4afc7c1
Fix flaky broadcast test ( #14329 )
2020-12-29 12:35:04 -08:00
sakridge
2074e407cd
Add poh speed check and tick speed calibration ( #14292 )
2020-12-29 09:35:57 -08:00
Michael Vines
444ed768dc
Upgrade in-tree tokio 0.2 usage to tokio 0.3
2020-12-29 09:23:23 -08:00
Tyera Eulberg
3f10fb993b
Retry durable-nonce transactions ( #14308 )
...
* Retry durable-nonce transactions
* Add metric to track durable-nonce txs in queue
* Populate send-tx-service initial addresses with tpu_address if empty (primarily for testing)
* Reinstate last_valid_slot check for durable-nonce txs; use arbitrary future slot
2020-12-29 09:48:43 -07:00
Michael Vines
9ddd6f08e8
Persist gossip contact info
2020-12-27 20:46:54 -08:00
sakridge
c693ffaa08
Fix subtraction overflow in metrics ( #14290 )
2020-12-27 16:26:22 -08:00
behzad nouri
2fd38d9912
indexes votes in crds table ( #14272 )
2020-12-27 13:31:05 +00:00
behzad nouri
49019c6613
obtains staked-nodes from the root-bank ( #14257 )
...
... as opposed to the working bank
2020-12-27 13:28:05 +00:00
Trent Nelson
5b903318b2
vote: Add helper for creating current-versioned states
2020-12-22 19:37:26 -07:00
Michael Vines
ace360ade2
Multiple entrypoint support
2020-12-22 18:35:31 -08:00
Michael Vines
3373082ffa
Update entrypoint contact info even when shred version adoption is not requested
2020-12-22 18:35:31 -08:00
behzad nouri
a14cfd660a
removes &Arc<Self> receivers ( #14234 )
2020-12-22 23:51:53 +00:00
sakridge
baa9602411
Add shrink paths ( #14238 )
2020-12-21 21:33:37 -08:00
behzad nouri
7b08cb1f0d
improves performance in replay-stage ( #14217 )
...
bank::vote_accounts returns a hash-map which is slow to iterate, but all uses
only require an iterator:
https://github.com/solana-labs/solana/blob/b3dc98856/runtime/src/bank.rs#L4300-L4306
Similarly, calculate_stake_weighted_timestamp takes a hash-map whereas it only
requires an iterator:
https://github.com/solana-labs/solana/blob/b3dc98856/sdk/src/stake_weighted_timestamp.rs#L21-L28
2020-12-21 19:18:19 +00:00
Michael Vines
57b03c5bc1
Restore Content-Length header for streaming snapshot download
2020-12-20 23:22:26 -08:00
Michael Vines
b3dc988564
Stream RPC snapshot downloads
2020-12-19 15:54:23 -08:00
Michael Vines
0090106f60
getBlockTime RPC method now falls back to BigTable in all cases
2020-12-18 20:51:49 +00:00
behzad nouri
691031fefd
limits number of crds values returned when responding to pull requests ( #13739 )
...
Crds values buffered when responding to pull-requests can be very large taking a lot of memory.
Added a limit for number of buffered crds values based on outbound data budget.
2020-12-18 18:45:12 +00:00
behzad nouri
6a3797e164
adds crds-value for broadcasting duplicate shreds through gossip ( #14133 )
...
In gossip, the header overhead we get from:
https://github.com/solana-labs/solana/blob/de9ac43eb/core/src/cluster_info.rs#L434-L435
https://github.com/solana-labs/solana/blob/de9ac43eb/core/src/crds_value.rs#L31-L36
https://github.com/solana-labs/solana/blob/de9ac43eb/core/src/crds_value.rs#L73
already exceeds SIZE_OF_NONCE in shreds. We also need aditional
meta-data (wallclock, source pubkey, ...). Which means that given the
SHRED_PAYLOAD_SIZE, we cannot fit all these in PACKET_DATA_SIZE:
https://github.com/solana-labs/solana/blob/de9ac43eb/ledger/src/shred.rs#L80
On top of that, we need 2 shred payloads as the proof of duplicate. So
each DuplicateShred crds value includes only a chunk of the payload,
along with the meta-data to reconstruct the full payload from the chunks
on the receiving end.
2020-12-18 14:32:43 +00:00
sakridge
a5db6399ad
Fix test_max_hashes ( #14189 )
2020-12-17 15:12:18 -08:00
sakridge
da7d1e2302
Improved Transaction Forwarding ( #13944 )
...
* Forwarding
* Dedupe leaders
* Use consistent commitment for last_valid_slot in rpc send_transaction
* Plumb rpc send_transaction options into solana-validator
* Extend num slots banking-stage holds forwarded txs
Co-authored-by: Tyera Eulberg <tyera@solana.com>
2020-12-17 15:37:22 -07:00
behzad nouri
d6d76219b6
caches staked nodes computed from vote-accounts ( #13929 )
2020-12-17 21:22:50 +00:00
Michael Vines
0b92720fdb
Don't require increased open file limit in solana-test-validator
...
Travis CI in particular does not allow the open file limit to be
increased.
2020-12-16 20:43:08 -08:00
Trent Nelson
6875113d82
Bump SPL crates
2020-12-17 01:54:22 +00:00
Tyera Eulberg
ac0d32bc7e
Add blockstore skipped api ( #14145 )
...
* Add blockstore api to determine if a slot was skipped
* Return custom rpc error if slot is skipped
2020-12-16 13:40:36 -07:00
sakridge
d4a174fb7c
Partial shred deserialize cleanup and shred type differentiation ( #14094 )
...
* Partial shred deserialize cleanup and shred type differentiation in retransmit
* consolidate packet hashing logic
2020-12-15 16:50:40 -08:00
carllin
75e9e321de
Fix race between setting tick height and calculating accounts hash ( #14101 )
...
Co-authored-by: Carl Lin <carl@solana.com>
2020-12-15 12:45:40 -08:00
Michael Vines
1e977ac025
Bump version to v1.6.0
2020-12-15 18:28:04 +00:00
behzad nouri
c2b7115031
indexes crds values associated with a pubkey ( #14088 )
...
record_labels returns all the possible labels for a record identified by
a pubkey, used in updating timestamp of crds values:
https://github.com/solana-labs/solana/blob/1792100e2/core/src/crds_value.rs#L560-L577
https://github.com/solana-labs/solana/blob/1792100e2/core/src/crds.rs#L240-L251
The code relies on CrdsValueLabel to be limited to a small deterministic
set of possible values for a fixed pubkey. As we expand crds values to
include duplicate shreds, this limits what the duplicate proofs can be
keyed by in the table.
In addition the computation of these labels is inefficient and will
become more so as duplicate shreds and more types of crds values are
added. An alternative is to maintain an index of all crds values
associated with a pubkey.
2020-12-15 01:49:22 +00:00
sakridge
5294f70189
Remove some non-warnings ( #14115 )
2020-12-14 16:26:17 -08:00
Michael Vines
7143aaa89b
Clippy
2020-12-14 08:03:29 -08:00
Michael Vines
cdd3e7d856
Remove solana-vote-signer ( #14099 )
2020-12-13 19:12:20 -08:00
Michael Vines
9f2d154588
Rework TestValidator API to be more like ProgramTest
2020-12-13 07:45:55 +00:00
dependabot[bot]
88d950e5a8
chore: bump log from 0.4.8 to 0.4.11 ( #13691 )
...
* chore: bump log from 0.4.8 to 0.4.11
Bumps [log](https://github.com/rust-lang/log ) from 0.4.8 to 0.4.11.
- [Release notes](https://github.com/rust-lang/log/releases )
- [Changelog](https://github.com/rust-lang/log/blob/master/CHANGELOG.md )
- [Commits](https://github.com/rust-lang/log/compare/0.4.8...0.4.11 )
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2020-12-13 13:48:23 +09:00
carllin
55fc963595
Move slot cleanup to AccountsBackgroundService ( #13911 )
...
* Move bank drop to AccountsBackgroundService
* Send to ABS on drop instead, protects against other places banks are dropped
* Fix Abi
* test
Co-authored-by: Carl Lin <carl@solana.com>
2020-12-13 01:22:34 +00:00
Tyera Eulberg
8541ffa328
Add subscription cap ( #14011 )
...
* Add subscription cap
* Elide unwraps
2020-12-12 01:57:40 +00:00
Trent Nelson
546c6a4531
Override yanked miow crate to 0.2.2
2020-12-11 23:44:03 +00:00
Trent Nelson
277b6790c7
Override yanked net2 0.2.37
2020-12-11 23:44:03 +00:00
Ryo Onodera
09bd412b13
Wrong rpc inflation rate ( #14063 )
...
* Fix wrong inflation calculation in rpc
* Reorder
* Fix test
2020-12-11 07:42:39 +00:00
Michael Vines
4fba7e6865
Move validator_block_0_fee_workaround into TestValidator
2020-12-11 04:17:38 +00:00
Michael Vines
bbad3fe501
TestValidator now implements Drop, no need to close() it
2020-12-11 04:17:38 +00:00
Michael Vines
0a9ff1dc9d
Initial solana-test-validator command-line program
2020-12-11 04:17:38 +00:00
Josh
13db3eca9f
SPL token balance in transaction metadata ( #13673 )
...
* feat: store pre / post token balances
* move helper functions into separate include
* move token balance functionality to transaction-status crate
* fix blockstore processor test
* fix bigtable legacy test
* add caching to decimals
2020-12-10 19:25:07 -08:00
sakridge
aa2751e614
Check shred type in is_duplicate ( #14050 )
2020-12-10 18:20:08 -08:00
behzad nouri
409fe3bca1
adds the instance token to crds-labels for node-instance crds-values ( #14037 )
...
If a node "a" receives instance-info from node "b1" it will override any
instance-info associated with "b1" pubkey in its crds table. This makes
it less likely that when "b1" receives crds values from "a" (either
through pull or push), it sees other instances of itself (because node
"a" discarded them when it received "b1" instance info).
In order for the crds table to contain all instance-info associated with
the same pubkey at the same time, we need to add the instance tokens to
the keys in the crds table (i.e. the CrdsValueLabel).
2020-12-10 17:01:55 +00:00
sakridge
5c95d8e963
Shred filter ( #14030 )
2020-12-10 07:54:15 -08:00
sakridge
c5fe076432
Better dupe detection ( #13992 )
2020-12-09 23:14:31 -08:00
behzad nouri
1d267eae6b
std::process::exit to kill all threads
2020-12-09 10:24:23 -08:00
behzad nouri
895d7d6a65
removes RwLock on ClusterInfo.instance
2020-12-09 10:24:23 -08:00
behzad nouri
542198180a
pushes node-instance along with version early in gossip
2020-12-09 10:24:23 -08:00
behzad nouri
8cd5eb9863
checks for duplicate validator instances using gossip
2020-12-09 10:24:23 -08:00
sakridge
f6600810d7
Use LRU cache and blake3 hash of shreds to filter duplicates ( #13976 )
2020-12-07 16:42:39 -08:00
Michael Vines
6e9dbb4f6e
Add --rpc-max-multiple-accounts to override the getMultipleAccounts JSON RPC maximum
2020-12-07 16:31:01 -08:00
carllin
239a191612
Remove unneeded BankWeight fork choice ( #13978 )
...
Co-authored-by: Carl Lin <carl@solana.com>
2020-12-07 13:47:14 -08:00
Tyera Eulberg
6ae4d2e5cb
Fix logsSubscribe ( #13996 )
2020-12-07 19:00:52 +00:00