Commit Graph

2232 Commits

Author SHA1 Message Date
Jeff Washington (jwash) ddc758439e
metrics for poh_recorder.record (#15998) 2021-03-19 09:48:55 -05:00
Michael Vines 02b81dd05d Update to reqwest 0.11.2 2021-03-18 11:59:41 -07:00
Tyera Eulberg aa54c468ea
rpc: Add config options limiting getConfirmedBlock response data (#15970)
* Add new confirmed block struct

* Add RpcConfirmedBlockConfig options

* Configure block response based on new options

* Add client api, use in cli fetch_epoch_rewards

* Update docs

* Apply review suggestions
2021-03-18 17:58:20 +00:00
Michael Vines 04c99cf7ea Add --slots-per-epoch argument 2021-03-17 22:56:41 +00:00
carllin f548a04fae
Allow unbounded wallclock processing time in tests (#15961) 2021-03-17 15:48:50 -07:00
Michael Vines 59c19d9fbf Notice the user when the --mint, --bpf-program, or --clone arguments are ignored 2021-03-17 20:04:53 +00:00
Michael Vines 8a9b51952e Ignore flaky test_banking_stage_entries_only and test_banking_stage_entryfication 2021-03-17 11:28:56 -07:00
Jeff Washington (jwash) 40997d0aef
add metrics for tick producer and poh_recorder (#15931) 2021-03-17 10:38:26 -05:00
Jeff Washington (jwash) 5460fb10a2
drop poh lock after record (#15930) 2021-03-17 10:37:20 -05:00
Michael Vines a2eb655322 =1.7.0 2021-03-16 07:51:07 +00:00
Jeff Washington (jwash) efee8b62d7
a few missed set_data calls (#15846)
* a few missed set_data calls

* another set data call
2021-03-15 21:57:23 -05:00
Jeff Washington (jwash) c09ea2c314
More AccountSharedData construction (#15844)
* one more AccountSharedData construction

* one more construct
2021-03-15 19:27:17 -05:00
carllin c1ba265dd9
Wallclock BankingStage Throttle (#15731) 2021-03-15 17:11:15 -07:00
Michael Vines 0c9ca5522c Bump version to v1.7.0 2021-03-13 09:01:21 +00:00
Tyera Eulberg 5b2da19c93
Rpc: support extended config for getConfirmedBlock (#15827)
* Add rpc confirmed-block config wrapper to support struct of extended config

* Update docs

* Make config wrapper generic and use in getConfirmedTransaction as well

* Update/clean confirmed-tx docs
2021-03-12 22:19:45 +00:00
behzad nouri f2865dfd63
requires stakes for propagating crds values through gossip (#15561) 2021-03-12 15:50:14 +00:00
Justin Starry 918d04e3f0
Add more slot update notifications (#15734)
* Add more slot update notifications

* fix merge

* Address feedback and add integration test

* switch to datapoint

* remove unused shred method

* fix clippy

* new thread for rpc completed slots

* remove extra constant

* fixes

* rely on channel closing

* fix check
2021-03-12 21:44:06 +08:00
Ryo Onodera 4bbeb9c033
Remove old feature: simple_capitalization (#15763)
* Remove old feature: simple_capitalization

* Fix another failing test in core

* Finish up test cleanup

* Further clean up a bit
2021-03-12 11:12:40 +09:00
Jeff Washington (jwash) 952c3bcbb7
AccountSharedData construction (#15790) 2021-03-11 18:09:04 -06:00
Trent Nelson 24d18b3cf2 docs: add docs links for crates published to crates.io 2021-03-11 08:38:18 +00:00
Jeff Washington (jwash) 1135ffd595
mut data refs as slice (#15782) 2021-03-10 15:28:03 -06:00
behzad nouri 56923c91bf
limits number of unique pubkeys in the crds table (#15539) 2021-03-10 20:46:05 +00:00
Jeff Washington (jwash) 52e54e1100
account.data -> data() (#15778) 2021-03-09 22:31:33 +00:00
Jeff Washington (jwash) 8a3135d17b
Account->AccountSharedData (#15691) 2021-03-09 15:06:07 -06:00
carllin 2bee9435f3
Add tracer key for tracing transaction path through the network (#15732) 2021-03-08 19:31:00 -08:00
carllin 331c45decf
Report datapoint on number of retransmit shreds (#15694) 2021-03-08 17:54:53 -08:00
sakridge d09112fa6d
PoH batch size calibration (#15717) 2021-03-05 16:01:21 -08:00
Michael Vines 4a3ab77baf Remove unused id field 2021-03-05 19:07:59 +00:00
Michael Vines 66b781eec3 Add 'unknown' health check state 2021-03-05 17:46:50 +00:00
Tyera Eulberg 7e65289729
Convert blockstore TransactionStatus column family to protobufs (#15733)
* Prevent panic if TransactionStatus can't be deserialized

* Convert Blockstore TransactionStatus column to protobuf

* Add compatability test
2021-03-05 09:05:35 -07:00
Michael Vines bd13262b42 Add validator startup process reporting before RPC is available 2021-03-05 08:03:36 -08:00
Michael Vines 24ab84936e Break up RPC API into three categories: minimal, full and admin 2021-03-04 16:39:44 -08:00
Jeff Washington (jwash) 34bebb7d09
report execution details in replay time (#15693) 2021-03-04 11:38:12 -06:00
Jeff Washington (jwash) be35c1c1b7
add execute detail timings (#15638) 2021-03-03 17:07:45 -06:00
behzad nouri 658951e680
sends only the latest vote of each validator to the banking stage (#15629) 2021-03-03 19:07:16 +00:00
carllin aacb28c453
Only report metrics every second (#15652) 2021-03-03 10:58:47 -08:00
sakridge 830be855dc
Forward and hold packets (#15634) 2021-03-03 10:23:05 -08:00
Tyera Eulberg 19ac79b5cc
Deprecate UiTokenAmount::ui_amount (#15616)
* Add TokenAmount::ui_amount_string

* Fixup solana-tokens

* Update docs
2021-03-02 22:51:41 -07:00
Tyera Eulberg a4f0033bd7
Remove ValidatorConfig derive Clone, and fix local-cluster tests (#15647)
* Remove ValidatorConfig derive Clone

* Add local-cluster ValidatorConfig helpers

* Fix benches
2021-03-03 04:21:30 +00:00
behzad nouri 0bd0084b0d
adds more metrics for tx counts and batch sizes (#15642) 2021-03-03 01:28:15 +00:00
behzad nouri 416ea38028
adds metrics for the size and number of batches in bank_send_loop (#15627) 2021-03-02 15:44:35 +00:00
Greg Fitzgerald 2463cc1e6a
Fix typos (#15610) 2021-03-02 06:36:49 -08:00
Michael Vines 640e36287e Move ValidatorExit into ValidatorConfig, making it accessible from the solana-validator crate 2021-03-01 16:49:56 -08:00
sakridge f1223fb783
Lower blockstore processor error severity (#15578) 2021-03-01 14:57:37 -08:00
Trent Nelson ef63ffce7d Bump ed25519-dalek to 1.0.1 2021-03-01 21:10:46 +00:00
carllin ae96ba3459
Plumb slot update pubsub notifications (#15488) 2021-02-28 23:29:11 -08:00
behzad nouri f7a049f87f
coalesces vote packets into one Packets (#15566) 2021-02-26 23:23:08 +00:00
sakridge 05409e51ce
Increase tpu coalescing and add parameter (#15536)
Should create larger entries on average
2021-02-26 09:15:45 -08:00
behzad nouri 5a9896706c
indexes epoch slots in crds table (#15459)
ClusterInfo::get_epoch_slots_since scans the entire crds table to obtain
epoch-slots inserted since a timestamp:
https://github.com/solana-labs/solana/blob/013daa8f4/core/src/cluster_info.rs#L1245-L1262
The alternative is to index epoch-slots in crds table ordered by their
insert timestamp.
2021-02-26 14:12:04 +00:00
Tyera Eulberg 1ad2c9f741
Revert "Make UiTokenAmount::ui_amount a String (#15447)" (#15542)
This reverts commit d14374bc9f.
2021-02-25 21:53:40 +00:00
Michael Vines 5b54aed1c0 Speed up getLeaderSchedule 2021-02-24 11:17:25 -08:00
Justin Starry 61ed980ac0
Fix received notifications for gossip signature subscriptions (#15506) 2021-02-24 16:59:22 +08:00
carllin c2e8814dce
Add limit and shrink policy for recycler (#15320) 2021-02-24 00:15:58 -08:00
Tyera Eulberg 52f2d425e5
Count if optimistically confirmed slot is already rooted (#15492) 2021-02-23 22:03:22 +00:00
sakridge 1b59b163dd
Add max retransmit and shred insert slot (#15475) 2021-02-23 13:06:33 -08:00
Michael Vines 4b0114b991 Limit the number of getProgramAccounts filters 2021-02-23 18:43:22 +00:00
Tyera Eulberg d14374bc9f
Make UiTokenAmount::ui_amount a String (#15447)
* Make UiTokenAmount::ui_amount a String

* Fixup solana-tokens

* Ignore spl downstream-project
2021-02-22 13:05:45 -07:00
Ryo Onodera 5ccaa6336a
Print original error from accounts dir remove (#15458) 2021-02-22 21:24:09 +09:00
Ivan Mironov 013daa8f47 RPC: Improve snapshot path sanitization 2021-02-20 13:06:07 -08:00
Michael Vines 5df36aec7d Pacify clippy 2021-02-19 20:08:41 -08:00
Michael Vines fd3b71a2c6 cargo fmt 2021-02-19 20:08:41 -08:00
behzad nouri aa3aac766f
adds metrics for inbound/outbound gossip packets counts (#15407) 2021-02-19 22:49:35 +00:00
Justin Starry 15bbe6436d
Bump SPL token version to v3.1.0 (#15429)
* Bump SPL token version to v3.1.0

* Cargo.lock
2021-02-19 09:15:41 +00:00
Tyera Eulberg 170cb792eb
Return blockstore error if previous_blockhash cannot be determined (#15382)
* Return blockstore error if previous_blockhash cannot be determined

* Add require_previous_blockshash flag
2021-02-18 01:04:52 +00:00
Trent Nelson 7f7370c306 Re-allow clippy::integer_arithmetic at crate-level 2021-02-17 13:55:08 -07:00
Michael Vines 9ba69a7381 Adapt to fs_extra 1.2.0 2021-02-17 12:44:58 -08:00
sakridge b24cb9840e
Speedup ledger cleanup test (#15304)
Just clone to produce shreds and use a separate insert thread.
2021-02-17 08:59:25 -08:00
Tyera Eulberg 98e3e570d2 Add --force arg for bigtable upload 2021-02-16 17:24:07 -08:00
Jeff Washington (jwash) ba02452d75
add validator flag no-accounts-db-index-hashing (#15350)
* add validator flag no_accounts_db_index_hashing

* add validator flag no_accounts_db_index_hashing
2021-02-16 21:13:48 +00:00
behzad nouri f79c9d4094
adds an upper bound on cluster-slots size (#15300)
https://github.com/solana-labs/solana/issues/14366#issuecomment-769096305
2021-02-16 21:12:13 +00:00
behzad nouri 076c20f1ca
checks that prune-messages have the same inner/outer pubkey (#15352) 2021-02-16 21:06:18 +00:00
Trent Nelson b3256aa722 Bump rand_core to 0.6.2
https://rustsec.org/advisories/RUSTSEC-2021-0023
2021-02-15 22:51:35 +00:00
sakridge 5b8f046c67
More configurable rocksdb compaction (#15213)
rocksdb compaction can cause long stalls, so
make it more configurable to try and reduce those stalls
and also to coordinate between multiple nodes to not induce
stall at the same time.
2021-02-14 10:16:30 -08:00
carllin 629dcd0f39
Cleanup buffered packets (#15210) 2021-02-12 03:27:37 -08:00
carllin 990bb426a9
Fix flaky test test_concurrent_snapshot_packaging (#15252) 2021-02-11 16:03:51 -08:00
Josh 4013f91dbe
RPC: add caching to getLargestAccounts (#15154)
* introduce get largest accounts cache

* remove cache size and change hash key

* remove eq and hash derivation from commitment config

* add slot to the cache
2021-02-11 11:32:46 -08:00
Tyera Eulberg 948819dfa8
Use spl-token-mint secondary index for relevant getProgramAccounts requests (#15219) 2021-02-09 15:49:13 -07:00
Tyera Eulberg da6753b8c0
Warp timestamp and extend max-allowable-drift for accommodate slow blocks (#15204)
* Remove timestamp_correction feature gating

* Remove timestamp_bounding feature gating

* Remove unused deprecated ledger code

* Remove unused deprecated unbounded-timestamp code

* Enable independent adjustment of fast/slow timestamp bounding

* Update timestamp bounds to 25% fast, 80% slow; warp timestamp

* Update bank hash test

* Add PR number to feature

Co-authored-by: Michael Vines <mvines@gmail.com>

Co-authored-by: Michael Vines <mvines@gmail.com>
2021-02-09 15:49:00 -07:00
behzad nouri 2758588ddd
uses btree-map instead of hash-map for cluster-slots (#15194)
retain traverses all values in the hashmap which is slow:
https://github.com/solana-labs/solana/blob/88f22c360/core/src/cluster_slots.rs#L45
btree-map instead allows more efficient prunning there.

In addition there is potential race condition here:
https://github.com/solana-labs/solana/blob/88f22c360/core/src/cluster_slots.rs#L68-L74
If another thread inserts a value at the same slot key between the read
and write lock, current thread will discard the inserted value.
2021-02-09 22:04:41 +00:00
Michael Vines f063f02c41 solana-test-validator now uses the BPF JIT by default, `--no-bpf-jit` to disable 2021-02-09 20:24:14 +00:00
Michael Vines 2caed6bd4c Complete --rpc-pubsub-max-in-buffer-capacity/--rpc-pubsub-max-out-buffer-capacity plumbing 2021-02-09 07:24:30 +00:00
behzad nouri b6f231b60e
removes locked pubkey references (#15152) 2021-02-08 02:07:00 +00:00
sakridge bbae23358c
ledger-tool cleanup and additions (#15179)
* Plumb allow-dead-slots to ledger-tool verify

* ledger-tool cleanup and add some useful missing args

Print root slots and how many unrooted past last root.
2021-02-06 17:26:42 -08:00
Jeff Washington (jwash) fbf9dc47e9
Threadpool2 (#15151)
* rework thread pool for hash calculation

* rename
2021-02-05 18:48:16 -06:00
Jeff Washington (jwash) fabecdc86c
use thread pool for non-index hash calculations (#15149) 2021-02-05 19:48:55 +00:00
behzad nouri 6fd5ec0e4c
caches descendants in bank forks (#15107) 2021-02-05 18:00:45 +00:00
Ryo Onodera c69027e5aa
chore: bump serde from 1.0.118 to 1.0.122 (#15126)
* chore: bump serde from 1.0.118 to 1.0.122

* Update programs/bpf/Cargo.lock
2021-02-05 19:23:24 +09:00
Tyera Eulberg d1563f0ccd
Bump tonic, prost, tarpc, tokio (#15013)
* Update tonic & prost, and regenerate proto

* Reignore doc code

* Revert pull #14367, but pin tokio to v0.2 for jsonrpc

* Bump backoff and goauth -> and therefore tokio

* Bump tokio in faucet, net-utils

* Bump remaining tokio, plus tarpc
2021-02-05 00:21:53 -07:00
Jeff Washington (jwash) 600ff0d915
calculate hash from store instead of index (#15034)
* calculate hash from store instead of index

* restore update hash in abs
2021-02-04 09:00:33 -06:00
behzad nouri 86467d825a
removes pubkey references (#15050) 2021-02-03 23:02:11 +00:00
behzad nouri 0ad063f4e9
adds flag to disable duplicate instance check (#15006) 2021-02-03 16:26:17 +00:00
Tyera Eulberg 98aa1fa4ea
Upgrade jsonrpc crates to v17.0.0 (#15018)
* Upgrade to jsonrpc 17.0.0

* Fix test

* tree

Co-authored-by: Michael Vines <mvines@gmail.com>
2021-02-02 19:53:08 -07:00
dependabot[bot] 1df93fa2be
chore: bump serde from 1.0.112 to 1.0.118 (#14828)
* chore: bump serde from 1.0.112 to 1.0.122

Bumps [serde](https://github.com/serde-rs/serde) from 1.0.112 to 1.0.122.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.112...v1.0.122)

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

* Update frozen_abi digest following serde update

* Revert "chore: bump serde from 1.0.112 to 1.0.122"

This reverts commit a3ef4442a4c985144ae2bd7ceaf8899a7ab8d7c0.

* Revert "[auto-commit] Update all Cargo lock files"

This reverts commit c41c3b005fb1ccade55155302c52cd5736c4b55f.

* chore: bump serde from 1.0.112 to 1.0.118

Bumps [serde](https://github.com/serde-rs/serde) from 1.0.112 to 1.0.118.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.112...v1.0.118)

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

* Remove serum-dex pinning

* blind commit!

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
Co-authored-by: Ryo Onodera <ryoqun@gmail.com>
2021-02-02 23:28:16 +09:00
Tyera Eulberg cbb8b79a60
Add validator flag to opt in to cpi and logs storage (#14922)
* Add validator flag to opt in to cpi and logs storage

* Default TestValidator to opt-in; allow using in multinode-demo

* No clone

Co-authored-by: Carl Lin <carl@solana.com>
2021-02-01 14:00:51 -07:00
Tom Parker-Shemilt 01230a0105
Remove serial_test_derive dependency (#14891) 2021-01-28 22:35:31 -07:00
Ryo Onodera d6873b82ab
Remove potentially too costly Packets::default() (#14821)
* Remove potentially too costly Packets::default()

* Fix test...

* Restore Packets::default()

* Restore Packets::default() more
2021-01-29 09:32:38 +09:00
Michael Vines 119e2c75dd cli now supports a custodian for stake authorize operations 2021-01-26 11:48:28 -08:00
Tyera Eulberg ffa5c7dcc8
Deprecate commitment variants (#14797)
* Deprecate commitment variants

* Add new CommitmentConfig builders

* Add helpers to avoid allowing deprecated variants

* Remove deprecated transaction-status code

* Include new commitment variants in runtime commitment; allow deprecated as long as old variants persist

* Remove deprecated banks code

* Remove deprecated variants in core; allow deprecated in rpc/rpc-subscriptions for now

* Heavier hand with rpc/rpc-subscription commitment

* Remove deprecated variants from local-cluster

* Remove deprecated variants from various tools

* Remove deprecated variants from validator

* Update docs

* Remove deprecated client code

* Add new variants to cli; remove deprecated variants as possible

* Don't send new commitment variants to old clusters

* Retain deprecated method in test_validator_saves_tower

* Fix clippy matches! suggestion for BPF solana-sdk legacy compile test

* Refactor node version check to handle commitment variants and transaction encoding

* Hide deprecated variants from cli help

* Add cli App comments
2021-01-26 19:23:07 +00:00
behzad nouri d1df9da7d3
fixes test_filter_current flakiness (#14816) 2021-01-25 15:57:46 +00:00
Ryo Onodera bd0433c373
Clean up VerifiedVotePackets (#14822) 2021-01-25 09:01:47 +00:00