blockworks-foundation
/
solana
mirror of https://github.com/blockworks-foundation/solana.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
13,416 Commits 181 Branches 498 Tags 574 MiB
Commit Graph

2173 Commits

Author SHA1 Message Date
Michael Vines 5df36aec7d Pacify clippy 2021-02-19 20:08:41 -08:00
Michael Vines fd3b71a2c6 cargo fmt 2021-02-19 20:08:41 -08:00
behzad nouri aa3aac766f
adds metrics for inbound/outbound gossip packets counts (#15407) 2021-02-19 22:49:35 +00:00
Justin Starry 15bbe6436d
Bump SPL token version to v3.1.0 (#15429) 
* Bump SPL token version to v3.1.0

* Cargo.lock
 2021-02-19 09:15:41 +00:00
Tyera Eulberg 170cb792eb
Return blockstore error if previous_blockhash cannot be determined (#15382) 
* Return blockstore error if previous_blockhash cannot be determined

* Add require_previous_blockshash flag
 2021-02-18 01:04:52 +00:00
Trent Nelson 7f7370c306 Re-allow clippy::integer_arithmetic at crate-level 2021-02-17 13:55:08 -07:00
Michael Vines 9ba69a7381 Adapt to fs_extra 1.2.0 2021-02-17 12:44:58 -08:00
sakridge b24cb9840e
Speedup ledger cleanup test (#15304) 
Just clone to produce shreds and use a separate insert thread.
 2021-02-17 08:59:25 -08:00
Tyera Eulberg 98e3e570d2 Add --force arg for bigtable upload 2021-02-16 17:24:07 -08:00
Jeff Washington (jwash) ba02452d75
add validator flag no-accounts-db-index-hashing (#15350) 
* add validator flag no_accounts_db_index_hashing

* add validator flag no_accounts_db_index_hashing
 2021-02-16 21:13:48 +00:00
behzad nouri f79c9d4094
adds an upper bound on cluster-slots size (#15300) 
https://github.com/solana-labs/solana/issues/14366#issuecomment-769096305
 2021-02-16 21:12:13 +00:00
behzad nouri 076c20f1ca
checks that prune-messages have the same inner/outer pubkey (#15352) 2021-02-16 21:06:18 +00:00
Trent Nelson b3256aa722 Bump rand_core to 0.6.2 
https://rustsec.org/advisories/RUSTSEC-2021-0023
 2021-02-15 22:51:35 +00:00
sakridge 5b8f046c67
More configurable rocksdb compaction (#15213) 
rocksdb compaction can cause long stalls, so
make it more configurable to try and reduce those stalls
and also to coordinate between multiple nodes to not induce
stall at the same time.
 2021-02-14 10:16:30 -08:00
carllin 629dcd0f39
Cleanup buffered packets (#15210) 2021-02-12 03:27:37 -08:00
carllin 990bb426a9
Fix flaky test test_concurrent_snapshot_packaging (#15252) 2021-02-11 16:03:51 -08:00
Josh 4013f91dbe
RPC: add caching to getLargestAccounts (#15154) 
* introduce get largest accounts cache

* remove cache size and change hash key

* remove eq and hash derivation from commitment config

* add slot to the cache
 2021-02-11 11:32:46 -08:00
Tyera Eulberg 948819dfa8
Use spl-token-mint secondary index for relevant getProgramAccounts requests (#15219) 2021-02-09 15:49:13 -07:00
Tyera Eulberg da6753b8c0
Warp timestamp and extend max-allowable-drift for accommodate slow blocks (#15204) 
* Remove timestamp_correction feature gating

* Remove timestamp_bounding feature gating

* Remove unused deprecated ledger code

* Remove unused deprecated unbounded-timestamp code

* Enable independent adjustment of fast/slow timestamp bounding

* Update timestamp bounds to 25% fast, 80% slow; warp timestamp

* Update bank hash test

* Add PR number to feature

Co-authored-by: Michael Vines <mvines@gmail.com>

Co-authored-by: Michael Vines <mvines@gmail.com>
 2021-02-09 15:49:00 -07:00
behzad nouri 2758588ddd
uses btree-map instead of hash-map for cluster-slots (#15194) 
retain traverses all values in the hashmap which is slow:
https://github.com/solana-labs/solana/blob/88f22c360/core/src/cluster_slots.rs#L45
btree-map instead allows more efficient prunning there.

In addition there is potential race condition here:
https://github.com/solana-labs/solana/blob/88f22c360/core/src/cluster_slots.rs#L68-L74
If another thread inserts a value at the same slot key between the read
and write lock, current thread will discard the inserted value.
 2021-02-09 22:04:41 +00:00
Michael Vines f063f02c41 solana-test-validator now uses the BPF JIT by default, `--no-bpf-jit` to disable 2021-02-09 20:24:14 +00:00
Michael Vines 2caed6bd4c Complete --rpc-pubsub-max-in-buffer-capacity/--rpc-pubsub-max-out-buffer-capacity plumbing 2021-02-09 07:24:30 +00:00
behzad nouri b6f231b60e
removes locked pubkey references (#15152) 2021-02-08 02:07:00 +00:00
sakridge bbae23358c
ledger-tool cleanup and additions (#15179) 
* Plumb allow-dead-slots to ledger-tool verify

* ledger-tool cleanup and add some useful missing args

Print root slots and how many unrooted past last root.
 2021-02-06 17:26:42 -08:00
Jeff Washington (jwash) fbf9dc47e9
Threadpool2 (#15151) 
* rework thread pool for hash calculation

* rename
 2021-02-05 18:48:16 -06:00
Jeff Washington (jwash) fabecdc86c
use thread pool for non-index hash calculations (#15149) 2021-02-05 19:48:55 +00:00
behzad nouri 6fd5ec0e4c
caches descendants in bank forks (#15107) 2021-02-05 18:00:45 +00:00
Ryo Onodera c69027e5aa
chore: bump serde from 1.0.118 to 1.0.122 (#15126) 
* chore: bump serde from 1.0.118 to 1.0.122

* Update programs/bpf/Cargo.lock
 2021-02-05 19:23:24 +09:00
Tyera Eulberg d1563f0ccd
Bump tonic, prost, tarpc, tokio (#15013) 
* Update tonic & prost, and regenerate proto

* Reignore doc code

* Revert pull #14367, but pin tokio to v0.2 for jsonrpc

* Bump backoff and goauth -> and therefore tokio

* Bump tokio in faucet, net-utils

* Bump remaining tokio, plus tarpc
 2021-02-05 00:21:53 -07:00
Jeff Washington (jwash) 600ff0d915
calculate hash from store instead of index (#15034) 
* calculate hash from store instead of index

* restore update hash in abs
 2021-02-04 09:00:33 -06:00
behzad nouri 86467d825a
removes pubkey references (#15050) 2021-02-03 23:02:11 +00:00
behzad nouri 0ad063f4e9
adds flag to disable duplicate instance check (#15006) 2021-02-03 16:26:17 +00:00
Tyera Eulberg 98aa1fa4ea
Upgrade jsonrpc crates to v17.0.0 (#15018) 
* Upgrade to jsonrpc 17.0.0

* Fix test

* tree

Co-authored-by: Michael Vines <mvines@gmail.com>
 2021-02-02 19:53:08 -07:00
dependabot[bot] 1df93fa2be
chore: bump serde from 1.0.112 to 1.0.118 (#14828) 
* chore: bump serde from 1.0.112 to 1.0.122

Bumps [serde](https://github.com/serde-rs/serde) from 1.0.112 to 1.0.122.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.112...v1.0.122)

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

* Update frozen_abi digest following serde update

* Revert "chore: bump serde from 1.0.112 to 1.0.122"

This reverts commit a3ef4442a4c985144ae2bd7ceaf8899a7ab8d7c0.

* Revert "[auto-commit] Update all Cargo lock files"

This reverts commit c41c3b005fb1ccade55155302c52cd5736c4b55f.

* chore: bump serde from 1.0.112 to 1.0.118

Bumps [serde](https://github.com/serde-rs/serde) from 1.0.112 to 1.0.118.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.112...v1.0.118)

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

* Remove serum-dex pinning

* blind commit!

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
Co-authored-by: Ryo Onodera <ryoqun@gmail.com>
 2021-02-02 23:28:16 +09:00
Tyera Eulberg cbb8b79a60
Add validator flag to opt in to cpi and logs storage (#14922) 
* Add validator flag to opt in to cpi and logs storage

* Default TestValidator to opt-in; allow using in multinode-demo

* No clone

Co-authored-by: Carl Lin <carl@solana.com>
 2021-02-01 14:00:51 -07:00
Tom Parker-Shemilt 01230a0105
Remove serial_test_derive dependency (#14891) 2021-01-28 22:35:31 -07:00
Ryo Onodera d6873b82ab
Remove potentially too costly Packets::default() (#14821) 
* Remove potentially too costly Packets::default()

* Fix test...

* Restore Packets::default()

* Restore Packets::default() more
 2021-01-29 09:32:38 +09:00
Michael Vines 119e2c75dd cli now supports a custodian for stake authorize operations 2021-01-26 11:48:28 -08:00
Tyera Eulberg ffa5c7dcc8
Deprecate commitment variants (#14797) 
* Deprecate commitment variants

* Add new CommitmentConfig builders

* Add helpers to avoid allowing deprecated variants

* Remove deprecated transaction-status code

* Include new commitment variants in runtime commitment; allow deprecated as long as old variants persist

* Remove deprecated banks code

* Remove deprecated variants in core; allow deprecated in rpc/rpc-subscriptions for now

* Heavier hand with rpc/rpc-subscription commitment

* Remove deprecated variants from local-cluster

* Remove deprecated variants from various tools

* Remove deprecated variants from validator

* Update docs

* Remove deprecated client code

* Add new variants to cli; remove deprecated variants as possible

* Don't send new commitment variants to old clusters

* Retain deprecated method in test_validator_saves_tower

* Fix clippy matches! suggestion for BPF solana-sdk legacy compile test

* Refactor node version check to handle commitment variants and transaction encoding

* Hide deprecated variants from cli help

* Add cli App comments
 2021-01-26 19:23:07 +00:00
behzad nouri d1df9da7d3
fixes test_filter_current flakiness (#14816) 2021-01-25 15:57:46 +00:00
Ryo Onodera bd0433c373
Clean up VerifiedVotePackets (#14822) 2021-01-25 09:01:47 +00:00
behzad nouri e1021d9f83
removes redundant epoch stakes cache in retransmit (#14781) 
Following d6d76219b, staked nodes computed from vote accounts are
already cached in runtime::Stakes, so the caching in retransmit_stage is
redundant.
 2021-01-24 21:15:09 +00:00
behzad nouri 491b059755
broadcasts duplicate shreds through gossip (#14699) 2021-01-24 15:47:43 +00:00
Michael Vines cbffab7850 Upgrade to Rust v1.49.0 2021-01-23 19:16:36 -08:00
Michael Vines bf1943e489 Add solana-test-validator --warp-slot argument 2021-01-22 21:17:02 -08:00
Michael Vines cbb9ac19b9 Add ability to clone accounts from an RPC endpoint 2021-01-22 13:29:36 -08:00
Tyera Eulberg 71e9958e06
Rpc: Add custom error for BigTable data not found (#14762) 
* Expose not-found bigtable error

* Add custom rpc error for bigtable data not found

* Return custom rpc error when bigtable block is not found

* Generalize long-term storage
 2021-01-22 04:40:47 +00:00
behzad nouri e4da6761a7
fixes test_filter_current flakiness (#14749) 2021-01-21 21:53:10 +00:00
behzad nouri 8e581601d6
patches crds vote-index assignment bug (#14438) 
If tower is full, old votes are evicted from the front of the deque:
https://github.com/solana-labs/solana/blob/2074e407c/programs/vote/src/vote_state/mod.rs#L367-L373
whereas recent votes if expire are evicted from the back:
https://github.com/solana-labs/solana/blob/2074e407c/programs/vote/src/vote_state/mod.rs#L529-L537

As a result, from a single tower_index scalar, we cannot infer which crds-vote
should be overwritten:
https://github.com/solana-labs/solana/blob/2074e407c/core/src/crds_value.rs#L576

In addition there is an off by one bug in the existing code. tower_index is
bounded by MAX_LOCKOUT_HISTORY - 1:
https://github.com/solana-labs/solana/blob/2074e407c/core/src/consensus.rs#L382
So, it is at most 30, whereas MAX_VOTES is 32:
https://github.com/solana-labs/solana/blob/2074e407c/core/src/crds_value.rs#L29
Which means that this branch is never taken:
https://github.com/solana-labs/solana/blob/2074e407c/core/src/crds_value.rs#L590-L593
so crds table alwasys keeps 29 **oldest** votes by wallclock, and then
only overrides the 30st one each time. (i.e a tally of only two most
recent votes).
 2021-01-21 13:08:07 +00:00
Michael Vines 04ce33a04e Ensure sanitary transactions 2021-01-20 23:59:32 -08:00
Ryo Onodera dcaa025822
Configure Bigtable's timeout, enabling by default (#14657) 
* Configure bigtable's timeout when read-only

* Review comments

* Apply nits (thanks!)

Co-authored-by: Michael Vines <mvines@gmail.com>

* Timeout in the streamed decoding as well

Co-authored-by: Michael Vines <mvines@gmail.com>
 2021-01-19 13:57:16 +00:00
behzad nouri b5fd0ed859
rewrites turbine retransmit peers computation (#14584) 2021-01-19 04:18:47 +00:00
behzad nouri c6ae0667e6
feature gates turbine retransmit peers patch (#14631) 2021-01-19 04:16:19 +00:00
Michael Vines 5d9dc609b1 Rename RpcNodeUnhealthy error to NodeUnhealthy, generalize `getHealth` RPC error object for the future 2021-01-18 19:57:25 -08:00
Tyera Eulberg cbf8ef7480
Make Bigtable::get_confirmed_blocks inclusive of requested start_slot and end_slot (#14651) 
* Fix off-by-one error

* Filter out blocks greater than end slot
 2021-01-18 19:14:10 -07:00
carllin 5f14f45282
More generic accounts purge functions (#14595) 
Co-authored-by: Carl Lin <carl@solana.com>
 2021-01-17 20:31:03 -08:00
Ryo Onodera 8d4ab1bab1 Clean up and add comment 2021-01-17 19:59:59 -08:00
Ryo Onodera 5cf9094bb9 WIP fix the occasional stuck RPC request 2021-01-17 19:59:59 -08:00
Michael Vines 4003f86f04 Add `getSnapshotSlot` RPC method 2021-01-16 19:31:21 +00:00
Michael Vines dacb95083d Add `getHealth` RPC method 2021-01-16 10:10:48 -08:00
Tyera Eulberg 9a89689ad3
Use optimistic confirmation in getSignatureStatuses, and various downstream client methods (#14430) 
* Add optimistically_confirmed field to TransactionStatus

* Update docs

* Convert new field to confirmation_status

* Update docs to confirmationStatus

* Update variants

* Update docs

* Just Confirmed
 2021-01-15 16:05:05 +00:00
dependabot[bot] 4ecc0f4fa6
chore: bump rayon from 1.4.1 to 1.5.0 (#14177) 
* chore: bump rayon from 1.4.1 to 1.5.0

Bumps [rayon](https://github.com/rayon-rs/rayon) from 1.4.1 to 1.5.0.
- [Release notes](https://github.com/rayon-rs/rayon/releases)
- [Changelog](https://github.com/rayon-rs/rayon/blob/master/RELEASES.md)
- [Commits](https://github.com/rayon-rs/rayon/compare/rayon-core-v1.4.1...rayon-core-v1.5.0)

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
 2021-01-15 18:07:35 +09:00
Michael Vines 1c2ae15b1d Improve solana-test-validator output 2021-01-14 23:07:24 -08:00
sakridge 907f518f6d
Add load/execute/store timings (#14561) 2021-01-14 14:14:16 -08:00
behzad nouri cfcca1cd3c
patches bug in turbine's neighbors computation (#14565) 
Removing local node's index early from the set here:
https://github.com/solana-labs/solana/blob/e1b59ded4/core/src/retransmit_stage.rs#L346
distorts the order of nodes depending on which node is computing the
turbine fan-out tree, and results in incorrect neighbors computation.
 2021-01-13 22:25:29 +00:00
Michael Vines 11daaadc93 Add --rpc-threads argument 2021-01-13 13:34:46 -08:00
Tyera Eulberg e1b59ded4b
Use leader_forward_count for tx retries too (#14547) 2021-01-13 11:14:22 -07:00
Jeff Washington (jwash) 935dfdf0f6
fill in timing gaps in replay_stage (#14550) 
* fill in timing gaps in replay_stage

* add replay_stage bank_count metric

* formatting

* handle another gap

* cleanup wait_receive_time to be more straightforward
 2021-01-13 10:08:53 -06:00
carllin 6dfad0652f
Cache account stores, flush from AccountsBackgroundService (#13140) 2021-01-11 17:00:23 -08:00
Tyera Eulberg b0e6e29527
Update timestamp max allowable drift to 50% of PoH (#14531) 
* Repurpose warp-timestamp feature for general bump

* Change max_allowable_drift to 50%

* Fill in PR#

* Fix rpc test setup
 2021-01-11 23:27:30 +00:00
Michael Vines a95675a7ce
Avoid tmp snapshot backlog in SnapshotPackagerService under high load (#14516) 2021-01-11 10:21:15 -08:00
Michael Vines 7be6770808 Rename CompressionType to ArchiveFormat 2021-01-09 09:07:49 -08:00
Michael Vines 141e6706e6 Rename AccountsPackage::root to AccountsPackage::slot 2021-01-09 09:07:49 -08:00
behzad nouri 766195dded
limits number of crds values associated with a pubkey (#14467) 2021-01-08 18:54:40 +00:00
Jeff Washington (jwash) 773b21b34e
consolidate DEFAULT_HASHES_PER_TICK (#14463) 2021-01-07 09:49:42 -06:00
sakridge c282586753
Add fixed tick rate adjustment (#14447) 2021-01-05 19:25:44 -08:00
Tyera Eulberg ce1766d798
Enable program-id account index for supply calculations (#14444) 
* Enable program-id account index for supply calculations

* Fixup comments
 2021-01-06 02:42:33 +00:00
carllin 5affd8aa72
Add secondary indexes (#14212) 2020-12-31 18:06:03 -08:00
Michael Vines 4a3d217839 Only initialize BigTable upload service when requested 2020-12-31 17:40:51 -08:00
Michael Vines 0b23abd479 Bind RPC and faucet to 0.0.0.0 2020-12-31 07:55:26 +00:00
Michael Vines 3d077fb656 Revert "Upgrade in-tree tokio 0.2 usage to tokio 0.3" 
This reverts commit 444ed768dc.
 2020-12-31 06:26:53 +00:00
Michael Vines fb6c660cfd Port ip-echo-server to tokio 0.3 2020-12-30 09:29:16 -08:00
Trent Nelson fe667db910 validator: Add experimental flag to select PoH pinned core 2020-12-29 19:15:44 -07:00
Trent Nelson 82f61c0c4a core: Update stale error message 2020-12-29 19:15:44 -07:00
sakridge f8a4afc7c1
Fix flaky broadcast test (#14329) 2020-12-29 12:35:04 -08:00
sakridge 2074e407cd
Add poh speed check and tick speed calibration (#14292) 2020-12-29 09:35:57 -08:00
Michael Vines 444ed768dc Upgrade in-tree tokio 0.2 usage to tokio 0.3 2020-12-29 09:23:23 -08:00
Tyera Eulberg 3f10fb993b
Retry durable-nonce transactions (#14308) 
* Retry durable-nonce transactions

* Add metric to track durable-nonce txs in queue

* Populate send-tx-service initial addresses with tpu_address if empty (primarily for testing)

* Reinstate last_valid_slot check for durable-nonce txs; use arbitrary future slot
 2020-12-29 09:48:43 -07:00
Michael Vines 9ddd6f08e8 Persist gossip contact info 2020-12-27 20:46:54 -08:00
sakridge c693ffaa08
Fix subtraction overflow in metrics (#14290) 2020-12-27 16:26:22 -08:00
behzad nouri 2fd38d9912
indexes votes in crds table (#14272) 2020-12-27 13:31:05 +00:00
behzad nouri 49019c6613
obtains staked-nodes from the root-bank (#14257) 
... as opposed to the working bank
 2020-12-27 13:28:05 +00:00
Trent Nelson 5b903318b2 vote: Add helper for creating current-versioned states 2020-12-22 19:37:26 -07:00
Michael Vines ace360ade2 Multiple entrypoint support 2020-12-22 18:35:31 -08:00
Michael Vines 3373082ffa Update entrypoint contact info even when shred version adoption is not requested 2020-12-22 18:35:31 -08:00
behzad nouri a14cfd660a
removes &Arc<Self> receivers (#14234) 2020-12-22 23:51:53 +00:00
sakridge baa9602411
Add shrink paths (#14238) 2020-12-21 21:33:37 -08:00
behzad nouri 7b08cb1f0d
improves performance in replay-stage (#14217) 
bank::vote_accounts returns a hash-map which is slow to iterate, but all uses
only require an iterator:
https://github.com/solana-labs/solana/blob/b3dc98856/runtime/src/bank.rs#L4300-L4306
Similarly, calculate_stake_weighted_timestamp takes a hash-map whereas it only
requires an iterator:
https://github.com/solana-labs/solana/blob/b3dc98856/sdk/src/stake_weighted_timestamp.rs#L21-L28
 2020-12-21 19:18:19 +00:00
Michael Vines 57b03c5bc1 Restore Content-Length header for streaming snapshot download 2020-12-20 23:22:26 -08:00
Michael Vines b3dc988564 Stream RPC snapshot downloads 2020-12-19 15:54:23 -08:00