Commit Graph

236 Commits

Author SHA1 Message Date
Jeff Biseda bafdb7dd62
Revert handle start_http failure in rpc_service (#25400) (#26130)
* revert e263be2000
2022-06-22 10:52:27 -07:00
Michael Vines f3639b76ce Remove some clippy lints 2022-06-22 09:23:22 -07:00
Dan Coombs 32a58dd9e0
Add ability to use a non-default app profile id in bigtable requests (#25968)
* Add ability to use a non-default app profile id in bigtable requests

* Only run subcommand once when getting global configs

* Remove unneded scoping on option type
2022-06-16 00:58:16 +00:00
behzad nouri 3c1ce3cc93
permanently disables durable nonces with chain blockhash domain (#25788)
https://github.com/solana-labs/solana/pull/25744
separated durable nonce and blockhash domains, which will stop double
execution going forward. However it is possible that a durable
transaction has *already* been executed once as a normal transaction and
it is now a valid durable transaction. #25744 cannot stop such
transactions to be re-executed until the nonce accounts are advanced.

This commit adds a new nonce version indicating that the nonce is moved
out of the blockhash domain, and permanently disables durable
transactions for legacy nonces which are in the blockhash domain.
2022-06-09 15:28:37 +00:00
Jon Cinque 79a8ecd0ac
client: Remove static connection cache, plumb it instead (#25667)
* client: Remove static connection cache, plumb it instead

* Add TpuClient::new_with_connection_cache to not break downstream

* Refactor get_connection and RwLock into ConnectionCache

* Fix merge conflicts from new async TpuClient

* Remove `ConnectionCache::set_use_quic`

* Move DEFAULT_TPU_USE_QUIC to client, use ConnectionCache::default()
2022-06-08 13:57:12 +02:00
behzad nouri 5ee157f43d separates durable nonce and blockhash domains
AdvanceNonceAccount instruction updates nonce to blockhash. This makes it
possible that a durable transaction is executed twice both as a normal
transaction and a nonce transaction if it uses blockhash (as opposed to nonce)
for its recent_blockhash field.

The commit prevents this double execution by separating nonce and blockhash
domains; when advancing nonce account, blockhash is hashed with a fixed string.
As a result a blockhash cannot be a valid nonce value; and if transaction was
once executed as a normal transaction it cannot be re-executed as a durable
transaction again and vice-versa.
2022-06-04 12:39:31 +00:00
Michael Vines b05c7d91ed Fix derive_partial_eq_without_eq clippy lint 2022-05-22 22:22:21 -07:00
Jeff Biseda e263be2000
handle start_http failure in rpc_service (#25400) 2022-05-20 17:59:23 -07:00
xiangzhu70 1295a95de6
Problem (#25348)
https://buildkite.com/solana-labs/solana/builds/73576#2847bbc5-6af4-4968-817d-f32b03d2f16d
The current timeout is 5 sec.  The timeout happened in this test run.
But it is not easy to manually reproduce this error.  I saw it
happened once out of about 20 runs.

Summary of Changes

Increase broadcast receiver timeout to mitigate test flakiness.
The chance of exceeding the 10sec timeout should be much lower.

We can debug the timeout further once we have the cargo nextest
stress run set up.

Co-authored-by: Xiang Zhu <xiangzhu@Xiangs-MacBook-Pro.local>
2022-05-20 08:09:52 -07:00
Michael Vines c54e06355f
voteSubscribe pubsub notification now includes the vote transaction signature (#25291) 2022-05-19 18:28:46 -07:00
Brooks Prumo 021276c955
Move updating accounts data len from execute to commit (#25256) 2022-05-19 12:24:13 -05:00
Michael Vines edd090f4a7
Avoid "Dropped RPC Notification" log spam (#25341) 2022-05-18 12:45:45 -07:00
Michael Vines 795f6eda44
`--rpc-pubsub-notification-threads 0` now turns off the internal PubSub notification machinery (#25307) 2022-05-17 20:23:51 -07:00
Jon Cinque 0820065c98
rpc: Support token-2022 in token-specific calls (#25150)
* rpc: Support token-2022 in token-specific calls

* Address feedback
2022-05-17 21:02:43 +02:00
Justin Starry 4690956849
RPC: Add minContextSlot param to ensure consistent responses (#25161)
* RPC: Add minContextSlot param to ensure consistent responses

* revert changes to deprecated endpoints

* update docs
2022-05-17 14:46:02 +08:00
Michael Vines e718c80a21 Avoid multiple base58 encodes for all votes when no `voteSubscribe` is disabled 2022-05-16 19:56:48 -07:00
Tyera Eulberg bc005e3408
Add configurable limit to number of blocks to check before Bigtable upload (#24716)
* Add ConfirmedBlockUploadConfig, no behavior changes

* Add comment

* A little DRY cleanup

* Add configurable limit to number of blocks to check in Blockstore and Bigtable before uploading

* Limit blockstore and bigtable look-ahead

* Exit iterator early when reach ending_slot

* Use rooted_slot_iterator instead of slot_meta_iterator

* Only check blocks in the ledger
2022-05-13 07:34:02 +00:00
Justin Starry a118af069e
RPC: Add API version to context response (#25134)
* RPC: Add API version to context response

* restore backwards compatibility
2022-05-12 12:17:21 +08:00
DimAn 2fa9bc3e70
Add options to store full and/or incremental snapshots in separate locations (#24247) 2022-05-10 16:37:41 -04:00
Justin Starry 082502d4f3
Fail tx sanitization when ix program id uses lookup table (#25035)
* Fail tx sanitization when ix program id uses lookup table

* feedback
2022-05-07 03:19:50 +08:00
Lijun Wang aae13c63cb
Fixed a minor typo (#24982) 2022-05-04 14:13:46 -07:00
Jon Cinque 326e53be97
stake: Allow initialized stakes to be below the min delegation (#24670)
* stake: Allow initialized stakes to be below the min delegation

* Add PR number in feature

* Fixup RPC subscription test

* Address feedback pt 1

* Address feedback pt 2

* Update FrozenAbi Digest

* Address feedback: no new error type, more comments
2022-05-04 12:17:29 +02:00
Justin Starry a61652104b
Avoid holding lock guards in match expressions (#24805)
* Avoid holding bank forks read lock for RPC requests

* Avoid using lock guards in temporaries

* revert fetch stage change
2022-04-29 16:32:46 +08:00
Justin Starry 4e58b3870c
Update all BankForks methods to return owned values (#24801) 2022-04-28 18:51:00 +00:00
Tyera Eulberg b6a18e0e87
Fix get_first_available_block for genesis; also make get_blocks(_with_limit) consistent (#24760)
* Handle genesis more appropriately in get_first_available_block

* Add unit test

* Make get_blocks starting-slots consistent with other methods
2022-04-28 00:36:19 -06:00
Steven Luscher 3007f233f4
Log subscription for `All` no longer clobbers `Some` subscription for pubkey (#24215)
* fix: subscribing for all logs no longer clobbers pubkey-specific subscriptions
* test: write tests to cover logsSubscribe
2022-04-26 10:31:11 -07:00
Justin Starry b6bc5d2fa1 Aggregate executor update timing metrics 2022-04-26 23:15:22 +08:00
Brooks Prumo d6869773bb
Make rpc test_account_subscribe aware of stake minimum delegation (#24659) 2022-04-25 18:35:05 -04:00
Tyera Eulberg d2dad51b8c
Support token-2022 in RPC account parsing (#24619)
* Use token-2022 in parsed_token_account

* Parse token-2022 accounts
2022-04-25 13:25:21 -06:00
Lijun Wang 7c61e438fc
Batch tpu calls in send-transaction-service (#24083)
Introduced flag --tpu-do-batch2.
Introduced flag to control the batch size-- by default 100
The default batch timeout is 200ms -- configurable. If either it time out or the batch size is filled, a new batch is sent
The batch honor the retry rate on the transaction already sent before.
Introduced two threads in STS: one for receiving new transactions and doing batch send and one for retrying old transactions and doing batch.6.
Fixes #
2022-04-21 12:43:08 -07:00
Tao Zhu a21fc3f303
Apply transaction actual execution units to cost_tracker (#24311)
* Pass the sum of consumed compute units to cost_tracker

* cost model tracks builtins and bpf programs separately, enabling adjust block cost by actual bpf programs execution costs

* Copied nightly-only experimental `checked_add_(un)signed` implementation to sdk

* Add function to update cost tracker with execution cost adjustment

* Review suggestion - using enum instead of struct for CommitTransactionDetails
Co-authored-by: Justin Starry <justin.m.starry@gmail.com>

* review - rename variable to distinguish accumulated_consumed_units from individual compute_units_consumed

* not to use signed integer operations

* Review - using saturating_add_assign!(), and checked_*().unwrap_or()

* Review - using Ordering enum to cmp

* replace checked_ with saturating_

* review - remove unnecessary Option<>

* Review - add function to report number of non-zero units account to metrics
2022-04-21 07:38:07 +00:00
Tyera Eulberg b42f34a7b8
Update Rpc filters to support token-2022 (#24541)
* Add TokenAccountState rpc_filter

* Use a method on RpcFliterType instead of helper
2022-04-20 19:18:12 -06:00
steviez 2a770e9726
Use unsigned_abs() instead of casting abs() (#24521) 2022-04-20 12:01:33 -05:00
Tyera Eulberg 77f6ec5625
Add program_id to TokenBalances structs (#24513)
* Add program_id to TokenBalances structs

* Update docs
2022-04-19 22:01:22 -06:00
Trent Nelson 50fba01842 rpc-pubsub: reduce metrics/log spam 2022-04-15 01:16:58 -06:00
Tyera Eulberg f7d557d5ae
Update simulateTransaction rpc handling of return_data, and update docs (#24355)
* Stringify return_data program_id; also camel-case all fields

* Update simulateTransaction json-rpc docs

* Base64-encode return data in simulation RPC responses
2022-04-14 23:42:08 -06:00
Rachael Pai aea17c35ae
Add a stringified credential option for LedgerStorage (#24314)
* add a stringified credential option for LedgerStorage

* fix clippy::useless-format warning

* change CredentialOption to enum CredentialType

* rename credential_option to credential_type

* restore LedgerStorage new fn signature

* fmt

Co-authored-by: Tyera Eulberg <tyera@solana.com>
2022-04-13 14:35:06 -06:00
Tyera Eulberg fb67ff14de
Remove replica-node crates (#24152) 2022-04-06 16:52:19 -06:00
HaoranYi c9a476e24d
handle channel disconnect (#24036) 2022-04-01 13:47:06 -05:00
HaoranYi 51b37f0184
Modify rpc_completed_slot_service to be non-blocking (#24007)
* timeout for validator exits

* clippy

* print backtrace when panic

* add backtrace package

* increase time out to 30s

* debug logging

* make rpc complete service non blocking

* reduce log level

* remove logging

* recv_timeout

* remove backtrace

* remove sleep

* remove unused variable

* add comments

* Update core/src/validator.rs

Co-authored-by: Trent Nelson <trent.a.b.nelson@gmail.com>

* Update core/src/validator.rs

Co-authored-by: Trent Nelson <trent.a.b.nelson@gmail.com>

* whitespace

* more whitespace

* fix build

Co-authored-by: Trent Nelson <trent.a.b.nelson@gmail.com>
2022-03-31 16:44:23 -05:00
Jon Cinque 7af48465fa
transaction-status: Add return data to meta (#23688)
* transaction-status: Add return data to meta

* Add return data to simulation results

* Use pretty-hex for printing return data

* Update arg name, make TransactionRecord struct

* Rename TransactionRecord -> ExecutionRecord
2022-03-22 23:17:05 +01:00
Trent Nelson 9b32b72990 bigtable: allow custom instance names 2022-03-21 11:26:49 -06:00
Trent Nelson f513195468 bigtable: add a config ctor for `LedgerStorage` 2022-03-21 11:26:49 -06:00
Tyera Eulberg 63ee00e647 Refactor validator bigtable config 2022-03-21 11:26:49 -06:00
DimAn 739e43ba58
Add ability to get the latest incremental snapshot via RPC (#23788) 2022-03-21 11:48:49 -05:00
Lijun Wang ae76fe2bd7
Made connection cache configurable. (#23783)
Added command-line argument tpu-use-quic argument.
Changed connection cache to return different connections based on the config.
2022-03-21 09:31:37 -07:00
Justin Starry 8c8f9694e0
Refactor: Sanitized transaction creation (#23558)
* Refactor: SanitizedTransaction::try_create optionally computes hash

* Refactor: Add SimpleAddressLoader
2022-03-15 12:02:22 +08:00
HaoranYi 0c684721d8
Separate remotely downloaded snapshot archives (#23510)
* seperate remotely downloaded snapshot archives

* add str const for snapshot download dir

* only walk remote sub directory

* move directory creation outside of loop

* move is_remote to traits

* clippy simplify

* clippy

* clippy

* add unittest

* fix local cluster tests

* look for remote snapshot archive in remote foler

* create remote dir in tests

* use snapshot download dir constant

* extract build_remote_dir fn

* fix build

* code review - walking snapshot archived dirs explicitly

* fix build

* fix build

* fix comments

* Update runtime/src/snapshot_utils.rs

Co-authored-by: Brooks Prumo <brooks@prumo.org>

* clippy

* borrow to avoid copy

Co-authored-by: Brooks Prumo <brooks@prumo.org>
2022-03-14 14:03:59 -05:00
Justin Starry 249d926d1b
Refactor: Add transaction binary encoding enum (#23546) 2022-03-09 16:09:08 +08:00
Justin Starry 3114c199bd
Add RPC support for versioned transactions (#22530)
* Add RPC support for versioned transactions

* fix doc tests

* Add rpc test for versioned txs

* Switch to preflight bank
2022-03-08 15:20:34 +08:00
Justin Starry 38db1dead4
Refactor RPC tests (#23483) 2022-03-04 18:20:11 +08:00
Tyera Eulberg 3b5b71ce44
Improve UX querying rpc for blocks at or before the snapshot from boot (#23403)
* Bump first-available block to first complete block

* Remove obsolete purges in tests (PrimaryIndex toggling no longer in use

* Check first-available block in Rpc check_slot_cleaned_up
2022-02-28 23:57:41 -07:00
Trent Nelson d4292774c5 checks 2022-02-25 08:05:28 +00:00
Tyera Eulberg 7e08ae1d0c
Revert "Add simulation detection countermeasure (#22880)" (#23261)
This reverts commit c42b80f099.
2022-02-21 21:15:37 +00:00
buffalu 70ebab2c82
Add rustfmt.toml and `cargo fmt` (#23238)
* fmt

* formatted

Co-authored-by: Lucas B <buffalu@jito.network>
2022-02-19 13:32:29 +08:00
Trent Nelson 4213ece7bd rpc: make `getGenesisHash` part of minimal api 2022-02-16 10:05:46 -07:00
Michael Vines c42b80f099
Add simulation detection countermeasure (#22880)
* Add simulation detection countermeasures

* Add program and test using TestValidator

* Remove incinerator deposit

* Remove incinerator

* Update Cargo.lock

* Add more features to simulation bank

* Update Cargo.lock per rebase

Co-authored-by: Jon Cinque <jon.cinque@gmail.com>
2022-02-15 13:09:59 +01:00
Lijun Wang c04438be4b
Retaining transaction logs when transaction plugin is loaded. (#22874)
Transaction logs are not being saved to the database through the plugin interface.

Summary of Changes

Retain the transaction logs when transaction notification plugin is loaded.

Fixes #
lijunwangs/solana-accountsdb-plugin-postgres#6
2022-02-11 20:29:07 -08:00
Jack May 3d9874b95a
Add fees to tx-wide caps (#22081) 2022-02-11 16:23:16 -08:00
Justin Starry d5dec989b9
Enforce tx metadata upload with static types (#23028) 2022-02-10 13:28:18 +08:00
Michael Vines 86d465c531 Prepare RPC subsystem for multiple SPL Token program ids 2022-02-09 08:16:00 -08:00
Kirill Fomichev 541b5a4826
Rename struct to imported name (#22814)
* use imported RpcResponse

* Remove extra map method

* rename result to accounts

* fmt
2022-02-08 12:01:40 -07:00
Justin Starry ba215e94f6
Refactor: Add AccountKeys struct for static and dynamic message keys (#22960) 2022-02-05 20:00:31 +08:00
Justin Starry 60af1a4cce
Refactor: Add trait for loading addresses (#22903) 2022-02-03 11:00:12 +00:00
Trent Nelson c62f9839a2 test-validator-bin: reinstate full rpc method set 2022-02-03 02:43:03 +00:00
Trent Nelson eac4a6df68 rpc: use minimal mode by default 2022-02-01 19:00:06 -07:00
Kirill Fomichev 4bc3a1195f
Use internal function for encoding accounts (#22822)
* Use internal function for encoding accounts

* fix tests
2022-02-01 14:10:21 -07:00
Michael Vines 331b953551 Add vote account address to vote subscription 2022-01-27 08:22:29 -08:00
Michael Vines 6d5bbca630 Pacify clippy 2022-01-21 19:12:57 -08:00
Justin Starry 7f20c6149e
Refactor: move simple vote parsing to runtime (#22537) 2022-01-20 10:39:21 +08:00
Justin Starry f804ccdece
Store address table lookups in blockstore and bigtable (#22402) 2022-01-14 15:24:41 +08:00
Tyera Eulberg 637e366b18
Prevent rent-paying account creation (#22292)
* Fixup typo

* Add new feature

* Add new TransactionError

* Add framework for checking account state before and after transaction processing

* Fail transactions that leave new rent-paying accounts

* Only check rent-state of writable tx accounts

* Review comments: combine process_result success behavior; log and metrics before feature activation

* Fix tests that assume rent-exempt accounts are okay

* Remove test no longer relevant

* Remove native/sysvar special case

* Move metrics submission to report legacy->legacy rent paying transitions as well
2022-01-11 11:32:25 -07:00
Jeff Biseda 8b66625c95
convert std::sync::mpsc to crossbeam_channel (#22264) 2022-01-11 02:44:46 -08:00
pieceofr d9220652ad
[ledger-tool]compare_blocks (#22229)
* 1.made load_credentials accept credential path as a parameter. 2.partial implement bigtable comparasion function

* finding missing blocks in bigtables in a specified range

* refactor compare-blocks,add unit test for missing_blocks and fmt

* compare-block fix last block bug

* refactor compare-block and improve wording

* Update ledger-tool/src/bigtable.rs

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>

* update compare-block command-line description

* style:improve wording/naming/code style

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
2022-01-06 14:36:03 +08:00
Nikita c1995c647b
fix(rpc): recreate dead and uncleaned subscriptions (#22281) 2022-01-05 00:15:21 -07:00
Justin Starry 45458e7139
Refactor: Improve type safety and readability of transaction execution (#22215)
* Refactor Bank::load_and_execute_transactions

* Refactor: improve type safety of TransactionExecutionResult

* Add enum for extra type safety in execution results

* feedback
2022-01-05 10:15:15 +08:00
Justin Starry 2b5e00d36d
Limit number of accounts that a transaction can lock (#22201) 2022-01-04 14:25:23 +08:00
Justin Starry 2a00382d71
Refactor: cleanup solana_transaction_status crate (#22230) 2022-01-03 15:45:18 +00:00
Tyera Eulberg 0b1b36f088
Exit early on BigTable error (#22200) 2021-12-31 13:36:57 -07:00
Justin Starry b1d9a2e60e
Don't forward packets received from TPU forwards port (#22078)
* Don't forward packets received from TPU forwards port

* Add banking stage test
2021-12-29 19:34:31 +01:00
Omar Kilani bac6821e19
get_signatures_for_address does not correctly account for result sets that span local and Bigtable sources (#22115)
* get_signatures_for_address does not correctly account for result sets that span Blockstore and Bigtable.

This causes Bigtable to return `RowNotFound` until the new tx is uploaded.

Check that `before` exists in Bigtable, and if not, set it to `None` to return the full data set.

References #21442
Closes #22110

* Differentiate between before sig not found and no newer signatures

* Dedupe bigtable results to account for potential upload race

Co-authored-by: Tyera Eulberg <tyera@solana.com>
2021-12-29 10:25:10 -07:00
Yueh-Hsuan Chiang b89cd8cd1a
Avoid cloning Vec<Entry> when calling entries_to_test_shreds() (#22093) 2021-12-24 12:32:43 -08:00
behzad nouri 4d62f03297
uses enum instead of trait for VoteTransaction (#22019)
Box<dyn Trait> involves runtime dispatch, has significant overhead and
is slow. It also requires hacky boilerplate code for implementing Clone
or other basic traits:
https://github.com/solana-labs/solana/blob/e92a81b74/programs/vote/src/vote_state/mod.rs#L70-L102

Only limited known types can be VoteTransaction and they are all defined
in the same crate. So using a trait here only adds overhead.
https://github.com/solana-labs/solana/blob/e92a81b74/programs/vote/src/vote_state/mod.rs#L125-L165
https://github.com/solana-labs/solana/blob/e92a81b74/programs/vote/src/vote_state/mod.rs#L221-L264
2021-12-22 14:25:46 +00:00
segfaultdoctor 76098dd42a
RPC Block Subscription (#21787)
* add stuff

* compiling

* add notify block

* wip

* feat: add blockSubscribe pubsub method

* address PR comments

Co-authored-by: Lucas B <buffalu@jito.network>
Co-authored-by: Zano <segfaultdoctor@protonmail.com>
2021-12-17 16:03:09 -07:00
Justin Starry 6ff0be6a82
Clean up demote program write lock feature (#21949)
* Clean up demote program write lock feature

* fix test
2021-12-16 17:27:22 -05:00
Justin Starry e5476913fe
Remove activated feature that checks tx signature len (#21747) 2021-12-14 09:23:05 -05:00
Michael Vines 824994db69 simulateTransaction now returns the correct error code if accounts are provided as input 2021-12-08 14:03:21 -08:00
Ashwin Sekar f0acf7681e
Add vote instructions that directly update on chain vote state (#21531)
* Add vote state instructions

UpdateVoteState and UpdateVoteStateSwitch

* cargo tree

* extract vote state version conversion to common fn
2021-12-07 16:47:26 -08:00
Michael Vines b8837c04ec Reformat imports to a consistent style for imports
rustfmt.toml configuration:
  imports_granularity = "One"
  group_imports = "One"
2021-12-03 09:19:13 -08:00
Tyera Eulberg 9e043d2055
Use max_complete_transaction_status_slot in BigTableUploadService (#21401)
* Use max_complete_transaction_status_slot in BigTableUploadService

* Use node root to limit BigTableUploadService
2021-11-23 17:36:13 -07:00
Lijun Wang c29838fce1
Accountsdb plugin transaction part 3: Transaction Notifier (#21374)
The TransactionNotifierInterface interface for notifying transactions.
Changes to transaction_status_service to notify the notifier of the transaction data.
Interface to query the plugin's interest in transaction data
2021-11-23 09:55:53 -08:00
Jon Cinque 02bc4e3fc1
spl-token: New program feature flag (#21354)
* spl-token: Add feature flag for new release

* Remove all spl token version declarations
2021-11-21 14:27:03 +01:00
Jeff Washington (jwash) 79d21d6805
add --accounts-index-scan-results-limit-mb to allow scans to abort (#21327)
* ScanConfig -> &ScanConfig

* add --accounts-index-scan-results-limit-mb to allow scans to abort

* feedback
2021-11-19 09:00:19 -06:00
Jack May 03c36d240a
Nonce naming cleanup (#21336) 2021-11-18 16:07:17 -08:00
Justin Starry 66fa062f13
rename process_entries to indicate it's only for tests (#21321) 2021-11-17 20:53:40 +01:00
Jeff Washington (jwash) 0f69a14247
Add ability to abort scan (#21314) 2021-11-17 13:10:29 -06:00
Jack May cb0bb5bd1e
Nonce accounts must be writeable (#21260)
* Nonce accounts must be writeable

* feedback

* feedback
2021-11-16 23:01:00 +00:00
Michael Keleti b0ca335463
Rename "trusted" to "known" in `validators/` (#21197)
* Replaced trusted with known validator

* Format Convention
2021-11-12 11:57:55 -07:00
Jeff Biseda 7659a2edc2
report pubsub stats every 2s (#21192) 2021-11-05 19:59:54 -07:00
Ivan Mironov 0e751ef7df Add validator option to change niceness of RPC server threads
Fixes https://github.com/solana-labs/solana/issues/14556
2021-11-04 17:16:46 -06:00