Commit Graph

397 Commits

Author SHA1 Message Date
Tyera Eulberg d1563f0ccd
Bump tonic, prost, tarpc, tokio (#15013)
* Update tonic & prost, and regenerate proto

* Reignore doc code

* Revert pull #14367, but pin tokio to v0.2 for jsonrpc

* Bump backoff and goauth -> and therefore tokio

* Bump tokio in faucet, net-utils

* Bump remaining tokio, plus tarpc
2021-02-05 00:21:53 -07:00
Tyera Eulberg 98aa1fa4ea
Upgrade jsonrpc crates to v17.0.0 (#15018)
* Upgrade to jsonrpc 17.0.0

* Fix test

* tree

Co-authored-by: Michael Vines <mvines@gmail.com>
2021-02-02 19:53:08 -07:00
Tyera Eulberg cbb8b79a60
Add validator flag to opt in to cpi and logs storage (#14922)
* Add validator flag to opt in to cpi and logs storage

* Default TestValidator to opt-in; allow using in multinode-demo

* No clone

Co-authored-by: Carl Lin <carl@solana.com>
2021-02-01 14:00:51 -07:00
Tyera Eulberg ffa5c7dcc8
Deprecate commitment variants (#14797)
* Deprecate commitment variants

* Add new CommitmentConfig builders

* Add helpers to avoid allowing deprecated variants

* Remove deprecated transaction-status code

* Include new commitment variants in runtime commitment; allow deprecated as long as old variants persist

* Remove deprecated banks code

* Remove deprecated variants in core; allow deprecated in rpc/rpc-subscriptions for now

* Heavier hand with rpc/rpc-subscription commitment

* Remove deprecated variants from local-cluster

* Remove deprecated variants from various tools

* Remove deprecated variants from validator

* Update docs

* Remove deprecated client code

* Add new variants to cli; remove deprecated variants as possible

* Don't send new commitment variants to old clusters

* Retain deprecated method in test_validator_saves_tower

* Fix clippy matches! suggestion for BPF solana-sdk legacy compile test

* Refactor node version check to handle commitment variants and transaction encoding

* Hide deprecated variants from cli help

* Add cli App comments
2021-01-26 19:23:07 +00:00
Michael Vines cbffab7850 Upgrade to Rust v1.49.0 2021-01-23 19:16:36 -08:00
Tyera Eulberg 71e9958e06
Rpc: Add custom error for BigTable data not found (#14762)
* Expose not-found bigtable error

* Add custom rpc error for bigtable data not found

* Return custom rpc error when bigtable block is not found

* Generalize long-term storage
2021-01-22 04:40:47 +00:00
Michael Vines 04ce33a04e Ensure sanitary transactions 2021-01-20 23:59:32 -08:00
Ryo Onodera dcaa025822
Configure Bigtable's timeout, enabling by default (#14657)
* Configure bigtable's timeout when read-only

* Review comments

* Apply nits (thanks!)

Co-authored-by: Michael Vines <mvines@gmail.com>

* Timeout in the streamed decoding as well

Co-authored-by: Michael Vines <mvines@gmail.com>
2021-01-19 13:57:16 +00:00
Michael Vines 5d9dc609b1 Rename RpcNodeUnhealthy error to NodeUnhealthy, generalize `getHealth` RPC error object for the future 2021-01-18 19:57:25 -08:00
Tyera Eulberg cbf8ef7480
Make Bigtable::get_confirmed_blocks inclusive of requested start_slot and end_slot (#14651)
* Fix off-by-one error

* Filter out blocks greater than end slot
2021-01-18 19:14:10 -07:00
Michael Vines 4003f86f04 Add `getSnapshotSlot` RPC method 2021-01-16 19:31:21 +00:00
Michael Vines dacb95083d Add `getHealth` RPC method 2021-01-16 10:10:48 -08:00
Tyera Eulberg 9a89689ad3
Use optimistic confirmation in getSignatureStatuses, and various downstream client methods (#14430)
* Add optimistically_confirmed field to TransactionStatus

* Update docs

* Convert new field to confirmation_status

* Update docs to confirmationStatus

* Update variants

* Update docs

* Just Confirmed
2021-01-15 16:05:05 +00:00
Michael Vines 11daaadc93 Add --rpc-threads argument 2021-01-13 13:34:46 -08:00
Tyera Eulberg b0e6e29527
Update timestamp max allowable drift to 50% of PoH (#14531)
* Repurpose warp-timestamp feature for general bump

* Change max_allowable_drift to 50%

* Fill in PR#

* Fix rpc test setup
2021-01-11 23:27:30 +00:00
Tyera Eulberg ce1766d798
Enable program-id account index for supply calculations (#14444)
* Enable program-id account index for supply calculations

* Fixup comments
2021-01-06 02:42:33 +00:00
carllin 5affd8aa72
Add secondary indexes (#14212) 2020-12-31 18:06:03 -08:00
Michael Vines 3d077fb656 Revert "Upgrade in-tree tokio 0.2 usage to tokio 0.3"
This reverts commit 444ed768dc.
2020-12-31 06:26:53 +00:00
Michael Vines 444ed768dc Upgrade in-tree tokio 0.2 usage to tokio 0.3 2020-12-29 09:23:23 -08:00
Tyera Eulberg 3f10fb993b
Retry durable-nonce transactions (#14308)
* Retry durable-nonce transactions

* Add metric to track durable-nonce txs in queue

* Populate send-tx-service initial addresses with tpu_address if empty (primarily for testing)

* Reinstate last_valid_slot check for durable-nonce txs; use arbitrary future slot
2020-12-29 09:48:43 -07:00
behzad nouri 49019c6613
obtains staked-nodes from the root-bank (#14257)
... as opposed to the working bank
2020-12-27 13:28:05 +00:00
Michael Vines 0090106f60 getBlockTime RPC method now falls back to BigTable in all cases 2020-12-18 20:51:49 +00:00
sakridge da7d1e2302
Improved Transaction Forwarding (#13944)
* Forwarding

* Dedupe leaders

* Use consistent commitment for last_valid_slot in rpc send_transaction

* Plumb rpc send_transaction options into solana-validator

* Extend num slots banking-stage holds forwarded txs

Co-authored-by: Tyera Eulberg <tyera@solana.com>
2020-12-17 15:37:22 -07:00
Tyera Eulberg ac0d32bc7e
Add blockstore skipped api (#14145)
* Add blockstore api to determine if a slot was skipped

* Return custom rpc error if slot is skipped
2020-12-16 13:40:36 -07:00
Michael Vines 7143aaa89b Clippy 2020-12-14 08:03:29 -08:00
carllin 55fc963595
Move slot cleanup to AccountsBackgroundService (#13911)
* Move bank drop to AccountsBackgroundService

* Send to ABS on drop instead, protects against other places banks are dropped

* Fix Abi

* test

Co-authored-by: Carl Lin <carl@solana.com>
2020-12-13 01:22:34 +00:00
Ryo Onodera 09bd412b13
Wrong rpc inflation rate (#14063)
* Fix wrong inflation calculation in rpc

* Reorder

* Fix test
2020-12-11 07:42:39 +00:00
Michael Vines 6e9dbb4f6e Add --rpc-max-multiple-accounts to override the getMultipleAccounts JSON RPC maximum 2020-12-07 16:31:01 -08:00
Tyera Eulberg 8fd1e55805
Add logging in check_blockstore_max_root (#13887) 2020-12-01 07:44:18 +00:00
behzad nouri e1793e5a13
caches vote-state de-serialized from vote accounts (#13795)
Gossip and other places repeatedly de-serialize vote-state stored in
vote accounts. Ideally the first de-serialization should cache the
result.

This commit adds new VoteAccount type which lazily de-serializes
VoteState from Account data and caches the result internally.

Serialize and Deserialize traits are manually implemented to match
existing code. So, despite changes to frozen_abi, this commit should be
backward compatible.
2020-11-30 17:18:33 +00:00
Tyera Eulberg 4ff0f0949a
Separate blockstore checks for not (yet) rooted and cleaned up (#13814) 2020-11-25 22:59:38 +00:00
Michael Vines 215ddecaa5 Add base64+zstd encoding for RPC account data 2020-11-25 02:03:23 +00:00
Tyera Eulberg 7befad2f6d
Check SlotNotRooted if confirmed block not found in blockstore or bigtable (#13776) 2020-11-24 03:36:20 +00:00
Trent Nelson f2a1a0ac5c RPC: Demote missing block error to warning
It frightens the tourists
2020-11-19 04:54:49 +00:00
Michael Vines baa6b3a261 Add stable program logging for BPF and native programs 2020-11-14 08:26:01 -08:00
Ryo Onodera 89b474e192
Fix slow/stuck unstaking due to toggling in epoch (#13501)
* Fix slow/stuck unstaking due to toggling in epoch

* nits

* nits

* Add stake_program_v2 feature status check to cli

Co-authored-by: Tyera Eulberg <tyera@solana.com>
2020-11-11 14:11:57 -07:00
Michael Vines 8838a55a1a Bump spl-token and spl-memo crate versions 2020-11-04 21:44:33 +00:00
Tyera Eulberg eb2560e782 Prevent block times from ever going backward 2020-10-31 21:30:42 -07:00
Tyera Eulberg 90778615f6 Use bounded timestamp-correction when feature enabled 2020-10-31 21:30:42 -07:00
Jack May c458d4b213
move Account to solana-sdk (#13198) 2020-10-28 22:01:07 -07:00
Tyera Eulberg 39686ef098
Use bank timestamp to populate Blockstore::blocktime_cf when correction active (#13158) 2020-10-26 19:23:45 +00:00
Josh 766406fd23
add precompile verification to simulate_transaction (#13080) 2020-10-23 20:47:51 -07:00
Michael Vines 959880db60 Remove unused pubkey::Pubkey imports 2020-10-21 19:08:13 -07:00
Michael Vines 7bc073defe Run `codemod --extensions rs Pubkey::new_rand solana_sdk::pubkey::new_rand` 2020-10-21 19:08:13 -07:00
Trent Nelson c26512255d RPC: Add metrics for TX encoding 2020-10-14 12:24:11 -06:00
Michael Vines 247228ee61 Implementation-defined RPC server errors are now accessible to client/ users 2020-10-13 10:05:44 -07:00
Michael Vines 649fe6d3b6 get_vote_accounts: access HashMap directly instead of turning it into an iterator 2020-10-13 04:12:10 +00:00
Trent Nelson e35889542b RPC: Support base64 encoded transactions
Defaults to base58
2020-10-06 22:41:06 -06:00
Trent Nelson 7f67d36777 RPC: Check encoded transaction size before decoding 2020-10-06 22:41:06 -06:00
Michael Vines 75b621160e Add GetConfirmedBlocksWithLimit RPC method 2020-10-01 22:56:17 -07:00
Tyera Eulberg 96a7d4dbd8
Query BigTable for block time if does not exist in blockstore (#12560) 2020-09-29 21:39:36 +00:00
Tyera Eulberg 89621adca7
Rpc -> proper optimistic confirmation (#12514)
* Add service to track the most recent optimistically confirmed bank

* Plumb service into ClusterInfoVoteListener and ReplayStage

* Clean up test

* Use OptimisticallyConfirmedBank in RPC

* Remove superfluous notifications from RpcSubscriptions

* Use crossbeam to avoid mpsc recv_timeout panic

* Review comments

* Remove superfluous last_checked_slots, but pass in OptimisticallyConfirmedBank for complete correctness
2020-09-28 20:43:05 -06:00
sakridge 6583c8cffe
Add precompile verification to preflight (#12486) 2020-09-27 22:29:00 -07:00
Michael Vines 35f5f9fc7b Add feature set identifier to gossiped version information 2020-09-25 11:40:36 -07:00
Josh 1d04c1db94
introduce RpcPerfSample and modify getPerformanceSamples output (#12434)
* introduce RpcPerfSample and modify getPerformanceSamples output

* camelCase test results
2020-09-24 14:22:22 -07:00
Justin Starry 731a943239
Remove transaction encoding from storage layer (#12404) 2020-09-24 13:10:29 +08:00
Tyera Eulberg e1a212fb79
Bump spl-token (#12395) 2020-09-22 17:08:54 -06:00
Josh 65a6bfad09
Add blockstore column to store performance sampling data (#12251)
* Add blockstore column to store performance sampling data

* introduce timer and write performance metrics to blockstore

* introduce getRecentPerformanceSamples rpc

* only run on rpc nodes enabled with transaction history

* add unit tests for get_recent_performance_samples

* remove RpcResponse from rpc call

* refactor to use Instant::now and elapsed for timer

* switch to root bank and ensure not negative subraction

* Add PerfSamples to purge/compaction

* refactor to use Instant::now and elapsed for timer

* switch to root bank and ensure not negative subraction

* remove duplicate constants

Co-authored-by: Tyera Eulberg <tyera@solana.com>
2020-09-22 12:26:32 -07:00
Michael Vines c4913e3c9e SendTransactionServices now exit their thread on channel drop instead of by a flag 2020-09-18 17:29:10 +00:00
Trent Nelson 32dcce0ac1 RPC: Limit request payload size to 50kB 2020-09-16 20:21:59 +00:00
Justin Starry f6cda2579f Fix off-by-one max payload checks 2020-09-16 12:46:06 -07:00
Michael Vines c6eea94edc Remove stale comment 2020-09-16 08:42:26 -07:00
Michael Vines 749208fa32 RPC sendTransaction now returns transaction logs on simulation failure 2020-09-16 08:42:26 -07:00
sakridge 3930cb865a
Add keccak-secp256k1 instruction (#11839)
* Implement keccak-secp256k1 instruction

Verifies eth addreses with ecrecover function

* Move secp256k1 test
2020-09-15 18:23:21 -07:00
Ryo Onodera de4a613610
Check bank capitalization (#11927)
* Check bank capitalization

* Simplify and unify capitalization calculation

* Improve and add tests

* Avoid overflow and inhibit automatic restart

* Fix test

* Tweak checked sum for cap. and add tests

* Fix broken build after merge conflicts..

* Rename to ClusterType

* Rename confusing method

* Clarify comment

* Verify cap. in rent and inflation tests

Co-authored-by: Stephen Akridge <sakridge@gmail.com>
2020-09-12 01:48:06 +09:00
Michael Vines 10ce839ec0 Speed up `bigtable transaction-history` command 2020-09-10 05:22:10 +00:00
Tyera Eulberg 05db41fe9c
Cache block time in Blockstore (#11955)
* Add blockstore column to cache block times

* Add method to cache block time

* Add service to cache block time

* Update rpc getBlockTime to use new method, and refactor blockstore slightly

* Return block_time with confirmed block, if available

* Add measure and warning to cache-block-time
2020-09-09 09:33:14 -06:00
Michael Vines 9e96180ce4 getMinimumBalanceForRentExemption now only responds to valid account lengths 2020-09-08 17:48:09 +00:00
anatoly yakovenko c67f8bd821
Forward transactions to the expected leader instead of your own TPU port (#12004)
* Use PoHRecorder to send to the right leader

* cleanup

* fmt

* clippy

* Cleanup, fix bug

Co-authored-by: Carl <carl@solana.com>
2020-09-08 17:00:49 +08:00
Justin Starry 9940870c89
Fix RPC transaction method configs serialization (#12100) 2020-09-08 13:08:09 +08:00
Michael Vines d8e2038dda Add --enable-bigtable-ledger-upload flag 2020-09-04 16:01:49 -07:00
Tyera Eulberg 90fad36613
Use new_response for consistency 2020-09-03 12:02:26 -06:00
Tyera Eulberg b22de369b7
Rpc: add getMultipleAccounts endpoint (#12005)
* Add rpc endpoint to return the state of multiple accounts from the same bank

* Add docs

* Review comments: Dedupe account code, default to base64, add max const

* Add get_multiple_accounts to rpc-client
2020-09-03 11:35:06 -06:00
Michael Vines 7341e60043 Ensure that the spl-token 2 native mint account is owned by the spl-token 2 program.
Workaround for https://github.com/solana-labs/solana-program-library/issues/374 until spl-token 3 is shipped
2020-09-01 17:09:36 +00:00
Tyera Eulberg 60c7ac6f95
Fix get_parsed_token_accounts (#11907) 2020-08-29 12:38:27 -06:00
Tyera Eulberg 2eff9a19c3
Update to token pack/unpack changes (#11900) 2020-08-28 22:06:30 -06:00
Tyera Eulberg 76be36c9ce
Update spl-token to v2.0 (#11884)
* Update account-decoder to spl-token v2.0

* Update transaction-status to spl-token v2.0

* Update rpc to spl-token v2.0

* Update getTokenSupply to pull from Mint directly

* Fixup to spl-token v2.0.1
2020-08-28 15:54:57 -06:00
Tyera Eulberg 1988ee9cd6
Rpc: Filter accounts with invalid mints from get_parsed_token_accounts (#11844)
* Filter out accounts with invalid mints from get_parsed_token_accounts

* Explicit docs
2020-08-26 11:39:42 -06:00
Michael Vines b660704faa Allow the sendTransaction preflight commitment level to be configured 2020-08-22 08:19:50 -07:00
Tyera Eulberg 747f8d5877
Rpc: Return error if block does not exist (#11743)
* Return error if block does not exist

* Update docs
2020-08-20 15:05:30 -06:00
Tyera Eulberg 2fd2aceeb2
Bump spl-token to clean up magic number (#11726) 2020-08-20 09:51:12 -06:00
carllin b7ed06b17a
Cleanup test utilities (#11723)
* Add voting utility

* Add blockstore utility

Co-authored-by: Carl <carl@solana.com>
2020-08-20 05:04:38 +00:00
Michael Vines adc984a225 Rename Binary64 to Base64. Establish Base58 encoding 2020-08-17 17:26:29 -07:00
Michael Vines 757e147b3b Rework UiAccountData encode/decode such that it works from Rust 2020-08-17 17:26:29 -07:00
Tyera Eulberg 6c5b8f324a
Rpc: Add until parameter for getConfirmedSignaturesForAddress2 (#11644)
* Refactor bigtable apis to accept start and end keys

* Make helper fn to deserialize cell data

* Refactor get_confirmed_signatures_for_address to use get_row_data range

* Add until param to get_confirmed_signatures_for_address

* Add until param to blockstore api

* Plumb until through client/cli

* Simplify client params
2020-08-15 10:42:17 -06:00
Greg Fitzgerald 574c356863
Add CommitmentSlots::new_from_slot() (#11600) 2020-08-13 03:51:15 +00:00
Tyera Eulberg b1e452f876
Return blockstore signatures-for-address despite bigtable error (#11594) 2020-08-12 17:57:05 -06:00
Tyera Eulberg 88ca04dbdb
Add config param to specify offset/length for single and program account info (#11515)
* Add config param to specify dataSlice for account info and program accounts

* Use match instead of if
2020-08-10 22:35:29 +00:00
Tyera Eulberg da210ddd51
Fix parsing of spl-token Mint (#11512)
* Add failing test

* Fix jsonParsed mint
2020-08-10 20:35:36 +00:00
Tyera Eulberg a9f76862fb
Decode native-program and sysvar accounts (#11463)
* Pass pubkey in to account-decoder for sysvars

* Decode sysvar accounts

* Decode config accounts; move validator-info lower

* Decode stake accounts

* Review comments

* Stringify any account lamports and epochs that can be set to u64::MAX
2020-08-09 07:50:45 +00:00
sakridge 068d23f298
Add Binary64 option for account data (#11474)
* Add Binary64 option for account data

* Decode into binary64

* Reword docs
2020-08-08 22:40:13 -07:00
Tyera Eulberg b7c2681903
Token Accounts: return ui_amount, decimals with decoded account (#11407)
* Return ui_amount, decimals from token client methods

* Return ui_amount, decimals in RPC jsonParsed token accounts

* Fixup docs

* Return ui_amount, decimals in pubsub jsonParsed token accounts

* Remove unnecessary duplicate struct

* StringAmount rename
2020-08-07 11:37:39 -06:00
Michael Vines 4222932e08 Plug getConfirmedSignaturesForAddress2 into bigtable storage 2020-08-05 23:04:16 -07:00
Michael Vines 02c0981ecf Rename startAfter to before 2020-08-05 14:33:14 -07:00
Michael Vines 087fd32ce3 Reimplement transaction-history command with getConfirmedSignaturesForAddress2 2020-08-05 14:33:14 -07:00
Michael Vines 1b2276520b Add getConfirmedSignaturesForAddress2 RPC method 2020-08-05 14:33:14 -07:00
Michael Vines dfae9a9864 Plumb Bigtable ledger storage into the RPC subsystem 2020-08-05 14:27:12 -07:00
Michael Vines 0e02740565 Setup a tokio 0.2 runtime for RPC usage 2020-08-05 14:27:12 -07:00
Tyera Eulberg 86e3f96f16
Return token amounts as floats (#11370)
* Return token amounts as floats

* Floating-point equality

* Return float and raw token amounts

* Fix decimals and token rpcs for native-mint tokens

* Fixup docs and review comments
2020-08-05 06:48:09 +00:00
Trent Nelson 14339dec0a Harden RPC preflight test inputs 2020-08-05 06:06:57 +00:00