Commit Graph

125 Commits

Author SHA1 Message Date
Justin Starry 85eb37fab0
Merge pull request from GHSA-8v47-8c53-wwrc
* Track transaction check time separately from account loads

* banking packet process metrics

* Remove signature clone in status cache lookup

* Reduce allocations when converting packets to transactions

* Add blake3 hash of transaction messages in status cache

* Bug fixes

* fix tests and run fmt

* Address feedback

* fix simd tx entry verification

* Fix rebase

* Feedback

* clean up

* Add tests

* Remove feature switch and fall back to signature check

* Bump programs/bpf Cargo.lock

* clippy

* nudge benches

* Bump `BankSlotDelta` frozen ABI hash`

* Add blake3 to sdk/programs/Cargo.lock

* nudge bpf tests

* short circuit status cache checks

Co-authored-by: Trent Nelson <trent@solana.com>
2021-04-13 00:28:08 -06:00
Tyera Eulberg ae7bc8299d
Use spl-memo v3.0.1 (#16384)
* Use memo v3.0.1, which simplifies id imports

* tree
2021-04-06 02:06:44 +00:00
Tyera Eulberg a902505810
Parse SPL associated-token-account instructions (#16318) 2021-04-01 22:48:05 +00:00
behzad nouri 3f63ed9a72
removes OrderedIterator and transaction batch iteration order (#16153)
In TransactionBatch,
https://github.com/solana-labs/solana/blob/e50f59844/runtime/src/transaction_batch.rs#L4-L11
lock_results[i] is aligned with transactions[iteration_order[i]]:
https://github.com/solana-labs/solana/blob/e50f59844/runtime/src/bank.rs#L2414-L2424
https://github.com/solana-labs/solana/blob/e50f59844/runtime/src/accounts.rs#L788-L817

However load_and_execute_transactions is iterating over
  lock_results[iteration_order[i]]
https://github.com/solana-labs/solana/blob/e50f59844/runtime/src/bank.rs#L2878-L2889
and then returning i as for the index of the retryable transaction.

If iteratorion_order is [1, 2, 0], and i is 0, then:
  lock_results[iteration_order[i]] = lock_results[1]
which corresponds to
  transactions[iteration_order[1]] = transactions[2]
so neither i = 0, nor iteration_order[i] = 1 gives the correct index for the
corresponding transaction (which is 2).

This commit removes OrderedIterator and transaction batch iteration order
entirely. There is only one place in blockstore processor which the
iteration order is not ordinal:
https://github.com/solana-labs/solana/blob/e50f59844/ledger/src/blockstore_processor.rs#L269-L271
It seems like, instead of using an iteration order, that can shuffle entry
transactions in-place.
2021-03-31 23:59:19 +00:00
sakridge 54c68ea83f
Drop write lock on sysvars (#15497)
* Drop write lock on sysvars

* adds env var for demoting sysvar write lock demotion

* moves demote logic to is_writable

* feature gates sysvar write lock demotion

* adds builtins to write lock demotion

* adds system program id to builtins

* adds Feature111...

* adds an abi-freeze test

* mvines set of builtin program keys

Co-authored-by: Michael Vines <mvines@gmail.com>

* update tests

* adds bpf loader keys

* Add test sysvar

* Plumb demote_sysvar to is_writable

* more plumbing of demote_sysvar_write_locks to is_writable

* patches test_program_bpf_instruction_introspection

* hard codes demote_sysvar_write_locks to false for serialization/encoding methods

* Revert "hard codes demote_sysvar_write_locks to false for serialization/encoding methods"

This reverts commit ae3e2d2e777437bddd753933097a210dcbc1b1fc.

* change the hardcoded ones to demote_sysvar_write_locks=true

* Use data_as_mut_slice

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
Co-authored-by: Michael Vines <mvines@gmail.com>
2021-03-30 10:05:09 -07:00
Tyera Eulberg 433f1ead1c
Rpc: enable getConfirmedBlock and getConfirmedTransaction to return confirmed (not yet finalized) data (#16142)
* Add Blockstore block and tx apis that allow unrooted responses

* Add TransactionStatusMessage, and send on bank freeze; also refactor TransactionStatusSender

* Track highest slot with tx-status writes complete

* Rename and unpub fn

* Add commitment to GetConfirmed input configs

* Support confirmed blocks in getConfirmedBlock

* Support confirmed txs in getConfirmedTransaction

* Update sigs-for-addr2 comment

* Enable confirmed block in cli

* Enable confirmed transaction in cli

* Review comments

* Rename blockstore method
2021-03-26 16:47:35 -06:00
Tyera Eulberg aa54c468ea
rpc: Add config options limiting getConfirmedBlock response data (#15970)
* Add new confirmed block struct

* Add RpcConfirmedBlockConfig options

* Configure block response based on new options

* Add client api, use in cli fetch_epoch_rewards

* Update docs

* Apply review suggestions
2021-03-18 17:58:20 +00:00
Jack May 7f500d610c
Add Close instrruction and tooling to upgradeable loader (#15887) 2021-03-17 21:39:29 -07:00
Josh 61112d4826
fix: compute pre/post token balances on all accounts if token program present (#15900)
* fix: compute pre/post token balances on all accounts if token program present

* fix: skip token program in balance query

* fix: prevent program ids from being collected
2021-03-16 16:49:22 +00:00
Jeff Washington (jwash) 52e54e1100
account.data -> data() (#15778) 2021-03-09 22:31:33 +00:00
Tyera Eulberg 19ac79b5cc
Deprecate UiTokenAmount::ui_amount (#15616)
* Add TokenAmount::ui_amount_string

* Fixup solana-tokens

* Update docs
2021-03-02 22:51:41 -07:00
Tyera Eulberg 1ad2c9f741
Revert "Make UiTokenAmount::ui_amount a String (#15447)" (#15542)
This reverts commit d14374bc9f.
2021-02-25 21:53:40 +00:00
Tyera Eulberg d14374bc9f
Make UiTokenAmount::ui_amount a String (#15447)
* Make UiTokenAmount::ui_amount a String

* Fixup solana-tokens

* Ignore spl downstream-project
2021-02-22 13:05:45 -07:00
Justin Starry 15bbe6436d
Bump SPL token version to v3.1.0 (#15429)
* Bump SPL token version to v3.1.0

* Cargo.lock
2021-02-19 09:15:41 +00:00
Trent Nelson 7f7370c306 Re-allow clippy::integer_arithmetic at crate-level 2021-02-17 13:55:08 -07:00
Tyera Eulberg c0a6272afd
Parse upgradeable loader instructions and accounts (#15195)
* Parse upgradeable-loader instructions

* Parse upgradeable-loader accounts
2021-02-08 17:18:10 -07:00
Tyera Eulberg 34dfcc9c6f
Parse SPL Memo v3 (#14979)
* Parse memo v3 too

* tree
2021-02-02 04:02:57 +00:00
Michael Vines 119e2c75dd cli now supports a custodian for stake authorize operations 2021-01-26 11:48:28 -08:00
Tyera Eulberg ffa5c7dcc8
Deprecate commitment variants (#14797)
* Deprecate commitment variants

* Add new CommitmentConfig builders

* Add helpers to avoid allowing deprecated variants

* Remove deprecated transaction-status code

* Include new commitment variants in runtime commitment; allow deprecated as long as old variants persist

* Remove deprecated banks code

* Remove deprecated variants in core; allow deprecated in rpc/rpc-subscriptions for now

* Heavier hand with rpc/rpc-subscription commitment

* Remove deprecated variants from local-cluster

* Remove deprecated variants from various tools

* Remove deprecated variants from validator

* Update docs

* Remove deprecated client code

* Add new variants to cli; remove deprecated variants as possible

* Don't send new commitment variants to old clusters

* Retain deprecated method in test_validator_saves_tower

* Fix clippy matches! suggestion for BPF solana-sdk legacy compile test

* Refactor node version check to handle commitment variants and transaction encoding

* Hide deprecated variants from cli help

* Add cli App comments
2021-01-26 19:23:07 +00:00
Michael Vines 8359f4f5ff Include Clock sysvar in AuthorizeWithSeed instruction 2021-01-25 19:38:10 -08:00
Michael Vines e9b5d65f40 `solana decode-transaction` no longer panics on unsanitary transactions 2021-01-20 23:59:32 -08:00
Josh 1de6d28eaf
Add block_time to getConfirmedSignaturesForAddress2 and getConfirmedTransaction (#14572)
* add block_time to get_confirmed_signatures_for_address2 and protobuf implementation for tx_by_addr

* add tests for convert

* update cargo lock

* run cargo format after rebase

* introduce legacy TransactionByAddrInfo

* move LegacyTransactionByAddrInfo back to storage-bigtable
2021-01-20 22:10:35 -08:00
Tyera Eulberg 0e87572eb0
Return confirmation-status (#14709) 2021-01-20 18:32:48 -07:00
Jack May e3bd9e5300
Prevent the invoke and upgrade of programs in the same tx batch (#14653)
* Prevent the invoke and upgrade of programs in the same tx batch

* Pass program address as writable in the upgrade instruction
2021-01-19 16:24:44 -08:00
Tyera Eulberg 9a89689ad3
Use optimistic confirmation in getSignatureStatuses, and various downstream client methods (#14430)
* Add optimistically_confirmed field to TransactionStatus

* Update docs

* Convert new field to confirmation_status

* Update docs to confirmationStatus

* Update variants

* Update docs

* Just Confirmed
2021-01-15 16:05:05 +00:00
Michael Vines 7143aaa89b Clippy 2020-12-14 08:03:29 -08:00
Josh 13db3eca9f
SPL token balance in transaction metadata (#13673)
* feat: store pre / post token balances

* move helper functions into separate include

* move token balance functionality to transaction-status crate

* fix blockstore processor test

* fix bigtable legacy test

* add caching to decimals
2020-12-10 19:25:07 -08:00
Tyera Eulberg 2ef4369237
Distribute spl tokens (#13559)
* Add helpers to covert between sdk types

* Add distribute-spl-tokens to args and arg-parsing

* Build spl-token transfer-checked instructions

* Check spl-token balances properly

* Add display handling to support spl-token

* Small refactor to allow failures in allocation iter

* Use Associated Token Account for spl-token distributions

* Add spl token support to balances command

* Update readme

* Add spl-token tests

* Rename spl-tokens file

* Move a couple more things out of commands

* Stop requiring lockup_date heading for non-stake distributions

* Use epsilon for allocation retention
2020-11-19 10:32:31 -07:00
Tyera Eulberg 91f4e99b4c
Fix parsing CreateAccountWithSeed instructions (#13513)
* Reduce required num_system_accounts and handle 2-account instructions properly

* Update CreateAccountWithSeed account docs to be correct

* Add CreateAccountWithSeed test
2020-11-10 23:51:53 +00:00
Michael Vines 8838a55a1a Bump spl-token and spl-memo crate versions 2020-11-04 21:44:33 +00:00
Tyera Eulberg c4962af9eb
Parse vote instructions (#13202) 2020-10-27 17:53:17 -06:00
Michael Vines 959880db60 Remove unused pubkey::Pubkey imports 2020-10-21 19:08:13 -07:00
Michael Vines 7bc073defe Run `codemod --extensions rs Pubkey::new_rand solana_sdk::pubkey::new_rand` 2020-10-21 19:08:13 -07:00
Tyera Eulberg 46d0019955
Parse stake and system instructions (#13035)
* Fix token account check

* Add helper to check num accounts

* Add parse_stake

* Add parse_system

* Fix AuthorizeNonce docs

* Remove jsonParsed unstable markers

* Clippy
2020-10-20 21:02:17 -06:00
Tyera Eulberg 6c55aaf4c7
_ 2020-10-20 09:00:41 -06:00
Tyera Eulberg 6dc5be17b2
Remove errant print 2020-10-20 08:56:36 -06:00
Tyera Eulberg 942e4273ba
Parse bpf loader instructions (#12998)
* Add parsing for BpfLoader2 instructions

* Skip info if null

* Return account address in info map
2020-10-19 23:13:02 -06:00
Tyera Eulberg 359707c85e
Convert Blockstore Rewards cf to protobuf (#12860)
* Add Blockstore protobuf cf type

* Add Rewards message to proto and make generated pub

* Convert Rewards cf to ProtobufColumn

* Add bench

* Adjust tags

* Move solana proto definitions and conversion methods to new crate
2020-10-15 18:04:10 -06:00
Michael Vines c5c8da1ac0 Expose all rewards (fees, rent, voting and staking) in RPC getConfirmedBlock and the cli 2020-10-09 21:54:13 -07:00
Josh 8f5431551e
Store program logs in blockstore / bigtable (TransactionWithStatusMeta) (#12678)
* introduce store program logs in blockstore / bigtable

* fix test, transaction logs created for successful transactions

* fix test for legacy bincode implementation around log_messages

* only api nodes should record logs

* truncate transaction logs to 100KB

* refactor log truncate for improved coverage
2020-10-08 12:06:15 -07:00
Trent Nelson e35889542b RPC: Support base64 encoded transactions
Defaults to base58
2020-10-06 22:41:06 -06:00
Tyera Eulberg c31a34fbcb
Include post balance information for rewards (#12598)
* Include post balance information for rewards

* Add post-balance to stored Reward struct

* Handle extended Reward in bigtable

Co-authored-by: Michael Vines <mvines@gmail.com>
2020-09-30 17:57:06 -06:00
Justin Starry ce598c5c98
Use protobufs to store confirmed blocks in BigTable (#12526)
* Use protobufs to store confirmed blocks in BigTable

* Cleanup

* Reorganize proto

* Clean up use statements

* Split out function for unit testing

* s/utils/convert

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
2020-09-30 17:55:22 +00:00
Tyera Eulberg 865d01c38d
Fix TransactionStatusMeta breakage in blockstore (#12587)
* Add helper to facilitate deserializing legacy structs

* Use default_on_eof to fix blockstore vis-a-vis TransactionStatusMeta

* Add should-panic test and comments
2020-09-30 11:49:35 -06:00
Justin Starry 6601ec8f26
Record and store invoked instructions in transaction meta (#12311)
* Record invoked instructions and store in transaction meta

* Enable cpi recording if transaction sender is some

* Rename invoked to innerInstructions
2020-09-24 22:36:22 +08:00
Justin Starry 731a943239
Remove transaction encoding from storage layer (#12404) 2020-09-24 13:10:29 +08:00
Tyera Eulberg e1a212fb79
Bump spl-token (#12395) 2020-09-22 17:08:54 -06:00
Tyera Eulberg b940da4040
Update token amounts in parsed instructions to retain full precision (#12020) 2020-09-03 11:34:52 -06:00
Tyera Eulberg 76be36c9ce
Update spl-token to v2.0 (#11884)
* Update account-decoder to spl-token v2.0

* Update transaction-status to spl-token v2.0

* Update rpc to spl-token v2.0

* Update getTokenSupply to pull from Mint directly

* Fixup to spl-token v2.0.1
2020-08-28 15:54:57 -06:00
Michael Vines d15173ad9d Address latest nightly clippy lints, but globally disable stable_sort_primitive 2020-08-17 22:36:10 -07:00
Michael Vines adc984a225 Rename Binary64 to Base64. Establish Base58 encoding 2020-08-17 17:26:29 -07:00
Michael Vines 757e147b3b Rework UiAccountData encode/decode such that it works from Rust 2020-08-17 17:26:29 -07:00
Michael Vines 2ebc68a9e2 decode-transaction now supports binary64 2020-08-17 17:26:29 -07:00
Michael Vines b5f3ced860 Add base64 (binary64) encoding for getConfirmedTransaction/getConfirmedBlock 2020-08-17 17:26:29 -07:00
Michael Vines 1b2276520b Add getConfirmedSignaturesForAddress2 RPC method 2020-08-05 14:33:14 -07:00
Michael Vines 494968be66 UiTransactionEncoding is now copy 2020-08-05 14:27:12 -07:00
Tyera Eulberg 9d4f9be1fe
Update instruction encoding format (#11363)
* Rework parsed instruction format

* Rework parsed message accounts

* Review comments
2020-08-05 00:58:58 -06:00
Tyera Eulberg 3f6f1adb5b
Make accounts explicit in unrecognized jsonParsed instructions (#11351) 2020-08-03 20:27:15 +00:00
Tyera Eulberg 0f551d4f75
Decode token instructions (#11281)
* Token->SplToken

* Add spl_token instruction parsing

* Rebase on master

* Gracefully fail key len mismatches
2020-07-31 13:26:09 -06:00
Tyera Eulberg b45ac5d4db
Add SPL Token-specific rpc endpoints (#11231)
* Simplify account-decoder program ids + spl_token helper

* Spl program namespace version

* Add getTokenAccountBalance endpoint

* Remove token program id from getTokenAccountBalance request

* Add getTokenSupply endpoint

* Remove token program id from getTokenSupply request

* Add getTokenAccountsByOwner/Delegate endpoints

* Remove token program id from getTokenAccountsByOwner/Delegate requests

* Named parameter
2020-07-28 23:00:48 -06:00
Michael Vines 1437660686 Revert "Add UiTransactionEncoding::Raw"
This reverts commit a4a5438b6d.
2020-07-12 08:59:50 -07:00
Michael Vines a4a5438b6d Add UiTransactionEncoding::Raw 2020-07-10 12:18:41 -07:00
Michael Vines 491f5ae61a Add block time placeholder to getConfirmedBlock 2020-07-10 14:47:11 +00:00
Tyera Eulberg 2669ccb864
Update struct prefixes to Ui (#10874)
* Update comments

* Use Ui prefix
2020-07-01 20:06:40 +00:00
Tyera Eulberg d97850f1d9
Add account-decoder utilities (#10846)
* Fix comment and make less pub

* Add account-decoder crate and use to decode vote and system (nonce) accounts

* Update docs

* Rename RpcAccount struct

* s/Rpc/Display

* Call it jsonParsed and update docs

* Revert "s/Rpc/Display"

This reverts commit 6e7149f503f560f1e9237981058ff05642bb7db5.

* s/Rpc/Ui

* Add tests

* Ui more things

* Comments
2020-07-01 04:55:11 +00:00
Tyera Eulberg fd2508b09e
Add jsonParsed option for EncodedTransactions; add memo parser (#10711)
* Add jsonParsed option for EncodedTransactions; add memo parser

* Use kebab case for program names

* Add account-key parsing

* Add parse test
2020-06-19 16:15:13 -06:00
Michael Vines c11abf88b7
Clean up `use` to keep rust 1.43.0 from complaining (#9740) 2020-04-27 16:54:11 -07:00
Michael Vines 7e7cbec8a1 Passing -v/--verbose to `solana confirm` now displays the full transaction 2020-04-16 08:19:30 -07:00
Tyera Eulberg e1aa247548
Rpc: Add getConfirmedTransaction (#9381)
* Add blockstore method to return a complete transaction by signature

* Plumb getConfirmedTransaction rpc

* Add doc
2020-04-09 00:57:30 -06:00
Tyera Eulberg acf64f8476
Update getSignatureStatuses to return historical statuses (#9314)
automerge
2020-04-06 03:04:54 -07:00
Tyera Eulberg 49e2cc6593
Rework TransactionStatus index in blockstore (#9281)
automerge
2020-04-04 20:24:06 -07:00
Michael Vines ad0997e15f
RPC: add `err` field to TransactionStatus, alongside the now deprecated `status` field (#9296)
automerge
2020-04-04 16:13:26 -07:00
Michael Vines 3ae6e0b8ab
Add solana-stake-monitor program (#9081) 2020-03-27 22:55:55 -07:00
Tyera Eulberg 4b97e58cba
Consolidate signature-status rpcs (#9069)
* getSignatureStatus: return confirmations for non-rooted transactions

* Remove getNumConfirmations.. rpc

* Remove getSignatureConfirmation

* Review comments

* More review comments
2020-03-26 19:21:01 -06:00
sakridge b7b4aa5d4d
move rpc types from client to client-types crate (#9039)
* Separate client types into own crate, so ledger does not need it

Removes about 50 crates of dependency from ledger

* Drop Rpc name from transaction-status types
2020-03-26 13:29:30 -07:00