Commit Graph

58 Commits

Author SHA1 Message Date
Justin Starry d5dec989b9
Enforce tx metadata upload with static types (#23028) 2022-02-10 13:28:18 +08:00
Justin Starry ba215e94f6
Refactor: Add AccountKeys struct for static and dynamic message keys (#22960) 2022-02-05 20:00:31 +08:00
Justin Starry f804ccdece
Store address table lookups in blockstore and bigtable (#22402) 2022-01-14 15:24:41 +08:00
pieceofr d9220652ad
[ledger-tool]compare_blocks (#22229)
* 1.made load_credentials accept credential path as a parameter. 2.partial implement bigtable comparasion function

* finding missing blocks in bigtables in a specified range

* refactor compare-blocks,add unit test for missing_blocks and fmt

* compare-block fix last block bug

* refactor compare-block and improve wording

* Update ledger-tool/src/bigtable.rs

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>

* update compare-block command-line description

* style:improve wording/naming/code style

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
2022-01-06 14:36:03 +08:00
Justin Starry 2a00382d71
Refactor: cleanup solana_transaction_status crate (#22230) 2022-01-03 15:45:18 +00:00
Michael Vines b8837c04ec Reformat imports to a consistent style for imports
rustfmt.toml configuration:
  imports_granularity = "One"
  group_imports = "One"
2021-12-03 09:19:13 -08:00
Tyera Eulberg f3f8d2e4f3
Add BigTable query logs and counter (#21394)
* Add LedgerStorage logs

* Add storage-bigtable metric
2021-11-22 16:22:46 -07:00
Tyera Eulberg f0de3e9bf0
chore: bump tonic from 0.5.2 to 0.6.1 (#21035)
* Bump tonic and prost crates

* connect_lazy no longer infallible
2021-10-28 06:59:36 +00:00
Ryo Onodera 795dde109c
Fix memo handling and "blocks" key formatting (#20044)
* Fix memo handling and "blocks" key formatting

* Skip memo equality check

* Define slot_to_tx_by_addr_key

Co-authored-by: Justin Starry <justin@solana.com>
2021-09-22 00:46:46 +09:00
Justin Starry c71fab6cb3
Add `delete` subcommand to `ledger-tool bigtable` (#19931)
* Add `delete` subcommand to `ledger-tool bigtable` command

* feedback
2021-09-16 23:37:45 +00:00
Tyera Eulberg f4ae450f34
Populate memo in bigtable transaction structs (#19512)
* Populate memo in bigtable transaction structs

* Preface memos with len
2021-08-30 21:14:18 -06:00
Michael Vines 4098af3b5b Record vote account commission with voting/staking rewards and surface in RPC 2021-07-12 15:09:44 -07:00
Tyera Eulberg c2e7d39154
Add storage-proto build.rs and readme (#18353)
* Use build.rs for storage-proto generation

* Add readme

* Single use statements
2021-07-09 20:06:06 +00:00
Tyera Eulberg 761de8b1a3
Bump prost, prost-types, and tonic (#18537)
* Bump prost+tonic and accommodate generated service changes

* Unignore advisory

* Fixup .proto error list
2021-07-09 17:31:26 +00:00
dependabot[bot] 0426c2d28e
chore: bump bzip2 from 0.3.3 to 0.4.3 (#18361)
* chore: bump bzip2 from 0.3.3 to 0.4.3

Bumps [bzip2](https://github.com/alexcrichton/bzip2-rs) from 0.3.3 to 0.4.3.
- [Release notes](https://github.com/alexcrichton/bzip2-rs/releases)
- [Commits](https://github.com/alexcrichton/bzip2-rs/commits)

---
updated-dependencies:
- dependency-name: bzip2
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

* Update to new api

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
2021-07-01 13:10:52 -06:00
Michael Vines 4a12c715a3 Drop Error suffix from enum values to avoid the enum_variant_names clippy lint 2021-06-18 23:02:13 +00:00
Alexander Meißner 6514096a67 chore: cargo +nightly clippy --fix -Z unstable-options 2021-06-18 10:42:46 -07:00
Tyera Eulberg c57d1b44ef
Properly handle block_height in Bigtable bincode deserialization (#17990)
* Default block_height on eof

* Add comment to prevent future errors
2021-06-15 17:34:07 -06:00
Tyera Eulberg ab581dafc2
Add block height to ConfirmedBlock structs (#17523)
* Add BlockHeight CF to blockstore

* Rename CacheBlockTimeService to be more general

* Cache block-height using service

* Fixup previous proto mishandling

* Add block_height to block structs

* Add block-height to solana block

* Fallback to BankForks if block time or block height are not yet written to Blockstore

* Add docs

* Review comments
2021-05-26 22:16:16 -06:00
Michael Vines 9541411c15 Plumb transaction-level rewards (aka "rent debits") into the `getTransaction` RPC method 2021-05-27 03:05:05 +00:00
François Garillot b08cff9e77
Simplify some pattern-matches (#16402)
When those match an exact combinator on Option / Result.

Tool-aided by [comby-rust](https://github.com/huitseeker/comby-rust).
2021-04-08 12:40:37 -06:00
Trent Nelson 7f7370c306 Re-allow clippy::integer_arithmetic at crate-level 2021-02-17 13:55:08 -07:00
Tyera Eulberg d1563f0ccd
Bump tonic, prost, tarpc, tokio (#15013)
* Update tonic & prost, and regenerate proto

* Reignore doc code

* Revert pull #14367, but pin tokio to v0.2 for jsonrpc

* Bump backoff and goauth -> and therefore tokio

* Bump tokio in faucet, net-utils

* Bump remaining tokio, plus tarpc
2021-02-05 00:21:53 -07:00
Michael Vines cbffab7850 Upgrade to Rust v1.49.0 2021-01-23 19:16:36 -08:00
Tyera Eulberg 71e9958e06
Rpc: Add custom error for BigTable data not found (#14762)
* Expose not-found bigtable error

* Add custom rpc error for bigtable data not found

* Return custom rpc error when bigtable block is not found

* Generalize long-term storage
2021-01-22 04:40:47 +00:00
Josh 1de6d28eaf
Add block_time to getConfirmedSignaturesForAddress2 and getConfirmedTransaction (#14572)
* add block_time to get_confirmed_signatures_for_address2 and protobuf implementation for tx_by_addr

* add tests for convert

* update cargo lock

* run cargo format after rebase

* introduce legacy TransactionByAddrInfo

* move LegacyTransactionByAddrInfo back to storage-bigtable
2021-01-20 22:10:35 -08:00
Ryo Onodera dcaa025822
Configure Bigtable's timeout, enabling by default (#14657)
* Configure bigtable's timeout when read-only

* Review comments

* Apply nits (thanks!)

Co-authored-by: Michael Vines <mvines@gmail.com>

* Timeout in the streamed decoding as well

Co-authored-by: Michael Vines <mvines@gmail.com>
2021-01-19 13:57:16 +00:00
Tyera Eulberg 2eb19fa5e5
Improve docs around bigtable read limit (#14660) 2021-01-19 06:33:51 +00:00
Tyera Eulberg 9a89689ad3
Use optimistic confirmation in getSignatureStatuses, and various downstream client methods (#14430)
* Add optimistically_confirmed field to TransactionStatus

* Update docs

* Convert new field to confirmation_status

* Update docs to confirmationStatus

* Update variants

* Update docs

* Just Confirmed
2021-01-15 16:05:05 +00:00
Michael Vines 7143aaa89b Clippy 2020-12-14 08:03:29 -08:00
Josh 13db3eca9f
SPL token balance in transaction metadata (#13673)
* feat: store pre / post token balances

* move helper functions into separate include

* move token balance functionality to transaction-status crate

* fix blockstore processor test

* fix bigtable legacy test

* add caching to decimals
2020-12-10 19:25:07 -08:00
Michael Vines 959880db60 Remove unused pubkey::Pubkey imports 2020-10-21 19:08:13 -07:00
Michael Vines 7bc073defe Run `codemod --extensions rs Pubkey::new_rand solana_sdk::pubkey::new_rand` 2020-10-21 19:08:13 -07:00
Tyera Eulberg 359707c85e
Convert Blockstore Rewards cf to protobuf (#12860)
* Add Blockstore protobuf cf type

* Add Rewards message to proto and make generated pub

* Convert Rewards cf to ProtobufColumn

* Add bench

* Adjust tags

* Move solana proto definitions and conversion methods to new crate
2020-10-15 18:04:10 -06:00
Tyera Eulberg 42943ab86d
Update get-block method in get_confirmed_transaction (#12923)
* Update get-block method in get_confirmed_transaction

* Remove superfluous into()
2020-10-15 13:56:32 -06:00
Tyera Eulberg 67ed44c007
Add log_messages to proto file (#12859) 2020-10-14 00:11:52 +00:00
Michael Vines c5c8da1ac0 Expose all rewards (fees, rent, voting and staking) in RPC getConfirmedBlock and the cli 2020-10-09 21:54:13 -07:00
Josh 8f5431551e
Store program logs in blockstore / bigtable (TransactionWithStatusMeta) (#12678)
* introduce store program logs in blockstore / bigtable

* fix test, transaction logs created for successful transactions

* fix test for legacy bincode implementation around log_messages

* only api nodes should record logs

* truncate transaction logs to 100KB

* refactor log truncate for improved coverage
2020-10-08 12:06:15 -07:00
Tyera Eulberg c31a34fbcb
Include post balance information for rewards (#12598)
* Include post balance information for rewards

* Add post-balance to stored Reward struct

* Handle extended Reward in bigtable

Co-authored-by: Michael Vines <mvines@gmail.com>
2020-09-30 17:57:06 -06:00
Justin Starry ce598c5c98
Use protobufs to store confirmed blocks in BigTable (#12526)
* Use protobufs to store confirmed blocks in BigTable

* Cleanup

* Reorganize proto

* Clean up use statements

* Split out function for unit testing

* s/utils/convert

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
2020-09-30 17:55:22 +00:00
Justin Starry 6601ec8f26
Record and store invoked instructions in transaction meta (#12311)
* Record invoked instructions and store in transaction meta

* Enable cpi recording if transaction sender is some

* Rename invoked to innerInstructions
2020-09-24 22:36:22 +08:00
Justin Starry 731a943239
Remove transaction encoding from storage layer (#12404) 2020-09-24 13:10:29 +08:00
Michael Vines 10ce839ec0 Speed up `bigtable transaction-history` command 2020-09-10 05:22:10 +00:00
Michael Vines 1db5c9d346 Include transaction index 2020-09-10 05:22:10 +00:00
Tyera Eulberg 879c98efeb
Bigtable bug fixes (#12058)
* Accommodate stricted get_bincode_cell in get_confirmed_signatures_for_address

* Sort signatures newest-oldest, even within slot
2020-09-04 20:29:11 -06:00
Tyera Eulberg b041afe1be
Bigtable method to return a single row of data (#11999) 2020-09-02 13:41:25 -06:00
Tyera Eulberg 55ce2ebd53
Get index (#11694) 2020-08-18 17:24:25 +00:00
Tyera Eulberg 6c5b8f324a
Rpc: Add until parameter for getConfirmedSignaturesForAddress2 (#11644)
* Refactor bigtable apis to accept start and end keys

* Make helper fn to deserialize cell data

* Refactor get_confirmed_signatures_for_address to use get_row_data range

* Add until param to get_confirmed_signatures_for_address

* Add until param to blockstore api

* Plumb until through client/cli

* Simplify client params
2020-08-15 10:42:17 -06:00
Tyera Eulberg 820af533a4
Bigtable: Use index to filter address-signatures correctly (#11622)
* Use index to filter address-signatures correctly

* Pull additional keys to account for filtered records

* Clarify variable name
2020-08-14 19:41:27 +00:00
Michael Vines 4222932e08 Plug getConfirmedSignaturesForAddress2 into bigtable storage 2020-08-05 23:04:16 -07:00