Commit Graph

80 Commits

Author SHA1 Message Date
behzad nouri 9ee53e594d
patches clippy errors from new rust nightly release (#28028) 2022-09-23 20:57:27 +00:00
dependabot[bot] e22842f5db
chore: bump enum-iterator from 0.8.1 to 1.2.0 (#27496)
* chore: bump enum-iterator from 0.8.1 to 1.2.0

Bumps [enum-iterator](https://github.com/stephaneyfx/enum-iterator) from 0.8.1 to 1.2.0.
- [Release notes](https://github.com/stephaneyfx/enum-iterator/releases)
- [Commits](https://github.com/stephaneyfx/enum-iterator/compare/0.8.1...1.2.0)

---
updated-dependencies:
- dependency-name: enum-iterator
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

* Make versions consistent, and a-z toml

* Accommodate new apis

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
2022-08-31 15:07:29 -06:00
Tyera Eulberg 7a11571e7f
Update BigTable apis to respect a limit of zero (#27380)
Update apis to respect a limit of zero
2022-08-24 12:44:19 -06:00
apfitzge caca525695
ignore clippy derive_partial_eq_without_eq in prost generated code (#27337)
ignore clippy derive_partial_eq_without_eq in prost generated code
2022-08-23 14:21:24 -05:00
Richard Patel 270315a7f6
transaction-status, storage-proto: add compute_units_consumed (#26528)
* transaction-status, storage-proto: add compute_units_consumed

* fix bpf test

Co-authored-by: Justin Starry <justin@solana.com>
2022-08-06 17:14:31 +00:00
Kirill Fomichev 0c98b06249
bigtable: set_nodelay for http connector (#26398) 2022-07-25 15:16:55 -06:00
Kirill Fomichev 90d9118048
Add `get_confirmed_transactions` to `storage-bigtable` (#25404)
* Add get_confirmed_transactions to storage-bigtable

* remove zip

* HashMap::new instead of default

* extract txs in order
2022-07-05 22:52:11 +00:00
lwus d12e566a4c
bigtable: allow forward proxy in gRPC connection (#25918)
* bigtable: allow forward proxy in tonic connection

* bigtable: try fixing ci

* bigtable: cargo fmt

* bigtable: add docs for BIGTABLE_PROXY env var

* bigtable: fix readme header level

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
2022-06-23 10:51:31 -06:00
Arthur Burkart 49648c16c4
Add metric for tracking Bigtable block uploads (#26027) 2022-06-21 18:27:02 -06:00
Dan Coombs 32a58dd9e0
Add ability to use a non-default app profile id in bigtable requests (#25968)
* Add ability to use a non-default app profile id in bigtable requests

* Only run subcommand once when getting global configs

* Remove unneded scoping on option type
2022-06-16 00:58:16 +00:00
Michael Vines b05c7d91ed Fix derive_partial_eq_without_eq clippy lint 2022-05-22 22:22:21 -07:00
buffalu 6bcadc755e
Speedup bigtable block upload by factor of 8-10x (#24534)
Added multiple blockstore read threads.
Run the bigtable upload in tokio::spawn context.
Run bigtable tx and tx-by-addr uploads in tokio::spawn context.
2022-05-17 00:21:05 -06:00
dependabot[bot] 4520c25330
chore: bump tonic-build from 0.7.1 to 0.7.2 (#25037)
* chore: bump tonic-build from 0.7.1 to 0.7.2

Bumps [tonic-build](https://github.com/hyperium/tonic) from 0.7.1 to 0.7.2.
- [Release notes](https://github.com/hyperium/tonic/releases)
- [Changelog](https://github.com/hyperium/tonic/blob/master/CHANGELOG.md)
- [Commits](https://github.com/hyperium/tonic/compare/v0.7.1...v0.7.2)

---
updated-dependencies:
- dependency-name: tonic-build
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

* Regenerate proto

* Bump tonic and remove deprecated method

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
2022-05-09 12:26:58 -06:00
Rachael Pai aea17c35ae
Add a stringified credential option for LedgerStorage (#24314)
* add a stringified credential option for LedgerStorage

* fix clippy::useless-format warning

* change CredentialOption to enum CredentialType

* rename credential_option to credential_type

* restore LedgerStorage new fn signature

* fmt

Co-authored-by: Tyera Eulberg <tyera@solana.com>
2022-04-13 14:35:06 -06:00
Edgar Xi d8be0d9430 make get_protobuf_or_bincode_cells accept IntoIter on row_keys, make get_confirmed_blocks_with_data return an Iterator 2022-03-22 22:47:25 -06:00
Edgar Xi f717fda9a3 modify get_protobuf_or_bincode_cells to accept and return an iterator 2022-03-22 22:47:25 -06:00
Edgar Xi fbcf6a0802 use &[T] instead of Vec<T> where appropriate
clippy
2022-03-22 22:47:25 -06:00
Edgar Xi 5533e9393c appease clippy 2022-03-22 22:47:25 -06:00
Edgar Xi f3219fb695 add get_confirmed_blocks_with_data and get_protobuf_or_bincode_cells 2022-03-22 22:47:25 -06:00
Jon Cinque 7af48465fa
transaction-status: Add return data to meta (#23688)
* transaction-status: Add return data to meta

* Add return data to simulation results

* Use pretty-hex for printing return data

* Update arg name, make TransactionRecord struct

* Rename TransactionRecord -> ExecutionRecord
2022-03-22 23:17:05 +01:00
Trent Nelson 9b32b72990 bigtable: allow custom instance names 2022-03-21 11:26:49 -06:00
Trent Nelson f513195468 bigtable: add a config ctor for `LedgerStorage` 2022-03-21 11:26:49 -06:00
Justin Starry d5dec989b9
Enforce tx metadata upload with static types (#23028) 2022-02-10 13:28:18 +08:00
Justin Starry ba215e94f6
Refactor: Add AccountKeys struct for static and dynamic message keys (#22960) 2022-02-05 20:00:31 +08:00
Justin Starry f804ccdece
Store address table lookups in blockstore and bigtable (#22402) 2022-01-14 15:24:41 +08:00
pieceofr d9220652ad
[ledger-tool]compare_blocks (#22229)
* 1.made load_credentials accept credential path as a parameter. 2.partial implement bigtable comparasion function

* finding missing blocks in bigtables in a specified range

* refactor compare-blocks,add unit test for missing_blocks and fmt

* compare-block fix last block bug

* refactor compare-block and improve wording

* Update ledger-tool/src/bigtable.rs

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>

* update compare-block command-line description

* style:improve wording/naming/code style

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
2022-01-06 14:36:03 +08:00
Justin Starry 2a00382d71
Refactor: cleanup solana_transaction_status crate (#22230) 2022-01-03 15:45:18 +00:00
Michael Vines b8837c04ec Reformat imports to a consistent style for imports
rustfmt.toml configuration:
  imports_granularity = "One"
  group_imports = "One"
2021-12-03 09:19:13 -08:00
Tyera Eulberg f3f8d2e4f3
Add BigTable query logs and counter (#21394)
* Add LedgerStorage logs

* Add storage-bigtable metric
2021-11-22 16:22:46 -07:00
Tyera Eulberg f0de3e9bf0
chore: bump tonic from 0.5.2 to 0.6.1 (#21035)
* Bump tonic and prost crates

* connect_lazy no longer infallible
2021-10-28 06:59:36 +00:00
Ryo Onodera 795dde109c
Fix memo handling and "blocks" key formatting (#20044)
* Fix memo handling and "blocks" key formatting

* Skip memo equality check

* Define slot_to_tx_by_addr_key

Co-authored-by: Justin Starry <justin@solana.com>
2021-09-22 00:46:46 +09:00
Justin Starry c71fab6cb3
Add `delete` subcommand to `ledger-tool bigtable` (#19931)
* Add `delete` subcommand to `ledger-tool bigtable` command

* feedback
2021-09-16 23:37:45 +00:00
Tyera Eulberg f4ae450f34
Populate memo in bigtable transaction structs (#19512)
* Populate memo in bigtable transaction structs

* Preface memos with len
2021-08-30 21:14:18 -06:00
Michael Vines 4098af3b5b Record vote account commission with voting/staking rewards and surface in RPC 2021-07-12 15:09:44 -07:00
Tyera Eulberg c2e7d39154
Add storage-proto build.rs and readme (#18353)
* Use build.rs for storage-proto generation

* Add readme

* Single use statements
2021-07-09 20:06:06 +00:00
Tyera Eulberg 761de8b1a3
Bump prost, prost-types, and tonic (#18537)
* Bump prost+tonic and accommodate generated service changes

* Unignore advisory

* Fixup .proto error list
2021-07-09 17:31:26 +00:00
dependabot[bot] 0426c2d28e
chore: bump bzip2 from 0.3.3 to 0.4.3 (#18361)
* chore: bump bzip2 from 0.3.3 to 0.4.3

Bumps [bzip2](https://github.com/alexcrichton/bzip2-rs) from 0.3.3 to 0.4.3.
- [Release notes](https://github.com/alexcrichton/bzip2-rs/releases)
- [Commits](https://github.com/alexcrichton/bzip2-rs/commits)

---
updated-dependencies:
- dependency-name: bzip2
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

* Update to new api

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
2021-07-01 13:10:52 -06:00
Michael Vines 4a12c715a3 Drop Error suffix from enum values to avoid the enum_variant_names clippy lint 2021-06-18 23:02:13 +00:00
Alexander Meißner 6514096a67 chore: cargo +nightly clippy --fix -Z unstable-options 2021-06-18 10:42:46 -07:00
Tyera Eulberg c57d1b44ef
Properly handle block_height in Bigtable bincode deserialization (#17990)
* Default block_height on eof

* Add comment to prevent future errors
2021-06-15 17:34:07 -06:00
Tyera Eulberg ab581dafc2
Add block height to ConfirmedBlock structs (#17523)
* Add BlockHeight CF to blockstore

* Rename CacheBlockTimeService to be more general

* Cache block-height using service

* Fixup previous proto mishandling

* Add block_height to block structs

* Add block-height to solana block

* Fallback to BankForks if block time or block height are not yet written to Blockstore

* Add docs

* Review comments
2021-05-26 22:16:16 -06:00
Michael Vines 9541411c15 Plumb transaction-level rewards (aka "rent debits") into the `getTransaction` RPC method 2021-05-27 03:05:05 +00:00
François Garillot b08cff9e77
Simplify some pattern-matches (#16402)
When those match an exact combinator on Option / Result.

Tool-aided by [comby-rust](https://github.com/huitseeker/comby-rust).
2021-04-08 12:40:37 -06:00
Trent Nelson 7f7370c306 Re-allow clippy::integer_arithmetic at crate-level 2021-02-17 13:55:08 -07:00
Tyera Eulberg d1563f0ccd
Bump tonic, prost, tarpc, tokio (#15013)
* Update tonic & prost, and regenerate proto

* Reignore doc code

* Revert pull #14367, but pin tokio to v0.2 for jsonrpc

* Bump backoff and goauth -> and therefore tokio

* Bump tokio in faucet, net-utils

* Bump remaining tokio, plus tarpc
2021-02-05 00:21:53 -07:00
Michael Vines cbffab7850 Upgrade to Rust v1.49.0 2021-01-23 19:16:36 -08:00
Tyera Eulberg 71e9958e06
Rpc: Add custom error for BigTable data not found (#14762)
* Expose not-found bigtable error

* Add custom rpc error for bigtable data not found

* Return custom rpc error when bigtable block is not found

* Generalize long-term storage
2021-01-22 04:40:47 +00:00
Josh 1de6d28eaf
Add block_time to getConfirmedSignaturesForAddress2 and getConfirmedTransaction (#14572)
* add block_time to get_confirmed_signatures_for_address2 and protobuf implementation for tx_by_addr

* add tests for convert

* update cargo lock

* run cargo format after rebase

* introduce legacy TransactionByAddrInfo

* move LegacyTransactionByAddrInfo back to storage-bigtable
2021-01-20 22:10:35 -08:00
Ryo Onodera dcaa025822
Configure Bigtable's timeout, enabling by default (#14657)
* Configure bigtable's timeout when read-only

* Review comments

* Apply nits (thanks!)

Co-authored-by: Michael Vines <mvines@gmail.com>

* Timeout in the streamed decoding as well

Co-authored-by: Michael Vines <mvines@gmail.com>
2021-01-19 13:57:16 +00:00
Tyera Eulberg 2eb19fa5e5
Improve docs around bigtable read limit (#14660) 2021-01-19 06:33:51 +00:00