Commit Graph

5225 Commits

Author SHA1 Message Date
Elijah Hampton 49fca309cf
Adds logging of column family size and database size on startup and s… (#8336)
* Adds logging of column family size and database size on startup and shutdown

* Changes log level of column families size strings to debug. Adds TODO comment to use human_bytes crate for human-readable format of metrics. Adds print_db_metrics function to ZebraDb struct.

* Calls enumerate() on column_families to access index var

* Resolves cargo fmt checker results

* Resolves clippy lint

* Runs and fixes changes from fmt, clippy, check and test

* Removes prop.txt

* minor doc changes

---------

Co-authored-by: Elijah Hampton <elijahhampton@MBP-de-Elijah-2.lan>
Co-authored-by: Pili Guerra <mpguerra@users.noreply.github.com>
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
2024-03-29 00:00:28 +00:00
dependabot[bot] 7723736736
build(deps): bump the devops group with 2 updates (#8381)
Bumps the devops group with 2 updates: [codecov/codecov-action](https://github.com/codecov/codecov-action) and [tj-actions/changed-files](https://github.com/tj-actions/changed-files).


Updates `codecov/codecov-action` from 4.1.0 to 4.1.1
- [Release notes](https://github.com/codecov/codecov-action/releases)
- [Changelog](https://github.com/codecov/codecov-action/blob/main/CHANGELOG.md)
- [Commits](https://github.com/codecov/codecov-action/compare/v4.1.0...v4.1.1)

Updates `tj-actions/changed-files` from 43.0.0 to 44.0.0
- [Release notes](https://github.com/tj-actions/changed-files/releases)
- [Changelog](https://github.com/tj-actions/changed-files/blob/main/HISTORY.md)
- [Commits](https://github.com/tj-actions/changed-files/compare/v43.0.0...v44.0.0)

---
updated-dependencies:
- dependency-name: codecov/codecov-action
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: devops
- dependency-name: tj-actions/changed-files
  dependency-type: direct:production
  update-type: version-update:semver-major
  dependency-group: devops
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-03-27 15:59:30 +00:00
Arya c54fb84c97
makes some service requests in `getblock` method in parallel (#8376)
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
2024-03-26 11:04:13 +00:00
dependabot[bot] 24914edda9
build(deps): bump the devops group with 2 updates (#8369)
Bumps the devops group with 2 updates: [tj-actions/changed-files](https://github.com/tj-actions/changed-files) and [docker/build-push-action](https://github.com/docker/build-push-action).


Updates `tj-actions/changed-files` from 42.1.0 to 43.0.0
- [Release notes](https://github.com/tj-actions/changed-files/releases)
- [Changelog](https://github.com/tj-actions/changed-files/blob/main/HISTORY.md)
- [Commits](https://github.com/tj-actions/changed-files/compare/v42.1.0...v43.0.0)

Updates `docker/build-push-action` from 5.2.0 to 5.3.0
- [Release notes](https://github.com/docker/build-push-action/releases)
- [Commits](https://github.com/docker/build-push-action/compare/v5.2.0...v5.3.0)

---
updated-dependencies:
- dependency-name: tj-actions/changed-files
  dependency-type: direct:production
  update-type: version-update:semver-major
  dependency-group: devops
- dependency-name: docker/build-push-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: devops
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
2024-03-25 14:54:09 +00:00
Arya 949594c25a
fixes a new clippy warnings (#8373) 2024-03-25 12:57:08 +00:00
Marek f79fc6aa8e
Bump prod group (#8370) 2024-03-21 18:23:41 +00:00
idky137 4579722833
change(chain): Remove `Copy` trait impl from `Network` (#8354)
* removed derive copy from network, address and ledgerstate

* changed is_max_block_time_enforced to accept ref

* changed NetworkUpgrade::Current to accept ref

* changed NetworkUpgrade::Next to accept ref

* changed NetworkUpgrade::IsActivationHeight to accept ref

* changed NetworkUpgrade::TargetSpacingForHeight to accept ref

* changed NetworkUpgrade::TargetSpacings to accept ref

* changed NetworkUpgrade::MinimumDifficultySpacing_forHeight to accept ref

* changed NetworkUpgrade::IsTestnetMinDifficultyBlock to accept ref

* changed NetworkUpgrade::AveragingWindowTimespanForHeight to accept ref

* changed NetworkUpgrade::ActivationHeight to accept ref

* changed sapling_activation_height to accept ref

* fixed lifetime for target_spacings

* fixed sapling_activation_height

* changed transaction_to_fake_v5 and fake_v5_transactions_for_network to accept ref to network

* changed Input::vec_strategy to accept ref to network

* changed functions in zebra-chain/src/primitives/zcash_history.rs to accept ref to network

* changed functions in zebra-chain/src/history_tree.rs to accept ref to network

* changed functions in zebra-chain/src/history_tree.rs to accept ref to network

* changed functions in zebra-chain/src/primitives/address.rs to accept ref to network

* changed functions in zebra-chain/src/primitives/viewing_key* to accept ref to network

* changed functions in zebra-chain/src/transparent/address.rs to accept ref to network

* changed functions in zebra-chain/src/primitives/zcash_primitives.rs to accept ref to network

* changed functions in zebra-chain/src/primitives/zcash_note_encryption.rs to accept ref to network

* changed functions in zebra-chain/src/primitives/history_tree* to accept ref to network

* changed functions in zebra-chain/src/block* to accept ref to network

* fixed errors in zebra-chain::parameters::network

* fixed errors in zebra-chain::parameters::network

* fixed errors in zebra-chain

* changed NonEmptyHistoryTree and InnerHistoryTree to hold value instead of ref

* changed NonEmptyHistoryTree and InnerHistoryTree to hold value instead of ref

* fixed errors in zebra-chain/src/block/arbitrary.rs

* finished fixing errors in zebra-chain - all crate tests pass

* changed functions in zebra-state::service::finalized_state to accept &Network

* changed functions in zebra-state::service::non_finalized_state to accept &Network

* zebra-state tests run but fail with overflow error

* zebra-state tests all pass

* converted zebra-network -- all crate tests pass

* applied all requested changes from review

* converted zebra-consensus -- all crate tests pass

* converted zebra-scan -- all crate tests pass

* converted zebra-rpc -- all crate tests pass

* converted zebra-grpc -- all crate tests pass

* converted zebrad -- all crate tests pass

* applied all requested changes from review

* fixed all clippy errors

* fixed build error in zebrad/src/components/mempool/crawler.rs
2024-03-19 20:45:27 +00:00
dependabot[bot] 0c77b40ccc
build(deps): bump the devops group with 6 updates (#8347)
Bumps the devops group with 6 updates:

| Package | From | To |
| --- | --- | --- |
| [actions/checkout](https://github.com/actions/checkout) | `4.1.1` | `4.1.2` |
| [docker/login-action](https://github.com/docker/login-action) | `3.0.0` | `3.1.0` |
| [tj-actions/changed-files](https://github.com/tj-actions/changed-files) | `42.0.5` | `42.1.0` |
| [reviewdog/action-actionlint](https://github.com/reviewdog/action-actionlint) | `1.42.0` | `1.43.0` |
| [baptiste0928/cargo-install](https://github.com/baptiste0928/cargo-install) | `3.0.0` | `3.0.1` |
| [docker/build-push-action](https://github.com/docker/build-push-action) | `5.1.0` | `5.2.0` |


Updates `actions/checkout` from 4.1.1 to 4.1.2
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/v4.1.1...v4.1.2)

Updates `docker/login-action` from 3.0.0 to 3.1.0
- [Release notes](https://github.com/docker/login-action/releases)
- [Commits](https://github.com/docker/login-action/compare/v3.0.0...v3.1.0)

Updates `tj-actions/changed-files` from 42.0.5 to 42.1.0
- [Release notes](https://github.com/tj-actions/changed-files/releases)
- [Changelog](https://github.com/tj-actions/changed-files/blob/main/HISTORY.md)
- [Commits](https://github.com/tj-actions/changed-files/compare/v42.0.5...v42.1.0)

Updates `reviewdog/action-actionlint` from 1.42.0 to 1.43.0
- [Release notes](https://github.com/reviewdog/action-actionlint/releases)
- [Commits](https://github.com/reviewdog/action-actionlint/compare/v1.42.0...v1.43.0)

Updates `baptiste0928/cargo-install` from 3.0.0 to 3.0.1
- [Release notes](https://github.com/baptiste0928/cargo-install/releases)
- [Changelog](https://github.com/baptiste0928/cargo-install/blob/main/CHANGELOG.md)
- [Commits](https://github.com/baptiste0928/cargo-install/compare/v3.0.0...v3.0.1)

Updates `docker/build-push-action` from 5.1.0 to 5.2.0
- [Release notes](https://github.com/docker/build-push-action/releases)
- [Commits](https://github.com/docker/build-push-action/compare/v5.1.0...v5.2.0)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: devops
- dependency-name: docker/login-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: devops
- dependency-name: tj-actions/changed-files
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: devops
- dependency-name: reviewdog/action-actionlint
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: devops
- dependency-name: baptiste0928/cargo-install
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: devops
- dependency-name: docker/build-push-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: devops
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Gustavo Valverde <gustavo@iterativo.do>
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
2024-03-19 15:34:01 +00:00
Alfredo Garcia c22404f639
fix openapi now showing up in book summary (#8357) 2024-03-18 22:05:22 +00:00
Pili Guerra 5b9e88f547
Project management action is no longer necessary (#8353) 2024-03-15 10:55:27 +00:00
Arya 5e86ebb699
fix(scan): Improve gRPC method errors and add timeout to scan service to avoid hanging (#8318)
* updates ScanTask.subscribe() to return rsp_tx instead of awaiting it, returns service errors from grpc methods

* Adds a timeout to scan service and improves error message when subscribe responders are dropped

* Returns an error from `RegisterKeys` svc request when no keys are registered

* tests register keys error message

* Adds test for timeouts

* adds TODOs for fixing bugs

* Orders scan service requests in scan method

* removes CheckKeyHashes request

* fixes bad merge conflict resolution

* update test to check downcast to Elapsed error

* update errors in scan method

* update error msg in register_keys method

* fixes compiler error
2024-03-14 20:27:33 +00:00
Alfredo Garcia 652b4c3263
add openapiu.md to the book summary (#8351) 2024-03-14 16:32:53 +00:00
Alfredo Garcia f2be73eebb
feat(rpc): OpenAPI spec (#8342)
* add an openapi generator

* fix run command

Co-authored-by: Marek <mail@marek.onl>

---------

Co-authored-by: Marek <mail@marek.onl>
2024-03-14 15:04:19 +00:00
dependabot[bot] 5d6fe9134f
build(deps): bump mio from 0.8.10 to 0.8.11 (#8339)
Bumps [mio](https://github.com/tokio-rs/mio) from 0.8.10 to 0.8.11.
- [Release notes](https://github.com/tokio-rs/mio/releases)
- [Changelog](https://github.com/tokio-rs/mio/blob/master/CHANGELOG.md)
- [Commits](https://github.com/tokio-rs/mio/compare/v0.8.10...v0.8.11)

---
updated-dependencies:
- dependency-name: mio
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
2024-03-14 13:19:17 +00:00
Arya 00a3120c7b
minor/trivial cleanup (#8348) 2024-03-14 11:54:01 +00:00
Za Wilcox 3bef54b764
change(consensus): Refactor production code for network consensus rules to `Network` methods (#8340)
* begin refactor suggested as "step 2": https://github.com/ZcashFoundation/zebra/issues/7968#issue-2003245309
Squashed from multiple commits to enable partial rebase

* break out more little traits

* add activation implementation leveraging From<Network> for lrz::cons::

* for transfer of ownership I cannot return a type that's owned by the method

* hrp_sapling_extended_full_viewing_key

* complete implementation of interface of Parameters on Network reuse Parameters on zcash Network where possible

* move doc-comments to trait declarations (from impls)

* Simplify/complete Parameters impl for Network

* Add checkpoint_list method, move documentation, etc

* move last match network to inside network method

* add back comment

* use zcash_address for parameter types in zebra-chain

* use inherent methods instead of big parameters passthrough

* revert to implementation of From on zcash_primitives::..::Network vs &zcash_prim...

* move match

* add test to block maximum time rule

* update changelog

* finish porting target_difficutly_limit

* remove obscelete code comment

* revert non-functional change

* finish migrating target_difficulty_limit, checkpoint_list

* update changelog

---------

Co-authored-by: Hazel OHearn <gygaxis@zingolabs.org>
2024-03-12 21:41:44 +00:00
idky137 9b91d4bc0e
change(tests): Remove Matches on Network From Tests (#8295)
* added functions for fetching block vectors

* inserted new network methods for vector fetching into zebra-chain

* changed tag back to test/feature=branch

* changed tag back to test/feature=branch

* changed tag back to test/feature=branch

* changed feature tag to proptest-impl, started implementing in zebra-consensus tests

* adding methods to zebra-consensus, lifetime error in src/transaction/tests.rs, needs refactoring

* finished adding methods to zebra-consensus

* finished adding new methods to zebrad

* added new methods to zebra-rpc and zebra-scan

* finished removing statements matching on Network from tests

* updated new error message

* changed get_block_bytes() and get_block_sapling_roots_bytes to return option and removed serialization error types as per requested changes in PR review

* removed match statements from zebra_chain::transaction::arbitrary::test_transactions() and new zebra-grpc tests

* moved zebra-chain::test_utils to zebra-chain::test

* removed get_ prefix from getter methods

* renamed zebra-chain::test to zebra-chain::tests

* renamed zebra-chain::test to zebra-chain::tests

* fixed clippy warnings

* changed block_map to return clone
2024-03-05 09:12:25 -05:00
Arya e64f3dcddd
add(test): Check that scan task removes keys, unify scan task acceptance tests, and add test to CI (#8308)
* merges scan task tests and checks that the scan task removes keys

* Adds test to CI

* Skip to expected block height and assert that the result is from that height

* Updates test docs and uses zebrad state path for scan cache

* Log "Zcash network: {network}" from scan task commands test
2024-03-04 21:25:50 +00:00
dependabot[bot] 6b4d61feb5
build(deps): bump the devops group with 5 updates (#8334)
Bumps the devops group with 5 updates:

| Package | From | To |
| --- | --- | --- |
| [google-github-actions/auth](https://github.com/google-github-actions/auth) | `2.1.1` | `2.1.2` |
| [actions/add-to-project](https://github.com/actions/add-to-project) | `0.5.0` | `0.6.0` |
| [codecov/codecov-action](https://github.com/codecov/codecov-action) | `4.0.1` | `4.1.0` |
| [tj-actions/changed-files](https://github.com/tj-actions/changed-files) | `42.0.4` | `42.0.5` |
| [reviewdog/action-actionlint](https://github.com/reviewdog/action-actionlint) | `1.41.0` | `1.42.0` |


Updates `google-github-actions/auth` from 2.1.1 to 2.1.2
- [Release notes](https://github.com/google-github-actions/auth/releases)
- [Changelog](https://github.com/google-github-actions/auth/blob/main/CHANGELOG.md)
- [Commits](https://github.com/google-github-actions/auth/compare/v2.1.1...v2.1.2)

Updates `actions/add-to-project` from 0.5.0 to 0.6.0
- [Release notes](https://github.com/actions/add-to-project/releases)
- [Commits](https://github.com/actions/add-to-project/compare/v0.5.0...v0.6.0)

Updates `codecov/codecov-action` from 4.0.1 to 4.1.0
- [Release notes](https://github.com/codecov/codecov-action/releases)
- [Changelog](https://github.com/codecov/codecov-action/blob/main/CHANGELOG.md)
- [Commits](https://github.com/codecov/codecov-action/compare/v4.0.1...v4.1.0)

Updates `tj-actions/changed-files` from 42.0.4 to 42.0.5
- [Release notes](https://github.com/tj-actions/changed-files/releases)
- [Changelog](https://github.com/tj-actions/changed-files/blob/main/HISTORY.md)
- [Commits](https://github.com/tj-actions/changed-files/compare/v42.0.4...v42.0.5)

Updates `reviewdog/action-actionlint` from 1.41.0 to 1.42.0
- [Release notes](https://github.com/reviewdog/action-actionlint/releases)
- [Commits](https://github.com/reviewdog/action-actionlint/compare/v1.41.0...v1.42.0)

---
updated-dependencies:
- dependency-name: google-github-actions/auth
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: devops
- dependency-name: actions/add-to-project
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: devops
- dependency-name: codecov/codecov-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: devops
- dependency-name: tj-actions/changed-files
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: devops
- dependency-name: reviewdog/action-actionlint
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: devops
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-03-02 00:49:43 +00:00
Gustavo Valverde 56fca2c9ee
ref(build)!: use official lightwallted image for CI (#8323) 2024-02-29 15:50:50 +00:00
Marek 6a6f2865d3
Refactor creating GCP disks (#8330) 2024-02-29 14:46:29 +00:00
Marek 58bfe974e1
chore: Release v1.6.0 (#8319)
* Prepare the CHANGELOG for v1.6.0

* chore: Release

* Set `ESTIMATED_RELEASE_HEIGHT` to 2_413_000

* Revert "chore: Release"

This reverts commit 5261e85c37.

* chore: Release

* Fix release-crates-dry-run

* Temporarily turn off bumps for `zebra-{scan,grpc}`

* Apply suggestions from code review

Co-authored-by: Arya <aryasolhi@gmail.com>

* Re-enable test bump for `zebra-scan`

* Re-enable test bumps for `zebra-{scan, grpc}`

* Turn off dry-run bumps for `zebra-{scan, grpc}`

* Re-enable bumps and disable dry-run publishing

* Re-enable dry-run publishing

* Exclude `zebra-scan` from publishing

---------

Co-authored-by: Arya <aryasolhi@gmail.com>
2024-02-23 23:28:44 +00:00
Marek c2cfde0749
chore: Update docs for building Zebra (#8315)
* Update build instructions with scanning enabled

* Remove `pkg-config` from build deps in README

I tried compiling Zebra with `--all-features` on a Linux machine without
`pkg-config` or `pkfgconf` (which is an alternative to `pkg-config`),
and it worked.

* Add `protoc` to build deps in README
2024-02-23 16:49:53 +00:00
bishopcheckmate d1d128ac15
chore: allow crater runs (#8171)
* chore(dev-deps): always specify the version for dev-dependencies

* test(tower-batch-control): remove zebra-consensus dev dependency

Copies the Ed25519Verifier code directly to tower-batch-control tests

* test(zebra-consensus): update ed25519 verifier tests with ones from batch-control

* test(zebra-consensus): restore previous timeout values

The timeouts copied from tower-batch-control seems too low
and there must have been a reasoning why were they introduced

* chore: update dev-deps versions to beta.33

* chore(tower-batch-control): remove dev-dependency on metrics

* chore(tower-batch-control): remove zebra-chain dev-dependency

* Update zebra-scan/Cargo.toml

* bump all versions to match current release

* fix missed commas in version bumps

---------

Co-authored-by: Arya <aryasolhi@gmail.com>
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
Co-authored-by: Pili Guerra <mpguerra@users.noreply.github.com>
2024-02-23 12:47:10 +00:00
Arya 4ebd7a80d0
add(scan): Adds gRPC reflection and documents how to use the zebra-scan gRPC server (#8288)
* adds tonic-reflection

* adds listen_addr to log

* Adds user guide for scan server to zebra book

* fixes typo

* Applies suggestions from code review

* update link

Co-authored-by: Marek <mail@marek.onl>

---------

Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
Co-authored-by: Marek <mail@marek.onl>
2024-02-22 18:51:14 +00:00
Alfredo Garcia 790e4bd1b1
update dry-run release commands (#8298) 2024-02-22 12:41:28 +00:00
Marek 6b2768c021
chore: Update Checkpoints (#8309)
* Update Testnet checkpoints

* Update Mainnet checkpoints
2024-02-22 12:19:32 +00:00
Pili Guerra 9167093320
Update CONTRIBUTING.md (#8304)
We have deprecated the RFC process
2024-02-22 10:52:35 +00:00
Marek a6f8a9b192
Update `Cargo.lock` (#8310) 2024-02-22 01:20:05 +00:00
Gustavo Valverde d6314471b9
ref(docker): publish less tags in Docker Hub (#8300)
* ref(docker): publish less tags in Docker Hub

* fix: remove extra versions
2024-02-22 00:16:06 +00:00
Arya 5a9281a7a8
fix(scan): Fix minor concurrency bug in the `scan` gRPC method (#8303)
* update scan task to notify subscribe caller once the new subscribed_keys are sent on the watch channel

* Fixes timing bug in scan gRPC method:

Joins register/subscribe scan service calls, sends SubscribeResults request first, and filters out duplicate results from channel

* Removes outdated TODO

* wraps subscribed_keys in an Arc before sending to watch channel, fixes typo

* Remove result senders for keys that have been removed
2024-02-21 23:29:13 +00:00
dependabot[bot] c6b56b492a
build(deps): bump the devops group with 1 update (#8306)
Bumps the devops group with 1 update: [tj-actions/changed-files](https://github.com/tj-actions/changed-files).


Updates `tj-actions/changed-files` from 42.0.2 to 42.0.4
- [Release notes](https://github.com/tj-actions/changed-files/releases)
- [Changelog](https://github.com/tj-actions/changed-files/blob/main/HISTORY.md)
- [Commits](https://github.com/tj-actions/changed-files/compare/v42.0.2...v42.0.4)

---
updated-dependencies:
- dependency-name: tj-actions/changed-files
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: devops
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-02-21 20:40:55 +00:00
Gustavo Valverde 1a0d5ffec7
fix(ci): keep `gcloud` arguments consistency (#8305)
* fix(ci): keep `gcloud` arguments consistency

* Fix the syntax for setting `DISK_OPTION`

---------

Co-authored-by: Marek <mail@marek.onl>
2024-02-21 13:53:51 +00:00
Alfredo Garcia 2f0b284997
tests(grpc): Unit tests (#8293)
* add grpc method unit tests

* add test module description

* runs tests in sequence, adds some messages to assertions, and minor cleanups (#8296)

* fix field name change

---------

Co-authored-by: Arya <aryasolhi@gmail.com>
2024-02-20 22:45:55 +00:00
Arya a7de137a7a
add(scan): Implement `scan` gRPC method (#8268)
* Adds scan method

* fix clippy warning

* Skip empty results and don't panic if the gRPC client disconnects

* moves `Results` request above `SubscribeResults` request to avoid duplicate results.

* returns early if there's a send error when sending initial results and updates response type

* move responder buffer size to const

* Adds a snapshot test for the `scan` method

* updates snapshots
2024-02-20 19:16:20 +00:00
dependabot[bot] 3a67e81300
build(deps): bump the devops group with 1 update (#8270)
Bumps the devops group with 1 update: [shimataro/ssh-key-action](https://github.com/shimataro/ssh-key-action).


Updates `shimataro/ssh-key-action` from 2.6.1 to 2.7.0
- [Release notes](https://github.com/shimataro/ssh-key-action/releases)
- [Changelog](https://github.com/shimataro/ssh-key-action/blob/v2/CHANGELOG.md)
- [Commits](https://github.com/shimataro/ssh-key-action/compare/v2.6.1...v2.7.0)

---
updated-dependencies:
- dependency-name: shimataro/ssh-key-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: devops
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-02-20 10:40:46 +00:00
kai 6c0aa042ad
fix(docker): set log rotation to avoid docker bugs (#8269)
Containers can generate a significant amount of logs,
consuming disk space over time. Here I added configuration
for maximum log file size (max-size) and the number of retained
log files (max-file) in the logging driver options
to control disk usage and prevent log files from
growing indefinitely.
2024-02-20 10:40:38 +00:00
Marek 663c57700b
add(scan): Test the `RegisterKeys` scan service call (#8281)
* Impl generating continuous deserialized blocks

* Make `sapling_efvk_hrp` `pub`

* Don't wait for Sapling activation height in tests

* Set the sleep interval for scan service to 10 secs

* Simplify `sapling_key_to_scan_block_keys`

* Enable mocking Sapling scanning keys for Testnet

* Test the `RegisterKeys` scan service call

* Enable `shielded-scan` for `zebra-chain`

* Use an ephemeral database so results don't persist

* Don't generate blocks when mocking the state

* Improve error messages

* Simplify seeding mocked Sapling viewing keys

* Apply suggestions from code review

Co-authored-by: Arya <aryasolhi@gmail.com>

* Use a manual iterator over `Network`

---------

Co-authored-by: Arya <aryasolhi@gmail.com>
2024-02-19 23:45:38 +00:00
dependabot[bot] a8dcd98b6e
build(deps): bump the prod group with 3 updates (#8283)
Bumps the prod group with 3 updates: [clap](https://github.com/clap-rs/clap), [semver](https://github.com/dtolnay/semver) and [rocksdb](https://github.com/rust-rocksdb/rust-rocksdb).


Updates `clap` from 4.5.0 to 4.5.1
- [Release notes](https://github.com/clap-rs/clap/releases)
- [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md)
- [Commits](https://github.com/clap-rs/clap/compare/clap_complete-v4.5.0...clap_complete-v4.5.1)

Updates `semver` from 1.0.21 to 1.0.22
- [Release notes](https://github.com/dtolnay/semver/releases)
- [Commits](https://github.com/dtolnay/semver/compare/1.0.21...1.0.22)

Updates `rocksdb` from 0.21.0 to 0.22.0
- [Release notes](https://github.com/rust-rocksdb/rust-rocksdb/releases)
- [Changelog](https://github.com/rust-rocksdb/rust-rocksdb/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-rocksdb/rust-rocksdb/compare/v0.21.0...v0.22.0)

---
updated-dependencies:
- dependency-name: clap
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: prod
- dependency-name: semver
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: prod
- dependency-name: rocksdb
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: prod
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-02-19 18:17:34 +00:00
Marek 96c0d42d74
Add the `experimental_allow_proto3_optional` flag (#8282) 2024-02-19 13:09:52 +00:00
Alfredo Garcia fbddec3cb3
test(grpc): Add snapshots (#8277)
* add grpc snapshot tests

* replaces ScanService with MockService in snapshot tests

* removes dev-dep in zebra-grpc on zebra-scan and updates snapshots

* Apply suggestions from code review

Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>

---------

Co-authored-by: Arya <aryasolhi@gmail.com>
2024-02-16 19:41:04 +00:00
Arya f20834c373
improve error message in non_blocking_logger test (#8276) 2024-02-15 22:46:04 +00:00
Marek 85a7cc977a
add(scan): Implement gRPC support for registering viewing keys for scanning (#8266)
* Use `use tonic::Request` for consistency

* Add definitions of `proto` types

* Impl `register_keys` gRPC

* Fix a typo

---------

Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
2024-02-15 18:02:32 +00:00
Marek 584bd23718
Update Debian from Bullseye to Bookworm (#8273) 2024-02-14 23:02:06 +00:00
dependabot[bot] f342f50938
build(deps): bump the prod group with 14 updates (#8264)
* build(deps): bump the prod group with 14 updates

Bumps the prod group with 14 updates:

| Package | From | To |
| --- | --- | --- |
| [clap](https://github.com/clap-rs/clap) | `4.4.18` | `4.5.0` |
| [chrono](https://github.com/chronotope/chrono) | `0.4.33` | `0.4.34` |
| [indexmap](https://github.com/indexmap-rs/indexmap) | `2.2.2` | `2.2.3` |
| [toml](https://github.com/toml-rs/toml) | `0.8.9` | `0.8.10` |
| [thiserror](https://github.com/dtolnay/thiserror) | `1.0.56` | `1.0.57` |
| [metrics](https://github.com/metrics-rs/metrics) | `0.22.0` | `0.22.1` |
| [num-integer](https://github.com/rust-num/num-integer) | `0.1.45` | `0.1.46` |
| [metrics-exporter-prometheus](https://github.com/metrics-rs/metrics) | `0.13.0` | `0.13.1` |
| [indicatif](https://github.com/console-rs/indicatif) | `0.17.7` | `0.17.8` |
| [tempfile](https://github.com/Stebalien/tempfile) | `3.9.0` | `3.10.0` |
| [tonic](https://github.com/hyperium/tonic) | `0.10.2` | `0.11.0` |
| [tonic-build](https://github.com/hyperium/tonic) | `0.10.2` | `0.11.0` |
| [x25519-dalek](https://github.com/dalek-cryptography/curve25519-dalek) | `2.0.0` | `2.0.1` |
| [serde_with](https://github.com/jonasbb/serde_with) | `3.6.0` | `3.6.1` |


Updates `clap` from 4.4.18 to 4.5.0
- [Release notes](https://github.com/clap-rs/clap/releases)
- [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md)
- [Commits](https://github.com/clap-rs/clap/compare/v4.4.18...clap_complete-v4.5.0)

Updates `chrono` from 0.4.33 to 0.4.34
- [Release notes](https://github.com/chronotope/chrono/releases)
- [Changelog](https://github.com/chronotope/chrono/blob/main/CHANGELOG.md)
- [Commits](https://github.com/chronotope/chrono/compare/v0.4.33...v0.4.34)

Updates `indexmap` from 2.2.2 to 2.2.3
- [Changelog](https://github.com/indexmap-rs/indexmap/blob/master/RELEASES.md)
- [Commits](https://github.com/indexmap-rs/indexmap/compare/2.2.2...2.2.3)

Updates `toml` from 0.8.9 to 0.8.10
- [Commits](https://github.com/toml-rs/toml/compare/toml-v0.8.9...toml-v0.8.10)

Updates `thiserror` from 1.0.56 to 1.0.57
- [Release notes](https://github.com/dtolnay/thiserror/releases)
- [Commits](https://github.com/dtolnay/thiserror/compare/1.0.56...1.0.57)

Updates `metrics` from 0.22.0 to 0.22.1
- [Changelog](https://github.com/metrics-rs/metrics/blob/main/release.toml)
- [Commits](https://github.com/metrics-rs/metrics/compare/metrics-v0.22.0...metrics-v0.22.1)

Updates `num-integer` from 0.1.45 to 0.1.46
- [Changelog](https://github.com/rust-num/num-integer/blob/master/RELEASES.md)
- [Commits](https://github.com/rust-num/num-integer/compare/num-integer-0.1.45...num-integer-0.1.46)

Updates `metrics-exporter-prometheus` from 0.13.0 to 0.13.1
- [Changelog](https://github.com/metrics-rs/metrics/blob/main/release.toml)
- [Commits](https://github.com/metrics-rs/metrics/compare/metrics-exporter-prometheus-v0.13.0...metrics-v0.13.1)

Updates `indicatif` from 0.17.7 to 0.17.8
- [Release notes](https://github.com/console-rs/indicatif/releases)
- [Commits](https://github.com/console-rs/indicatif/commits)

Updates `tempfile` from 3.9.0 to 3.10.0
- [Changelog](https://github.com/Stebalien/tempfile/blob/master/CHANGELOG.md)
- [Commits](https://github.com/Stebalien/tempfile/compare/v3.9.0...v3.10.0)

Updates `tonic` from 0.10.2 to 0.11.0
- [Changelog](https://github.com/hyperium/tonic/blob/master/CHANGELOG.md)
- [Commits](https://github.com/hyperium/tonic/compare/v0.10.2...v0.11.0)

Updates `tonic-build` from 0.10.2 to 0.11.0
- [Changelog](https://github.com/hyperium/tonic/blob/master/CHANGELOG.md)
- [Commits](https://github.com/hyperium/tonic/compare/v0.10.2...v0.11.0)

Updates `x25519-dalek` from 2.0.0 to 2.0.1
- [Release notes](https://github.com/dalek-cryptography/curve25519-dalek/releases)
- [Commits](https://github.com/dalek-cryptography/curve25519-dalek/compare/2.0.0...x25519-2.0.1)

Updates `serde_with` from 3.6.0 to 3.6.1
- [Release notes](https://github.com/jonasbb/serde_with/releases)
- [Commits](https://github.com/jonasbb/serde_with/compare/v3.6.0...v3.6.1)

---
updated-dependencies:
- dependency-name: clap
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: prod
- dependency-name: chrono
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: prod
- dependency-name: indexmap
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: prod
- dependency-name: toml
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: prod
- dependency-name: thiserror
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: prod
- dependency-name: metrics
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: prod
- dependency-name: num-integer
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: prod
- dependency-name: metrics-exporter-prometheus
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: prod
- dependency-name: indicatif
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: prod
- dependency-name: tempfile
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: prod
- dependency-name: tonic
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: prod
- dependency-name: tonic-build
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: prod
- dependency-name: x25519-dalek
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: prod
- dependency-name: serde_with
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: prod
...

Signed-off-by: dependabot[bot] <support@github.com>

* update deny.toml

* update deny.toml missing dup

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
2024-02-14 12:24:33 +00:00
Arya 3929a526e5
add(scan): Implement SubscribeResults request for scan service (#8253)
* processes SubscribeResults messages

* send tx ids of results to the subscribe channel

* replaces BoxError with Report in scan_range

* adds a watch channel for using subscribed_keys in scan_range

* updates args to process_messages in test

* adds a `subscribe` method to ScanTask for sending a SubscribeResults cmd

* updates test for process_messages to cover subscribe cmds

* impls SubscribeResult service request and updates sender type

* adds test for SubscribeResults scan service request

* adds acceptance test

* updates tests and imports

* fixes acceptance test by using spawn_blocking to avoid blocking async executor and setting an appropriate start height

* fixes test

* Applies suggestions from code review.

* use tokio mpsc channel in scan task instead of std/blocking mpsc

* use tokio mpsc channel for results sender

* adds `was_parsed_keys_empty` instead of checking that all the parsed keys are new keys

* fixes test failures related to send errors in scan task

* returns height and key for scan results from subcribe_results results receiver

* hide scan_service mod in zebra-node-service behind feature
2024-02-13 00:42:40 +00:00
dependabot[bot] c69befda2f
build(deps): bump the cargo group across 1 directories with 3 updates (#8263)
Bumps the cargo group with 3 updates in the /. directory: [h2](https://github.com/hyperium/h2), [libgit2-sys](https://github.com/rust-lang/git2-rs) and [shlex](https://github.com/comex/rust-shlex).


Updates `h2` from 0.3.23 to 0.3.24
- [Release notes](https://github.com/hyperium/h2/releases)
- [Changelog](https://github.com/hyperium/h2/blob/v0.3.24/CHANGELOG.md)
- [Commits](https://github.com/hyperium/h2/compare/v0.3.23...v0.3.24)

Updates `libgit2-sys` from 0.16.1+1.7.1 to 0.16.2+1.7.2
- [Changelog](https://github.com/rust-lang/git2-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/git2-rs/commits)

Updates `shlex` from 1.2.0 to 1.3.0
- [Changelog](https://github.com/comex/rust-shlex/blob/master/CHANGELOG.md)
- [Commits](https://github.com/comex/rust-shlex/commits)

---
updated-dependencies:
- dependency-name: h2
  dependency-type: indirect
  dependency-group: cargo-security-group
- dependency-name: libgit2-sys
  dependency-type: indirect
  dependency-group: cargo-security-group
- dependency-name: shlex
  dependency-type: indirect
  dependency-group: cargo-security-group
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-02-12 21:42:42 +00:00
Alfredo Garcia d59c7305b0
add(grpc): `get_results` (#8255)
* add `get_results` grpc call

* nitpick
2024-02-09 23:41:26 +00:00
Marek 6b8cbf9904
add(scan): Implement `RegisterKeys` service request to register new keys (#8251)
* Refactor obtaining of activation heights

* Impl the `RegisterKeys` service request

* Mock viewing keys for tests

* Refactor tests

* Apply suggestions from code review

Co-authored-by: Arya <aryasolhi@gmail.com>

* Remove a redundant comment

I don't think we need to assume the genesis block doesn't contain
shielded data as the comment says.

* Avoid using a single-letter variable

* Refactor mocking Sapling scanning keys

---------

Co-authored-by: Arya <aryasolhi@gmail.com>
2024-02-09 15:23:19 +00:00
dependabot[bot] 0c2c421bec
build(deps): bump the devops group with 4 updates (#8246)
Bumps the devops group with 4 updates: [google-github-actions/auth](https://github.com/google-github-actions/auth), [codecov/codecov-action](https://github.com/codecov/codecov-action), [baptiste0928/cargo-install](https://github.com/baptiste0928/cargo-install) and [release-drafter/release-drafter](https://github.com/release-drafter/release-drafter).


Updates `google-github-actions/auth` from 2.1.0 to 2.1.1
- [Release notes](https://github.com/google-github-actions/auth/releases)
- [Changelog](https://github.com/google-github-actions/auth/blob/main/CHANGELOG.md)
- [Commits](https://github.com/google-github-actions/auth/compare/v2.1.0...v2.1.1)

Updates `codecov/codecov-action` from 3.1.6 to 4.0.1
- [Release notes](https://github.com/codecov/codecov-action/releases)
- [Changelog](https://github.com/codecov/codecov-action/blob/main/CHANGELOG.md)
- [Commits](https://github.com/codecov/codecov-action/compare/v3.1.6...v4.0.1)

Updates `baptiste0928/cargo-install` from 2.2.0 to 3.0.0
- [Release notes](https://github.com/baptiste0928/cargo-install/releases)
- [Changelog](https://github.com/baptiste0928/cargo-install/blob/main/CHANGELOG.md)
- [Commits](https://github.com/baptiste0928/cargo-install/compare/v2.2.0...v3.0.0)

Updates `release-drafter/release-drafter` from 5 to 6
- [Release notes](https://github.com/release-drafter/release-drafter/releases)
- [Commits](https://github.com/release-drafter/release-drafter/compare/v5...v6)

---
updated-dependencies:
- dependency-name: google-github-actions/auth
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: devops
- dependency-name: codecov/codecov-action
  dependency-type: direct:production
  update-type: version-update:semver-major
  dependency-group: devops
- dependency-name: baptiste0928/cargo-install
  dependency-type: direct:production
  update-type: version-update:semver-major
  dependency-group: devops
- dependency-name: release-drafter/release-drafter
  dependency-type: direct:production
  update-type: version-update:semver-major
  dependency-group: devops
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-02-09 12:08:31 +00:00