Commit Graph

4277 Commits

Author SHA1 Message Date
Conrado Gouvea ff0122044f
change(doc): add transaction index diagram to RFC-0005 (#4330)
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-05-06 03:32:55 +00:00
Janito Vaqueiro Ferreira Filho 9538ad29e5
feat(util): add a `zebra-tip-height` utility (#4289)
* Move `init_tracing` to `lib.rs`

Allow it to be reused by other binaries.

* Fix `hex` dependency inclusion in `zebra-chain`

It would previously fail because `hex` was being included without the
`serde` feature, even though we required that for `transparent::Script`.

* Implement `FromStr` for `Network`

Make it easy to receive it as a command-line parameter.

* Add `zebra-tip-height` utility

Obtains the chain tip height of a directory containing Zebra state.

* Remove Tokio dependency from `zebra-utils`

It wasn't actually used by the `zebra-tip-height` utility.

* Remove `BoxStateService` type alias

It's not needed if we don't return the unused state service. This also
allows removing the `tower` dependency.

* Remove unnecessary attribute

Leftover from copied code.

* Make `cache_dir` argument optional

Fallback to the default Zebra state cache directory.

* Remove added newline

Minor formatting fix, to avoid adding an unnecessary newline.

* Move `tip-height` command into `zebrad`

Make it available as a sub-command in `zebrad`.

* Make some zebrad sub-commands only log warnings

Co-authored-by: teor <teor@riseup.net>
2022-05-06 03:31:52 +00:00
dependabot[bot] f3f26ff85f
build(deps): bump tonic from 0.7.1 to 0.7.2 (#4325)
Bumps [tonic](https://github.com/hyperium/tonic) from 0.7.1 to 0.7.2.
- [Release notes](https://github.com/hyperium/tonic/releases)
- [Changelog](https://github.com/hyperium/tonic/blob/master/CHANGELOG.md)
- [Commits](https://github.com/hyperium/tonic/compare/v0.7.1...v0.7.2)

---
updated-dependencies:
- dependency-name: tonic
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-05-06 03:31:38 +00:00
dependabot[bot] 6d9f7befc0
build(deps): bump tonic-build from 0.7.1 to 0.7.2 (#4326)
Bumps [tonic-build](https://github.com/hyperium/tonic) from 0.7.1 to 0.7.2.
- [Release notes](https://github.com/hyperium/tonic/releases)
- [Changelog](https://github.com/hyperium/tonic/blob/master/CHANGELOG.md)
- [Commits](https://github.com/hyperium/tonic/compare/v0.7.1...v0.7.2)

---
updated-dependencies:
- dependency-name: tonic-build
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-05-06 03:31:23 +00:00
teor b008509a9e
Skip tests when doing a manual full sycn (#4333)
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-05-06 02:08:41 +00:00
Janito Vaqueiro Ferreira Filho a2ebc914ac
Update `crossbeam-channel` to `0.5.4` (#4331)
The previously locked version (`0.5.2`) was yanked from `crates.io`.

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-05-06 00:35:40 +00:00
teor da232ec224
Update cached state conditions and docs (#4332) 2022-05-06 07:52:56 +10:00
dependabot[bot] 89fe4d73fc
build(deps): bump prost from 0.10.1 to 0.10.2 (#4321)
Bumps [prost](https://github.com/tokio-rs/prost) from 0.10.1 to 0.10.2.
- [Release notes](https://github.com/tokio-rs/prost/releases)
- [Commits](https://github.com/tokio-rs/prost/compare/v0.10.1...v0.10.2)

---
updated-dependencies:
- dependency-name: prost
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-05-05 19:56:26 +00:00
Gustavo Valverde 322f68d81a
fix(ci): run tests according to the right conditions (#4313) 2022-05-05 18:04:51 +00:00
dependabot[bot] f6692d9154
build(deps): bump num-integer from 0.1.44 to 0.1.45 (#4260)
Bumps [num-integer](https://github.com/rust-num/num-integer) from 0.1.44 to 0.1.45.
- [Release notes](https://github.com/rust-num/num-integer/releases)
- [Changelog](https://github.com/rust-num/num-integer/blob/master/RELEASES.md)
- [Commits](https://github.com/rust-num/num-integer/compare/num-integer-0.1.44...num-integer-0.1.45)

---
updated-dependencies:
- dependency-name: num-integer
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-05-05 12:44:00 +00:00
Gustavo Valverde 55a73c554c
fix(lint): stop actionlint from failing in main (#4317) 2022-05-05 07:18:27 -04:00
Gustavo Valverde e72260db52
fix(ci): add cached state version to disk images (#4314) 2022-05-05 07:10:50 -04:00
Gustavo Valverde 9856e2ad17
refactor(lint): check specific files for each job (#4311)
* refactor(lint): check specific files for each job

* refactor(lint): use an approach which requires less code

* fix(lint): validate against true string not boolean
2022-05-05 07:09:21 -04:00
Gustavo Valverde a23420be88
feat(ci): add `fully_synced_rpc_test` test to CI (#4223)
* feat(ci): add lightwalletd RPC call test

Co-authored-by: teor <teor@riseup.net>
2022-05-05 05:27:07 -04:00
teor d720ab011a
fix(test): make the full sync tests cache state at `/zebrad-cache` (#4308)
* Rewrite the full sync tests to cache at /zebrad-cache

* Improve error reporting from cached state tests
2022-05-05 05:25:46 -04:00
teor 7cc8f7e745
Fix confusing Rust "environmental variable name" variable names (#4301)
```sh
fastmod _VAR "" zebra*
```
2022-05-05 05:59:31 +00:00
teor 5cd1133584
change(test): Stop requiring cached lightwalletd state for the send transaction test (#4302)
* Expect a Zebra cached state with at least a million blocks

* Set the zebrad timeout and failure messages based on the test type

* Temporarily stop requiring cached lightwalletd state for the send transaction tests
2022-05-05 04:44:14 +00:00
Alfredo Garcia d0e81001bc
fix(rpc): Use a structure for parameters of getaddresstxids (#4264)
* add info to lightwalletd_state_path()

* fix getaddresstxids rpc

* normalize more the env vars in lightwalletd tests

* extra logging

Co-authored-by: teor <teor@riseup.net>

Co-authored-by: teor <teor@riseup.net>
2022-05-05 01:08:27 +00:00
dependabot[bot] 32056e0e5c
build(deps): bump thiserror from 1.0.30 to 1.0.31 (#4259)
Bumps [thiserror](https://github.com/dtolnay/thiserror) from 1.0.30 to 1.0.31.
- [Release notes](https://github.com/dtolnay/thiserror/releases)
- [Commits](https://github.com/dtolnay/thiserror/compare/1.0.30...1.0.31)

---
updated-dependencies:
- dependency-name: thiserror
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-05-05 01:08:06 +00:00
Janito Vaqueiro Ferreira Filho 79d58285fb
Increase block validation timeouts (#4156)
* Increase UTXO lookup timeout

Avoid block validation failures because UTXOs aren't available on time.

* Increase the block verification timeout

Attempt to reduce the synchronization restarts and consequently improve
performance.

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-05-05 00:01:12 +00:00
Gustavo Valverde 9c94c0d6e4
feat(ci)!: add a reusable workflow for deployable integration tests (#4271)
Co-authored-by: teor <teor@riseup.net>
2022-05-04 18:55:02 -04:00
teor 56f766f9b8
fix(sync): fix testnet syncer loop on large Orchard blocks (#4286)
* Return BlockDownloadVerifyError from download_and_verify

* Check block requests and genesis for temporary errors

* Ignore DuplicateBlockQueuedForDownload as a temporary error

* Propagate error info to the syncer main loop

* Sleep after temporary genesis download and verify errors
2022-05-04 22:04:34 +00:00
dependabot[bot] e9d37c6275
build(deps): bump serde_json from 1.0.80 to 1.0.81 (#4290)
Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.80 to 1.0.81.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.80...v1.0.81)

---
updated-dependencies:
- dependency-name: serde_json
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-05-04 17:19:59 +00:00
Alfredo Garcia 25712f1935
tests(rpc): Add wallet grpc tests (#4253)
* add a test that call all the lightwalletd grpc methods

* fix some docs

* use ZF address in tests for balance and utxos

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-05-04 04:00:46 +00:00
dependabot[bot] 2e643c8631
build(deps): bump serde from 1.0.136 to 1.0.137 (#4261)
Bumps [serde](https://github.com/serde-rs/serde) from 1.0.136 to 1.0.137.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.136...v1.0.137)

---
updated-dependencies:
- dependency-name: serde
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-05-04 04:00:30 +00:00
dependabot[bot] 1fe66685cb
build(deps): bump serde_json from 1.0.79 to 1.0.80 (#4262)
Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.79 to 1.0.80.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.79...v1.0.80)

---
updated-dependencies:
- dependency-name: serde_json
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-05-03 23:34:58 +00:00
dependabot[bot] 47f144205c
build(deps): bump tokio from 1.18.0 to 1.18.1 (#4280)
Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.18.0 to 1.18.1.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.18.0...tokio-1.18.1)

---
updated-dependencies:
- dependency-name: tokio
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-05-03 20:55:43 +00:00
dependabot[bot] 88533d9c40
build(deps): bump semver from 1.0.7 to 1.0.9 (#4258)
Bumps [semver](https://github.com/dtolnay/semver) from 1.0.7 to 1.0.9.
- [Release notes](https://github.com/dtolnay/semver/releases)
- [Commits](https://github.com/dtolnay/semver/compare/1.0.7...1.0.9)

---
updated-dependencies:
- dependency-name: semver
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-05-03 20:55:20 +00:00
Deirdre Connolly 239230989d
feat(ci): clarify lint workflow names (#4279)
* Lint workflow names

* Rename lint.patch.yml display name to Lint, add stub for Actionlint
2022-05-03 17:17:58 +00:00
Gustavo Valverde 8b68651ba8
fix(build): avoid docker cache contamination and invalidation (#4254)
* imp(build): reduce docker cache invalidation

Use scoped caching and more file ignores to reduce cache invalidation

* fix(build): add entrypoint.sh as a required file

* fix(build): do not logout if the build takes too long

* Add 'doc comment' about .dockerignore

Co-authored-by: Deirdre Connolly <durumcrustulum@gmail.com>
2022-05-03 10:13:28 -04:00
Gustavo Valverde 93f40fd0ba
fix(ci): typo causing docker run command to split (#4277) 2022-05-03 08:00:10 -04:00
Gustavo Valverde 6331722c12
fix(ci): use debian for faster mounting, and increase timeout (#4276) 2022-05-03 06:49:24 -04:00
teor be4e065afb
change(rpc): Simplify RPC types and add documentation (#4218)
* Simplify RPC types and add documentation

* Derive serde traits in production code
2022-05-03 04:10:21 +00:00
Gustavo Valverde 9b9578c999
refactor(ci): use docker instead of Konlet for GCP deployments in CI (#4252)
* refactor(ci): use docker in docker

This is a workaround for an issue related to disk partitioning, caused by a GCP service called Konlet, while mounting the cached disks to the VM and then to the container

* fix(build): persist docker login credentials

* fix(ci): get sync height from docker logs instead of gcp

* try: use gha cache for faster building

* fix(ci): mount disk in container to make it available in vm

* fix(build): do not invalidate cache between images

* try(docker): invalidate cache as less as possible

* fix(ci): GHA terminal is not a TTY

* fix(build): do not ignore entrypoint.sh

* fix

* fix(ci): mount using root priveleges

* fix(ci): use existing disk as cached state

* fix(ci): wait for disks to get mounted

* force rebuild

* fix failed force

* fix(ci): some tests does not use a cached state

* fix(ci): do not name boot and attached disk the same

* fix(ci): attach a disk to full sync, to snapshot the state

* fix(ci): use appropiate grep text depending on the test

* reduce diff

* fix(ci): use correct GCP disk source attribute

* imp(ci): reduce diff

* fix(ci): revert wrong deletion

* fix: revert uneeded changes

* fix: reduce main diff

* fix

* fix(ci): reduce diff
2022-05-03 02:47:04 +00:00
Gustavo Valverde fe97e14fe8
fix(ci): create a full sync disk with cached state inside (#4266) 2022-05-02 21:28:33 -04:00
dependabot[bot] 6dc8bea3f5
build(deps): bump tonic-build from 0.7.0 to 0.7.1 (#4249)
Bumps [tonic-build](https://github.com/hyperium/tonic) from 0.7.0 to 0.7.1.
- [Release notes](https://github.com/hyperium/tonic/releases)
- [Changelog](https://github.com/hyperium/tonic/blob/master/CHANGELOG.md)
- [Commits](https://github.com/hyperium/tonic/compare/v0.7.0...v0.7.1)

---
updated-dependencies:
- dependency-name: tonic-build
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-05-02 22:58:12 +00:00
Gustavo Valverde b3bccd6655
fix(ci): garbage collect instances no matter previous steps status (#4255)
* fix(ci): garbage collect instances no matter the status

As we're not going to reuse test instances, the safest method to apply is to always delete this instances if they fail, get skipped or succeed running a workflow

* Apply suggestions from code review

Co-authored-by: Deirdre Connolly <durumcrustulum@gmail.com>

* docs(ci): imrpove comment

Co-authored-by: Deirdre Connolly <durumcrustulum@gmail.com>
2022-05-01 16:52:57 +00:00
teor 2d783f3138
test(rpc): Add a script for comparing zcashd and zebrad RPC responses (#4219)
* Add a tool for comparing zcashd and zebrad RPC responses

* Add the network to the RPC diff logs

This helps diagnose main/test mismatches.

* Warn when heights or networks are different

* Move diff script to utils

* Document zcash-rpc-diff script

* Rename a confusing internal script env var
2022-05-01 15:45:32 +00:00
teor 86b252aad7
fix(state): Tweak ephemeral state directory path and improve docs (#4248)
* Fix state directory and docs

* Explain state versions better, and fix a docs bug

* fix docs

Co-authored-by: Marek <mail@marek.onl>

* add extra info to doc

Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
Co-authored-by: Marek <mail@marek.onl>
2022-04-29 22:02:09 +00:00
teor 59bdab17bf
test(rpc): Add Rust tests for lightwalletd sync from Zebra (#4177)
* Make the lightwalletd integration test take a test type

* Configure lightwalletd tests based on the test type

* Remove obsolete kill_on_error() in the lightwalletd test

* Refactor to simplify the test function

* Move LightwalletdTestType to the lightwalletd module

* Create a test function that runs the full lightwalletd test suite

* Actually use the cached Zebra state

* Add checks for the new integration test modes

* Populate the lightwalletd state dir in the FullSyncFromGenesis test

* Fix up state handling, fail earlier if state is invalid

* Adjust timeouts and regex escapes

* Make state requirements for each test stricter

* Move configs to the top of the test function

* Allow unexpected lightwalletd cached state in some tests

* Speed up tests slightly by removing an intermittent log check

* Move timeout selection into test type methods

* Move failure messages into test type methods

* Turn a function argument into an enum field

* Check lightwalletd state directly, rather than Zebra RPC results

* Update gRPC tests for function argument changes

* Remove duplicate env var constant and redundant code
2022-04-29 19:56:11 +00:00
teor 23ff00b246
Increase deploy docs job timeout (#4246) 2022-04-28 22:01:23 +00:00
dependabot[bot] a922688947
build(deps): bump docker/metadata-action from 3.7.0 to 3.8.0 (#4241)
Bumps [docker/metadata-action](https://github.com/docker/metadata-action) from 3.7.0 to 3.8.0.
- [Release notes](https://github.com/docker/metadata-action/releases)
- [Commits](https://github.com/docker/metadata-action/compare/v3.7.0...v3.8.0)

---
updated-dependencies:
- dependency-name: docker/metadata-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-04-28 17:54:48 +00:00
dependabot[bot] 1691d9e855
build(deps): bump tokio from 1.17.0 to 1.18.0 (#4242)
Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.17.0 to 1.18.0.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.17.0...tokio-1.18.0)

---
updated-dependencies:
- dependency-name: tokio
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-04-28 17:54:33 +00:00
dependabot[bot] 646bedbdb6
build(deps): bump tj-actions/changed-files from 18.7 to 19 (#4240)
Bumps [tj-actions/changed-files](https://github.com/tj-actions/changed-files) from 18.7 to 19.
- [Release notes](https://github.com/tj-actions/changed-files/releases)
- [Changelog](https://github.com/tj-actions/changed-files/blob/main/HISTORY.md)
- [Commits](https://github.com/tj-actions/changed-files/compare/v18.7...v19)

---
updated-dependencies:
- dependency-name: tj-actions/changed-files
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-04-28 17:54:28 +00:00
teor d38eb66eca
Decrease the peer handshake timeout to 3 seconds (#4212) 2022-04-28 17:32:16 +00:00
teor d021f50df1
Fix zcash-params Dockerfile build failure (#4235) 2022-04-28 11:55:36 +00:00
teor 7506655774
fix(rpc): Fix some RPC response formats to match `zcashd` (#4217)
* Match `zcashd`'s version format in `getinfo`

* Remove an incorrect array wrapper in getaddressbalance

* Use a stable sort order in getrawmempool

Because we can't match zcashd's arbitrary order,
and probably shouldn't try to.

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-04-28 10:19:02 +00:00
Gustavo Valverde 83d26890f5
feat(ci)!: implement reusable workflows for image building (#4173)
* refactor(ci): test building in a separate workflow

* force a change

* force a change

* fix(ci): send the correct variables to the reusable build

* fix(ci): variables are not allowed

* fix(ci): conditions are not allowed as input

* fix(ci): use expected value

* refactor(build): simplify the use of other dockerfiles

* fix(cd): depend on docker build yml

* fix(cd): use main branch as image name

* imp(actions): remove uneeded variable repetition

* imp(build): remove unused variables

* imp(actions): rename the image building workflow

Not all images are for zebra execution as we also have one for zcash-params

* fix(ci): add dependable workflow in paths filters

* docs(ci): remove TODO as this won't be needed at least an issue arises

* docs(ci): CARGO_INCREMENTAL can decrease build time when running from a cache

* fix: revert forced changes

* fix(build): remove unused build inputs in zcash-params

* imp(cd): as this is the production image, use the executable name

* imp(ci): reduce log level to improve speed

Co-authored-by: teor <teor@riseup.net>

* imp(ci): use the correct name for the workflow

Co-authored-by: teor <teor@riseup.net>

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: teor <teor@riseup.net>
2022-04-28 08:13:35 +00:00
dependabot[bot] 597f55319d
build(deps): bump spandoc from 0.2.1 to 0.2.2 (#4221)
Bumps [spandoc](https://github.com/yaahc/spandoc) from 0.2.1 to 0.2.2.
- [Release notes](https://github.com/yaahc/spandoc/releases)
- [Changelog](https://github.com/yaahc/spandoc/blob/master/CHANGELOG.md)
- [Commits](https://github.com/yaahc/spandoc/compare/v0.2.1...v0.2.2)

---
updated-dependencies:
- dependency-name: spandoc
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-04-28 07:08:30 +00:00
teor 1f2837c86e
Add missing application/json headers to RPC requests (#4215)
This enables compatibility with `zcash-cli`.
2022-04-28 00:57:13 +00:00