Commit Graph

252 Commits

Author SHA1 Message Date
teor f7202bfbc0
Download Zcash Sapling parameters and load them from cached files (#3057)
* Replace Zcash parameters crates with pre-downloaded local parameter files

* Download Zcash parameters using the `zcashd` script in CI and Docker

* Add a zcash_proofs dependency to zebra-consensus

* Download Sapling parameters using zcash_proofs, rather than fetch-params.sh

* Add a new `zebrad download` subcommand

This command isn't required for nomrmal usage.
But it's useful when testing, or launching multiple Zebra instances.

* Use `zebrad download` in CI to pre-download parameters

* Log a helpful hint if downloading fails

* Allow some duplicate dependencies currently hidden by orchard

* Spawn a separate task to download Groth16 parameters

* Run the parameter download with code coverage

This avoids re-compining Zebra with and without coverage.

* Update Cargo.lock after rebase

* Try to pass `download` as an argument to `zebrad` in coverage CI

* Fix copy and paste comment typos

* Add path and download examples, like zcash_proofs

* Download params in CI just like zcash_proofs does

* Delete a redundant build step

* Implement graceful shutdown for zebrad start

* Send coverage summary to /dev/null when getting the params path

* Use the correct parameters path and download commands in CI

* Explain pre-downloads

* Avoid calling params_folder twice

* Rename parameter types and methods for consistency

```sh
fastmod SaplingParams SaplingParameters zebra*
fastmod Groth16Params Groth16Parameters zebra*
fastmod PARAMS GROTH16_PARAMETERS zebra*
fastmod params_folder directory zebra*
```

And a manual variable name tweak.

* rustfmt

* Remove a redundant coverage step

Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>
2021-11-19 23:02:56 +00:00
dependabot[bot] bdf5f70557
Bump actions/checkout from 2.3.5 to 2.4.0 (#3009)
Bumps [actions/checkout](https://github.com/actions/checkout) from 2.3.5 to 2.4.0.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/v2.3.5...v2.4.0)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: teor <teor@riseup.net>
2021-11-05 15:08:39 +10:00
Janito Vaqueiro Ferreira Filho 0960e4fb0b
Update to Tokio 1.13.0 (#2994)
* Update `tower` to version `0.4.9`

Update to latest version to add support for Tokio version 1.

* Replace usage of `ServiceExt::ready_and`

It was deprecated in favor of `ServiceExt::ready`.

* Update Tokio dependency to version `1.13.0`

This will break the build because the code isn't ready for the update,
but future commits will fix the issues.

* Replace import of `tokio::stream::StreamExt`

Use `futures::stream::StreamExt` instead, because newer versions of
Tokio don't have the `stream` feature.

* Use `IntervalStream` in `zebra-network`

In newer versions of Tokio `Interval` doesn't implement `Stream`, so the
wrapper types from `tokio-stream` have to be used instead.

* Use `IntervalStream` in `inventory_registry`

In newer versions of Tokio the `Interval` type doesn't implement
`Stream`, so `tokio_stream::wrappers::IntervalStream` has to be used
instead.

* Use `BroadcastStream` in `inventory_registry`

In newer versions of Tokio `broadcast::Receiver` doesn't implement
`Stream`, so `tokio_stream::wrappers::BroadcastStream` instead. This
also requires changing the error type that is used.

* Handle `Semaphore::acquire` error in `tower-batch`

Newer versions of Tokio can return an error if the semaphore is closed.
This shouldn't happen in `tower-batch` because the semaphore is never
closed.

* Handle `Semaphore::acquire` error in `zebrad` test

On newer versions of Tokio `Semaphore::acquire` can return an error if
the semaphore is closed. This shouldn't happen in the test because the
semaphore is never closed.

* Update some `zebra-network` dependencies

Use versions compatible with Tokio version 1.

* Upgrade Hyper to version 0.14

Use a version that supports Tokio version 1.

* Update `metrics` dependency to version 0.17

And also update the `metrics-exporter-prometheus` to version 0.6.1.
These updates are to make sure Tokio 1 is supported.

* Use `f64` as the histogram data type

`u64` isn't supported as the histogram data type in newer versions of
`metrics`.

* Update the initialization of the metrics component

Make it compatible with the new version of `metrics`.

* Simplify build version counter

Remove all constants and use the new `metrics::incement_counter!` macro.

* Change metrics output line to match on

The snapshot string isn't included in the newer version of
`metrics-exporter-prometheus`.

* Update `sentry` to version 0.23.0

Use a version compatible with Tokio version 1.

* Remove usage of `TracingIntegration`

This seems to not be available from `sentry-tracing` anymore, so it
needs to be replaced.

* Add sentry layer to tracing initialization

This seems like the replacement for `TracingIntegration`.

* Remove unnecessary conversion

Suggested by a Clippy lint.

* Update Cargo lock file

Apply all of the updates to dependencies.

* Ban duplicate tokio dependencies

Also ban git sources for tokio dependencies.

* Stop allowing sentry-tracing git repository in `deny.toml`

* Allow remaining duplicates after the tokio upgrade

* Use C: drive for CI build output on Windows

GitHub Actions uses a Windows image with two disk drives, and the
default D: drive is smaller than the C: drive. Zebra currently uses a
lot of space to build, so it has to use the C: drive to avoid CI build
failures because of insufficient space.

Co-authored-by: teor <teor@riseup.net>
2021-11-02 18:46:57 +00:00
Pili Guerra 989ba51cb9
Update release-checklist.md (#3001)
Include potential changes to the README in the release checklist

Co-authored-by: Conrado Gouvea <conrado@zfnd.org>
2021-11-02 15:36:47 +00:00
teor 9963471b7c
Check for duplicate crate dependencies in CI (#2986)
* Add default deny.toml for "cargo deny check bans"

`cargo deny init`

* Delete unused "cargo deny" config entries

Also cleanup trailing whitespace.

* Deny duplicate crates and unexpected crate sources

Allow the current set of duplicates and sources,
with references to the tickets that will fix them.

* Check for duplicate dependencies in CI

Also check for:
- unexpected crate sources
- outdated Cargo.lock
  (required for accurate duplicate and source checks)

* Revert CI name changes so required statuses pass

* Fix ticket for sentry-tracing
2021-11-01 21:19:22 +00:00
Deirdre Connolly 0381c2347b
ZIP-401: weighted random mempool eviction (#2889)
* ZIP-401 weighted random mempool eviction

* rename zcash.mempool.total_cost.bytes to zcash.mempool.cost.bytes

Co-authored-by: teor <teor@riseup.net>

* Remove duplicated lines

* Add cost() method to UnminedTx

Update serialization failure messages

* More docs quoting ZIP-401 rules

* Change mempool::Storage::new() to handle Copy-less HashMap, HashSet

* mempool: tidy cost types and evict_one()

* More consensus rule docs

* Refactor calculating mempool costs for Unmined transactions

* Add a note on asympotic performance of calculating weights of txs in mempool

* Bump test mempool / storage config to avoid weighted random cost limits

* Use mempool tx_cost_limit = u64::MAX for some tests

* Remove failing tests for now

* Allow(clippy::field-reassign-with-default) because of a move on a type that doesn't impl Copy

* Fix mistaken doctest formatting

Co-authored-by: Conrado Gouvea <conrado@zfnd.org>

* Increase test timeout for Windows builds

Co-authored-by: teor <teor@riseup.net>
Co-authored-by: Conrado Gouvea <conrado@zfnd.org>
2021-10-27 00:21:19 +00:00
Deirdre Connolly 52357f7522
Remove dconnolly as default assignee and reviewer on dependabot PRs, and make them low priority (#2951)
* Remove dconnolly as default assignee and reviewer on dependabot PRs

* Assign dependabot PRs low priority by default

Co-authored-by: teor <teor@riseup.net>
2021-10-26 00:22:51 +00:00
teor 35290db052
Only run fake activation heights tests on zebra-state (#2929)
Also only run the zebrad acceptance tests on macOS.

Re-running the compiler and test binaries for unused crates is slow in CI.

Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>
2021-10-21 23:51:20 +00:00
Marek ffa6fed7b3
Improve the template for release versioning (#2906)
* Specify the location of the user agent string

* Specify the `cargo install` command

* Fix a typo

Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>
2021-10-20 18:02:07 +00:00
dependabot[bot] 0b25257bef
Bump actions/checkout from 2.3.4 to 2.3.5 (#2898)
Bumps [actions/checkout](https://github.com/actions/checkout) from 2.3.4 to 2.3.5.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/v2.3.4...v2.3.5)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: teor <teor@riseup.net>
2021-10-19 04:03:10 +00:00
dependabot[bot] d307d43e19 Bump codecov/codecov-action from 2.0.3 to 2.1.0
Bumps [codecov/codecov-action](https://github.com/codecov/codecov-action) from 2.0.3 to 2.1.0.
- [Release notes](https://github.com/codecov/codecov-action/releases)
- [Changelog](https://github.com/codecov/codecov-action/blob/master/CHANGELOG.md)
- [Commits](https://github.com/codecov/codecov-action/compare/v2.0.3...v2.1.0)

---
updated-dependencies:
- dependency-name: codecov/codecov-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-09-13 15:14:19 -04:00
Deirdre Connolly 0e60936ad3
Restore coverage (#2684)
* Update coverage workflow

* Remove cache layer, up timeout to 60 minutes anyway
2021-08-27 11:03:40 -04:00
Alfredo Garcia ac76d35cbb
Make some small changes to release-checklist.md (#2671)
* Update release-checklist.md

* give all alternatives to check what changed

Co-authored-by: teor <teor@riseup.net>
2021-08-26 22:45:19 +00:00
Conrado Gouvea bc4194fcb9
ZIP-221/244 auth data commitment validation in checkpoint verifier (#2633)
* Add validation of ZIP-221 and ZIP-244 commitments

* Apply suggestions from code review

Co-authored-by: teor <teor@riseup.net>

* Add auth commitment check in the finalized state

* Reset the verifier when comitting to state fails

* Add explanation comment

* Add test with fake activation heights

* Add generate_valid_commitments flag

* Enable fake activation heights using env var instead of feature

* Also update initial_tip_hash; refactor into progress_from_tip()

* Improve comments

* Add fake activation heights test to CI

* Fix bug that caused commitment trees to not match when generating partial arbitrary chains

* Add ChainHistoryBlockTxAuthCommitmentHash::from_commitments to organize and deduplicate code

* Remove stale comment, improve readability

* Allow overriding with PROPTEST_CASES

* partial_chain_strategy(): don't update note commitment trees when not needed; add comment

Co-authored-by: teor <teor@riseup.net>
2021-08-23 14:17:33 +00:00
Conrado Gouvea f6416ff8ee
Update to cache v10 (#2644) 2021-08-19 17:33:22 +00:00
Conrado Gouvea 4014e0fec8
Attempt to fix test.yml (#2635)
* Only use -t flag to docker run, set SSH keep alive

* Remove SSH flag for now

* Add ssh flag back to test.yml gcloud compute ssh command

Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>
2021-08-16 19:02:39 -04:00
Conrado Gouvea 32e1237df8
Update cache to v9 (#2598)
* Update cache to v9

* Update state disk image

* Rename

* Remove ending semicolon for remote commands

Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>
2021-08-11 17:11:59 -04:00
Deirdre Connolly 21d71a651e
Merge pull request #2601 from ZcashFoundation/dconnolly-patch-1
Sync to, not past, activation for regenerate-stateful-test-disks.yml
2021-08-11 12:15:22 -04:00
teor e2a3a38047
Disable Rust beta tests in CI, due to a rust bug (#2542) 2021-07-29 13:02:12 +10:00
teor 5684667a31
Update to google-github-actions/setup-gcloud in workflows (#2533)
* Update to google-github-actions/setup-gcloud in workflows

See https://github.com/google-github-actions/setup-gcloud#use-google-github-actionssetup-gcloud

* Also update the Google Cloud test workflow

* Also update regenerate stateful test disks

* Also update Manual Deploy

* Also update zcashd deploy
2021-07-27 12:13:04 -04:00
teor c8305fba1f
Stop using the wrong name for the coverage action (#2453)
Previously the action was called "CI", which could be confusing.
2021-07-07 11:36:54 -03:00
Pili Guerra 515dc4bf5c
Update versions for Zebra v1.0.0 alpha.12 release (#2415)
* Update versions for zebra v1.0.0-alpha.12 release

* Update Cargo.lock

* Update release checklist with latest version changes to help keep track for future releases

* Remove reference to the fact that tower-fallback was not updated
2021-07-01 08:59:32 +01:00
teor 4c321f6fa1 Skip IPv6 network tests in google cloud workflows
Also consistently use the zebra-test:latest docker image.
2021-06-28 23:34:57 -04:00
teor efb2ffbf33 Stop ignoring failed cached state tests
Previously, Zebra's cached state workflows would run all of Zebra's
tests, but they would ignore the results for most tests. They would only
fail if the mainnet cached state test failed.

After this fix, the tests fail if any test or build step fails.
2021-06-28 23:34:57 -04:00
teor 17e94bd241
Stop skipping the cached sync tests in CI (#2402)
This is a bugfix on PR #2314, which changed the name of the test
function in the code, but didn't change the CI workflow.
2021-06-29 12:49:42 +10:00
teor 22f2f06bad
Remove "solution" from the ticket template (#2385)
We want developers to do detailed designs and solutions in
RFCs and code comments, rather than tickets.
2021-06-24 10:43:27 +01:00
Pili Guerra e87933e167
Add CHANGELOG.md file to the zebra git repo (#2346)
* Add new CHANGELOG.md file to zebra git repo

* Update Release Checklist to add updates to CHANGELOG.md

* Add some explanation about the CHANGELOG.md file

* Fix headings to make them consistent with Keep a changelog format

* Small fix for clarity

* Add release dates to changelog

* Change order of steps to update the changelog
2021-06-22 14:31:30 +01:00
Alfredo Garcia 544d182d25
Add and use a function for mandatory checkpoint (#2314)
* add `mandatory_checkpoint_height()` function

* use mandatory checkpoint instead of canopy in acceptance tests
2021-06-18 10:05:28 +10:00
Deirdre Connolly ea15ad1131
Use Swatinem/rust-cache@v1 (#2291) 2021-06-15 11:36:33 +10:00
Deirdre Connolly 28c3a23f14
Replace bespoke source-based coverage config with cargo-llvm-cov (#2286)
* Replace bespoke source-based coverage config with cargo-llvm-cov

* Keep incremental builds disabled

* Use cargo-llvm-cov ^0.1.0-alpha.4

Co-authored-by: teor <teor@riseup.net>
2021-06-15 11:35:26 +10:00
teor 0b4f32a324 Maybe GitHub actions doesn't like the variable substitution 2021-06-14 20:24:37 -04:00
teor 130e348b0f Use a different name for clippy job and results 2021-06-14 20:24:37 -04:00
teor 0a34d506cf CI: Remove duplicate Clippy job name
Naming the job and the task makes it show up twice
2021-06-14 20:24:37 -04:00
Deirdre Connolly 71b41f0206 cargo install mdbook 2021-06-14 17:12:31 -04:00
Deirdre Connolly 45cbfa8e67 Upgrade mdbook to 0.4.10 2021-06-14 17:12:31 -04:00
teor 53e50821ec Remove outdated pinned nightly in coverage workflow
nightly-2021-03-24 is incompatible with semver 1.0.2.
2021-06-09 16:10:47 -04:00
dependabot[bot] aac0b5d60f build(deps): bump codecov/codecov-action from 1.5.0 to 1.5.2
Bumps [codecov/codecov-action](https://github.com/codecov/codecov-action) from 1.5.0 to 1.5.2.
- [Release notes](https://github.com/codecov/codecov-action/releases)
- [Changelog](https://github.com/codecov/codecov-action/blob/master/CHANGELOG.md)
- [Commits](https://github.com/codecov/codecov-action/compare/v1.5.0...v1.5.2)

---
updated-dependencies:
- dependency-name: codecov/codecov-action
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-06-09 15:55:36 -04:00
Deirdre Connolly 8b5b367c8a
Create workflow to regenerate cached state disks for tests (#2247)
* Create workflow to regenerate cached state disks for tests

* Mount testnet disk
2021-06-07 13:14:30 +10:00
Deirdre Connolly 584f35ce68 Update test job to use the updated state 2021-06-04 15:14:11 -04:00
teor 3d2fb1c13e
Update GitHub and RFC templates based on retrospectives (#2242)
Updates:
- GitHub Issue templates
- GitHub PR templates
- RFC template

Focusing on:
- consensus rule / network reference sections
- design sections
- review/test checklist

Process changes:
- add new team members to RFC approval
- change RFC approval to "most of the team"

And general cleanup:
- delete docs from the checklist, because we now `warn(missing_docs)`
- shorter explanations
- consistent headings
- consistent order
- consistent formatting
2021-06-04 14:40:53 +10:00
teor dd7747bc69
Security: Remove checkout credentials from GitHub actions (#2158)
* Remove checkout credentials from CD action
* Remove checkout credentials from CI action
* Remove checkout credentials from coverage action
* Remove checkout credentials from docs action
* Remove checkout credentials from manual deploy action
* Remove checkout credentials from test action
* Remove checkout credentials from zcashd action
2021-05-17 09:32:38 +10:00
dependabot[bot] 2ea1d8624d
build(deps): bump codecov/codecov-action from 1 to 1.5.0 (#2142)
Bumps [codecov/codecov-action](https://github.com/codecov/codecov-action) from 1 to 1.5.0.
- [Release notes](https://github.com/codecov/codecov-action/releases)
- [Changelog](https://github.com/codecov/codecov-action/blob/master/CHANGELOG.md)
- [Commits](https://github.com/codecov/codecov-action/compare/v1...v1.5.0)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-05-12 22:12:49 +00:00
dependabot[bot] 300ce12a29
build(deps): bump actions-rs/cargo from 1 to 1.0.3 (#2140)
Bumps [actions-rs/cargo](https://github.com/actions-rs/cargo) from 1 to 1.0.3.
- [Release notes](https://github.com/actions-rs/cargo/releases)
- [Changelog](https://github.com/actions-rs/cargo/blob/master/CHANGELOG.md)
- [Commits](https://github.com/actions-rs/cargo/compare/v1...v1.0.3)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-05-13 07:54:02 +10:00
dependabot[bot] 48657506c5
build(deps): bump actions-rs/clippy-check from 1 to 1.0.7 (#2141)
Bumps [actions-rs/clippy-check](https://github.com/actions-rs/clippy-check) from 1 to 1.0.7.
- [Release notes](https://github.com/actions-rs/clippy-check/releases)
- [Changelog](https://github.com/actions-rs/clippy-check/blob/master/CHANGELOG.md)
- [Commits](https://github.com/actions-rs/clippy-check/compare/v1...v1.0.7)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-05-13 07:52:10 +10:00
dependabot[bot] e4ed072c28
build(deps): bump actions/checkout from 2 to 2.3.4 (#2143)
Bumps [actions/checkout](https://github.com/actions/checkout) from 2 to 2.3.4.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/v2...v2.3.4)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-05-13 07:51:12 +10:00
teor f8693ee004
Explain how to review version bump PRs (#2138) 2021-05-12 10:42:48 +02:00
teor f33db69267 Remove broken ci-success check
Previously, Zebra made ci-success a required check for merges to main. And then we made ci-success depend on a bunch of other CI checks.

But this doesn't work as expected, because if the dependent checks fail, ci-success is skipped, and the branch protection rules allow the branch to be merged to main.
2021-04-29 18:28:47 -04:00
teor 7e395298f1 Just run builds on ubuntu stable
But keep the tests across ubuntu, macOS, Windows x Rust stable, beta
2021-04-30 07:35:46 +10:00
teor d0e6597fec Use minimal cargo components in docs workflow 2021-04-30 07:35:46 +10:00
teor ea4d3b23f9 Use minimal cargo components in CI 2021-04-30 07:35:46 +10:00