Commit Graph

188 Commits

Author SHA1 Message Date
dependabot[bot] 597f55319d
build(deps): bump spandoc from 0.2.1 to 0.2.2 (#4221)
Bumps [spandoc](https://github.com/yaahc/spandoc) from 0.2.1 to 0.2.2.
- [Release notes](https://github.com/yaahc/spandoc/releases)
- [Changelog](https://github.com/yaahc/spandoc/blob/master/CHANGELOG.md)
- [Commits](https://github.com/yaahc/spandoc/compare/v0.2.1...v0.2.2)

---
updated-dependencies:
- dependency-name: spandoc
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-04-28 07:08:30 +00:00
dependabot[bot] 3cbd49afc3
build(deps): bump serde_with from 1.12.1 to 1.13.0 (#4190)
Bumps [serde_with](https://github.com/jonasbb/serde_with) from 1.12.1 to 1.13.0.
- [Release notes](https://github.com/jonasbb/serde_with/releases)
- [Commits](https://github.com/jonasbb/serde_with/compare/v1.12.1...v1.13.0)

---
updated-dependencies:
- dependency-name: serde_with
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-04-25 23:30:32 +00:00
teor 75671c53a2
1.0.0-beta.8 Release (#4135)
* Update changelog for 1.0.0-beta.8

* Bump Zebra crate versions

The tower crates and their dependencies haven't changed,
so their versions don't need to be bumped.

Some Zebra crates haven't changed,
but I bumped all the versions to keep them consistent.

* Explicitly say that we support NU5 testnet 2

* Expand PR #3799 into changelog entries

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-04-19 13:02:17 +10:00
Conrado Gouvea d79e71e969
change(nu5): use new V5 transaction script verification API (#3799)
* update librustzcash; adapt to new API

* add ticket reference for removing zcash_proofs duplicated dependencies

* update to new zcash_script V5 API

* use zp_tx shorthand

* update to Zcash 4.7.0 dependencies

* update protocol versions

* feat(rpc): Implement `getblockchaininfo` RPC method (#3891)

* Implement `getblockchaininfo` RPC method

* add a test for `get_blockchain_info`

* fix tohex/fromhex

* move comment

* Update lightwalletd acceptance test for getblockchaininfo RPC (#3914)

* change(rpc): Return getblockchaininfo network upgrades in height order (#3915)

* Update lightwalletd acceptance test for getblockchaininfo RPC

* Update some doc comments for network upgrades

* List network upgrades in order in the getblockchaininfo RPC

Also:
- Use a constant for the "missing consensus branch ID" RPC value
- Simplify fetching consensus branch IDs
- Make RPC type derives consistent
- Update RPC type documentation

* Make RPC type derives consistent

* Fix a confusing test comment

* get hashand height at the same time

* fix estimated_height

* fix lint

* add extra check

Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>

* fix typo

Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>

* split test

Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>

* fix(rpc): ignore an expected error in the RPC acceptance tests (#3961)

* Add ignored regexes to test command failure regex methods

* Ignore empty chain error in getblockchaininfo

We expect this error when zebrad starts up with an empty state.

Co-authored-by: teor <teor@riseup.net>
Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

* Make sync error logs more user-friendly (#3944)

- use info level, there is nothing the user needs to do,
  particularly for a single error
- explain that the errors are temporary
- hide backtraces, because they look like crashes

* Update test.patch.yml with lightwalletd job (#3970)

* Update test.patch.yml with lightwalletd job

* Remove a workflow condition that will always be false

In general, patch workflows need the
opposite conditions to the original workflow.

But in this case, we know the result of the
condition will always be true, so we can just delete it.

Co-authored-by: teor <teor@riseup.net>

* fix(doc): Fix bugs in the lightwalletd database design (#3964)

* Re-order column families in design in dependency order

* Minor RFC design tweaks and fixes

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

* Repoint zebra image links to our new zfnd.org site for now (#3949)

* Repoint zebra image links to our new zfnd.org site for now

* Remove images/

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

* Fix typos (#3956)

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: teor <teor@riseup.net>

* bump database version to trigger testnet rollback

* reduce minimum protocol version for now (will be changed later)

* update dependencies

* Apply suggestions from code review

Co-authored-by: teor <teor@riseup.net>

* update versions to match zcash 4.7.0

* deny.toml: update 'darling'

Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
Co-authored-by: teor <teor@riseup.net>
Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Deirdre Connolly <durumcrustulum@gmail.com>
Co-authored-by: Dimitris Apostolou <dimitris.apostolou@icloud.com>
2022-04-19 10:14:16 +10:00
teor 43e80fd61c
6. feat(db): Add a transparent address UTXO index (#3999)
* Add test-only serialization, and make existing serialization test-only

* Make AddressLocations clearer in the API

* Add UnspentOutputAddressLocation

* Add the AddressLocation to the UTXO database value

* Update the snapshot test code for UnspentOutputAddressLocation

* Update the raw data snapshots

* Update the high-level data snapshots

* Increment the database version

* Make serialization clearer

Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>

* Fix code formatting

Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>

* Add an empty utxo_by_transparent_addr_loc column family

* Update snapshot data for the new column family

* Add an AddressUnspentOutputs type

* Add round-trip tests for AddressUnspentOutputs

* Move address balances into their own method

* Simplify updating address balances

* Fix utxo_by_out_loc column family name

* Implement reads and writes of address UTXOs

* Update raw data snapshots

* Update the snapshot tests for high-level address UTXOs

* Assert rather than taking empty address snapshots for genesis

* Update high-level address UTXO snapshot data, and delete empty snapshots

* Increment the database version

* Use typed values for all ReadDisk methods

* Implement test-only serialization for transparent::Address

* Implement FromDisk for ()

* Store AddressUnspentOutput as the column family key

* Update round-trip serialization tests for AddressUnspentOutput

* Update snapshot test code, and add a UTXO data snapshot

* Update existing snapshot data

* Add new UTXO snapshot data

* Update column family name

```sh
fastmod utxo_by_transparent_addr_loc utxo_loc_by_transparent_addr_loc zebra*
```

* cargo fmt --all

* cargo insta test --review --delete-unreferenced-snapshots

* Explain why it is ok to use invalid database iterator indexes

Co-authored-by: Conrado Gouvea <conrado@zfnd.org>

* Add explanations of UTXO database updates

* Simplify an assertion

* Remove UnspentOutputAddressLocation and just store transparent::Output

* Update snapshot test data

Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>
Co-authored-by: Conrado Gouvea <conrado@zfnd.org>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-04-13 04:06:52 +00:00
teor 13c216cd3c
1.0.0-beta.7 Release (#4039)
* Update changelog for v1.0.0-beta.7

* Increment all crate versions

* Remove redundant release test that is now covered by CI

* Remove completed NU5 README check task from the release template

* Add Merge Freeze tool to the release checklist

* Simplify release checklist by removing unused steps
2022-04-05 18:06:46 +00:00
Conrado Gouvea ee6a38dffc
build(deps): bump serde-big-array from 0.3.2 to 0.4.1 (#4004)
Bumps [serde-big-array](https://github.com/est31/serde-big-array) from 0.3.2 to 0.4.1.
- [Release notes](https://github.com/est31/serde-big-array/releases)
- [Commits](https://github.com/est31/serde-big-array/compare/v0.3.2...v0.4.1)

---
updated-dependencies:
- dependency-name: serde-big-array
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-03-31 01:33:52 +00:00
Alfredo Garcia 53bf4c6ba5
1.0.0-beta.6 Release (#3966)
* 1.0.0-beta.6 release version bump

* 1.0.0-beta.6 release update the changelog

* change date of the release

* change description

Co-authored-by: teor <teor@riseup.net>

* remove phrase from description

Co-authored-by: teor <teor@riseup.net>
2022-03-28 22:36:42 +00:00
teor b4deca2912
6. add(test): add more tests for height and transaction location serialization (#3879)
* Add round-trip test for OutputLocation disk serialization

* Make the transaction snapshot tests more accurate

Previously, we were showing the genesis transaction hash at height 0, index 0.

But its hash is actually not stored by location in the database,
because the genesis transaction is skipped due to a consensus rule.

* Update the transaction snapshot data

* Add history tree snapshot tests

At the current test heights, the history trees are all empty.

* Add the history tree snapshot data

* Update comments

* Simplify snapshot test code

* Make some serde::Serialize impls test-only, so we can hex-encode them

This should also speed up release compilation a bit.

* Add snapshot test code for UTXOs

* Add snapshot data for UTXOs
2022-03-18 20:30:16 +00:00
Marek 2cc7e158a2
Upgrade dependencies (#3625)
* Upgrade some dependencies

* Upgrade some dependencies

* Upgrade dependencies for zebrad

* Upgrade tracing dependencies

* Revert `tor` & `arti`

* Upgrade `criterion` & `pin-project` in `deny.toml`

* Remove some dependencies from `skip-tree` in `deny.toml`

* Revert some the versions of dependencies because of duplicates

* Revert proptest regressions

* Upgrade dependencies, then ignore some more duplicates (#3716)

* feat(actions)!: add full sync test (#3582)

* add(tests): full sync test

* fix(test): add build

* fix(deploy): escape double dashes '--' correctly

* fix(test): remove unexpected --no-capture arg

error: Found argument '--nocapture' which wasn't expected, or isn't valid in this context

* refactor(docker): use default executable as entrypoint

* refactor(startup): add a custom entrypoint

* fix(test): add missing TEST_FULL_SYNC variable

* test(timeout): use the biggest machine

* fix

* fix(deploy): use latest successful image

* typo

* refactor(docker): generate config file at startup

* revert(build): changes were made to docker

* fix(docker): send variables correctly to the entrypoint

* test different conf file approach

* fix(env): add RUN_TEST env variable

* ref: use previous approach

* fix(color): use environment variable

* fix(resources): use our normal machine size

* fix(ci): double CPU and RAM for full sync test

* fix(test): check for zebrad test output in the correct order

The mempool is only activated once, so we must check for that log first.
After mempool activation, the stop regex is logged at least once.
(It might be logged before as well, but we can't rely on that.)

When checking that the mempool didn't activate,
wait for the `zebrad` command to exit,
then check the entire log.

* fix(ci): run full sync test with full compiler optimisations

* fix(tests): reintroduce tests and run full sync on approval

* fix(tests): reduce the changelog

Co-authored-by: teor <teor@riseup.net>

* fix(ci): update CI job path triggers (#3692)

* ci(test): re-run tests when snapshot data changes

* fix(ci): rebuild state when disk format changes

* fix(ci): rebuild rust docs when code or dependencies change

* doc(ci): explain why we run jobs when files change

Co-authored-by: Gustavo Valverde <gustavo@iterativo.do>

* fix(build): use the right multistage target (#3700)

* fix(review): only assign one reviewer to general Rust reviews (#3708)

If we assign two teams, GitHub assigns two reviewers.

* fix(ci): change the color-eyre ignore to a tracing-subscriber ignore

* fix(ci): ignore duplicate darling dependencies

* doc(ci): remove an alternative resolution doc

Co-authored-by: Gustavo Valverde <gustavo@iterativo.do>

Co-authored-by: teor <teor@riseup.net>
Co-authored-by: Gustavo Valverde <gustavo@iterativo.do>
2022-03-07 12:07:25 +10:00
dependabot[bot] 30cc048166
build(deps): bump secp256k1 from 0.21.2 to 0.21.3 (#3632)
Bumps [secp256k1](https://github.com/rust-bitcoin/rust-secp256k1) from 0.21.2 to 0.21.3.
- [Release notes](https://github.com/rust-bitcoin/rust-secp256k1/releases)
- [Changelog](https://github.com/rust-bitcoin/rust-secp256k1/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-bitcoin/rust-secp256k1/commits)

---
updated-dependencies:
- dependency-name: secp256k1
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-03-02 00:49:09 +00:00
dependabot[bot] eaf0b6add4
build(deps): bump sha2 from 0.9.8 to 0.9.9 (#3585)
Bumps [sha2](https://github.com/RustCrypto/hashes) from 0.9.8 to 0.9.9.
- [Release notes](https://github.com/RustCrypto/hashes/releases)
- [Commits](https://github.com/RustCrypto/hashes/compare/sha2-v0.9.8...sha2-v0.9.9)

---
updated-dependencies:
- dependency-name: sha2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-02-22 05:36:14 +00:00
Marek 2253f0121b
v1.0.0 beta.5 release (#3586)
* Increment the version numbers

* Use the new tag in `README.md` and `install.md`

* Update README.md

* Update CHANGELOG

* Add new PRs to the CHANGELOG
2022-02-21 14:29:34 +00:00
teor a4dd3b7396
4. Avoid repeated requests to peers after partial responses or errors (#3505)
* fix(network): split synthetic NotFoundRegistry from message NotFoundResponse

* docs(network): Improve `notfound` message documentation

* refactor(network): Rename MustUseOneshotSender to MustUseClientResponseSender

```
fastmod MustUseOneshotSender MustUseClientResponseSender zebra*
```

* docs(network): fix a comment typo

* refactor(network): remove generics from MustUseClientResponseSender

* refactor(network): add an inventory collector to Client, but don't use it yet

* feat(network): register missing peer responses as missing inventory

We register this missing inventory based on peer responses,
or connection errors or timeouts.

Inbound message inventory tracking requires peers to send `notfound` messages.
But `zcashd` skips `notfound` for blocks, so we can't rely on peer messages.
This missing inventory tracking works regardless of peer `notfound` messages.

* refactor(network): rename ResponseStatus to InventoryResponse

```sh
fastmod ResponseStatus InventoryResponse zebra*
```

* refactor(network): rename InventoryStatus::inner() to to_inner()

* fix(network): remove a redundant runtime.enter() in a test

* doc(network): the exact time used to filter outbound peers doesn't matter

* fix(network): handle block requests slightly more efficiently

* doc(network): fix a typo

* fmt(network): `cargo fmt` after rename ResponseStatus to InventoryResponse

* doc(test): clarify some test comments

* test(network): test synthetic notfound from connection errors and peer inventory routing

* test(network): improve inbound test diagnostics

* feat(network): add a proptest-impl feature to zebra-network

* feat(network): add a test-only connect_isolated_with_inbound function

* test(network): allow a response on the isolated peer test connection

* test(network): fix failures in test synthetic notfound

* test(network): Simplify SharedPeerError test assertions

* test(network): test synthetic notfound from partially successful requests

* test(network): MissingInventoryCollector ignores local NotFoundRegistry errors

* fix(network): decrease the inventory rotation interval

This stops us waiting 3-4 sync resets (4 minutes) before we retry a missing block.

Now we wait 1-2 sync resets (2 minutes), which is still a reasonable rate limit.
This should speed up syncing near the tip, and on testnet.

* fmt(network): cargo fmt --all

* cleanup(network): remove unnecessary allow(dead_code)

* cleanup(network): stop importing the whole sync module into tests

* doc(network): clarify syncer inventory retry constraint

* doc(network): add a TODO for a fix to ensure API behaviour remains consistent

* doc(network): fix a function doc typo

* doc(network): clarify how we handle peers that don't send `notfound`

* docs(network): clarify a test comment

Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>

Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-02-15 01:44:33 +00:00
dependabot[bot] 7b401ddeb0
Bump uint from 0.9.2 to 0.9.3 (#3468)
Bumps [uint](https://github.com/paritytech/parity-common) from 0.9.2 to 0.9.3.
- [Release notes](https://github.com/paritytech/parity-common/releases)
- [Commits](https://github.com/paritytech/parity-common/compare/uint-v0.9.2...uint-v0.9.3)

---
updated-dependencies:
- dependency-name: uint
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-02-05 05:06:51 +00:00
dependabot[bot] 8939ddf3d8
Bump tokio from 1.15.0 to 1.16.1 (#3434)
Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.15.0 to 1.16.1.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.15.0...tokio-1.16.1)

---
updated-dependencies:
- dependency-name: tokio
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-01-30 19:53:07 +00:00
dependabot[bot] 188e440a96
Bump uint from 0.9.1 to 0.9.2 (#3435)
Bumps [uint](https://github.com/paritytech/parity-common) from 0.9.1 to 0.9.2.
- [Release notes](https://github.com/paritytech/parity-common/releases)
- [Commits](https://github.com/paritytech/parity-common/compare/uint-v0.9.1...uint-v0.9.2)

---
updated-dependencies:
- dependency-name: uint
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-01-28 21:40:01 +00:00
Janito Vaqueiro Ferreira Filho f6de7faae1
Release v1.0.0 beta.4 (#3403)
* Updating zebra-test to v1.0.0-beta.4

* Updating tower-fallback to v0.2.16

* Updating tower-batch to v0.2.20

* Updating zebra-chain to v1.0.0-beta.4

* Updating zebra-script to v1.0.0-beta.4

* Updating zebra-network to v1.0.0-beta.4

* Updating zebra-state to v1.0.0-beta.4

* Updating zebra-consensus to v1.0.0-beta.4

* Updating zebra-utils to v1.0.0-beta.4

* Updating zebrad to v1.0.0-beta.4

* Update Cargo.lock with updated crate versions

* Add section for 1.0.0-beta.4 in the CHANGELOG

* Update CHANGELOG.md

Co-authored-by: teor <teor@riseup.net>

* Update README.md

Co-authored-by: teor <teor@riseup.net>

* Update book/src/user/install.md

Co-authored-by: teor <teor@riseup.net>

* Update release description to mention Rust 2021

Update the changelog to mention that all crates now use Rust 2021
Edition.

* Elaborate on Section 3.6 documentation entry

Add the title of the section and mention that it's a section of the
Zcash protocol specification.

* Make changelog entries consistent

Use the same format for entries related to consensus rule documentation.

Co-authored-by: Marek <mail@marek.onl>

* Update README about validated consensus rules

Zebra now validates all documented consensus rules.

* Add changelog entry for newly merged PR

Describe the security fix in the changelog.

Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>
Co-authored-by: teor <teor@riseup.net>
Co-authored-by: Marek <mail@marek.onl>
2022-01-26 21:39:41 +00:00
teor 82b2b3e625
fix (zebra-chain): make test and bench targets build using 2021 edition (#3404)
The 2021 edition activates the v2 cargo resolver.

This broke implicit feature resolution,
but only when `zebra-chain`'s dev or bench targets were compiled by themselves.

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-01-26 05:23:22 +00:00
dependabot[bot] 3ba0326c51
Bump secp256k1 from 0.20.3 to 0.21.2 (#3326)
Bumps [secp256k1](https://github.com/rust-bitcoin/rust-secp256k1) from 0.20.3 to 0.21.2.
- [Release notes](https://github.com/rust-bitcoin/rust-secp256k1/releases)
- [Changelog](https://github.com/rust-bitcoin/rust-secp256k1/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-bitcoin/rust-secp256k1/commits)

---
updated-dependencies:
- dependency-name: secp256k1
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-01-14 14:25:12 +00:00
Deirdre Connolly 89b0403582
Enforce Rust edition 2021 (#3332)
* Rust edition 2021: zebra-network, cargo fix --edition and clippy --fix

* Rust edition 2021: zebra-chain, cargo fix --edition

* Rust edition 2021: tower-batch, cargo fix --edition

* Rust edition 2021: tower-fallback, cargo fix --edition

* Rust edition 2021: zebra-client, cargo fix --edition

* Rust edition 2021: zebra-consensus, cargo fix --edition

* Rust edition 2021: zebra-rpc, cargo fix --edition

* Rust edition 2021: zebra-state, cargo fix --edition

* Rust edition 2021: zebra-state, cargo fix --edition

* Rust edition 2021: zebra-test, cargo fix --edition

* Rust edition 2021: zebra-utils, cargo fix --edition

* Rust edition 2021: zebrad, cargo fix --edition

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-01-14 12:10:18 +00:00
Deirdre Connolly e9c2125e7e
Upgrade to latest redjubjub commit, hasn't been published to crates.io yet (#3333)
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-01-13 23:59:59 +00:00
teor d076b999f3
Fix syncer download order and add sync tests (#3168)
* Refactor so that RetryLimit::Future is std::marker::Sync

* Make the syncer future std::marker::Send by spawning tips futures

* Download synced blocks in chain order, not HashSet order

* Improve MockService failure messages

* Add closure-based responses to the MockService API

* Move MockChainTip to zebra-chain

* Add a MockChainTipSender type alias

* Support MockChainTip in ChainSync and its downloader

* Add syncer tests for obtain tips, extend tips, and wrong block hashes

* Add block too high tests for obtain tips and extend tips

* Add syncer tests for duplicate FindBlocks response hashes

* Allow longer request delays for mocked services in syncer tests
2022-01-11 14:11:35 -03:00
teor 534494f075
Upgrade zcash_proofs to get new derived traits (#3308) 2021-12-30 14:38:21 -03:00
Conrado Gouvea a5713da0af
Release Zebra v1.0.0-beta.3 (#3256)
* Release Zebra v1.0.0-beta.3

* Apply suggestions from code review

Co-authored-by: teor <teor@riseup.net>

* Bump date

* Add recently merged entry to changelog

* Bump date, add recent merged PR to changelog

* Apply suggestions from code review

Co-authored-by: teor <teor@riseup.net>

* Apply suggestions from code review

Co-authored-by: teor <teor@riseup.net>

Co-authored-by: teor <teor@riseup.net>
2021-12-21 19:42:39 -03:00
dependabot[bot] a6d56b2c08
Bump itertools from 0.10.1 to 0.10.3 (#3171)
Bumps [itertools](https://github.com/rust-itertools/itertools) from 0.10.1 to 0.10.3.
- [Release notes](https://github.com/rust-itertools/itertools/releases)
- [Changelog](https://github.com/rust-itertools/itertools/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-itertools/itertools/compare/v0.10.1...v0.10.3)

---
updated-dependencies:
- dependency-name: itertools
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: teor <teor@riseup.net>
2021-12-08 22:57:14 +00:00
teor 022808d028
Release Zebra v1.0.0-beta.2 (#3132)
Zebra's latest beta continues implementing zero-knowledge proof and note commitment tree validation. In this release, we have finished implementing transaction header, transaction amount, and Zebra-specific NU5 validation. (NU5 mainnet validation is waiting on an `orchard` crate update, and some consensus parameter updates.)

We also fix a number of security issues that could pose a local denial of service risk, or make it easier for an attacker to make a node follow a false chain.

As of this release, Zebra will automatically download and cache the Sprout and Sapling Groth16 circuit parameters. The cache uses around 1 GB of disk space. These cached parameters are shared across all Zebra and `zcashd` instances run by the same user.

See CHANGELOG.md for the full list of changes in this release.
2021-12-03 06:54:14 +10:00
Deirdre Connolly e6ffe374d4
Validate sapling, orchard anchors (#3084)
* Add Transaction::sprout_joinsplits()

* Add Anchor variants to ValidateContextError

* Make Chain anchor collections pub(crate)

* tracing::instrument several methods in state

* Add contains_*_anchors methods to FinalizedState

* Add check::anchors module and function

* Verify that anchors_refer_to_earlier_treestates in when updating chains in non-finalized state

* Update zebra-state/src/service/check/anchors.rs

Co-authored-by: teor <teor@riseup.net>

* Add anchors() to sapling::ShieldedData

* Add sapling_anchors() to Transaction

* Use Transaction::sapling_anchors() in the anchors_refer_to_earlier_treestates() check

* Whoops, itertools

* Add a comment for improvement

Co-authored-by: teor <teor@riseup.net>

* Add & use a cfg(test) method on FinalizedState to prep test state with anchors to allow other tests to pass contextual checks

* Allow test nullifier checks to pass by populating anchor sets, allowing test anchor checks to pass

* Add mainnet block 419202 and its sapling note commitment tree root to test vectors

* Test sapling anchor verification using the first few Sapling blocks data

* Correct comment

* assert_eq instead of assert(bool)

Co-authored-by: Conrado Gouvea <conrado@zfnd.org>

* Update zebra-state/src/service/non_finalized_state.rs

Co-authored-by: teor <teor@riseup.net>

Co-authored-by: teor <teor@riseup.net>
Co-authored-by: Marek <mail@marek.onl>
Co-authored-by: Conrado Gouvea <conrado@zfnd.org>
2021-11-30 16:05:35 +00:00
teor 0ef4629232
Automatically download and load sprout parameters (#3085)
* Download and load Sprout parameters using zcash_proofs

Also update some librustzcash dependencies, to avoid duplicate dependencies.

* Update upstream orchard to avoid a compilation error

* Skip librustzcash batch refactor for now, to avoid compilation errors

* Change the cache ID, so we actually cache Sprout

* Move existing file checks into zcash_proofs

* Add a 1 hour timeout to parameter file downloads

* Give other tasks priority, before spawning the download task

* Update to the latest version of our modified librustzcash fork

* Change the cache key for Sprout

* Add 40 minutes to CI timeouts for occasional sprout downloads

* Update to zcash_proofs with split downloads

* Check file sizes to help debug parameter load failures in zcash_proofs

* Start the second download once the first has finished in zcash_proofs

* Document the parameter download task

* Stop hashing existing files twice
2021-11-25 13:26:32 -03:00
Pili Guerra 26b3a50e01
Updates for zebra v1.0.0-beta.1 release (#3073)
* Update versions for zebra v1.0.0-beta.1 release

* Adding original PR list for comparison and tracking as PRs merge

* First pass at categorising changes

* Merge and clarify description of related changes

* Remove or merge trivial changes

* Improve change descriptions

* Add new PRs merged

* CHANGELOG: Improve release summary

* CHANGELOG: categorise changes further

* README: Remove resolved issues and items

* Update CHANGELOG.md

Co-authored-by: teor <teor@riseup.net>

* CHANGELOG: Add new PRs merged

* CHANGELOG: Move change category

* CHANGELOG: Update release date ready for tagging

Co-authored-by: teor <teor@riseup.net>
2021-11-19 13:05:11 +01:00
dependabot[bot] 1643d394bf
Bump fpe from 0.5.0 to 0.5.1 (#3066)
Bumps [fpe](https://github.com/str4d/fpe) from 0.5.0 to 0.5.1.
- [Release notes](https://github.com/str4d/fpe/releases)
- [Changelog](https://github.com/str4d/fpe/blob/main/CHANGELOG.md)
- [Commits](https://github.com/str4d/fpe/commits)

---
updated-dependencies:
- dependency-name: fpe
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: teor <teor@riseup.net>
2021-11-18 23:50:00 +00:00
Conrado Gouvea 88b09c812a
Check nSpendsSapling, nOutputsSapling, and nActionsOrchard 2^16 limit (#3069)
* Check nSpendsSapling, nOutputsSapling, and nActionsOrchard 2^16 limit

* Apply suggestions from code review

Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>

* Removed not required #[macro_use]

Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>
2021-11-18 15:06:07 -03:00
Deirdre Connolly eda83ebe0e
Async Halo2 verifier service (#2645)
* First pass at async Halo2 verification service

Stubs out a batch verifier for the future.
The dependencies for orchard, halo2, librustzcash, zcash_primitives, have
not been resolved.

* Halo2 verifier service and test

* Remove redundant conversion

* Test async halo2 verifier service with pre-computed Orchard shielded data test vectors

* Fix typo

Co-authored-by: Conrado Gouvea <conrado@zfnd.org>

* Assert future result is_ok() in Halo2 verifier test

Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>

* Shorten tower::Service trait constraints for Halo2 verifier tests

* Remove commented out trait constraints

* .expect() vs .unwrap() to parse orchard::redpallas::VerificationKey

* Use .to_vec() for some test vectors

* Fix self-referential Display impl

* Fix deps

* Distinguish orchard vs zebra_chain::orchard imports

* Add test that halo2 verifier fails with malformed proof inputs

* Use thiserror for Halo2Error

* Use ZcashFoundation/orchard instead of dconnolly/orchard

* Add a link to the issue to remove the zfnd fork of orchard crate

* Update zebra-consensus/Cargo.toml

Co-authored-by: teor <teor@riseup.net>

* Add note

* Move artificial Orchard shielded data test vectors to zebra-test

* Align brackets

* Tidy some trait constraints and debug statements

Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>

Co-authored-by: Conrado Gouvea <conrado@zfnd.org>
Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>
Co-authored-by: teor <teor@riseup.net>
2021-11-17 13:26:15 +10:00
Deirdre Connolly 90da94fff3
Upgrade crypto deps (#3059)
* Upgrade aes and fpe

* Upgrade bellman, bls12_381, jubjub to latest

* Upgrade x25519-dalek to 1.2.0 and curve25519-dalek to 3.2.0 in the Cargo.lock

* Skip outdated hdrhistogram rather than its dependencies

Co-authored-by: teor <teor@riseup.net>
2021-11-16 19:47:54 +00:00
Conrado Gouvea 6570ebeeb8
ZIP 212: validate Sapling and Orchard output of coinbase transactions (#3029)
* Part of ZIP 212: validate Sapling and Orchard output of coinbase transactions

* Add Orchard test vector

* Revert accidentally deleted link

* Apply suggestions from code review

Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>

* Use height from loop

* Apply suggestions from code review

Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>

* Fix formatting

Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>
2021-11-11 22:18:37 +00:00
Conrado Gouvea e54917ae7c
V1.0.0-beta.0 (#2973)
* V1.0.0-beta.0

* Bump version in install.md
2021-10-29 20:21:26 +00:00
Marek d2a5af0ea5
V1.0.0 alpha.19 (#2907)
* Increment the crates that have new commits since the last version

* Increment the crates that depend on crates that have changed

* Increment the version of `zebra-script`

* Use the `zebrad` version in the `zebra-network` user agent string

* Use the `v1.0.0-alpha.19` git tag in `README.md`

* Copy the draft changelog into `CHANGELOG.md`

* Delete bumps

* Update CHANGELOG.md

Co-authored-by: teor <teor@riseup.net>

* Add newly merged PRs

Co-authored-by: teor <teor@riseup.net>
2021-10-21 12:33:35 +02:00
Deirdre Connolly e36ec346e8
Upgrade to ed25519-zebra 3.0.0 (#2864)
* Upgrade to ed25519-zebra 3.0.0

* Revert unrelated extra space

Co-authored-by: teor <teor@riseup.net>
2021-10-12 23:42:44 +00:00
dependabot[bot] c8af72cd30
Bump tracing from 0.1.28 to 0.1.29 (#2836)
Bumps [tracing](https://github.com/tokio-rs/tracing) from 0.1.28 to 0.1.29.
- [Release notes](https://github.com/tokio-rs/tracing/releases)
- [Commits](https://github.com/tokio-rs/tracing/compare/tracing-0.1.28...tracing-0.1.29)

---
updated-dependencies:
- dependency-name: tracing
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-10-06 21:27:04 +00:00
Pili Guerra a85e80a0ec
Update versions for zebra v1.0.0-alpha.18 release (#2828)
* Update versions for zebra v1.0.0-alpha.18 release

* WIP: Initial PR list

* Remove uninteresting version bumps from CHANGELOG

* Categorise and group PRs in CHANGELOG, removing uninteresting PRs

* Further refine and categorise changelog entries

* Fix tag url

* Final changes to CHANGELOG

* Add a changelog description

* Spacing

* Clarify and fix changelog PR descriptions

* Add PRs that are about to be merged

* More slight clarifications

* Spacing

Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>

Co-authored-by: teor <teor@riseup.net>
Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>
2021-10-05 23:22:31 -03:00
Janito Vaqueiro Ferreira Filho 339fefb6e6
Update shared NU5 dependencies, set the NU5 testnet activation network upgrade parameters (#2825)
* Check return value of zcash_script_new_precomputed_tx

* Set the NU5 testnet activation height to 1_590_000

* Apply suggestions from code review

Co-authored-by: teor <teor@riseup.net>

* Update Nu5 constants to new values

* Update ZIP-244 test vectors for new branch ID

* Squashed commit of the following:

commit bdb120a249
Author: Deirdre Connolly <durumcrustulum@gmail.com>
Date:   Tue Oct 5 11:54:01 2021 -0400

    Use pallas::Base::from_str_vartime() in sinsemilla tests

commit e99fa49258
Author: Deirdre Connolly <durumcrustulum@gmail.com>
Date:   Tue Oct 5 11:45:24 2021 -0400

    Compiles

commit a520018114
Author: Deirdre Connolly <durumcrustulum@gmail.com>
Date:   Tue Oct 5 10:15:17 2021 -0400

    Incomplete upgrade of deps

* Squashed commit of the following:

commit 8d1b76ec5626517817c3a4d9f3950acc90a359df
Author: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>
Date:   Tue Oct 5 04:02:26 2021 +0000

    Update `zcash_script` to support V5 transactions

    Use a newer version of `zcash_script` that has been updated to support
    V5 transactions.

commit 371233628ae61e0c25d6ba8f31d9dba42823becb
Author: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>
Date:   Tue Oct 5 03:06:20 2021 +0000

    Update Zcash dependencies

    Update some Zcash crates:

    - `halo2`
    - `incrementalmerkletree' (patch version)
    - `orchard` (patch version)
    - `zcash_history` (patch version)
    - `zcash_note_encryption` (patch version)
    - `zcash_primitives` (patch version)

    And also update the `group` dependency so that the code remains
    compatible.

commit de5cf1ec40c3fc08670fc971cdf3e65e13d9f4c7
Author: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>
Date:   Tue Oct 5 03:04:13 2021 +0000

    Update error message assertion

    Use the updated message for the expected error variant.

* Update `zcash_script` to support V5 transactions

Use a newer version of `zcash_script` that has been updated to support
V5 transactions.

Co-authored-by: Conrado Gouvea <conrado@zfnd.org>
Co-authored-by: teor <teor@riseup.net>
2021-10-06 11:08:41 +10:00
dependabot[bot] d178e08986 Bump tracing from 0.1.27 to 0.1.28
Bumps [tracing](https://github.com/tokio-rs/tracing) from 0.1.27 to 0.1.28.
- [Release notes](https://github.com/tokio-rs/tracing/releases)
- [Commits](https://github.com/tokio-rs/tracing/compare/tracing-0.1.27...tracing-0.1.28)

---
updated-dependencies:
- dependency-name: tracing
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-09-20 16:33:13 -04:00
dependabot[bot] 21af477a37 Bump tracing from 0.1.26 to 0.1.27
Bumps [tracing](https://github.com/tokio-rs/tracing) from 0.1.26 to 0.1.27.
- [Release notes](https://github.com/tokio-rs/tracing/releases)
- [Commits](https://github.com/tokio-rs/tracing/compare/tracing-0.1.26...tracing-0.1.27)

---
updated-dependencies:
- dependency-name: tracing
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-09-14 21:01:01 -04:00
dependabot[bot] 9b3f6b3105
Bump sha2 from 0.9.6 to 0.9.8 (#2746)
Bumps [sha2](https://github.com/RustCrypto/hashes) from 0.9.6 to 0.9.8.
- [Release notes](https://github.com/RustCrypto/hashes/releases)
- [Commits](https://github.com/RustCrypto/hashes/compare/sha2-v0.9.6...sha2-v0.9.8)

---
updated-dependencies:
- dependency-name: sha2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>
2021-09-14 17:57:58 -04:00
Conrado Gouvea 8971d62727
Update version strings for v1.0.0 alpha.17 release (#2750)
* Change versions for v1.0.0-alpha.17 release
2021-09-14 17:41:50 +00:00
dependabot[bot] 424095096a Bump sha2 from 0.9.5 to 0.9.6
Bumps [sha2](https://github.com/RustCrypto/hashes) from 0.9.5 to 0.9.6.
- [Release notes](https://github.com/RustCrypto/hashes/releases)
- [Commits](https://github.com/RustCrypto/hashes/compare/sha2-v0.9.5...sha2-v0.9.6)

---
updated-dependencies:
- dependency-name: sha2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-08-29 21:01:15 -04:00
Alfredo Garcia 968f20d423
Update versions for zebra v1.0.0-alpha.16 release (#2670)
* bump crate versions

* update zebra-script

Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>
2021-08-27 22:13:54 +00:00
Pili Guerra 234953e620
Update versions for zebra v1.0.0-alpha.15 release (#2612) 2021-08-16 10:06:26 +00:00
Pili Guerra f59d552721
Update versions for zebra v1.0.0-alpha.14 release (#2537)
Co-authored-by: teor <teor@riseup.net>
2021-07-29 19:42:21 +00:00
Deirdre Connolly e719c46b1b
Track anchors and note commitment trees in zebra-state (#2458)
* Tidy chain Cargo.toml

* Organize imports

* Add method to get note commitments from all Actions in Orchard shielded data

* Add method to get note commitments from all JoinSplits in Sprout JoinSplitData

* Add Request and Response variants for awaiting anchors

* Add anchors and note commitment trees to finalized state db

* Add (From|Into)Disk impls for tree::Roots and stubs for NoteCommitmentTrees

* Track anchors and note commitment trees in Chain

Append note commitments to their trees when doing update_chain_state_with,
then use the resulting Sapling and Orchard roots to pass to history_tree, and add
new roots to the anchor sets.

* Handle errors when appending to note commitment trees

* Add comments explaining why note commitment are not removed from the tree in revert_chain_state_with

* Implementing note commitments in finalized state

* Finish serialization of Orchard tree; remove old tree when updating finalize state

* Add serialization and finalized state updates for Sprout and Sapling trees

* Partially handle trees in non-finalized state. Use Option for trees in Chain

* Rebuild trees when forking; change finalized state tree getters to not require height

* Pass empty trees to tests; use empty trees by default in Chain

* Also rebuild anchor sets when forking

* Use empty tree as default in finalized state tree getters (for now)

* Use HashMultiSet for anchors in order to make pop_root() work correctly

* Reduce DEFAULT_PARTIAL_CHAIN_PROPTEST_CASES and MAX_PARTIAL_CHAIN_BLOCKS

* Reduce DEFAULT_PARTIAL_CHAIN_PROPTEST_CASES and MAX_PARTIAL_CHAIN_BLOCKS even more

* Apply suggestions from code review

* Add comments about order of note commitments and related methods/fields

* Don't use Option for trees

* Set DEFAULT_PARTIAL_CHAIN_PROPTEST_CASES=1 and restore MAX_PARTIAL_CHAIN_BLOCKS

* Remove unneeded anchor set rebuilding in fork()

* Improve proptest formatting

* Add missing comparisons to eq_internal_state

* Renamed sprout::tree::NoteCommitmentTree::hash() to root()

* Improve comments

* Add asserts, add issues to TODOs

* Remove impl Default for Chain since it was only used by tests

* Improve documentation and assertions; add tree serialization tests

* Remove Sprout code, which will be moved to another branch

* Add todo! in Sprout tree append()

* Remove stub request, response *Anchor* handling for now

* Add test for validating Sapling note commitment tree using test blocks

* Increase database version (new columns added for note commitment trees and anchors)

* Update test to make sure the order of sapling_note_commitments() is being tested

* Improve comments and structure of the test

* Improve variable names again

* Rustfmt

Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>
Co-authored-by: Conrado P. L. Gouvea <conradoplg@gmail.com>
Co-authored-by: Conrado Gouvea <conrado@zfnd.org>
Co-authored-by: teor <teor@riseup.net>
2021-07-29 09:37:18 -04:00