Commit Graph

1075 Commits

Author SHA1 Message Date
teor be91ab29ee
fix(clippy): resolve various clippy warnings (#4473)
* clippy: unused import on non-linux platforms

* Fix some instances of clippy::derive_partial_eq_without_eq

* Move a deref to fix clippy::significant_drop_in_scrutinee

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-05-25 20:56:32 +00:00
Alfredo Garcia 8150a720ca
release: bump versions and create changelog entry for `v1.0.0-beta.10` (#4430)
* bump versions

* add the changelog for beta 10

* Update CHANGELOG.md

Co-authored-by: Marek <mail@marek.onl>

* Distinguish CI and Rust dependency updates

* Include all the dep bumps in PR #4405

* join 2 dependencies

* Include incrementalmerkletree dep bump in changelog

Co-authored-by: Deirdre Connolly <durumcrustulum@gmail.com>
Co-authored-by: Marek <mail@marek.onl>
2022-05-19 19:23:30 +00:00
Conrado Gouvea 9aaf0ed0ce
change: set NU5 mainnet activation height and current network protocol version (#4390)
* change: set NU5 mainnet activation height and minimum protocol version

* fix tests

* test if V5 coinbase expiry height is not bound by non-coinbase limit

* update get_blockchain_info snapshot for NU5

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-05-19 01:04:11 +00:00
Conrado Gouvea 55822dda41
change: NU5 mainnet dependency upgrades (#4405)
* upgrade dependencies to NU5 mainnet versions

* migrate to bitvec 1.0

* update deny.toml

* point to published redjubjub 0.5

* Point at zcash-script@0.1.6 on crates.io

* update Cargo.lock

* specify patch versions of dependencies

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Deirdre Connolly <durumcrustulum@gmail.com>
2022-05-19 01:03:55 +00:00
dependabot[bot] d0d611ef19
build(deps): bump bech32 from 0.8.1 to 0.9.0 (#4394)
Bumps [bech32](https://github.com/rust-bitcoin/rust-bech32) from 0.8.1 to 0.9.0.
- [Release notes](https://github.com/rust-bitcoin/rust-bech32/releases)
- [Commits](https://github.com/rust-bitcoin/rust-bech32/commits)

---
updated-dependencies:
- dependency-name: bech32
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-05-13 17:57:56 +00:00
Marek 7c726b246d
feat(rpc): Implement `z_gettreestate` RPC (#3990)
* Impl the elementary structure of the `z_gettreestate` RPC

* Fix merging bugs

* Fix a merge bug

* Fix a merge bug

* Move a derive attribute

Co-authored-by: teor <teor@riseup.net>

* Clarify the support of negative heights

* Add Orchard note commitment trees to the response

* Add the time to the response

* Finalize the `z_gettreestate` RPC

* Add a note that verified blocks have coinbase height

* Refactor `from_str` for `HashOrHeight`

* Fix a mistake in the docs

Co-authored-by: teor <teor@riseup.net>

* Clarify request types

Co-authored-by: teor <teor@riseup.net>

* Simplify `hash_or_height` conversion to height

Co-authored-by: teor <teor@riseup.net>

* Add a TODO about optimization

Co-authored-by: teor <teor@riseup.net>

* Add a doc comment

* Make sure Sapling & Orchard trees don't get mixed up

* Serialize Sapling commitment trees

* Refactor some comments

* Serialize Orchard commitment trees

* Serialize block heights

* Simplify the serialization of commitment trees

* Remove the block time from the RPC response

* Simplify the serialization of block heights

* Put Sapling & Orchard requests together

* Remove a redundant TODO

* Add block times to the RPC response

* Derive `Clone, Debug, Eq, PartialEq` for `GetTreestate`

Co-authored-by: teor <teor@riseup.net>

* Derive `Clone`, `Debug`, `Eq` and `PartialEq` for `SerializedTree`

* Document the fields of `GetTreestate`

* Skip the serialization of empty trees

This ensures compatibility with `zcashd` in the `z_gettreestate` RPC.

* Document the `impl` of `merkle_tree::Hashable` for nodes

* Make the structure of the JSON response consistent with `zcashd`

* Derive `Eq` for nodes

Co-authored-by: teor <teor@riseup.net>

* Convert Sapling commitment trees to a format compatible with zcashd

* Refactor the conversion of Sapling commitment trees

* Refactor some comments

* Refactor comments

* Add a description of the conversion

Co-authored-by: Conrado Gouvea <conrado@zfnd.org>

* Fix comment indenting

* Document the conversion between the dense and sparse formats

Co-authored-by: teor <teor@riseup.net>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Conrado Gouvea <conrado@zfnd.org>
2022-05-12 07:00:12 +00:00
dependabot[bot] cb0487cec9
build(deps): bump tokio from 1.18.1 to 1.18.2 (#4358)
Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.18.1 to 1.18.2.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.18.1...tokio-1.18.2)

---
updated-dependencies:
- dependency-name: tokio
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-05-10 03:42:08 +00:00
Conrado Gouvea c190e3027c
v1.0.0-beta.9 release (#4349) 2022-05-09 13:32:53 -03:00
Janito Vaqueiro Ferreira Filho 9538ad29e5
feat(util): add a `zebra-tip-height` utility (#4289)
* Move `init_tracing` to `lib.rs`

Allow it to be reused by other binaries.

* Fix `hex` dependency inclusion in `zebra-chain`

It would previously fail because `hex` was being included without the
`serde` feature, even though we required that for `transparent::Script`.

* Implement `FromStr` for `Network`

Make it easy to receive it as a command-line parameter.

* Add `zebra-tip-height` utility

Obtains the chain tip height of a directory containing Zebra state.

* Remove Tokio dependency from `zebra-utils`

It wasn't actually used by the `zebra-tip-height` utility.

* Remove `BoxStateService` type alias

It's not needed if we don't return the unused state service. This also
allows removing the `tower` dependency.

* Remove unnecessary attribute

Leftover from copied code.

* Make `cache_dir` argument optional

Fallback to the default Zebra state cache directory.

* Remove added newline

Minor formatting fix, to avoid adding an unnecessary newline.

* Move `tip-height` command into `zebrad`

Make it available as a sub-command in `zebrad`.

* Make some zebrad sub-commands only log warnings

Co-authored-by: teor <teor@riseup.net>
2022-05-06 03:31:52 +00:00
dependabot[bot] 32056e0e5c
build(deps): bump thiserror from 1.0.30 to 1.0.31 (#4259)
Bumps [thiserror](https://github.com/dtolnay/thiserror) from 1.0.30 to 1.0.31.
- [Release notes](https://github.com/dtolnay/thiserror/releases)
- [Commits](https://github.com/dtolnay/thiserror/compare/1.0.30...1.0.31)

---
updated-dependencies:
- dependency-name: thiserror
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-05-05 01:08:06 +00:00
dependabot[bot] 2e643c8631
build(deps): bump serde from 1.0.136 to 1.0.137 (#4261)
Bumps [serde](https://github.com/serde-rs/serde) from 1.0.136 to 1.0.137.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.136...v1.0.137)

---
updated-dependencies:
- dependency-name: serde
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-05-04 04:00:30 +00:00
dependabot[bot] 47f144205c
build(deps): bump tokio from 1.18.0 to 1.18.1 (#4280)
Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.18.0 to 1.18.1.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.18.0...tokio-1.18.1)

---
updated-dependencies:
- dependency-name: tokio
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-05-03 20:55:43 +00:00
teor be4e065afb
change(rpc): Simplify RPC types and add documentation (#4218)
* Simplify RPC types and add documentation

* Derive serde traits in production code
2022-05-03 04:10:21 +00:00
dependabot[bot] 1691d9e855
build(deps): bump tokio from 1.17.0 to 1.18.0 (#4242)
Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.17.0 to 1.18.0.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.17.0...tokio-1.18.0)

---
updated-dependencies:
- dependency-name: tokio
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-04-28 17:54:33 +00:00
dependabot[bot] 597f55319d
build(deps): bump spandoc from 0.2.1 to 0.2.2 (#4221)
Bumps [spandoc](https://github.com/yaahc/spandoc) from 0.2.1 to 0.2.2.
- [Release notes](https://github.com/yaahc/spandoc/releases)
- [Changelog](https://github.com/yaahc/spandoc/blob/master/CHANGELOG.md)
- [Commits](https://github.com/yaahc/spandoc/compare/v0.2.1...v0.2.2)

---
updated-dependencies:
- dependency-name: spandoc
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-04-28 07:08:30 +00:00
dependabot[bot] 3cbd49afc3
build(deps): bump serde_with from 1.12.1 to 1.13.0 (#4190)
Bumps [serde_with](https://github.com/jonasbb/serde_with) from 1.12.1 to 1.13.0.
- [Release notes](https://github.com/jonasbb/serde_with/releases)
- [Commits](https://github.com/jonasbb/serde_with/compare/v1.12.1...v1.13.0)

---
updated-dependencies:
- dependency-name: serde_with
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-04-25 23:30:32 +00:00
Alfredo Garcia 56aabb1db1
feat(rpc): Implement `getaddressutxos` RPC method. (#4087)
* implement display for `Script`

* implement `getaddressutxos`

* fix space

* normalize list of addresses as argument to rpc methods

* implement `AddressStrings`

* make a doc clearer

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-04-25 03:00:52 +00:00
Conrado Gouvea dff25473aa
refactor: document coinbase rules, refactor to ease understanding (#4056)
* refactor: document coinbase rules, refactor to ease understanding

* Update zebra-consensus/src/block/check.rs

Co-authored-by: teor <teor@riseup.net>

* remove no longer used contains_prevout_input()

* remove unused CoinbaseHasPrevOutInput

* update coinbase description in documentation

Co-authored-by: teor <teor@riseup.net>
2022-04-20 09:31:12 +00:00
teor 5054e048d4
Require a minimum network protocol version of 170_040 (#3976)
This is the `zcashd` network protocol version:
- after the first NU5 testnet activation, and
- after updating to the second NU5 testnet activation consensus rules,
- but before setting the second NU5 testnet activation height.
2022-04-19 16:14:24 +00:00
teor 75671c53a2
1.0.0-beta.8 Release (#4135)
* Update changelog for 1.0.0-beta.8

* Bump Zebra crate versions

The tower crates and their dependencies haven't changed,
so their versions don't need to be bumped.

Some Zebra crates haven't changed,
but I bumped all the versions to keep them consistent.

* Explicitly say that we support NU5 testnet 2

* Expand PR #3799 into changelog entries

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-04-19 13:02:17 +10:00
Conrado Gouvea d79e71e969
change(nu5): use new V5 transaction script verification API (#3799)
* update librustzcash; adapt to new API

* add ticket reference for removing zcash_proofs duplicated dependencies

* update to new zcash_script V5 API

* use zp_tx shorthand

* update to Zcash 4.7.0 dependencies

* update protocol versions

* feat(rpc): Implement `getblockchaininfo` RPC method (#3891)

* Implement `getblockchaininfo` RPC method

* add a test for `get_blockchain_info`

* fix tohex/fromhex

* move comment

* Update lightwalletd acceptance test for getblockchaininfo RPC (#3914)

* change(rpc): Return getblockchaininfo network upgrades in height order (#3915)

* Update lightwalletd acceptance test for getblockchaininfo RPC

* Update some doc comments for network upgrades

* List network upgrades in order in the getblockchaininfo RPC

Also:
- Use a constant for the "missing consensus branch ID" RPC value
- Simplify fetching consensus branch IDs
- Make RPC type derives consistent
- Update RPC type documentation

* Make RPC type derives consistent

* Fix a confusing test comment

* get hashand height at the same time

* fix estimated_height

* fix lint

* add extra check

Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>

* fix typo

Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>

* split test

Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>

* fix(rpc): ignore an expected error in the RPC acceptance tests (#3961)

* Add ignored regexes to test command failure regex methods

* Ignore empty chain error in getblockchaininfo

We expect this error when zebrad starts up with an empty state.

Co-authored-by: teor <teor@riseup.net>
Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

* Make sync error logs more user-friendly (#3944)

- use info level, there is nothing the user needs to do,
  particularly for a single error
- explain that the errors are temporary
- hide backtraces, because they look like crashes

* Update test.patch.yml with lightwalletd job (#3970)

* Update test.patch.yml with lightwalletd job

* Remove a workflow condition that will always be false

In general, patch workflows need the
opposite conditions to the original workflow.

But in this case, we know the result of the
condition will always be true, so we can just delete it.

Co-authored-by: teor <teor@riseup.net>

* fix(doc): Fix bugs in the lightwalletd database design (#3964)

* Re-order column families in design in dependency order

* Minor RFC design tweaks and fixes

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

* Repoint zebra image links to our new zfnd.org site for now (#3949)

* Repoint zebra image links to our new zfnd.org site for now

* Remove images/

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

* Fix typos (#3956)

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: teor <teor@riseup.net>

* bump database version to trigger testnet rollback

* reduce minimum protocol version for now (will be changed later)

* update dependencies

* Apply suggestions from code review

Co-authored-by: teor <teor@riseup.net>

* update versions to match zcash 4.7.0

* deny.toml: update 'darling'

Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
Co-authored-by: teor <teor@riseup.net>
Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Deirdre Connolly <durumcrustulum@gmail.com>
Co-authored-by: Dimitris Apostolou <dimitris.apostolou@icloud.com>
2022-04-19 10:14:16 +10:00
teor 8e29219565
8. feat(state): add a query function for transparent address balances (#4097)
* Make address index types consistent

* Simplify non-finalized address index updates

* Update snapshots for address index queries

* Simplify non-finalized UTXO query

* Add a query method for non-finalized address balance changes

* Add a query method for finalized state address balances

* Add a query function for address balances

* Refactor balance queries to make them repeatable

* Retry interrupted finalized balance queries

* Pop chain root blocks until it matches the finalized tip

* Avoid cloning the chain

It has already been cloned by the watch receiver

* Refactor and fix documentation of the balance query code
2022-04-14 13:34:31 +00:00
teor 43e80fd61c
6. feat(db): Add a transparent address UTXO index (#3999)
* Add test-only serialization, and make existing serialization test-only

* Make AddressLocations clearer in the API

* Add UnspentOutputAddressLocation

* Add the AddressLocation to the UTXO database value

* Update the snapshot test code for UnspentOutputAddressLocation

* Update the raw data snapshots

* Update the high-level data snapshots

* Increment the database version

* Make serialization clearer

Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>

* Fix code formatting

Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>

* Add an empty utxo_by_transparent_addr_loc column family

* Update snapshot data for the new column family

* Add an AddressUnspentOutputs type

* Add round-trip tests for AddressUnspentOutputs

* Move address balances into their own method

* Simplify updating address balances

* Fix utxo_by_out_loc column family name

* Implement reads and writes of address UTXOs

* Update raw data snapshots

* Update the snapshot tests for high-level address UTXOs

* Assert rather than taking empty address snapshots for genesis

* Update high-level address UTXO snapshot data, and delete empty snapshots

* Increment the database version

* Use typed values for all ReadDisk methods

* Implement test-only serialization for transparent::Address

* Implement FromDisk for ()

* Store AddressUnspentOutput as the column family key

* Update round-trip serialization tests for AddressUnspentOutput

* Update snapshot test code, and add a UTXO data snapshot

* Update existing snapshot data

* Add new UTXO snapshot data

* Update column family name

```sh
fastmod utxo_by_transparent_addr_loc utxo_loc_by_transparent_addr_loc zebra*
```

* cargo fmt --all

* cargo insta test --review --delete-unreferenced-snapshots

* Explain why it is ok to use invalid database iterator indexes

Co-authored-by: Conrado Gouvea <conrado@zfnd.org>

* Add explanations of UTXO database updates

* Simplify an assertion

* Remove UnspentOutputAddressLocation and just store transparent::Output

* Update snapshot test data

Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>
Co-authored-by: Conrado Gouvea <conrado@zfnd.org>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-04-13 04:06:52 +00:00
teor e49c1d7034
feat(state): add transparent address indexes to the non-finalized state (#4022)
* Derive Hash for transparent address index types

* Expose some types used by transparent address indexes

* Add an empty transparent transfers type for transparent address indexes

* Update TransparentTransfers with created UTXOs

* Add spent transparent outputs to ContextuallyValidBlock

* Update TransparentTransfers with spent transparent outputs

* Ignore missing spent outputs, so that tests pass

* Remove empty TransparentTransfers after a spend revert

* Update TransparentTransfers with creating and spending transaction IDs

* Ignore duplicate created UTXOs, so that tests pass

* Add some TODO comments

* Remove accidental doctest formatting

* Add address transfers index accessor methods

* Use TransactionLocation in the non-finalized state

* Apply more address index assertions to production code

* Refactor deeply nested code and apply more assertions

* Return UTXOs in chain order

* Return transaction hashes in chain order

* Stop indexing each transparent output multiple times

* Run some more asserts during tests

* Tidy TODO comments

* Fix an incorrect assert condition

* Use OrderedUtxos so that spent UTXOs can be stored in chain order

* Update tests to use OrderedUtxos

* Update the index API for the getaddressutxos query

* Remove redundant arguments in tests

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-04-12 17:21:46 +00:00
teor caac71a9d8
4. change(db): stop storing redundant transparent output fields in the database (#3992)
* Add Utxo constructors from output locations

* Store transparent outputs rather than Utxo structs

* Update raw data snapshots

* Increment the state version
2022-04-12 03:10:23 +00:00
teor 7e8194c63f
3. change(db): Store UTXOs by transaction location rather than transaction hash (#3978)
* Change OutputLocation to contain a TransactionLocation

* Change OutputLocation reads from the database

* Update some doc comments

* Update some TODOs

* Change deleting spent UTXOs and updating spent balances

* Change adding new UTXOs and adding their values to balances

* Disable dead code warnings

* Update snapshot test code

* Update round-trip tests for OutputLocations

* Update snapshot test data

* Increment the database format version

* Remove a redundant try_into()

Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>

* Refactor redundant code

Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>

* ci: attempt at fixing 'Regenerate stateful disks'

Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>
Co-authored-by: Conrado Gouvea <conrado@zfnd.org>
2022-04-08 22:42:05 +00:00
teor 7faa6a26c5
2. feat(db): Add address balance indexes to the finalized state (#3963)
* Add an empty balance_by_transparent_addr column family

* Add an AddressBalanceLocation type for balance_by_transparent_addr

* Add serialization for balance_by_transparent_addr types

* Add round-trip tests for the new serialized types

* Add missing round-trip and serialized equality tests

* Add a network field to DiskWriteBatch

* Refactor confusing all_utxos_spent_by_block argument

It was actually just the UTXOs from the state spent by the block,
excluding the UTXOs created and spent within the block.

But now we need it to contain all the spent outputs,
including the ones created by the block.

* Read and update address balances in the finalized state

* Update raw data snapshots for transparent address balances

* Add test-only deserialization for transparent addresses

* Add high-level snapshot test code for address balances

* Add high-level snapshots for address balances

* Increment the state version after NU5 testnet 2 rollback
2022-04-07 23:15:17 +00:00
teor 13c216cd3c
1.0.0-beta.7 Release (#4039)
* Update changelog for v1.0.0-beta.7

* Increment all crate versions

* Remove redundant release test that is now covered by CI

* Remove completed NU5 README check task from the release template

* Add Merge Freeze tool to the release checklist

* Simplify release checklist by removing unused steps
2022-04-05 18:06:46 +00:00
teor b5efeb4ab3
Encode CommitmentErrors as hex (#4021) 2022-04-01 22:31:03 +00:00
Conrado Gouvea ee6a38dffc
build(deps): bump serde-big-array from 0.3.2 to 0.4.1 (#4004)
Bumps [serde-big-array](https://github.com/est31/serde-big-array) from 0.3.2 to 0.4.1.
- [Release notes](https://github.com/est31/serde-big-array/releases)
- [Commits](https://github.com/est31/serde-big-array/compare/v0.3.2...v0.4.1)

---
updated-dependencies:
- dependency-name: serde-big-array
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-03-31 01:33:52 +00:00
Alfredo Garcia 53bf4c6ba5
1.0.0-beta.6 Release (#3966)
* 1.0.0-beta.6 release version bump

* 1.0.0-beta.6 release update the changelog

* change date of the release

* change description

Co-authored-by: teor <teor@riseup.net>

* remove phrase from description

Co-authored-by: teor <teor@riseup.net>
2022-03-28 22:36:42 +00:00
Deirdre Connolly 0821e30623
Repoint zebra image links to our new zfnd.org site for now (#3949)
* Repoint zebra image links to our new zfnd.org site for now

* Remove images/

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-03-27 23:42:47 +00:00
Alfredo Garcia f687ab947f
feat(rpc): Implement `getblockchaininfo` RPC method (#3891)
* Implement `getblockchaininfo` RPC method

* add a test for `get_blockchain_info`

* fix tohex/fromhex

* move comment

* Update lightwalletd acceptance test for getblockchaininfo RPC (#3914)

* change(rpc): Return getblockchaininfo network upgrades in height order (#3915)

* Update lightwalletd acceptance test for getblockchaininfo RPC

* Update some doc comments for network upgrades

* List network upgrades in order in the getblockchaininfo RPC

Also:
- Use a constant for the "missing consensus branch ID" RPC value
- Simplify fetching consensus branch IDs
- Make RPC type derives consistent
- Update RPC type documentation

* Make RPC type derives consistent

* Fix a confusing test comment

* get hashand height at the same time

* fix estimated_height

* fix lint

* add extra check

Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>

* fix typo

Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>

* split test

Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>

* fix(rpc): ignore an expected error in the RPC acceptance tests (#3961)

* Add ignored regexes to test command failure regex methods

* Ignore empty chain error in getblockchaininfo

We expect this error when zebrad starts up with an empty state.

Co-authored-by: teor <teor@riseup.net>
Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-03-25 12:25:31 +00:00
Conrado Gouvea e7c0a78d4d
feat(rpc): add getrawtransaction (#3908)
* feat(rpc): add getrawtransaction

* Apply suggestions from code review

Co-authored-by: teor <teor@riseup.net>
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>

* Apply suggestions from code review

Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>

* address review comments

* move SerializedTransaction to the right module

* Use try_into() instead of 'as'

* add proptests

Co-authored-by: teor <teor@riseup.net>
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-03-24 09:45:37 +00:00
teor b4deca2912
6. add(test): add more tests for height and transaction location serialization (#3879)
* Add round-trip test for OutputLocation disk serialization

* Make the transaction snapshot tests more accurate

Previously, we were showing the genesis transaction hash at height 0, index 0.

But its hash is actually not stored by location in the database,
because the genesis transaction is skipped due to a consensus rule.

* Update the transaction snapshot data

* Add history tree snapshot tests

At the current test heights, the history trees are all empty.

* Add the history tree snapshot data

* Update comments

* Simplify snapshot test code

* Make some serde::Serialize impls test-only, so we can hex-encode them

This should also speed up release compilation a bit.

* Add snapshot test code for UTXOs

* Add snapshot data for UTXOs
2022-03-18 20:30:16 +00:00
Conrado Gouvea b9640fbb55
fix(chain): make FromHex consistent with ToHex for tx/block hashes (#3893) 2022-03-17 20:24:48 +00:00
Janito Vaqueiro Ferreira Filho b563b2a1c1
feat(consensus): Validate ZIP-212 grace period blocks using checkpoints (#3889)
* Test if mandatory checkpoint is after grace period

Check that the mandatory checkpoint height is after the ZIP-212 grace
period for both mainnet and testnet, because Zebra is currently unable
to fully validate the blocks during that grace period.

* Update mandatory checkpoint height

Require it to be after the ZIP-212 grace period so that the blocks
during the grace period are validated by the checkpoints because Zebra
can't currently fully validate blocks during that period.
2022-03-17 00:37:50 +00:00
Alfredo Garcia f39ac48c59
feature(rpc): Implement `getbestblockhash` method (#3754)
* feature(rpc): start adding a `getblock` method

* fix(rpc): replace oneshot

* fix(rpc): replace a panic with error

* fix(rpc): fix test

* feature(rpc): add hex to response

* refactor(rpc): use generic instead of alias

* docs(rpc): improve docs for getblock method

* test(rpc): add a test for getblock method

* deps(rpc): remove non needed tower features

Co-authored-by: teor <teor@riseup.net>

* docs(rpc): add a note to getblock doc

* refactor(rpc): replace alias

* fix(rpc): use `zcash_serialize_to_vec()` instead of logging format

* tests(rpc): add network argument to `populated_state()`

* refactor(rpc): use an error for state service readiness

* fix(rpc): add parameter

* fix(rpc): clippy

* nit(rpc): remove new line from imports

* fix(rpc): remove commented code

* fix(rpc): simplify error

Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>

* Use a `SerializedBlock` type to help serializing blocks (#3725)

* Create a `SerializedBlock` helper type

Create a type that can be used as a byte slice, but is guaranteed to
represent a valid block.

* Use `into_iter` instead of `iter`

There's no need to borrow the elements, they can be moved out directly.
This will be necessary because `&Arc<T>` doesn't implement `Borrow<T>`,
so a `SerializedBlock` can't be built directly from an `&Arc<Block>`.

* Use `SerializedBlock` in `GetBlock`

Make the type stricter to avoid storing possibly invalid values. The
bytes are still serialized as a hexadecimal string, through the usage of
`hex`.

The `serde::Deserialize` can't be derived because `hex` requires the
type to also implement `FromHex`.

* feature(rpc): add suggestions from code review

Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>

* tests(rpc): make sure mempool has no requests in get_block test

* fix(rpc): change height argument type in getblock method

* fix(rpc): rustfmt

* fix(rpc): replace panic

* fix(rpc): change getblock response

* fix(rpc): fix lightwalletd test

* tests(rpc): add a getblock error test

* fix(rpc): try another regex

* feature(rpc): add `getbestblockhash` RPC method

* feature(rpc): Add a `pub struct SerializedBlockHash` type

* tests(rpc): add a unit test for `getbestblockhash` method

* tests(rpc): make sure no requests are sent to mempool in getbestblockhash test

* tests(rpc): refactor check

Co-authored-by: teor <teor@riseup.net>

* fix(rpc): fixes after rebase

* refactor(rpc): refactor `GetBestBlockHash`

* fix(rpc): unused variables

Co-authored-by: teor <teor@riseup.net>

* docs(rpc): update

* fix(rpc): add panic

* fix(rpc): fix panic

Co-authored-by: teor <teor@riseup.net>
Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>
2022-03-11 05:13:08 +00:00
Conrado Gouvea 9862f6e5cf
feat: get addresses from transparent outputs (#3802)
* feat: add Output::address()

* test transactions in test blocks
2022-03-11 03:23:04 +00:00
Alfredo Garcia 833560411f
feature(rpc): implement getblock api call (#3707)
* feature(rpc): start adding a `getblock` method

* fix(rpc): replace oneshot

* fix(rpc): replace a panic with error

* fix(rpc): fix test

* feature(rpc): add hex to response

* refactor(rpc): use generic instead of alias

* docs(rpc): improve docs for getblock method

* test(rpc): add a test for getblock method

* deps(rpc): remove non needed tower features

Co-authored-by: teor <teor@riseup.net>

* docs(rpc): add a note to getblock doc

* refactor(rpc): replace alias

* fix(rpc): use `zcash_serialize_to_vec()` instead of logging format

* tests(rpc): add network argument to `populated_state()`

* refactor(rpc): use an error for state service readiness

* fix(rpc): add parameter

* fix(rpc): clippy

* nit(rpc): remove new line from imports

* fix(rpc): remove commented code

* fix(rpc): simplify error

Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>

* Use a `SerializedBlock` type to help serializing blocks (#3725)

* Create a `SerializedBlock` helper type

Create a type that can be used as a byte slice, but is guaranteed to
represent a valid block.

* Use `into_iter` instead of `iter`

There's no need to borrow the elements, they can be moved out directly.
This will be necessary because `&Arc<T>` doesn't implement `Borrow<T>`,
so a `SerializedBlock` can't be built directly from an `&Arc<Block>`.

* Use `SerializedBlock` in `GetBlock`

Make the type stricter to avoid storing possibly invalid values. The
bytes are still serialized as a hexadecimal string, through the usage of
`hex`.

The `serde::Deserialize` can't be derived because `hex` requires the
type to also implement `FromHex`.

* feature(rpc): add suggestions from code review

Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>

* tests(rpc): make sure mempool has no requests in get_block test

* fix(rpc): change height argument type in getblock method

* fix(rpc): rustfmt

* fix(rpc): replace panic

* fix(rpc): change getblock response

* fix(rpc): fix lightwalletd test

* tests(rpc): add a getblock error test

* fix(rpc): try another regex

Co-authored-by: teor <teor@riseup.net>
Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>
2022-03-10 01:12:41 +00:00
teor 7283b4bfd0
4. test(db): add large transaction tests (#3759)
* refactor(test/block): rename large single transaction function

```sh
fastmod single_transaction_block single_transaction_block_many_inputs
```

* rustfmt

* test(block): add a test block with many transparent outputs

* doc(db): explain why we can't just get the UTXOs right before they are deleted

* refactor(db): split out a block data write method

* refactor(block): add a height argument to new_outputs

* test(db): add block and transaction round-trip tests

Including large blocks and transactions.

* test(db): fix large block serialization instability in the tests

* doc(block): add TODOs for generating correct blocks

* Make transparent output functions which take a height test-only

* make sure generated blocks are actually over/under-sized

* replace println!() with an error!() log
2022-03-09 23:34:50 +00:00
teor 3291db35c0
fix(shielded): use RwLock for note commitment tree root caches (#3809)
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-03-09 23:26:49 +00:00
Janito Vaqueiro Ferreira Filho 0e0aefaa4e
Refactor `SentTransactionHash` to be a stricter type (#3706)
* Stub `sendrawtransaction` RPC method

Register the RPC method, and stub an implementation that currently just
panics. The method has a single `String` parameter with the hexadecimal
string of the raw transaction's bytes and returns a
`SentTransactionHash` wrapper type that's just a hexadecimal `String` of
the sent transaction's hash.

* Add mempool service instance to `RpcImpl`

Use a type parameter to represent the mempool service using the
interface defined by `zebra-node-services`.

* Update test vector to use a mock mempool service

Update the test to be compatible with the changes to `RpcImpl`. The mock
mempool service is expected to not be used during the test.

* Use a `tower::Buffer` for the mempool service

Make it simpler to send requests to the service in a concurrent manner.

* Return a `Future` from `send_raw_transaction`

Make the call asynchronous.

* Implement `sendrawtransaction` RPC

Deserialize the transaction and send it to be queued for verification
and subsequent inclusion in the mempool.

* Test if mempool receives sent raw transaction

Use a mock service as the mempool service and check that it receives a
sent raw transaction.

* Test using non-hexadecimal string parameter

The method should return an error.

* Test with bytes that fail deserialization

Check that the method returns an invalid parameters error if the input
can't be deserialized as a `Transaction`.

* Test if mempool errors are forwarded to caller

Mempool service errors should be sent back to the remote caller as
server errors.

* Test transactions rejected by the mempool service

Transactions that are rejected by the mempool service should result in
a server error being sent to the caller.

* Improve error message

Add the word "structurally" to make it clear that the issue is in the
transaction's deserialization.

Co-authored-by: Deirdre Connolly <durumcrustulum@gmail.com>

* Add note regarding missing `allowhighfees` param.

The parameter isn't supported yet because `lightwalletd` doesn't use it.

* Update the documentation to be consistent

Follow the convention adopted by the `get_info` RPC method.

* Implement `ToHex` and `FromHex` for `Hash`

Make it easier to generate hexadecimal strings from `transaction::Hash`
instances.

* Use `ToHex` in `Debug` and `Display`

Reduce repeated code.

* Refactor to add `bytes_in_display_order` method

Use it to remove repeated code and improve clarity a bit.

* Use `hex::serialize` to serialize transaction hash

Make the type stricter in its contents, while still serializing the
transaction has as a hexadecimal string.

* Simplify serialization attribute

Deserialization should also use `hex::deserialize`, so using the shorter
attribute makes things easier to read and more future proof.

* Update zebra-chain/src/transaction/hash.rs

* Remove unnecessary lifetime

The anonymous lifetime is automatically inferred by the compiler.

Co-authored-by: Deirdre Connolly <durumcrustulum@gmail.com>
2022-03-08 09:14:21 +00:00
teor cef146edbd
lint(clippy): warn on manual printing to stdout or stderr (#3767)
Most logging should use `tracing::trace!()` or `tracing::debug!()` instead.
2022-03-08 09:14:15 +00:00
teor d718f1d922
fix(zebrad/test): stop excessive logging which causes test hangs (#3755)
* fix(zebrad/test): stop excessive logging which causes test hangs

If we print a line every time the network upgrade heights are accessed,
some tests hang. This could happen because they are expecting different
log output, or because the standard output pipes fill up, blocking the
`zebrad` subprocess.

* doc(test/nu): explain cargo feature unification and test env vars
2022-03-07 08:44:03 -04:00
Marek 2cc7e158a2
Upgrade dependencies (#3625)
* Upgrade some dependencies

* Upgrade some dependencies

* Upgrade dependencies for zebrad

* Upgrade tracing dependencies

* Revert `tor` & `arti`

* Upgrade `criterion` & `pin-project` in `deny.toml`

* Remove some dependencies from `skip-tree` in `deny.toml`

* Revert some the versions of dependencies because of duplicates

* Revert proptest regressions

* Upgrade dependencies, then ignore some more duplicates (#3716)

* feat(actions)!: add full sync test (#3582)

* add(tests): full sync test

* fix(test): add build

* fix(deploy): escape double dashes '--' correctly

* fix(test): remove unexpected --no-capture arg

error: Found argument '--nocapture' which wasn't expected, or isn't valid in this context

* refactor(docker): use default executable as entrypoint

* refactor(startup): add a custom entrypoint

* fix(test): add missing TEST_FULL_SYNC variable

* test(timeout): use the biggest machine

* fix

* fix(deploy): use latest successful image

* typo

* refactor(docker): generate config file at startup

* revert(build): changes were made to docker

* fix(docker): send variables correctly to the entrypoint

* test different conf file approach

* fix(env): add RUN_TEST env variable

* ref: use previous approach

* fix(color): use environment variable

* fix(resources): use our normal machine size

* fix(ci): double CPU and RAM for full sync test

* fix(test): check for zebrad test output in the correct order

The mempool is only activated once, so we must check for that log first.
After mempool activation, the stop regex is logged at least once.
(It might be logged before as well, but we can't rely on that.)

When checking that the mempool didn't activate,
wait for the `zebrad` command to exit,
then check the entire log.

* fix(ci): run full sync test with full compiler optimisations

* fix(tests): reintroduce tests and run full sync on approval

* fix(tests): reduce the changelog

Co-authored-by: teor <teor@riseup.net>

* fix(ci): update CI job path triggers (#3692)

* ci(test): re-run tests when snapshot data changes

* fix(ci): rebuild state when disk format changes

* fix(ci): rebuild rust docs when code or dependencies change

* doc(ci): explain why we run jobs when files change

Co-authored-by: Gustavo Valverde <gustavo@iterativo.do>

* fix(build): use the right multistage target (#3700)

* fix(review): only assign one reviewer to general Rust reviews (#3708)

If we assign two teams, GitHub assigns two reviewers.

* fix(ci): change the color-eyre ignore to a tracing-subscriber ignore

* fix(ci): ignore duplicate darling dependencies

* doc(ci): remove an alternative resolution doc

Co-authored-by: Gustavo Valverde <gustavo@iterativo.do>

Co-authored-by: teor <teor@riseup.net>
Co-authored-by: Gustavo Valverde <gustavo@iterativo.do>
2022-03-07 12:07:25 +10:00
Conrado Gouvea 2f6a48bed8
fix(tests): use TEST_FAKE_ACTIVATION_HEIGHTS at runtime and fix tests (#3749)
* fix: check TEST_FAKE_ACTIVATION_HEIGHTS at runtime

* fix(tests): add TEST_FAKE_ACTIVATION_HEIGHTS variable

This variable ensures the test is activated in the `test-fake-activation-heights` step

* fix(docker): do not run specific tests by default in entrypoint.sh

* fix(test): remove extra TEST_FULL_SYNC argument

* imp(timeout): wait for an average build time

Co-authored-by: Gustavo Valverde <gustavo@iterativo.do>
2022-03-05 13:32:46 -04:00
dependabot[bot] 30cc048166
build(deps): bump secp256k1 from 0.21.2 to 0.21.3 (#3632)
Bumps [secp256k1](https://github.com/rust-bitcoin/rust-secp256k1) from 0.21.2 to 0.21.3.
- [Release notes](https://github.com/rust-bitcoin/rust-secp256k1/releases)
- [Changelog](https://github.com/rust-bitcoin/rust-secp256k1/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-bitcoin/rust-secp256k1/commits)

---
updated-dependencies:
- dependency-name: secp256k1
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-03-02 00:49:09 +00:00
dependabot[bot] eaf0b6add4
build(deps): bump sha2 from 0.9.8 to 0.9.9 (#3585)
Bumps [sha2](https://github.com/RustCrypto/hashes) from 0.9.8 to 0.9.9.
- [Release notes](https://github.com/RustCrypto/hashes/releases)
- [Commits](https://github.com/RustCrypto/hashes/compare/sha2-v0.9.8...sha2-v0.9.9)

---
updated-dependencies:
- dependency-name: sha2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-02-22 05:36:14 +00:00
Conrado Gouvea b7b62af2f2
Document consensus rules from 4.6 Action Descriptions (#3549)
* docs: document consensus rules from 4.6 Action Descriptions

* Apply suggestions from code review

Co-authored-by: Marek <mail@marek.onl>

* Apply suggestions from code review

Co-authored-by: Marek <mail@marek.onl>

Co-authored-by: Marek <mail@marek.onl>
2022-02-21 23:49:32 +00:00