Commit Graph

59 Commits

Author SHA1 Message Date
teor 8a3e8f8432
Match zcashd's getblockchaininfo capitalisation for NU5 (#7454) 2023-09-05 23:07:26 +00:00
teor c3f0f53256
refactor(app): De-duplicate and fix version handling code (#6996)
* De-duplicate app_version and user_agent code, rename to build_version

* Make RPC testnet flag forward-compatible with additional testnets

* Fix RPC tests with new argument

* Use "modified" rather than "dirty" for uncommitted changes in build metadata

* Split the vergen version into its own function
2023-06-20 02:42:06 +00:00
teor 04e96c2526
feat(net): Cache a list of useful peers on disk (#6739)
* Rewrite some state cache docs to clarify

* Add a zebra_network::Config.cache_dir for peer address caches

* Add new config test files and fix config test failure message

* Create some zebra-chain and zebra-network convenience functions

* Add methods for reading and writing the peer address cache

* Add cached disk peers to the initial peers list

* Add metrics and logging for loading and storing the peer cache

* Replace log of useless redacted peer IP addresses

* Limit the peer cache minimum and maximum size, don't write empty caches

* Add a cacheable_peers() method to the address book

* Add a peer disk cache updater task to the peer set tasks

* Document that the peer cache is shared by multiple instances unless configured otherwise

* Disable peer cache read/write in disconnected tests

* Make initial peer cache updater sleep shorter for tests

* Add unit tests for reading and writing the peer cache

* Update the task list in the start command docs

* Modify the existing persistent acceptance test to check for peer caches

* Update the peer cache directory when writing test configs

* Add a CacheDir type so the default config can be enabled, but tests can disable it

* Update tests to use the CacheDir config type

* Rename some CacheDir internals

* Add config file test cases for each kind of CacheDir config

* Panic if the config contains invalid socket addresses, rather than continuing

* Add a network directory to state cache directory contents tests

* Add new network.cache_dir config to the config parsing tests
2023-06-06 08:28:14 +00:00
teor 166526a088
feat(ui): Add a terminal-based progress bar to Zebra (#6235)
* Implement Display and to_string() for NetworkUpgrade

* Add a progress-bar feature to zebrad

* Add the progress bar writer to the tracing component

* Add a block progress bar transmitter

* Correctly shut down the progress bar, and shut it down on an interrupt

* Make it clearer that the progress task never exits

* Add a config for writing logs to a file

* Add a progress-bar feature to zebra-network

* Add a progress bar for the address book size

* Add progress bars for never attempted and failed peers

* Add an optional limit and label to connection counters

* Add open connection progress bars

* Improve CheckpointList API and CheckpointVerifier debugging

* Add checkpoint index and checkpoint queue progress bars

* Security: Limit the number of non-finalized chains tracked by Zebra

* Make some NonFinalizedState methods available with proptest-impl

* Add a non-finalized chain count progress bar

* Track the last fork height for newly forked chains

* Add a should_count_metrics to Chain

* Add a display method for PartialCumulativeWork

* Add a progress bar for each chain fork

* Add a NonFinalizedState::disable_metrics() method and switch to using it

* Move metrics out of Chain because we can't update Arc<Chain>

* Fix: consistently use best chain order when searching chains

* Track Chain progress bars in NonFinalizedState

* Display work as bits, not a multiple of the target difficulty

* Handle negative fork lengths by reporting "No fork"

* Correctly disable unused fork bars

* clippy: rewrite using `match _.cmp(_) { ... }`

* Initial mempool progress bar implementation

* Update Cargo.lock

* Add the actual transaction size as a description to the cost bar

* Only show mempool progress bars after first activation

* Add queued and rejected mempool progress bars

* Clarify cost note is actual size

* Add tracing.log_file config and progress-bar feature to zebrad docs

* Derive Clone for Chain

* Upgrade to howudoin 0.1.2 and remove some bug workarounds

* Directly call the debug formatter to Display a Network

Co-authored-by: Arya <aryasolhi@gmail.com>

* Rename the address count metric to num_addresses

Co-authored-by: Arya <aryasolhi@gmail.com>

* Simplify reverse checkpoint lookup

Co-authored-by: Arya <aryasolhi@gmail.com>

* Simplify progress bar shutdown code

Co-authored-by: Arya <aryasolhi@gmail.com>

* Remove unused MIN_TRANSPARENT_TX_MEMPOOL_SIZE

* Document that the progress task runs forever

* Fix progress log formatting

* If progress-bar is on, log to a file by default

* Create missing directories for log files

* Add file security docs for running Zebra with elevated permissions

* Document automatic log file, spell progress-bar correctly

---------

Co-authored-by: Arya <aryasolhi@gmail.com>
2023-04-13 08:42:17 +00:00
Marek 2a48d4cf25
change(chain): Refactor the handling of height differences (#6330)
* Unify the `impl`s of `Sub` and `Add` for `Height`

* Adjust tests for `Height` subtraction

* Use `Height` instead of `i32`

* Use `block:Height` in RPC tests

* Use `let .. else` statement

Co-authored-by: Arya <aryasolhi@gmail.com>

* Update zebra-consensus/src/block/subsidy/general.rs

* Refactor the handling of height differences

* Remove a redundant comment

* Update zebrad/src/components/sync/progress.rs

Co-authored-by: Arya <aryasolhi@gmail.com>

* Update progress.rs

* impl TryFrom<u32> for Height

* Make some test assertions clearer

* Refactor estimate_up_to()

* Restore a comment that was accidentally removed

* Document when estimate_distance_to_network_chain_tip() returns None

* Change HeightDiff to i64 and make Height.sub(Height) return HeightDiff (no Option)

* Update chain tip estimates for HeightDiff i64

* Update subsidy for HeightDiff i64

* Fix some height calculation test edge cases

* Fix the funding stream interval calculation

---------

Co-authored-by: Arya <aryasolhi@gmail.com>
Co-authored-by: teor <teor@riseup.net>
2023-03-29 23:06:31 +00:00
teor 256b1c0008
fix(clippy): Fix new lints in nightly clippy (#5959)
* Derive default using #[default]

* Implement PartialEq manually to satisfy clippy

* Allow a manual derive in test-only code

* Fix some missing docs warnings in the Docker build
2023-01-18 00:27:42 +00:00
teor e9d6e975b4
change(rpc): Refactor get block template RPC into stages (#5837)
* Add some TODOs

* Move and rename height_from_signed_int()

* Move get_block_template() support functions to a submodule

* Fix incorrect P2SH comments and logs

* Split initial checks into their own functions

* Split state fetch into its own function, do some cleanup

* Move get_block_template_opts to get_block_template::parameters

* Fix and simplify test imports

* Rename block_height to next_block_height

* Rename to chain_tip_and_local_time to make it clear what it contains

* Split fetching mempool transactions out, include them in long poll id

* Refactor coinbase generation

* Split default root calculation into a separate function

* Use DateTime32 for getblocktemplate times

* Use typed difficulty fields rather than strings

* Split out a generate coinbase and roots function

* Move GetBlockTemplate construction into a method

* Document what happens to unusual difficulty values

* Clean up some TODOs

* fastmod check_address check_miner_address

* cargo fmt --all

* Fix an incorrect panic message

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-12-13 21:25:04 +00:00
Conrado Gouvea 6fd750e168
build(deps): bump insta from 1.15.0 to 1.17.1 (#4884)
* build(deps): bump insta from 1.15.0 to 1.17.1

Bumps [insta](https://github.com/mitsuhiko/insta) from 1.15.0 to 1.17.1.
- [Release notes](https://github.com/mitsuhiko/insta/releases)
- [Changelog](https://github.com/mitsuhiko/insta/blob/master/CHANGELOG.md)
- [Commits](https://github.com/mitsuhiko/insta/compare/1.15.0...1.17.1)

---
updated-dependencies:
- dependency-name: insta
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* make zebra_test::init() return the insta drop guard

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-08-04 15:44:44 +00:00
teor 42ef8846b3
fix(clippy): resolve or disable new nightly clippy lints (#4739)
* Fix new dead_code lints in generated pin-project code

* Fix and ignore new needless_borrow lints
2022-07-05 11:04:47 +10:00
Marek b8b35f8da9
fix(doc): Fix various doc warnings, part 1 (#4514)
* Fix the syntax of links in comments

* Fix a mistake in the docs

Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>

* Remove unnecessary angle brackets from a link

* Revert the changes for links that serve as references

* Revert "Revert the changes for links that serve as references"

This reverts commit 8b091aa9fa.

* Remove `<` `>` from links that serve as references

This reverts commit 046ef25620.

* Don't use `<` `>` in normal comments

* Don't use `<` `>` for normal comments

* Revert changes for comments starting with `//`

* Fix some warnings produced by `cargo doc`

* Fix some rustdoc warnings

* Fix some warnings

* Refactor some changes

Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
2022-06-02 15:07:35 +00:00
Marek 6f896ef5a5
fix(doc): Fix the syntax of links in comments (#4494)
* Fix the syntax of links in comments

* Fix a mistake in the docs

Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>

* Remove unnecessary angle brackets from a link

* Revert the changes for links that serve as references

* Revert "Revert the changes for links that serve as references"

This reverts commit 8b091aa9fa.

* Remove `<` `>` from links that serve as references

This reverts commit 046ef25620.

* Don't use `<` `>` in normal comments

* Don't use `<` `>` for normal comments

* Revert changes for comments starting with `//`

Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
2022-05-30 20:12:11 +00:00
Conrado Gouvea 9aaf0ed0ce
change: set NU5 mainnet activation height and current network protocol version (#4390)
* change: set NU5 mainnet activation height and minimum protocol version

* fix tests

* test if V5 coinbase expiry height is not bound by non-coinbase limit

* update get_blockchain_info snapshot for NU5

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-05-19 01:04:11 +00:00
Janito Vaqueiro Ferreira Filho 9538ad29e5
feat(util): add a `zebra-tip-height` utility (#4289)
* Move `init_tracing` to `lib.rs`

Allow it to be reused by other binaries.

* Fix `hex` dependency inclusion in `zebra-chain`

It would previously fail because `hex` was being included without the
`serde` feature, even though we required that for `transparent::Script`.

* Implement `FromStr` for `Network`

Make it easy to receive it as a command-line parameter.

* Add `zebra-tip-height` utility

Obtains the chain tip height of a directory containing Zebra state.

* Remove Tokio dependency from `zebra-utils`

It wasn't actually used by the `zebra-tip-height` utility.

* Remove `BoxStateService` type alias

It's not needed if we don't return the unused state service. This also
allows removing the `tower` dependency.

* Remove unnecessary attribute

Leftover from copied code.

* Make `cache_dir` argument optional

Fallback to the default Zebra state cache directory.

* Remove added newline

Minor formatting fix, to avoid adding an unnecessary newline.

* Move `tip-height` command into `zebrad`

Make it available as a sub-command in `zebrad`.

* Make some zebrad sub-commands only log warnings

Co-authored-by: teor <teor@riseup.net>
2022-05-06 03:31:52 +00:00
teor 5054e048d4
Require a minimum network protocol version of 170_040 (#3976)
This is the `zcashd` network protocol version:
- after the first NU5 testnet activation, and
- after updating to the second NU5 testnet activation consensus rules,
- but before setting the second NU5 testnet activation height.
2022-04-19 16:14:24 +00:00
Conrado Gouvea d79e71e969
change(nu5): use new V5 transaction script verification API (#3799)
* update librustzcash; adapt to new API

* add ticket reference for removing zcash_proofs duplicated dependencies

* update to new zcash_script V5 API

* use zp_tx shorthand

* update to Zcash 4.7.0 dependencies

* update protocol versions

* feat(rpc): Implement `getblockchaininfo` RPC method (#3891)

* Implement `getblockchaininfo` RPC method

* add a test for `get_blockchain_info`

* fix tohex/fromhex

* move comment

* Update lightwalletd acceptance test for getblockchaininfo RPC (#3914)

* change(rpc): Return getblockchaininfo network upgrades in height order (#3915)

* Update lightwalletd acceptance test for getblockchaininfo RPC

* Update some doc comments for network upgrades

* List network upgrades in order in the getblockchaininfo RPC

Also:
- Use a constant for the "missing consensus branch ID" RPC value
- Simplify fetching consensus branch IDs
- Make RPC type derives consistent
- Update RPC type documentation

* Make RPC type derives consistent

* Fix a confusing test comment

* get hashand height at the same time

* fix estimated_height

* fix lint

* add extra check

Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>

* fix typo

Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>

* split test

Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>

* fix(rpc): ignore an expected error in the RPC acceptance tests (#3961)

* Add ignored regexes to test command failure regex methods

* Ignore empty chain error in getblockchaininfo

We expect this error when zebrad starts up with an empty state.

Co-authored-by: teor <teor@riseup.net>
Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

* Make sync error logs more user-friendly (#3944)

- use info level, there is nothing the user needs to do,
  particularly for a single error
- explain that the errors are temporary
- hide backtraces, because they look like crashes

* Update test.patch.yml with lightwalletd job (#3970)

* Update test.patch.yml with lightwalletd job

* Remove a workflow condition that will always be false

In general, patch workflows need the
opposite conditions to the original workflow.

But in this case, we know the result of the
condition will always be true, so we can just delete it.

Co-authored-by: teor <teor@riseup.net>

* fix(doc): Fix bugs in the lightwalletd database design (#3964)

* Re-order column families in design in dependency order

* Minor RFC design tweaks and fixes

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

* Repoint zebra image links to our new zfnd.org site for now (#3949)

* Repoint zebra image links to our new zfnd.org site for now

* Remove images/

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

* Fix typos (#3956)

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: teor <teor@riseup.net>

* bump database version to trigger testnet rollback

* reduce minimum protocol version for now (will be changed later)

* update dependencies

* Apply suggestions from code review

Co-authored-by: teor <teor@riseup.net>

* update versions to match zcash 4.7.0

* deny.toml: update 'darling'

Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
Co-authored-by: teor <teor@riseup.net>
Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Deirdre Connolly <durumcrustulum@gmail.com>
Co-authored-by: Dimitris Apostolou <dimitris.apostolou@icloud.com>
2022-04-19 10:14:16 +10:00
Alfredo Garcia f687ab947f
feat(rpc): Implement `getblockchaininfo` RPC method (#3891)
* Implement `getblockchaininfo` RPC method

* add a test for `get_blockchain_info`

* fix tohex/fromhex

* move comment

* Update lightwalletd acceptance test for getblockchaininfo RPC (#3914)

* change(rpc): Return getblockchaininfo network upgrades in height order (#3915)

* Update lightwalletd acceptance test for getblockchaininfo RPC

* Update some doc comments for network upgrades

* List network upgrades in order in the getblockchaininfo RPC

Also:
- Use a constant for the "missing consensus branch ID" RPC value
- Simplify fetching consensus branch IDs
- Make RPC type derives consistent
- Update RPC type documentation

* Make RPC type derives consistent

* Fix a confusing test comment

* get hashand height at the same time

* fix estimated_height

* fix lint

* add extra check

Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>

* fix typo

Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>

* split test

Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>

* fix(rpc): ignore an expected error in the RPC acceptance tests (#3961)

* Add ignored regexes to test command failure regex methods

* Ignore empty chain error in getblockchaininfo

We expect this error when zebrad starts up with an empty state.

Co-authored-by: teor <teor@riseup.net>
Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-03-25 12:25:31 +00:00
Janito Vaqueiro Ferreira Filho b563b2a1c1
feat(consensus): Validate ZIP-212 grace period blocks using checkpoints (#3889)
* Test if mandatory checkpoint is after grace period

Check that the mandatory checkpoint height is after the ZIP-212 grace
period for both mainnet and testnet, because Zebra is currently unable
to fully validate the blocks during that grace period.

* Update mandatory checkpoint height

Require it to be after the ZIP-212 grace period so that the blocks
during the grace period are validated by the checkpoints because Zebra
can't currently fully validate blocks during that period.
2022-03-17 00:37:50 +00:00
teor cef146edbd
lint(clippy): warn on manual printing to stdout or stderr (#3767)
Most logging should use `tracing::trace!()` or `tracing::debug!()` instead.
2022-03-08 09:14:15 +00:00
teor d718f1d922
fix(zebrad/test): stop excessive logging which causes test hangs (#3755)
* fix(zebrad/test): stop excessive logging which causes test hangs

If we print a line every time the network upgrade heights are accessed,
some tests hang. This could happen because they are expecting different
log output, or because the standard output pipes fill up, blocking the
`zebrad` subprocess.

* doc(test/nu): explain cargo feature unification and test env vars
2022-03-07 08:44:03 -04:00
Conrado Gouvea 2f6a48bed8
fix(tests): use TEST_FAKE_ACTIVATION_HEIGHTS at runtime and fix tests (#3749)
* fix: check TEST_FAKE_ACTIVATION_HEIGHTS at runtime

* fix(tests): add TEST_FAKE_ACTIVATION_HEIGHTS variable

This variable ensures the test is activated in the `test-fake-activation-heights` step

* fix(docker): do not run specific tests by default in entrypoint.sh

* fix(test): remove extra TEST_FULL_SYNC argument

* imp(timeout): wait for an average build time

Co-authored-by: Gustavo Valverde <gustavo@iterativo.do>
2022-03-05 13:32:46 -04:00
Janito Vaqueiro Ferreira Filho eb98b7a4b2
Estimate network chain tip height based on local node time and current best tip (#3492)
* Remove redundant documentation

The documentation was exactly the same as the documentation from the
trait.

* Calculate a mock time block delta for tests

Simulate a block being added to the chain with a random block time based
on the previous block time and the target spacing time.

* Add a `time` field to `ChainTipBlock`

Store the block time so that it's ready for a future chain that allows
obtaining the chain tip's block time.

* Add `ChainTip::best_tip_block_time` method

Allow obtaining the bes chain tip's block time.

* Add method to obtain both height and block time

Prevent any data races by returning both values so that they refer to
the same chain tip.

* Add `NetworkUpgrade::all_target_spacings` method

Returns all the target spacings defined for a network.

* Create a `NetworkChainTipEstimator` helper type

Isolate the code to calculate the height estimation in a new type, so
that it's easier to understand and doesn't decrease the readability of
the `chain_tip.rs` file.

* Add `ChainTip::estimate_network_chain_tip_height`

This is more of an extension method than a trait method. It uses the
`NetworkChainTipHeightEstimator` to actually perform the estimation, but
obtains the initial information from the current best chain tip.

* Fix typo in documentation

There was an extra closing bracket in the summary line.

* Refactor `MockChainTipSender` into a separate type

Prepare to allow mocking the block time of the best tip as well as the
block height.

* Allow sending mock best tip block times

Add a separate `watch` channel to send the best tip block times from a
`MockChainTipSender` to a `MockChainTip`.

The `best_tip_height_and_block_time` implementation will only return a
value if there's a height and a block time value for the best tip.

* Fix off-by-one height estimation error

Use Euclidean division to force the division result to round down
instead of rounding towards zero. This fixes an off-by-one error when
estimating a height that is lower than the current height, because the
fractionary result was being discarded, and it should have forced the
height to go one block back.

* Fix panics on local times very far in the past

Detect situations that might cause the block height estimate to
underflow, and return the genesis height instead.

* Fix another off-by-one height estimation error

The implementation of `chrono::Duration::num_seconds` adds one to the
number of seconds if it's negative. This breaks the division
calculation, so it has to be compensated for.

* Test network chain tip height estimation

Generate pairs of block heights and check that it's possible to estimate
the larger height from the smaller height and a displaced time
difference.
2022-02-11 01:27:02 +00:00
teor 4f0d7bd737
lint: add extra integer lints, and partially fix some code (#3409)
* lint: enable more clippy checks for bug-prone code

* fix(lint): stop denying lints, to avoid being excluded from Crater

Also categorise lints.

* lint: add some lints to the TODO list

* refactor(arithmetic): partial fixes for some integer arithmetic lints

* Document some weird lint behaviour
2022-01-27 11:34:15 -03:00
teor 6cbd7dce43
Fix task handling bugs, so peers are more likely to be available (#3191)
* Tweak crawler timings so peers are more likely to be available

* Tweak min peer connection interval so we try all peers

* Let other tasks run between fanouts, so we're more likely to choose different peers

* Let other tasks run between retries, so we're more likely to choose different peers

* Let other tasks run after peer crawler DemandDrop

This makes it more likely that peers will become ready.
2021-12-20 09:02:31 +10:00
Janito Vaqueiro Ferreira Filho 339fefb6e6
Update shared NU5 dependencies, set the NU5 testnet activation network upgrade parameters (#2825)
* Check return value of zcash_script_new_precomputed_tx

* Set the NU5 testnet activation height to 1_590_000

* Apply suggestions from code review

Co-authored-by: teor <teor@riseup.net>

* Update Nu5 constants to new values

* Update ZIP-244 test vectors for new branch ID

* Squashed commit of the following:

commit bdb120a249
Author: Deirdre Connolly <durumcrustulum@gmail.com>
Date:   Tue Oct 5 11:54:01 2021 -0400

    Use pallas::Base::from_str_vartime() in sinsemilla tests

commit e99fa49258
Author: Deirdre Connolly <durumcrustulum@gmail.com>
Date:   Tue Oct 5 11:45:24 2021 -0400

    Compiles

commit a520018114
Author: Deirdre Connolly <durumcrustulum@gmail.com>
Date:   Tue Oct 5 10:15:17 2021 -0400

    Incomplete upgrade of deps

* Squashed commit of the following:

commit 8d1b76ec5626517817c3a4d9f3950acc90a359df
Author: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>
Date:   Tue Oct 5 04:02:26 2021 +0000

    Update `zcash_script` to support V5 transactions

    Use a newer version of `zcash_script` that has been updated to support
    V5 transactions.

commit 371233628ae61e0c25d6ba8f31d9dba42823becb
Author: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>
Date:   Tue Oct 5 03:06:20 2021 +0000

    Update Zcash dependencies

    Update some Zcash crates:

    - `halo2`
    - `incrementalmerkletree' (patch version)
    - `orchard` (patch version)
    - `zcash_history` (patch version)
    - `zcash_note_encryption` (patch version)
    - `zcash_primitives` (patch version)

    And also update the `group` dependency so that the code remains
    compatible.

commit de5cf1ec40c3fc08670fc971cdf3e65e13d9f4c7
Author: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>
Date:   Tue Oct 5 03:04:13 2021 +0000

    Update error message assertion

    Use the updated message for the expected error variant.

* Update `zcash_script` to support V5 transactions

Use a newer version of `zcash_script` that has been updated to support
V5 transactions.

Co-authored-by: Conrado Gouvea <conrado@zfnd.org>
Co-authored-by: teor <teor@riseup.net>
2021-10-06 11:08:41 +10:00
Janito Vaqueiro Ferreira Filho 5d9893cf31
Send crawled transaction IDs to downloader (#2801)
* Rename type parameter to be more explicit

Replace the single letter with a proper name.

* Remove imports for `Request` and `Response`

The type names will conflict with the ones for the mempool service.

* Attach `Mempool` service to the `Crawler`

Add a field to the `Crawler` type to store a way to access the `Mempool`
service.

* Forward crawled transactions to downloader

The crawled transactions are now sent to the transaction downloader and
verifier, to be included in the mempool.

* Derive `Eq` and `PartialEq` for `mempool::Request`

Make it simpler to use the `MockService::expect_request` method.

* Test if crawled transactions are downloaded

Create some dummy crawled transactions, and let the crawler discover
them. Then check if they are forwarded to the mempool to be downloaded
and verified.

* Don't send empty transaction ID list to downloader

Ignore response from peers that don't provide any crawled transactions.

* Log errors when forwarding crawled transaction IDs

Calling the Mempool service should not fail, so if an error happens it
should be visible. However, errors when downloading individual
transactions can happen from time to time, so there's no need for them
to be very visible.

* Document existing `mempool::Crawler` test

Provide some depth as to what the test expect from the crawler's
behavior.

* Refactor to create `setup_crawler` helper function

Make it easier to reuse the common test setup code.

* Simplify code to expect requests

Now that `zebra_network::Request` implement `Eq`, the call can be
simplified into `expect_request`.

* Refactor to create `respond_with_transaction_ids`

A helper function that checks for a network crawl request and responds
with the given list of crawled transaction IDs.

* Refactor to create `crawler_iterator` helper

A function to intercept and respond to the fanned-out requests sent
during a single crawl iteration.

* Refactor to create `respond_to_queue_request`

Reduce the repeated code necessary to intercept and reply to a request
for queuing transactions to be downloaded.

* Add `respond_to_queue_request_with_error` helper

Intercepts a mempool request to queue transactions to be downloaded, and
responds with an error, simulating an internal problem in the mempool
service implementation.

* Derive `Arbitrary` for `NetworkUpgrade`

This is required for deriving `Arbitrary` for some error types.

* Derive `Arbitrary` for `TransactionError`

Allow random transaction errors to be generated for property tests.

* Derive `Arbitrary` for `MempoolError`

Allow random Mempool errors to be generated for property tests.

* Test if errors don't stop the mempool crawler

The crawler should be robust enough to continue operating even if the
mempool service fails to download transactions or even fails to handle
requests to enqueue transactions.

* Reduce the log level for download errors

They should happen regularly, so there's no need to have them with a
high visibility level.

Co-authored-by: teor <teor@riseup.net>

* Stop crawler if service stops

If `Mempool::poll_ready` returns an error, it's because the mempool
service has stopped and can't handle any requests, so the crawler should
stop as well.

Co-authored-by: teor <teor@riseup.net>
Co-authored-by: Conrado Gouvea <conrado@zfnd.org>
2021-10-05 10:55:42 +10:00
teor 44ac06775b
Reset `ChainTipChange`s on chain fork and network upgrade activation (#2721)
* Use `TipAction::Reset` for initialization and out-of-order blocks

Needs tests for:
- cloning a `ChainTipChange` resets the cloned instance
- skipped updates reset the cloned instance
- changing forks resets the cloned instance

* Use `TipAction::Reset` for network upgrade activation blocks

* Use an `if` expression

Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>

* Another if expression

Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>
2021-09-02 12:25:42 +10:00
Conrado Gouvea bc4194fcb9
ZIP-221/244 auth data commitment validation in checkpoint verifier (#2633)
* Add validation of ZIP-221 and ZIP-244 commitments

* Apply suggestions from code review

Co-authored-by: teor <teor@riseup.net>

* Add auth commitment check in the finalized state

* Reset the verifier when comitting to state fails

* Add explanation comment

* Add test with fake activation heights

* Add generate_valid_commitments flag

* Enable fake activation heights using env var instead of feature

* Also update initial_tip_hash; refactor into progress_from_tip()

* Improve comments

* Add fake activation heights test to CI

* Fix bug that caused commitment trees to not match when generating partial arbitrary chains

* Add ChainHistoryBlockTxAuthCommitmentHash::from_commitments to organize and deduplicate code

* Remove stale comment, improve readability

* Allow overriding with PROPTEST_CASES

* partial_chain_strategy(): don't update note commitment trees when not needed; add comment

Co-authored-by: teor <teor@riseup.net>
2021-08-23 14:17:33 +00:00
Alfredo Garcia 1624377da7
Legacy chain check and tests (#2366)
* add legacy chain check and tests
* improve has_network_upgrade check
* add docs to legacy_chain_check()
* change arbitrary module structure
* change the panic message
* move legacy chain acceptance into existing tests
* use a reduced_branch_id_strategy()
* add docs to strategy function
* add argument to check for legacy chain into sync_until()
2021-06-29 15:03:51 +10:00
Alfredo Garcia 544d182d25
Add and use a function for mandatory checkpoint (#2314)
* add `mandatory_checkpoint_height()` function

* use mandatory checkpoint instead of canopy in acceptance tests
2021-06-18 10:05:28 +10:00
teor 8ebb415e7c Clippy: remove needless borrows 2021-06-07 18:33:58 -04:00
teor 4c276dae64
Cleanup a few arbitrary impls (#2222) 2021-05-28 09:49:28 -03:00
Deirdre Connolly 376603d4c0 Flesh out Orchard note and nullifier derivation 2021-04-30 00:48:23 -04:00
Alfredo Garcia 9fc2388fbc
Parse ConsensusBranchId into NetworkUpgrade for transaction v5 (#2075)
* add consensus_branch_id field to transaction v5

* clippy

* rustfmt

* replace consensus_branch_id with network_upgrade

* remove unintended test files

* change method name

* some clanups

* add network_upgrade as a constant in tests

* use std in created function

* add comment to manual arbitrary impl

* create custom strategy to deal with NetworkUpgrade

* Add a missing TODO comment

Co-authored-by: teor <teor@riseup.net>
2021-04-29 01:55:29 +00:00
teor 9b3d56db0c
Update network_upgrade.rs (#2058) 2021-04-23 10:23:43 -03:00
teor 96b3c94dbc
Add the new commit count and git hash to the version (#2038)
* Use the git version + new commit count + hash for the app version

This helps diagnose bugs in versions of Zebra built from git branches,
rather than git version tags.

* Fill in assert

* Also log semver string

* Fix syntax

* Handle vergen using the cargo package version or raw git tag

* s/Semver/SemVer/

Co-authored-by: Deirdre Connolly <durumcrustulum@gmail.com>
2021-04-21 22:14:36 +00:00
Kirill Fomichev 43e792b9a4
Update to vergen 5, add branch, commit time, and build target to the panic metadata, automatically update app version from crate version (#2029)
* build(deps): bump vergen from 3.2.0 to 5.1.1

* fix hardcoded version for Tracing struct

* add additional metadata

* remove extra allocations for metadata

* Remove zebrad code version from release checklist

The zebrad code automatically uses the crate version now.

* Sort panic metadata into rough categories

Co-authored-by: teor <teor@riseup.net>
2021-04-20 06:48:14 +10:00
Deirdre Connolly c5bad9fac2
Rename NU5 to Nu5 to appease newly stable clippy::upper-case-acronyms (#1945) 2021-03-26 07:22:50 +10:00
Alfredo Garcia 8883543a85
Add transaction version 5 stubs (#1824)
* add transaction V5 stub
* add v5_strategy
* deduplicate version group ids
* Update comment for V5 transactions
* Add V5 transactions to non_finalized_state

Currently these are all `unimplemented!(...)`

* Fix struct matches
* Apply trivial panic message changes
* add zcash_deserialize for V5
* make all tx versions explicit in sprout and sapling nullifier functions
* match exhaustively in sprout and sapling nullifier functions
* fix matches in zebra-consensus
* fix NU5 strategy
* We're still deciding if v5 transactions support Sprout

Co-authored-by: teor <teor@riseup.net>
2021-03-04 07:56:41 +10:00
Jane Lusby e541746a50
Add initial support for NU5 to zebra (#1823)
* Add NU5 variant to NetworkUpgrade
* Add consensus branch ID for NU5
* Add network protocol versions for NU5
* Add NU5 to the protocol::version_consistent test
* Make unimplemented panic messages more specific
* Block target spacing doesn't change in NU5
* add comments for future updates for NU5

Co-authored-by: teor <teor@riseup.net>
2021-03-03 06:22:11 +10:00
Alfredo Garcia 436ce00fc2
Move height check to NetworkUpgrade (#1505)
* move height check to NetworkUpgrade
* refactor the max block time height check

Co-authored-by: teor <teor@riseup.net>
2020-12-15 08:30:38 +10:00
teor fb2f2a97f2 Delete an outdated canopy activation comment 2020-12-01 07:27:30 +10:00
teor d1ba1146d4 Add intra-doc links 2020-12-01 07:27:30 +10:00
teor b6ce509363 Make order of operations clearer 2020-12-01 07:27:30 +10:00
teor 750f096a99 Implement testnet minimum difficulty 2020-12-01 07:27:30 +10:00
teor 75519b0ae9 Implement averaging_window_timespan 2020-12-01 07:27:30 +10:00
Jane Lusby 17fdbe941b fix stdout issue with test framework for cached data tests 2020-11-24 11:04:30 -05:00
teor 76a9f5b8c4 Refactor block target spacing into NetworkUpgrade methods
And add a method for the minimum difficulty time gap threshold.
2020-11-12 12:30:54 +10:00
Alfredo Garcia 128643d81e
Call `zebra_test::init` where needed. (#1227)
* Add missing `zebra_test::init()` to zebra-chain
* Add missing `zebra_test::init()` to zebra-consensus
* Add missing `zebra_test::init()` to zebra-network
* Add missing `zebra_test::init()` to zebra-state
* Add missing `zebra_test::init()` to zebra-test
* Add missing `zebra_test::init()` to zebrad
2020-11-10 10:29:25 +10:00
Alfredo Garcia 21ad6ffc47
Reverse displayed endianness of transaction and block hashes (#1171)
* Reverse displayed endianness of transaction and block hashes
* fix zebra-checkpoints utility for new hash order
* Stop using "zebrad revhex" in zebrad-hash-lookup
* Rebuild checkpoint lists in new hash order
This change also adds additional checkpoints to the end of each list.

* Replace TransactionHash with transaction::Hash
This change should have been made in #905, but we missed Debug impls
and some docs.

Co-authored-by: Ramana Venkata <vramana@users.noreply.github.com>
Co-authored-by: teor <teor@riseup.net>
2020-10-22 07:54:02 +10:00
Alfredo Garcia c0a14ecc8c
move genesis parameters to zebra-chain (#1151) 2020-10-12 14:08:23 -07:00