Commit Graph

2452 Commits

Author SHA1 Message Date
teor 7e2c3a2fc7 Clarify a duplicate log message 2021-04-21 23:59:29 -04:00
teor 7b13d5573a Make String Zcash serialization consistent with deserialization
After recent changes, serialization was `write_string`, but
deserialization was `zcash_deserialize`.
2021-04-21 23:58:48 -04:00
Kirill Fomichev 5b2f1cdfd5
Add journald support through tracing-journald (#2034)
* Add journald support through tracing-journald

* change journald to use_journald

* more fixes
2021-04-22 09:31:06 +10:00
Kirill Fomichev afac2c2846
Use the default port for configured listen addresses with no port (#2043)
* Allow use listen address in config without port

* update comments

* remove not used alias

* use Network::default_port

* Move tests and use toml instead json

* change error message

* Make match more readable

Co-authored-by: teor <teor@riseup.net>
2021-04-21 23:14:29 +00:00
teor 96b3c94dbc
Add the new commit count and git hash to the version (#2038)
* Use the git version + new commit count + hash for the app version

This helps diagnose bugs in versions of Zebra built from git branches,
rather than git version tags.

* Fill in assert

* Also log semver string

* Fix syntax

* Handle vergen using the cargo package version or raw git tag

* s/Semver/SemVer/

Co-authored-by: Deirdre Connolly <durumcrustulum@gmail.com>
2021-04-21 22:14:36 +00:00
teor 0203d1475a Refactor and document correctness for std::sync::Mutex<AddressBook> 2021-04-21 17:14:47 -04:00
teor 905b90d6a1 Refactor and document correctness for std::sync::Mutex in ErrorSlot 2021-04-21 16:39:06 -04:00
teor 3f45735f3f Use futures:🔒:Mutex for the nonce set 2021-04-21 01:39:49 -04:00
teor 2ed8bb00cf Clarify CandidateSet state diagram
We get inbound connections on the listener port,
but the important part is the inbound connection
itself.
2021-04-21 01:37:43 -04:00
teor d3836e6fd8 Move spends_per_anchor to TransferData
And update its docs for ZIP-244
2021-04-21 01:32:00 -04:00
teor 9e45e6bb23 Revert "Revert "build(deps): bump vergen from 5.1.1 to 5.1.2""
This reverts commit ccf5d82517.
2021-04-20 19:30:20 -04:00
teor 79c0c4ec57 Stop assuming there will always be a git commit
Enable builds where:
* there is no google cloud git commit env var, and
* there is no `.git` directory.

By making all `vergen` env vars optional, and skipping any env vars that
don't exist.
2021-04-20 13:48:31 -04:00
Deirdre Connolly ccf5d82517 Revert "build(deps): bump vergen from 5.1.1 to 5.1.2"
This reverts commit 962cb9c579.
2021-04-20 10:19:02 -04:00
dependabot[bot] 962cb9c579 build(deps): bump vergen from 5.1.1 to 5.1.2
Bumps [vergen](https://github.com/rustyhorde/vergen) from 5.1.1 to 5.1.2.
- [Release notes](https://github.com/rustyhorde/vergen/releases)
- [Commits](https://github.com/rustyhorde/vergen/compare/5.1.1...5.1.2)

Signed-off-by: dependabot[bot] <support@github.com>
2021-04-20 10:18:45 -04:00
teor 53779d2a3c
Redesign Sapling data model for V5 shared anchor and spends (#2021)
* Redesign Sapling data model for V5 shared anchor and spends

The shared anchor is only present if there are any spends.

As part of this change, delete the manual PartialEq impl and its tests,
because we can derive PartialEq now.

* Stop creating a temporary Vec for the spend and output iterators

* Rename TransferData variants

Interactive rename using the following commands:
```sh
fastmod Spends SpendsAndMaybeOutputs
fastmod NoSpends JustOutputs
```

* Refactor out common sprout nullifier code

* Implement the AtLeastOne constrained vector type

This vector wrapper ensures that it always contains at least one element.

* Simplify Sapling TransferData using AtLeastOne

Also update the RFC to use AtLeastOne for Orchard.
2021-04-20 16:22:25 +10:00
teor ad272f2bee Make sure handshake version negotiation always has a timeout
As part of this change, refactor handshake version negotiation into its
own function.
2021-04-19 18:31:28 -04:00
Kirill Fomichev 43e792b9a4
Update to vergen 5, add branch, commit time, and build target to the panic metadata, automatically update app version from crate version (#2029)
* build(deps): bump vergen from 3.2.0 to 5.1.1

* fix hardcoded version for Tracing struct

* add additional metadata

* remove extra allocations for metadata

* Remove zebrad code version from release checklist

The zebrad code automatically uses the crate version now.

* Sort panic metadata into rough categories

Co-authored-by: teor <teor@riseup.net>
2021-04-20 06:48:14 +10:00
teor 2cecd52a10 Fix comment typo 2021-04-19 10:11:22 -04:00
teor 8fb12f07a1 Fix outdated comment 2021-04-19 10:11:22 -04:00
teor eabadb8301 Make heartbeats wait for the connection queue to empty, with a timeout
Also cleanup the heartbeat code, so each heartbeat request/response runs
in a future with a single timeout.
2021-04-19 10:11:22 -04:00
Kirill Fomichev b636cf8ac1
Benchmark Block struct serialization code (#2018)
* Benchmark Block struct serialization code

* benchmark generated block

* add deserialize benchmark

* Fix block/tests.rs conditional module imports

* add large_single_transaction_block

* Fix name for deserialise benchmark

Co-authored-by: teor <teor@riseup.net>
2021-04-19 12:15:02 +00:00
Kirill Fomichev 5f36be7223
build(deps): bump ripemd160 from 0.8.0 to 0.9.1 (#2027) 2021-04-19 20:43:38 +10:00
Kirill Fomichev fb799b202c
Update dependencies to support Apple M1, fix Windows CI LLVM paths (#2026)
* build(deps): update for Apple M1

* fix ci

* bump rocksdb

* bump zcash_script
2021-04-19 20:39:24 +10:00
teor b9ac221ad4
Implement Sapling serialization in Transaction V5 (#2020)
* serialize/deserialize spaling shielded data in v5 transaction

* fix serialize/deserialize fields order according to spec

* remove extra clone calls

* more serialize fixes

* clippy: fix empty array

* tidy comments

* Add v4 and v5 transaction tests

Also make sure that serialized bytes match if structs match.

* Test fake v5 blocks made out of pre-NU5 block test vectors

* Add outputs-only tests for v5 shared anchor serialization

* Refactor sapling::ShieldedData V5 serialization into its own impl

* Fix spec name typos

* Simplify sapling shielded data parsing

* Delete redundant V5 transaction wrappers in tests

And split out sapling ShieldedData serialization.

Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
2021-04-19 08:09:57 +10:00
Kirill Fomichev 32285faf56
Speedup proptests for Chain struct in zebra-state (#2012)
* Speedup proptests for Chain struct in zebra-state

* Add teor2345 requested changes

* Fix type for DEFAULT_PARTIAL_CHAIN_PROPTEST_CASES

* More costs for PROPTEST_CASES
2021-04-17 21:00:20 +10:00
dependabot[bot] 6f8a6a9bc0 build(deps): bump pretty_assertions from 0.7.1 to 0.7.2
Bumps [pretty_assertions](https://github.com/colin-kiegel/rust-pretty-assertions) from 0.7.1 to 0.7.2.
- [Release notes](https://github.com/colin-kiegel/rust-pretty-assertions/releases)
- [Changelog](https://github.com/colin-kiegel/rust-pretty-assertions/blob/main/CHANGELOG.md)
- [Commits](https://github.com/colin-kiegel/rust-pretty-assertions/compare/v0.7.1...v0.7.2)

Signed-off-by: dependabot[bot] <support@github.com>
2021-04-17 01:44:41 -04:00
teor 0def12f825
Add split array serialization functions for Transaction::V5 (#2017)
* Add functions for serializing and deserializing split arrays

In Transaction::V5, Zcash splits some types into multiple arrays, with a
single prefix count before the first array.

Add utility functions for serializing and deserializing the subsequent
arrays, with a paramater for the original array's length.

* Use zcash_deserialize_bytes_external_count in zebra-network

* Move some preallocate proptests to their own file

And fix the test module structure so it is consistent with the rest of
zebra-chain.

* Add a convenience alias zcash_serialize_external_count

* Explain why u64::MAX items will never be reached
2021-04-16 08:23:00 +10:00
Alfredo Garcia e42442d48b
Redesign Transaction V5 serialization, impl trusted vector security, nullifier utility functions (#1996)
* add sapling shielded data to transaction V5

* implement nullifiers

* test v5 in shielded_data_roundtrip

* Explicitly design serialization for Transaction V5

Implement serialization for V4 and V5 spends and outputs, to make sure
that the design works.

* Test serialization for v5 spends and outputs

Also add a few missing v4 tests.

* Delete a disabled proptest

* Make v5 transactions a top-level heading

And add a missing serialized type.

* Fix a comment typo

* v5 transaction RFC: split array serialization

Based on #2017

* RFC: explicitly describe serialized field order

And link to the spec

* RFC: add the shared anchor serialization rule test

Co-authored-by: teor <teor@riseup.net>
2021-04-16 08:19:28 +10:00
teor 3cb88c0957 Fix a nightly compile error by patching lexical-core to 0.7.5
lexical-core 0.7.4 is incompatible with rustc nightly >= ~2021-03-24,
see https://github.com/rust-lang/rust/issues/81654

While we're waiting for zcash_script, rocksdb, and their dependencies to
upgrade, patch lexical-core to 0.7.5 in the zebra workspace.
2021-04-15 13:47:54 -04:00
Deirdre Connolly 370d0480b4
Merge pull request #983 from ZcashFoundation/treestate-design
Draft RFC: Treestate management
2021-04-15 13:39:34 -04:00
teor 381c20b6af Security: change the GetAddr fanout to 3
Zebra avoids having a majority of addresses from a single peer by asking
3 peers for new addresses.

Also update a bunch of security comments and related documentation.
2021-04-15 13:09:14 -04:00
teor 59aa04c9b9 Stop panicking when Zebra sends a reject without extra data
Also add round-trip unit tests for extra data and no extra data.
2021-04-15 12:20:33 -04:00
teor 15843cfd6e Fix a test failure due to ' debug format changes in Rust
In general, tests shouldn't depend on debug formatting. But for now,
removing the ' fixes the issue.
2021-04-14 23:45:36 -04:00
teor 1e4e5924ca clippy: factor common code out of an if-else block 2021-04-14 23:16:45 -04:00
dependabot[bot] 7d36a5e2c3 build(deps): bump color-eyre from 0.5.10 to 0.5.11
Bumps [color-eyre](https://github.com/yaahc/color-eyre) from 0.5.10 to 0.5.11.
- [Release notes](https://github.com/yaahc/color-eyre/releases)
- [Changelog](https://github.com/yaahc/color-eyre/blob/v0.5.11/CHANGELOG.md)
- [Commits](https://github.com/yaahc/color-eyre/compare/v0.5.10...v0.5.11)

Signed-off-by: dependabot[bot] <support@github.com>
2021-04-14 09:22:27 -04:00
dependabot[bot] f03d69d65e build(deps): bump x25519-dalek from 1.1.0 to 1.1.1
Bumps [x25519-dalek](https://github.com/dalek-cryptography/x25519-dalek) from 1.1.0 to 1.1.1.
- [Release notes](https://github.com/dalek-cryptography/x25519-dalek/releases)
- [Changelog](https://github.com/dalek-cryptography/x25519-dalek/blob/main/CHANGELOG.md)
- [Commits](https://github.com/dalek-cryptography/x25519-dalek/compare/1.1.0...1.1.1)

Signed-off-by: dependabot[bot] <support@github.com>
2021-04-14 08:50:35 -04:00
teor a417c7c8c7 Use meaningful names for select! variables 2021-04-13 23:56:16 -04:00
teor e8e6d292c5 clippy: make to_* methods take self by value 2021-04-13 23:36:23 -04:00
teor fb95de99a6 Refactor the dial result into a From impl 2021-04-13 18:52:49 -04:00
dependabot[bot] e084d0f1dc build(deps): bump futures from 0.3.13 to 0.3.14
Bumps [futures](https://github.com/rust-lang/futures-rs) from 0.3.13 to 0.3.14.
- [Release notes](https://github.com/rust-lang/futures-rs/releases)
- [Changelog](https://github.com/rust-lang/futures-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/futures-rs/compare/0.3.13...0.3.14)

Signed-off-by: dependabot[bot] <support@github.com>
2021-04-12 20:45:35 -04:00
dependabot[bot] 3a93043d94 build(deps): bump bincode from 1.3.1 to 1.3.3
Bumps [bincode](https://github.com/servo/bincode) from 1.3.1 to 1.3.3.
- [Release notes](https://github.com/servo/bincode/releases)
- [Commits](https://github.com/servo/bincode/compare/v1.3.1...v1.3.3)

Signed-off-by: dependabot[bot] <support@github.com>
2021-04-12 16:00:59 -04:00
Alfredo Garcia 24bf952e98
missed bump of zebra-utils (#1992) 2021-04-08 23:49:21 +00:00
Alfredo Garcia 5ec05e91e1 update version strings for v1.0.0-alpha.6 2021-04-08 18:48:34 -04:00
teor 1626ec383a
Add InventoryHash and MetaAddr proptests (#1985)
* Make proptest dependencies consistent between chain and network

* Implement Arbitrary for InventoryHash and use it in tests

* Impl Arbitrary for MetaAddr and use it in tests

Also test some extreme times in MetaAddr sanitization.
2021-04-07 14:13:52 -03:00
teor f8094cdf5c
Test the manual eq impl on sapling::ShieldedData<PerSpend> (#1989) 2021-04-07 12:30:50 -03:00
teor 375c8d8700
Fix a deadlock between the crawler and dialer, and other hangs (#1950)
* Stop ignoring inbound message errors and handshake timeouts

To avoid hangs, Zebra needs to maintain the following invariants in the
handshake and heartbeat code:
- each handshake should run in a separate spawned task
  (not yet implemented)
- every message, error, timeout, and shutdown must update the peer address state
- every await that depends on the network must have a timeout

Once the Connection is created, it should handle timeouts.
But we need to handle timeouts during handshake setup.

* Avoid hangs by adding a timeout to the candidate set update

Also increase the fanout from 1 to 2, to increase address diversity.

But only return permanent errors from `CandidateSet::update`, because
the crawler task exits if `update` returns an error.

Also log Peers response errors in the CandidateSet.

* Use the select macro in the crawler to reduce hangs

The `select` function is biased towards its first argument, risking
starvation.

As a side-benefit, this change also makes the code a lot easier to read
and maintain.

* Split CrawlerAction::Demand into separate actions

This refactor makes the code a bit easier to read, at the cost of
sometimes blocking the crawler on `candidates.next()`.

That's ok, because `next` only has a short (< 100 ms) delay. And we're
just about to spawn a separate task for each handshake.

* Spawn a separate task for each handshake

This change avoids deadlocks by letting each handshake make progress
independently.

* Move the dial task into a separate function

This refactor improves readability.

* Fix buggy future::select function usage

And document the correctness of the new code.
2021-04-07 10:25:10 -03:00
teor 418575458e
Rename the () placeholder to FieldNotPresent (#1987)
* Rename the () placeholder to FieldNotPresent

* Use a unit struct

* Update the RFC
2021-04-07 09:34:58 -03:00
teor 5bf0a2954e
Update a test comment 2021-04-07 19:25:31 +10:00
teor bb3147dd1e
Update an outdated comment 2021-04-07 19:23:48 +10:00
teor de6d1c93f3
Clarify a comment 2021-04-07 18:56:38 +10:00