Add joinSplitPubKey and joinSplitSig to RPC
These two properties are required to fully reconstruct a Zcash transaction's binary form from the RPC data.
Add Foundation's and gtank's DNS seeders
This adds our new DNS seeders to the list. They're running [CoreDNS](https://coredns.io) with a [Zcash crawler plugin](https://github.com/ZcashFoundation/dnsseeder), the result of a Zcash Foundation in-house development effort to replace zcash-seeder with something memory safe and easier to maintain.
These are validly operated seeders per the existing policy (https://zcash.readthedocs.io/en/latest/rtd_pages/dnsseed_policy.html):
> A DNS seed operating organization or person is expected to follow good host security practices, maintain control of applicable infrastructure, and not sell or transfer control of the DNS seed. Any hosting services contracted by the operator are equally expected to uphold these expectations.
In both cases the code is running on well-operated public cloud infrastructure in either a container or the most sandboxing appropriate to the environment. The DNS records pointing to the seeders are controlled by reputable third-party DNS providers under accounts with 2FA enabled.
> The DNS seed results must consist exclusively of fairly selected and functioning Zcash nodes from the public network to the best of the operator’s understanding and capability.
The crawler attempts to connect to all discoverable Zcash peers and ensures their continued uptime on a regular basis. The results are always a uniformly randomized subset of all known live peers.
> For the avoidance of doubt, the results may be randomized but must not single out any group of hosts to receive different results unless due to an urgent technical necessity and disclosed.
See above. However, we reserve the right to begin offering [NU-targeted results](https://github.com/ZcashFoundation/dnsseeder/issues/3) based on opt-in client queries.
> The results may not be served with a DNS TTL of less than one minute.
Mainnet results are served with a TTL of 600 seconds, and Testnet results with a TTL of 300 seconds to account for greater flux on that network.
> Any logging of DNS queries should be only that which is necessary for the operation of the service or urgent health of the Zcash network and must not be retained longer than necessary nor disclosed to any third party.
There is no logging of DNS queries in either production configuration, which can be somewhat confirmed by examining the Corefile(s) [[1]](https://github.com/ZcashFoundation/coredns-zcash/blob/master/coredns/Corefile)[[2]](https://github.com/ZcashFoundation/coredns-zcash/blob/master/scripts/gcp-start.sh#L9-L27) we use.
> Information gathered as a result of the operators node-spidering (not from DNS queries) may be freely published or retained, but only if this data was not made more complete by biasing node connectivity (a violation of expectation (1)).
The seeder currently has no persistence outside of its static config file, so this data is neither retained nor shared by the operators.
> Operators are encouraged, but not required, to publicly document the details of their operating practices.
Our deployments are described in detail by the [coredns-zcash](https://github.com/ZcashFoundation/coredns-zcash) repo. Reader, you could run one too!
> A reachable email contact address must be published for inquiries related to the DNS seed operation.
For general questions related to either seeder, contact george@zfnd.org or mention @gtank in the Foundation's Discord. For bug reports, open an issue on the [dnsseeder](https://github.com/ZcashFoundation/dnsseeder) repo.
metrics: Add a progress bar when in Initial Block Download mode
The progress bar shows both headers (in green) and blocks (in white / inverse of background colour). It is only printed for TTY output.
Additionally, the "not mining" message is no longer shown on mainnet, as the built-in CPU miner is not effective at the current network difficulty.
This patch adds an option to configure the name and/or directory of the
debug log.
The user can specify either a relative path, in which case the path
is relative to the data directory. They can also specify an absolute
path to put the log anywhere else in the file system.
Report headers download
With current compile-time defaults, a Zcash node prefetches up to 160 block headers per request without a limit on how far it can prefetch, but only up to 16 full blocks at a time. For this and other reasons, it can get very far ahead in headers prefetch (and PoW verification on those, so it's quite some processing too) over full blocks fetch (such as 10x ahead) during initial blocks download. Let's report to the user on how many headers the node has fetched, and let's also use this information as additional input on estimating the total number of blocks to fetch: it can't be less than the number of headers already fetched.
While at it, also fix typos in related code.
Add options and help to AFL scripts
Closes https://github.com/zcash/zcash/issues/4315
For reference here is how the help of each command is script:
**afl-get.sh**:
```
$ ./zcutil/afl/afl-get.sh --help
Obtains and builds a copy of AFL from source.
Usage:
./zcutil/afl/afl-get.sh --afl-install=AFL_INSTALL_DIR
OPTIONS:
-h, --help Print this help message
-i, --afl-install Directory where AFL is going to be installed
EXAMPLE:
./zcutil/afl/afl-get.sh -i /tmp/afl
$
```
**afl-build.sh**:
```
$ ./zcutil/afl/afl-build.sh --help
A wrapper around ./zcutil/build.sh for instrumenting the build with AFL.
You may obtain a copy of AFL using ./zcutil/afl/afl-get.sh.
Additional arguments are passed-through to build.sh.
Usage:
./zcutil/afl/afl-build.sh --afl-install=AFL_INSTALL_DIR --fuzz-case=FUZZ_CASE [ OPTIONS ... ] [ ARGUMENTS ... ]
OPTIONS:
-a, --harden Turn off AFL_HARDEN. Default: 1
-c, --configure-flags Pass this flags to ./configure. Default: --enable-tests=no --enable-fuzz-main
-f, --fuzz-case Options are: CheckBlock, DecodeHexTx, DeserializeAddrMan, DeserializeTx or ReadFeeEstimates
-h, --help Print this help message
-l, --afl-log Directory to save AFL logs. Default: /home/oxarbitrage/zcash/zcash4/zcash
-i, --afl-install Directory where AFL is installed
-z, --zcutil The zcutil directory. Default /home/oxarbitrage/zcash/zcash4/zcash/zcutil
ARGUMENTS:
By default we are passing to build.sh the following flags:
CC=/home/oxarbitrage/zcash/zcash4/zcash/zcutil/afl/zcash-wrapper-gcc
CXX=/home/oxarbitrage/zcash/zcash4/zcash/zcutil/afl/zcash-wrapper-g++
EXAMPLE:
./zcutil/afl/afl-build.sh -i /tmp/afl -f DecodeHexTx
$
```
**afl-run.sh**:
```
$ ./zcutil/afl/afl-run.sh --help
Start fuzzing a case in a previously zcashd built for AFL.
Additional arguments are passed-through to AFL.
Usage:
./zcutil/afl/afl-run.sh --afl-install=AFL_INSTALL_DIR --fuzz-case=FUZZ_CASE [ ARGUMENTS... ]
OPTIONS:
-f, --fuzz-case Options are: CheckBlock, DecodeHexTx, DeserializeAddrMan, DeserializeTx or ReadFeeEstimates
-h, --help Print this help message
-i, --afl-install Directory where AFL is installed
EXAMPLE:
./zcutil/afl/afl-run.sh -i /tmp/afl -f DecodeHexTx
$
```
**afl-getbuildrun.sh**:
```
$ ./zcutil/afl/afl-getbuildrun.sh --help
Builds AFL and an instrumented zcashd, then begins fuzzing.
This script must be run from within the top level directory of a zcash clone.
Additional arguments are passed-through to AFL.
Usage:
./zcutil/afl/afl-getbuildrun.sh --fuzz-case=FUZZ_CASE [ OPTIONS ... ] [ ARGUMENTS... ]
OPTIONS:
-f, --fuzz-case Options are: CheckBlock, DecodeHexTx, DeserializeAddrMan, DeserializeTx or ReadFeeEstimates
-h, --help Print this help message
-i, --afl-install Directory where AFL is installed. Default: /home/oxarbitrage/zcash/zcash4/zcash/afl-temp
EXAMPLE:
./zcutil/afl/afl-getbuildrun.sh -f DecodeHexTx
$
```
Use the cached consensusBranchId in DisconnectBlock
If a node is started with a set of network upgrades that don't match the
serialized chain (such as when we implement NU rollbacks on testnet),
RewindBlockIndex will disconnect each block in the chain until it
reaches the most recent block that agrees with the node's set of network
upgrades. However, the blocks themselves should be disconnected using
the consensus branch ID that they were connected with, which is
persisted alongside the chain and reconstructed in LoadBlockIndex.
Homu
Rod Vagg
Daira Hopwood
George Tankersley
Alfredo Garcia
Jack Grigg
Anthony Towns
Wladimir J. van der Laan
Solar Designer
mdr0id
Eirik Ogilvie-Wigley