Report headers download
With current compile-time defaults, a Zcash node prefetches up to 160 block headers per request without a limit on how far it can prefetch, but only up to 16 full blocks at a time. For this and other reasons, it can get very far ahead in headers prefetch (and PoW verification on those, so it's quite some processing too) over full blocks fetch (such as 10x ahead) during initial blocks download. Let's report to the user on how many headers the node has fetched, and let's also use this information as additional input on estimating the total number of blocks to fetch: it can't be less than the number of headers already fetched.
While at it, also fix typos in related code.
Add options and help to AFL scripts
Closes https://github.com/zcash/zcash/issues/4315
For reference here is how the help of each command is script:
**afl-get.sh**:
```
$ ./zcutil/afl/afl-get.sh --help
Obtains and builds a copy of AFL from source.
Usage:
./zcutil/afl/afl-get.sh --afl-install=AFL_INSTALL_DIR
OPTIONS:
-h, --help Print this help message
-i, --afl-install Directory where AFL is going to be installed
EXAMPLE:
./zcutil/afl/afl-get.sh -i /tmp/afl
$
```
**afl-build.sh**:
```
$ ./zcutil/afl/afl-build.sh --help
A wrapper around ./zcutil/build.sh for instrumenting the build with AFL.
You may obtain a copy of AFL using ./zcutil/afl/afl-get.sh.
Additional arguments are passed-through to build.sh.
Usage:
./zcutil/afl/afl-build.sh --afl-install=AFL_INSTALL_DIR --fuzz-case=FUZZ_CASE [ OPTIONS ... ] [ ARGUMENTS ... ]
OPTIONS:
-a, --harden Turn off AFL_HARDEN. Default: 1
-c, --configure-flags Pass this flags to ./configure. Default: --enable-tests=no --enable-fuzz-main
-f, --fuzz-case Options are: CheckBlock, DecodeHexTx, DeserializeAddrMan, DeserializeTx or ReadFeeEstimates
-h, --help Print this help message
-l, --afl-log Directory to save AFL logs. Default: /home/oxarbitrage/zcash/zcash4/zcash
-i, --afl-install Directory where AFL is installed
-z, --zcutil The zcutil directory. Default /home/oxarbitrage/zcash/zcash4/zcash/zcutil
ARGUMENTS:
By default we are passing to build.sh the following flags:
CC=/home/oxarbitrage/zcash/zcash4/zcash/zcutil/afl/zcash-wrapper-gcc
CXX=/home/oxarbitrage/zcash/zcash4/zcash/zcutil/afl/zcash-wrapper-g++
EXAMPLE:
./zcutil/afl/afl-build.sh -i /tmp/afl -f DecodeHexTx
$
```
**afl-run.sh**:
```
$ ./zcutil/afl/afl-run.sh --help
Start fuzzing a case in a previously zcashd built for AFL.
Additional arguments are passed-through to AFL.
Usage:
./zcutil/afl/afl-run.sh --afl-install=AFL_INSTALL_DIR --fuzz-case=FUZZ_CASE [ ARGUMENTS... ]
OPTIONS:
-f, --fuzz-case Options are: CheckBlock, DecodeHexTx, DeserializeAddrMan, DeserializeTx or ReadFeeEstimates
-h, --help Print this help message
-i, --afl-install Directory where AFL is installed
EXAMPLE:
./zcutil/afl/afl-run.sh -i /tmp/afl -f DecodeHexTx
$
```
**afl-getbuildrun.sh**:
```
$ ./zcutil/afl/afl-getbuildrun.sh --help
Builds AFL and an instrumented zcashd, then begins fuzzing.
This script must be run from within the top level directory of a zcash clone.
Additional arguments are passed-through to AFL.
Usage:
./zcutil/afl/afl-getbuildrun.sh --fuzz-case=FUZZ_CASE [ OPTIONS ... ] [ ARGUMENTS... ]
OPTIONS:
-f, --fuzz-case Options are: CheckBlock, DecodeHexTx, DeserializeAddrMan, DeserializeTx or ReadFeeEstimates
-h, --help Print this help message
-i, --afl-install Directory where AFL is installed. Default: /home/oxarbitrage/zcash/zcash4/zcash/afl-temp
EXAMPLE:
./zcutil/afl/afl-getbuildrun.sh -f DecodeHexTx
$
```
Use the cached consensusBranchId in DisconnectBlock
If a node is started with a set of network upgrades that don't match the
serialized chain (such as when we implement NU rollbacks on testnet),
RewindBlockIndex will disconnect each block in the chain until it
reaches the most recent block that agrees with the node's set of network
upgrades. However, the blocks themselves should be disconnected using
the consensus branch ID that they were connected with, which is
persisted alongside the chain and reconstructed in LoadBlockIndex.
If a node is started with a set of network upgrades that don't match the
serialized chain (such as when we implement NU rollbacks on testnet),
RewindBlockIndex will disconnect each block in the chain until it
reaches the most recent block that agrees with the node's set of network
upgrades. However, the blocks themselves should be disconnected using
the consensus branch ID that they were connected with, which is
persisted alongside the chain and reconstructed in LoadBlockIndex.
librustzcash: make the header C compatible
The `librustzcash.h` file is compatible with both languages. However, only C++ is supported at the moment. By relying on the preprocessor to include or not the `extern "C"` piece, the interface becomes compatible with both.
Golden testing to verify no-reindex network upgrades.
This test was introduced to provide us a way to test network upgrades against previously-serialized versions of block and block-index data. At present the only thing that is checked is that the nodes start correctly after upgrading.
The commit history of this branch is built such that each persisted database artifact is constructed in a commit which has the correct history for constructing that particular artifact - basically, you get that artifact (disregarding randomness) by checking out that commit and creating it using the zcashd built from that commit.
Release v3.0.0-rc1
This is the release candidate for 3.0.0 Heartwood.
EOS halt manually configured to be (roughly) September 15, which is 2808 hours away (at the time of writing this) so with 48 blocks per hour, 2808 * 48 plus the current height 838929 equals 973713.
Enable Heartwood activation on mainnet
This sets the Heartwood activation height to `903000`, which follows the deprecation height of `v2.1.2-3` (which is set to deprecate on block `901475`, roughly 31 hours earlier, sometime mid-July).
Fix a null pointer dereference that occurs when formatting an error message
This fixes a bug in the error message printout for the case when we have sufficient chain work, but an expected network upgrade has not activated.
Jack Grigg
Homu
Solar Designer
mdr0id
Eirik Ogilvie-Wigley
therealyingtong
Taylor Hornby
Sean Bowe
Kris Nuttycombe
Alfredo Garcia
Daira Hopwood