Remove leveldb baseline coverage gathering.
Added filter rules to remove all of the subtress (leveldb, secp256k1, ctaes, univalue) and
benchmarking from the coverage report. These items are unnecessary as we do not test for any
of the subtrees and benchmark coverage is unneeded.
Instead of using lcov -r (which is extremely slow), first use a python script to perform bulk cleanup of the /usr/include/* coverage. Then use lcov -a to remove the duplicate entries. This has the same effect of lcov -r but runs significantly faster
Added an option to configure to allow for branch coverage statistics gathering.
Disabled logprint macro when coverage testing is on so that unnecessary branches are not analyzed.
bctest.py is only used as an import by bitcoin-util-test.py. There's no
value in keeping it as a separate module, so let's merge them into a
single module to keep building and packaging simpler.
bitcoin-test-util is importable as a module, so if any future modules
really want to import the code from bctest.py, they can import
bitcoin-test-util and call the bctest functions by name.
These were meant to help build subdir targets from the top builddir, but cause
infinite recursion when going the other way.
If anyone actually uses these, we can add back specific targets.
This was caused by an pyc files hanging around from previous
python2 invocations, when the matching .py missing from that path.
This should not be a problem with python3's tagged caches.
- Link pull-tester/rpc-tests.py to the build dir
- Add the build-dir's config to the python path so that tests can find it
- The tests themselves are in srcdir
- Clean up __pycache__ in 'make clean'
Unfortunately, the target namees defined at the Makefile.am level can't be used
for *.in substitution. So these new defines will have to stay synced up with
those targets.
Using the new variables for the deploy targets in the main Makefile.am will
ensure that they stay in sync, otherwise build tests will fail.
Ubuntu 16.04 "xenial xerus" does not come with Python 2.x by default.
It is possible to install a python-2.7 package, but this has its own
problem: no `python` or `python2` symlink (see #7717).
This fixes the following scripts to work with python 3:
- `make check` (bctest,py, bitcoin-util-test.py)
- `make translate` (extract_strings_qt.py)
- `make symbols-check` (symbol-check.py)
- `make security-check` (security-check.py)
Explicitly call the python commands using $(PYTHON) instead
of relying on the interpreter line at the top of the scripts.
The RPC tests (via `qa/pull-tester/rpc-tests.py`) are now executed,
when gathering code coverage data, for example with `make cov`.
Generating coverage data requires `lcov`, which can installed with:
sudo apt-get install lcov
To also use the BitcoinJ tests, get the test tool:
TOOL_URL=https://github.com/theuni/bitcoind-comparisontool/raw/master/pull-tests-8c6666f.jar
TOOL_HASH=a865332b3827abcde684ab79f5f43c083b0b6a4c97ff5508c79f29fee24f11cd
wget $TOOL_URL -O ./share/BitcoindComparisonTool.jar
echo "$TOOL_HASH ./share/BitcoindComparisonTool.jar" | shasum --algorithm 256 --check
The coverage data can be generated with:
./autogen.sh
./configure --enable-lcov --with-comparison-tool=./share/BitcoindComparisonTool.jar
make
make cov
Optionally the options `--enable-extended-rpc-tests` and
`--enable-comparison-tool-reorg-tests` may be used to enable more time
consuming tests.
It then runs the tests and generates two HTML reports:
- test_bitcoin.coverage/index.html
- total.coverage/index.html
Until now there were quite a few leftovers, and only the coverage
related files in `src/` were cleaned, while the ones in the other dirs
remained. `qa/tmp/` is related to the BitcoinJ tests, and `cache/` is
related to RPC tests.
1) created rpc-tests.py
2) deleted rpc-tests.sh
3) travis.yml points to rpc-tests.py
4) Modified Makefile.am
5) Updated README.md
6) Added tests_config.py and deleted tests-config.sh
7) Modified configure.ac with script to set correct path in tests_config.py
Andrew Chow
John Newbery
Cory Fields
Wladimir J. van der Laan
nomnombtc
Luke Dashjr
dexX7
ptschip
randy-waterhouse
Jonas Schnelli
randy-waterhouse