Commit Graph

13098 Commits

Author SHA1 Message Date
zancas 9f4c93a16c
replace "virtual" with "override" in subclasses
The use of virtual on these lines is obsolete.

I ran:

"rm -f ./src/zcash-gtest && rm -f ./src/gtest/*.o && make && ./src/zcash-gtest"

before an after making the change.   In both cases:

206 test ran
from
32 cases
and
1 test was DISABLED
2020-02-01 15:58:21 -07:00
Homu 529f102cac Auto merge of #4318 - str4d:configure-and-build-updates, r=str4d
More updates to the build system

Part of #58 and #4317.
2020-01-31 15:00:08 +00:00
Jack Grigg 795e1f40c5 Add build system changes to release notes 2020-01-31 14:58:52 +00:00
Jack Grigg 2e9028f1f1 Move default -g flag into configure.ac behind --enable-debug=no
This ensures that if --enable-debug is set, we correctly use -g3 if it
is available, instead of overriding it.

Part of #4317.
2020-01-31 13:39:38 +00:00
Jack Grigg 999b3a3235 Make -fwrapv conditional on --enable-debug=no
-fwrapv and -ftrapv conflict, and whichever is last takes precedence
(for gcc). This meant that -ftrapv was being ignored when --enable-debug
was set. This commit correctly sets only one of them based on whether
--enable-debug is set.

Part of #4317.
2020-01-31 13:22:07 +00:00
Jack Grigg d458cf627a zcutil/build.sh: Turn off verbosity by default
Users who want verbose output (like CI) can call ./zcutil/build.sh V=1
2020-01-31 12:12:07 +00:00
Jack Grigg 42a58f1e23 configure: Change default Proton to match build.sh 2020-01-31 12:12:07 +00:00
Jack Grigg 2ed62edfd6 zcutil/build.sh: Remove lcov and mining flags
These flags can be directly moved into CONFIGURE_FLAGS.
2020-01-31 12:12:07 +00:00
Homu ca55eb4794 Auto merge of #3870 - defuse:update-check-script, r=str4d
Add Python script for checking if dependencies have updates.
2020-01-30 18:27:56 +00:00
Taylor Hornby 1ecf2bae48 Make updatecheck.py compatible with python2 2020-01-30 11:11:37 -07:00
Homu 8e8a9350c3 Auto merge of #4222 - str4d:configure-updates, r=str4d
./configure updates

Includes code cherry-picked from the following upstream Bitcoin Core PRs:

- bitcoin/bitcoin#6748
- bitcoin/bitcoin#12373
- bitcoin/bitcoin#12692
- bitcoin/bitcoin#12901
- bitcoin/bitcoin#13005
- bitcoin/bitcoin#13445
- bitcoin/bitcoin#12686
- bitcoin/bitcoin#16435

Part of #2074.
2020-01-30 16:01:48 +00:00
Homu cafc8c7730 Auto merge of #4314 - oxarbitrage:patch-2, r=str4d
Remove travis

As per https://github.com/zcash/zcash/pull/4222#issuecomment-579717367 deleting travis file.
2020-01-29 16:20:19 +00:00
Alfredo Garcia 848526c170
Delete travis file 2020-01-29 12:11:46 -03:00
Taylor Hornby 5274f322b6 Cache releases to reduce network usage and improve performance 2020-01-28 14:53:36 -07:00
Taylor Hornby 992de545f9 Wrap long lines 2020-01-28 14:45:13 -07:00
Homu 37435aa2c2 Auto merge of #4220 - oxarbitrage:issue3748, r=str4d
Return address and type of imported key in z_importkey

Fixes #3748.
2020-01-25 00:45:00 +00:00
Homu 74ff73abab Auto merge of #3858 - str4d:microbench-1, r=str4d
Micro-benchmarking framework part 1

Cherry-picked from the following upstream PRs:

- bitcoin/bitcoin#6733
- bitcoin/bitcoin#6770
- bitcoin/bitcoin#6892
  - Excluding changes to `src/policy/policy.h` which we don't have yet.
- bitcoin/bitcoin#7934
  - Just the benchmark, not the performance improvements.
- bitcoin/bitcoin#8039
- bitcoin/bitcoin#8107
- bitcoin/bitcoin#8115
- bitcoin/bitcoin#8914
  - Required resolving several merge conflicts in code that had been refactored upstream. The changes were simple enough that I decided it was okay to impose merge conflicts on pulling in those refactors later.
- bitcoin/bitcoin#9200
- bitcoin/bitcoin#9202
  - Adds support for measuring CPU cycles, which is later removed in an upstream PR after the refactor. I am including it to reduce future merge conflicts.
- bitcoin/bitcoin#9281
  - Only changes to `src/bench/bench.cpp`
- bitcoin/bitcoin#9498
- bitcoin/bitcoin#9712
- bitcoin/bitcoin#9547
- bitcoin/bitcoin#9505
  - Just the benchmark, not the performance improvements.
- bitcoin/bitcoin#9792
  - Just the benchmark, not the performance improvements.
- bitcoin/bitcoin#10272
- bitcoin/bitcoin#10395
  - Only changes to `src/bench/`
- bitcoin/bitcoin#10735
  - Only changes to `src/bench/base58.cpp`
- bitcoin/bitcoin#10963
- bitcoin/bitcoin#11303
  - Only the benchmark backend change.
- bitcoin/bitcoin#11562
- bitcoin/bitcoin#11646
- bitcoin/bitcoin#11654

This pulls in all changes to the micro-benchmark framework prior to December 2017, when it was rewritten. The rewrite depends on other upstream PRs we have not pulled in yet.

This does not pull in all benchmarks prior to December 2017. It leaves out benchmarks that either test code we do not have yet (except for the `FastRandomContext` refactor, which I decided to pull in), or would require rewrites to work with our changes to the codebase.
2020-01-24 23:39:50 +00:00
Homu fbfa9df22a Auto merge of #3829 - MarkLTZ:master, r=str4d
Update gen_base58_test_vectors.py to work with multi-byte address and key prefixes

This PR closes https://github.com/zcash/zcash/issues/3772
2020-01-24 22:33:51 +00:00
Homu 599ebb9686 Auto merge of #3742 - defuse:pow-security-improvements, r=str4d
Equihash Macro and Testing Improvements
2020-01-24 20:52:30 +00:00
LitecoinZ 7604a3bfae Fix issue #3772
Co-authored-by: Jack Grigg <jack@z.cash>
2020-01-24 20:16:45 +00:00
Homu c3e6c06812 Auto merge of #4308 - zancas:4307_update_comment, r=daira
update comment, to correctly specify number of methods injected

ADD_SERIALIZE_METHODS was incorrectly documented as adding three methods, it now adds two, and the associated documentation correctly asserts this.
2020-01-24 16:02:40 +00:00
zancas d2c24e47ba
update comment, to correctly specify number of methods injected 2020-01-23 19:26:17 -07:00
Cory Fields 204384c016 build: Split hardening/fPIE options out
This allows for fPIE to be used selectively.

Zcash: Second half of upstream commit 17c4d9d1647bbac4b0557136b1c3d98c951feb79
First half was pulled in as c459de2f03
2020-01-22 21:41:28 +00:00
practicalswift 9f94f493f5 Initialize recently introduced non-static class member lastCycles to zero in constructor
lastCycles was introduced in 35328187463a7078b4206e394c21d5515929c7de which was merged into master yesterday.

Also initialize beginCycles to zero for consistency and completeness.
2020-01-22 21:41:28 +00:00
Matt Corallo 3182cde049 Require a steady clock for bench with at least micro precision 2020-01-22 21:41:28 +00:00
Cory Fields 671432ca2a bench: prefer a steady clock if the resolution is no worse 2020-01-22 21:41:28 +00:00
Cory Fields 101cab5ce3 bench: switch to std::chrono for time measurements
std::chrono removes portability issues.

Rather than storing doubles, store the untouched time_points. Then
convert to nanoseconds for display. This allows for maximum precision, while
keeping results comparable between differing hardware/operating systems.

Also, display full nanosecond counts rather than sub-second floats.
2020-01-22 21:41:28 +00:00
Matt Corallo 3954f0cdc0 Remove countMaskInv caching in bench framework
We were saving a div by caching the inverse as a float, but this
ended up requiring a int -> float -> int conversion, which takes
almost as much time as the difference between float mul and div.

There are lots of other more pressing issues with the bench
framework which probably require simply removing the adaptive
iteration count stuff anyway.
2020-01-22 21:41:28 +00:00
practicalswift 550cb05f94 Restore default format state of cout after printing with std::fixed/setprecision 2020-01-22 21:41:28 +00:00
practicalswift 95b6695ba2 Avoid static analyzer warnings regarding uninitialized arguments
Avoid static analyzer warnings regarding "Function call argument
is a pointer to uninitialized value" in cases where we are
intentionally using such arguments.

This is achieved by using ...

`f(b.begin(), b.end())` (`std::array<char, N>`)

... instead of ...

`f(b, b + N)` (`char b[N]`)

Rationale:
* Reduce false positives by guiding static analyzers regarding our
  intentions.

Before this commit:

```
$ clang-tidy-3.5 -checks=* src/bench/base58.cpp
bench/base58.cpp:23:9: warning: Function call argument is a pointer to uninitialized value [clang-analyzer-core.CallAndMessage]
        EncodeBase58(b, b + 32);
        ^
$ clang-tidy-3.5 -checks=* src/bench/verify_script.cpp
bench/verify_script.cpp:59:5: warning: Function call argument is a pointer to uninitialized value [clang-analyzer-core.CallAndMessage]
    key.Set(vchKey, vchKey + 32, false);
    ^
$
```

After this commit:

```
$ clang-tidy-3.5 -checks=* src/bench/base58.cpp
$ clang-tidy-3.5 -checks=* src/bench/verify_script.cpp
$
```

Zcash: Only applied changes to src/bench/base58.cpp
2020-01-22 21:41:28 +00:00
practicalswift 0c89713553 Replace boost::function with std::function (C++11)
Zcash: Only applied changes to src/bench/
2020-01-22 21:41:28 +00:00
Pavel Janík f84f56610e Prevent warning: variable 'x' is uninitialized 2020-01-22 21:41:28 +00:00
Pieter Wuille 8f23ab088d FastRandom benchmark 2020-01-22 21:41:28 +00:00
Pieter Wuille 2070adad85 Introduce FastRandomContext::randbool() 2020-01-22 21:41:28 +00:00
Jeremy Rubin a908ba71c4 Add prevector destructor benchmark 2020-01-22 21:41:28 +00:00
practicalswift 2edcfea7da Assert that what might look like a possible division by zero is actually unreachable 2020-01-22 21:41:28 +00:00
Wladimir J. van der Laan f619b42ca9 bench: Fix initialization order in registration
The initialization order of global data structures in different
implementation units is undefined. Making use of this is essentially
gambling on what the linker does, the so-called [Static initialization
order fiasco](https://isocpp.org/wiki/faq/ctors#static-init-order).

In this case it apparently worked on Linux but failed on OpenBSD and
FreeBSD.

To create it on first use, make the registration structure local to
a function.

Fixes #8910.
2020-01-22 21:41:28 +00:00
Jeremy Rubin f53305ec8a Address ryanofsky feedback on CCheckQueue benchmarks. Eliminated magic numbers, fixed scoping of vectors (and memory movement component of benchmark). 2020-01-22 21:41:28 +00:00
Jeremy Rubin 7e1cebe44f Add Basic CheckQueue Benchmark 2020-01-22 21:41:28 +00:00
Karl-Johan Alm d2b7c97cb5 Refactoring: Removed using namespace <xxx> from bench/ and test/ source files.
Zcash: Only apply changes to src/bench/bench.cpp
2020-01-22 21:41:28 +00:00
Wladimir J. van der Laan 2561e2664d bench: Add support for measuring CPU cycles
This adds cycle min/max/avg to the statistics.

Supported on x86 and x86_64 (natively through rdtsc), as well as Linux
(perf syscall).
2020-01-22 21:41:28 +00:00
Wladimir J. van der Laan 7266c84e16 bench: Fix subtle counting issue when rescaling iteration count
Make sure that the count is a zero modulo the new mask before
scaling, otherwise the next time until a measure triggers
will take only 1/2 as long as accounted for. This caused
the 'min time' to be potentially off by as much as 100%.
2020-01-22 21:41:28 +00:00
Wladimir J. van der Laan 957e5d216f Kill insecure_random and associated global state
There are only a few uses of `insecure_random` outside the tests.
This PR replaces uses of insecure_random (and its accompanying global
state) in the core code with an FastRandomContext that is automatically
seeded on creation.

This is meant to be used for inner loops. The FastRandomContext
can be in the outer scope, or the class itself, then rand32() is used
inside the loop. Useful e.g. for pushing addresses in CNode or the fee
rounding, or randomization for coin selection.

As a context is created per purpose, thus it gets rid of
cross-thread unprotected shared usage of a single set of globals, this
should also get rid of the potential race conditions.

- I'd say TxMempool::check is not called enough to warrant using a special
  fast random context, this is switched to GetRand() (open for
  discussion...)

- The use of `insecure_rand` in ConnectThroughProxy has been replaced by
  an atomic integer counter. The only goal here is to have a different
  credentials pair for each connection to go on a different Tor circuit,
  it does not need to be random nor unpredictable.

- To avoid having a FastRandomContext on every CNode, the context is
  passed into PushAddress as appropriate.

There remains an insecure_random for test usage in `test_random.h`.

Zcash: Resolved conflicts with the following files

	src/addrman.cpp
	src/main.cpp
	src/net.cpp
	src/net.h
	src/policy/fees.cpp
	src/policy/fees.h
	src/random.cpp
	src/test/merkle_tests.cpp
	src/test/net_tests.cpp
	src/test/prevector_tests.cpp
	src/test/sighash_tests.cpp
	src/test/skiplist_tests.cpp
	src/test/test_bitcoin.cpp
	src/test/versionbits_tests.cpp
	src/wallet/test/crypto_tests.cpp
2020-01-22 21:41:26 +00:00
Gregory Maxwell d442711d58 Avoid integer division in the benchmark inner-most loop.
Previously the benchmark code used an integer division (%) with
 a non-constant in the inner-loop.  This is quite slow on many
 processors, especially ones like ARM that lack a hardware divide.

Even on fairly recent x86_64 like haswell an integer division can
 take something like 100 cycles-- making it comparable to the
 runtime of siphash.

This change avoids the division by using bitmasking instead. This
 was especially easy since the count was only increased by doubling.

This change also restarts the timing when the execution time was
 very low this avoids mintimes of zero in cases where one execution
 ends up below the timer resolution. It also reduces the impact of
 the overhead on the final result.

The formatting of the prints is changed to not use scientific
 notation make it more machine readable (in particular, gnuplot
 croaks on the non-fixedpoint, and it doesn't sort correctly).

This also hoists out all the floating point divisions out of the
 semi-hot path because it was easy to do so.

It might be prudent to break out the critical test into a macro
 just to guarantee that it gets inlined.  It might also make sense
 to just save out the intermediate counts and times and get the
 floating point completely out of the timing loop (because e.g.
 on hardware without a fast hardware FPU like some ARM it will
 still be slow enough to distort the results). I haven't done
 either of these in this commit.
2020-01-22 21:40:38 +00:00
Yuri Zhykin 9585d5a64f bench: Added base58 encoding/decoding benchmarks 2020-01-22 21:40:38 +00:00
Wladimir J. van der Laan 6daf01e402 bench: Add crypto hash benchmarks
Add benchmarks for the cryptographic hash algorithms:

- RIPEMD160
- SHA1
- SHA256
- SHA512

Continues work on #7883.
2020-01-22 21:40:38 +00:00
Pieter Wuille 3161960fc2 Benchmark rolling bloom filter 2020-01-22 21:40:38 +00:00
Philip Kaufmann e72de1864f [Trivial] ensure minimal header conventions
- ensure header namespaces and end comments are correct
- add missing header end comments
- ensure minimal formatting (add newlines etc.)

Zcash: left out change to src/policy/policy.h which we don't yet have.
2020-01-22 21:40:38 +00:00
Pavel Janík 9233b8a15e Ignore bench_bitcoin binary. 2020-01-22 21:40:38 +00:00
Gavin Andresen a81da44a8f Support very-fast-running benchmarks
Avoid calling gettimeofday every time through the benchmarking loop, by keeping
track of how long each loop takes and doubling the number of iterations done
between time checks when they take less than 1/16'th of the total elapsed time.
2020-01-22 21:40:38 +00:00