Commit Graph

14496 Commits

Author SHA1 Message Date
Homu 311a079dd5 Auto merge of #4827 - str4d:commit-script-check, r=str4d
Verifier for scriptable changes

Includes changes from the following upstream PRs:
- bitcoin/bitcoin#10189
  - Excluding the `CNode` scripted changes.
- bitcoin/bitcoin#10480
- bitcoin/bitcoin#11390
- bitcoin/bitcoin#13281
  - Only the lint scripts we already have.
- bitcoin/bitcoin#13454
  - Only changes to scripts we already have.
- bitcoin/bitcoin#14864
- bitcoin/bitcoin#16327
  - Includes some portability fixes to other shell scripts.
- bitcoin/bitcoin#20069
2020-10-27 00:06:46 +00:00
Jack Grigg 1b42734b9f tests: Update chained_joinsplits test for HaveShieldedRequirements API change 2020-10-26 23:46:19 +00:00
str4d 8aadc745c2
Use HTTPS in script license headers
Co-authored-by: Daira Hopwood <daira@jacaranda.org>
2020-10-26 18:04:19 +00:00
str4d e9b836d612
Update example scripted-diff comit in doc/developer-notes.md 2020-10-26 17:59:18 +00:00
Jack Grigg 1b01bd942b test/lint: Check for working changes before checking scripted diffs 2020-10-26 17:57:26 +00:00
Wladimir J. van der Laan 001e803b92 test: Mention commit id in scripted diff error 2020-10-26 17:36:26 +00:00
Hennadii Stepanov 1d3af17140 Enable ShellCheck rules
Enabled ShellCheck rules:
  SC1087
  SC2001
  SC2004
  SC2005
  SC2006
  SC2016
  SC2028
  SC2048
  SC2066 (note that IFS already contains only a line feed)
  SC2116
  SC2166
  SC2181
  SC2206
  SC2207
  SC2230
  SC2236

Zcash: Only the changes that applied to the versions of the scripts we have.
2020-10-26 17:36:26 +00:00
Carl Dong ccd074a59b scripted-diff: Run scripted-diff in subshell
-BEGIN VERIFY SCRIPT-
sed -i 's/\bi\b/commit/g' test/lint/commit-script-check.sh
sed -i '34s/eval "$SCRIPT"/(eval "$SCRIPT")/' test/lint/commit-script-check.sh
-END VERIFY SCRIPT-
2020-10-26 17:36:26 +00:00
practicalswift 3b1bf4b4af Add "export LC_ALL=C" to all shell scripts
Zcash: Only for shell scripts we have from upstream.
2020-10-26 17:36:26 +00:00
MarcoFalke 6506fbacf1 test: Move linters to test/lint, add readme 2020-10-26 17:36:26 +00:00
John Newbery 9645ae5be0 [docs] document scripted-diff 2020-10-26 17:20:55 +00:00
Pieter Wuille 6b6f3b5649 Fail on commit with VERIFY SCRIPT but no scripted-diff 2020-10-26 17:20:55 +00:00
Cory Fields 553c9e6850 devtools: add script to verify scriptable changes 2020-10-26 15:54:33 +00:00
Jack Grigg 8f56306359 Add txid to "shielded requirements not met" messages 2020-10-26 14:12:57 +00:00
Homu 3605406358 Auto merge of #4815 - daira:fix-checksum-message, r=daira
Fix an error reporting bug in "Checksum missing or mismatched ..."

The sense of the test was accidentally inverted in my change to #4733. The message should be shown if any of the files exist but have an incorrect checksum. fixes #4813

No documentation needed.
Test plan: as for #4733, and also check that there are no false positive "Checksum missing or mismatched ..." warnings.

Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2020-10-26 12:12:15 +00:00
Daira Hopwood e86f83c627 Fix an error reporting bug in "Checksum missing or mismatched ..."
The sense of the test was accidentally inverted in my change to #4733.
The message should be shown if any of the files exist but have an incorrect checksum.
We also now correctly handle the case where there are no package source files.

Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2020-10-26 07:44:21 +00:00
Homu 40d5f0aa57 Auto merge of #4805 - str4d:fs-abstraction, r=str4d
Lightweight abstraction of boost::filesystem

This is a refactor backport ahead of replacing most of our Boost dependencies with C++17 code.

Cherry-picked from the following upstream PRs:
- bitcoin/bitcoin#7667
  - Removes merge conflicts.
- bitcoin/bitcoin#9902
- bitcoin/bitcoin@2300a5e13a
- bitcoin/bitcoin#10546
  - Only the changes to `src/fs.cpp`
2020-10-26 02:03:34 +00:00
Jack Grigg d7c80e760c utils: Remove unnecessary GetTempPath() 2020-10-26 02:02:18 +00:00
Homu fef4b911d1 Auto merge of #4804 - str4d:update-secp256k1-again, r=daira
Update secp256k1 again

This migrates us to the same dependency version that upstream Bitcoin
Core migrated to in bitcoin/bitcoin#20147.
2020-10-25 00:45:09 +00:00
Jack Grigg 6fa2d1b73e Add logging to CCoinsViewCache::HaveShieldedRequirements
Enable with -debug=consensus
2020-10-23 20:16:07 +01:00
Jack Grigg 47c0c65326 Improve reject reasons for unmet shielded requirements
These reject messages end up bubbling up to users via the RPC interface.
Distinguishing between the various failure cases will help users figure
out why their transaction is being rejected.

Uses operator* instead of std::optional::value because the latter was
introduced in macOS 10.14, and our current minimum is 10.12.

Closes zcash/zcash#3114.
2020-10-23 20:16:07 +01:00
Homu a4b6aaa333 Auto merge of #4810 - daira:fix-death-tests, r=daira
Fix death gtests on macOS by switching to the threadsafe style

[Tested on macOS](https://github.com/zcash/zcash/issues/4802#issuecomment-714770192). fixes #4802

Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2020-10-23 17:26:33 +00:00
Homu 693def7bac Auto merge of #4733 - nathan-at-least:download-stamp-native_rust-fix, r=daira
depends download: fix a logging bug for multi-archive packages:

No documentation change needed.

The test plan appears below. Since this bug output should appear on all builds without a pre-existing source cache, then an easier test plan might be to verify the bug output is not present from an infrastructure build log that doesn't rely on cached sources.

# Bug Behavior

While initially fetching packages, I saw `sh: test:` error messages in the make output for only two packages. However, it appears all packages are correctly fetched.

See detailed output appendix for demonstration of the bug output and conditions compared to with this patch for the same conditions.

# Analysis

## Design Requirements

The intent of the `check_or_remove_sources` is to fetch sources if either existing archive hashes do not match or the archives aren't present.

Additionally, this should report hash mismatches and be quiet about missing files.

## Bug Source

The second design requirement goal has a bug for multi-archive packages which define `$(package)_extra_sources)` because `test -f $($(package)_all_sources)` passes more than one argument to `test -f` which is not supported. If sha256sum fails in any case for multi-archive packages, a `sh: test` error line is always printed.

Aside from this spurious output, this bug has no effect, I believe. Fortunately this bug does not bypass the hash check!

## Testing

I tested this patch manually by running the same three tests against the `v4.0.0` tag and then again repeating those same three steps with this patch. In all 6 cases I visually inspected the output.

1. Starting with a pre-downloaded source cash, run the `download` target.
2. Remove an archive from a multi-archive package, then rerun `download`.
3. Alter a hash to cause a hash mismatch condition, then rerun `download`.

I believe after each step both with and without this patch the resulting source cache should be identical (except for filesystem timestamps).

### Testing v4.0.0 without this patch

Notice in the second and third steps the same bug output:

```
/bin/sh: 1: test: rust-std-1.42.0-x86_64-apple-darwin.tar.gz: unexpected operator
```
2020-10-23 15:43:57 +00:00
Daira Hopwood f23528ddd0 Fix death gtests on macOS by switching to the threadsafe style.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2020-10-23 16:08:59 +01:00
Homu 888039caf1 Auto merge of #4808 - nuttycom:fix_release_notes, r=daira
Update other incorrect uses of `python` instead of `python3`

This also adds explicit replacement of invalid characters in file read/write operations.
2020-10-22 19:07:35 +00:00
Kris Nuttycombe 4567e6f5fb Replace invalid characters in log message decoding. 2020-10-22 10:20:24 -06:00
Homu d264471f2a Auto merge of #4807 - nuttycom:release-fix-updatecheck, r=nuttycom
Fix invocation of updatecheck.py in make-release.py
2020-10-22 13:54:05 +00:00
Kris Nuttycombe b28c5353e2 Fix invocation of updatecheck.py in make-release.py 2020-10-22 07:44:08 -06:00
Nate Wilcox 94fa873853 depends: fix a logging bug for multi-archive packages.
While initially fetching packages, I saw `sh: test:` error messages in the make output for only two packages.
However, it appears all packages are correctly fetched.

I tested this patch manually by running these three tests against the `v4.0.0` tag and then with this patch.
In all 6 cases I visually inspected the output.

1. Starting with a pre-downloaded source cash, run the `download` target.
2. Remove an archive from a multi-archive package, then rerun `download`.
3. Alter a hash to cause a hash mismatch condition, then rerun `download`.

I believe after each step both with and without this patch the resulting source cache should be identical
(except for filesystem timestamps).

Co-authored-by: Nathan Wilcox <nathan@electriccoin.co>
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2020-10-22 14:43:22 +01:00
practicalswift 3cf87df03c Remove unused Boost includes
Zcash: Only the changes to src/fs.cpp
2020-10-22 01:11:37 +01:00
Ahmad Kazi e2273f1011 [Trivial] Add BITCOIN_FS_H endif footer in fs.h 2020-10-22 01:11:37 +01:00
Wladimir J. van der Laan 0d3bba071d Remove `namespace fs=fs`
Having these inside functions is silly and redundant now.
2020-10-22 01:11:37 +01:00
Wladimir J. van der Laan 532a0c5c52 torcontrol: Use fs::path instead of std::string for private key path 2020-10-22 01:11:37 +01:00
Wladimir J. van der Laan c0603a9fa6 Use fsbridge for fopen and freopen
Abstracts away how a path is opened to a `FILE*`.

Reduces the number of places where path is converted to a string
for anything else but printing.
2020-10-22 01:11:37 +01:00
Wladimir J. van der Laan 73bc7a068d Replace uses of boost::filesystem with fs
Step two in abstracting away boost::filesystem.

To repeat this, simply run:
```
git ls-files \*.cpp \*.h | xargs sed -i 's/boost::filesystem/fs/g'
```
2020-10-22 01:11:37 +01:00
Wladimir J. van der Laan 7044e39a57 Replace includes of boost/filesystem.h with fs.h
This is step one in abstracting the use of boost::filesystem.
2020-10-22 01:11:37 +01:00
Wladimir J. van der Laan 082f530d6e Add fs.cpp/h 2020-10-22 01:11:37 +01:00
Mustafa dafc0d35eb Move GetTempPath() to testutil. 2020-10-22 01:11:37 +01:00
Mustafa 42929fae9d Add a source file for unit test utils. 2020-10-22 01:11:37 +01:00
Jack Grigg 0522f6f0d0 build: Update secp256k1 configure flags 2020-10-22 00:45:25 +01:00
Jack Grigg eec9e2dcac Update libsecp256k1 subtree to latest master 2020-10-22 00:42:47 +01:00
Jack Grigg 52ac87242d Squashed 'src/secp256k1/' changes from 8ab24e8da..c6b6b8f1b
c6b6b8f1b Merge #830: Rip out non-endomorphism code + dependencies
c582abade Consistency improvements to the comments
63c6b7161 Reorder comments/function around scalar_split_lambda
2edc514c9 WNAF of lambda_split output has max size 129
4232e5b7d Rip out non-endomorphism code
ebad8414b Check correctness of lambda split without -DVERIFY
fe7fc1fda Make lambda constant accessible
9d2f2b44d Add tests to exercise lambda split near bounds
9aca2f7f0 Add secp256k1_split_lambda_verify
acab934d2 Detailed comments for secp256k1_scalar_split_lambda
76ed922a5 Increase precision of g1 and g2
6173839c9 Switch to our own memcmp function
63150ab4d Merge #827: Rename testrand functions to have test in name
c5257aed0 Merge #821: travis: Explicitly set --with-valgrind
bb1f54280 Merge #818: Add static assertion that uint32_t is unsigned int or wider
a45c1fa63 Rename testrand functions to have test in name
5006895bd Merge #808: Exhaustive test improvements + exhaustive schnorrsig tests
4eecb4d6e travis: VALGRIND->RUN_VALGRIND to avoid confusion with WITH_VALGRIND
66a765c77 travis: Explicitly set --with-valgrind
d7838ba6a Merge #813: Enable configuring Valgrind support
7ceb0b761 Merge #819: Enable -Wundef warning
8b7dcdd95 Add exhaustive test for extrakeys and schnorrsig
08d7d8929 Make pubkey parsing test whether points are in the correct subgroup
87af00b51 Abstract out challenge computation in schnorrsig
63e1b2aa7 Disable output buffering in tests_exhaustive.c
39f67dd07 Support splitting exhaustive tests across cores
e99b26fcd Give exhaustive_tests count and seed cmdline inputs
49e6630bc refactor: move RNG seeding to testrand
b110c106f Change exhaustive test groups so they have a point with X=1
cec7b18a3 Select exhaustive lambda in function of order
78f6cdfaa Make the curve B constant a secp256k1_fe
d7f39ae4b Delete gej_is_valid_var: unused outside tests
8bcd78cd7 Make secp256k1_scalar_b32 detect overflow in scalar_low
c498366e5 Move exhaustive tests for recovery to module
be3179154 Make group order purely compile-time in exhaustive tests
e73ff3092 Enable -Wundef warning
c0041b5cf Add static assertion that uint32_t is unsigned int or wider
4ad408faf Merge #782: Check if variable=yes instead of if var is set in travis.sh
412bf874d configure: Allow specifying --with[out]-valgrind explicitly
34debf7a6 Modify .travis.yml to explictly pass no in env vars instead of setting to nothing
a0e99fc12 Merge #814: tests: Initialize random group elements fully
5738e8622 tests: Initialize random group elements fully
c9939ba55 Merge #812: travis: run bench_schnorrsig
a51f2af62 travis: run bench_schnorrsig
ef37761fe Change travis.sh to check if variables are equal to yes instead of not-empty. Before this, setting `VALGRIND=wat` was considered as true, and to make it evaluate as false you had to unset the variable `VALGRIND=` but not it checks if `VALGRIND=yes` and if it's not `yes` then it's evaluated to false

git-subtree-dir: src/secp256k1
git-subtree-split: c6b6b8f1bb044d7d1aa065ebb674adde98a36a8e
2020-10-22 00:41:56 +01:00
Homu afe85096cb Auto merge of #4803 - therealyingtong:postpone-ccache, r=therealyingtong
Postpone native_ccache 4.0
2020-10-21 23:12:45 +00:00
Homu cafc622a22 Auto merge of #4757 - str4d:update-secp256k1, r=str4d
Update secp256k1

This migrates us to the same dependency version that upstream Bitcoin
Core migrated to in bitcoin/bitcoin#19944.

Also enables the endomorphism optimization now that the patents have
expired.
2020-10-21 16:55:47 +00:00
therealyingtong d8f20860de Postpone native_ccache 4.0 2020-10-21 23:15:19 +08:00
Homu 43ab084854 Auto merge of #4611 - str4d:c++17, r=str4d
Require and build with C++17

Closes zcash/zcash#4607.
2020-10-20 14:28:42 +00:00
Jack Grigg d0cb0b74aa Switch from std::random_shuffle to std::shuffle
std::random_shuffle is removed in C++17; it is replaced by std::shuffle,
which was introduced in C++11. The new ZcashRandomEngine class provides
the Uniform Random Number Generator interface, as inferred from
std::random_device.
2020-10-20 13:58:23 +01:00
Daira Hopwood c656ae1303 Add new copyright entries for build-aux/m4/ax_cxx_compile_stdcxx.m4
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2020-10-20 13:55:14 +01:00
Jack Grigg 9052751c21 depends: Build C++ dependencies with C++ 17 2020-10-20 13:55:14 +01:00
Jack Grigg 9d1a11a0e4 build: Require and build with C++ 17 2020-10-20 13:55:14 +01:00