Commit Graph

1178 Commits

Author SHA1 Message Date
MarcoFalke 8b3c39ea80 [qa] rpc-tests: Apply random offset to portseed
This helps to skip over resources, which are blocked by regtest bitcoind
zombie nodes
2020-11-21 03:03:15 +00:00
MarcoFalke 0a73a0fbd9 [qa] test_framework: Exit when tmpdir exists 2020-11-21 03:03:15 +00:00
MarcoFalke d014a30566 [qa] pull-tester: Only print output when failed 2020-11-21 03:03:15 +00:00
MarcoFalke bcbf03ebe9 [qa] Refactor RPCTestHandler to prevent TimeoutExpired 2020-11-21 03:03:15 +00:00
MarcoFalke 352f2dc54b [qa] create_cache: Delete temp dir when done 2020-11-21 03:03:15 +00:00
isle2983 2d90c8f0d8 [doc] - clarify statement about parallel jobs in rpc-tests.py 2020-11-21 03:03:12 +00:00
MarcoFalke 4500a75346 [qa] pull-tester: Don't mute zmq ImportError
Zcash: We didn't backport the mute, so this is just conflict removal.
2020-11-21 03:02:41 +00:00
MarcoFalke e4d1dcc753 [qa] Remove unused code 2020-11-21 03:02:41 +00:00
MarcoFalke b25a63593e [qa] Use single cache dir for chains 2020-11-21 03:02:41 +00:00
MarcoFalke dbb41d1082 [qa] Adjust timeouts for micro-optimization of run time 2020-11-21 03:02:41 +00:00
whythat 1b3866fd0e [qa]: enable rpcbind_test 2020-11-21 03:02:41 +00:00
whythat 3c7f4ca809 [qa]: add parsing for '<host>:<port>' argument form to rpc_url() 2020-11-21 03:02:41 +00:00
fanquake 2fc450d94e [doc] Add OS X ZMQ requirement to QA readme 2020-11-21 03:02:37 +00:00
MarcoFalke 7366e81375 [qa] pull-tester: Start longest test first 2020-11-21 02:34:47 +00:00
MarcoFalke fd7689ff07 [qa] pull-tester: Fix assertion and check for run_parallel 2020-11-21 02:34:47 +00:00
MarcoFalke 63a3c9aceb [qa] test_framework: Use different rpc_auth_pair for each node 2020-11-21 02:34:47 +00:00
Cory Fields ecf9d8eb2b build: a few ugly hacks to get the rpc tests working out-of-tree
- Link pull-tester/rpc-tests.py to the build dir
- Add the build-dir's config to the python path so that tests can find it
- The tests themselves are in srcdir
- Clean up __pycache__ in 'make clean'
2020-11-21 02:34:47 +00:00
Suhas Daftuar 2399cfc7ba Tests: add timeout to sync_blocks() and sync_mempools()
Previously these functions would infinitely loop if sync failed;
now they have a default timeout of 60 seconds, after which an
AssertionError is raised.

sync_blocks() has also been improved and now compares the tip
hash of each node, rather than just using block count.

Zcash: Kept block count check for a couple of tests where we use it.
2020-11-21 02:34:47 +00:00
MarcoFalke 17caf1ddd1 [qa] test_framework: Append portseed to tmpdir
This makes it possible to specify a tmpdir while running tests in
parallel
2020-11-21 02:34:47 +00:00
MarcoFalke 818d2de39b [qa] Remove hardcoded "4 nodes" from test_framework
Zcash: Applied changes to our RPC tests.
2020-11-21 02:34:47 +00:00
MarcoFalke 5a12bc9d83 [qa] Add option --portseed to test_framework 2020-11-21 02:33:27 +00:00
MarcoFalke cbe91ea855 [qa] pull-tester: Run rpc test in parallel 2020-11-21 02:33:27 +00:00
MarcoFalke 0d8a62925a [qa] pull-tester: Adjust comment 2020-11-21 02:33:27 +00:00
MarcoFalke 5f13446dee [qa] Stop other nodes, even when one fails to stop 2020-11-21 02:33:27 +00:00
MarcoFalke 065255d3cb [qa] Update README.md 2020-11-21 02:33:27 +00:00
MarcoFalke 08256c9ec6 [qa] Refactor test_framework and pull tester
* log to stdout
* increase range for p2p and rpc ports
* UPPERCASE_CONSTANTS
* Stop nodes on CTRL+C
2020-11-21 02:33:27 +00:00
MarcoFalke 15721275d0 [qa] Switch to py3
Zcash: We already did this, so most of the backported changes are
ancillary (license header updates and style), but a few are relevant.
2020-11-21 02:33:27 +00:00
MarcoFalke d362b632bf [qa] rpc-tests: Fix link in comment and label error msg 2020-11-21 02:33:27 +00:00
MarcoFalke cc44368315 [qa] pull-tester: Exit early when no tests are run 2020-11-21 02:33:27 +00:00
MarcoFalke e7a6c49c00 [qa] pull-tester: Don't mute zmq ImportError 2020-11-21 02:33:27 +00:00
MarcoFalke db761077e5 [qa] mininode: Catch exceptions in got_data 2020-11-21 02:33:27 +00:00
MarcoFalke 7108d6aaa6 [qa] rpc-tests: Properly use integers, floats 2020-11-21 02:33:27 +00:00
MarcoFalke 65a6249d41 [qa] Use python2/3 syntax 2020-11-21 02:33:27 +00:00
Elliot Olds 845b31a760 Check if zmq is installed in tests, update docs
If ZMQ is enabled, check whether it's installed before running ZMQ tests.
If it isn't, disable ZMQ and print a warning.

Also add dependency info to test docs, so users know ZMQ is required
before running tests, and so they know how to install it.

When following the build instructions before this change then trying
to run the RPC tests, a unix user would get an error when python
tried to import zmq.

There may be other dependencies that should be added to the docs,
particularly ones for non-unix systems. This is the only unlisted
dependency I encountered using linux.
2020-11-21 02:33:27 +00:00
Wladimir J. van der Laan 239857d7d9 test: don't override BITCOIND and BITCOINCLI if they're set
In rpc-tests.py, don't override BITCOIND and BITCOINCLI if they're
already set. Makes it possible to run the tests with either another tree
or the GUI.
2020-11-21 02:33:27 +00:00
MarcoFalke fb1f76f330 [qa] pull-tester: Cleanup (run keypool, tidy stdout)
* Run keypool (takes 6 seconds)
* Print duration of each rpc test
* Structure output (bold, new lines)
2020-11-21 02:33:27 +00:00
MarcoFalke db27db2e43 [qa] keypool: DRY: Use test framework 2020-11-21 02:33:27 +00:00
MarcoFalke fa44509d6c [qa] keypool: Fix white space to prepare transition to test framework 2020-11-21 02:33:27 +00:00
MarcoFalke 01c36295c4 [qa] Extend README.md 2020-11-21 02:33:27 +00:00
MarcoFalke 80b2274ddf [qa] Split README.md to /qa and /qa/rpc-tests
+ Update with new -help message
2020-11-21 02:33:27 +00:00
Suhas Daftuar 800bf0a45f Remove unmaintained example test script_test.py 2020-11-21 02:33:27 +00:00
James O'Beirne eaf4dfe280 Add basic coverage reporting for RPC tests
Thanks to @MarcoFalke @dexX7 @laanwj for review.

Zcash: Just the --coverage flag for rpc-tests.py, as we backported the
rest of the coverage backend in zcash/zcash#4411.
2020-11-21 02:33:27 +00:00
MarcoFalke 760e500811 [rpc-tests] Check return code 2020-11-20 03:32:53 +00:00
MarcoFalke 6aa4f19e87 [rpc-tests] fundrawtransaction: Update fee after minRelayTxFee increase
Zcash: Added floor, since our minRelayTxFee is actually lower.
2020-11-20 03:32:53 +00:00
James O'Beirne 480f1bf43e Add tests for gettxoutsetinfo, CLevelDBBatch, CLevelDBIterator
Thanks @dexX7.

Zcash: Rest of this commit was cherry-picked in zcash/zcash#2598.
2020-11-20 03:32:53 +00:00
MarcoFalke 8e58eddab1 [doc] trivial: fix markdown syntax in qa/rpc-tests/README.md 2020-11-20 03:32:53 +00:00
Jeff Garzik 6618580194 qa/pull-tester/rpc-tests.py: chmod 0755
Fix file mode to be executable.

Include dummy whitespace change to force git to change mode.
2020-11-20 03:32:53 +00:00
ptschip 772d780730 Migrated rpc-tests.sh to all python rpc-tests.py
1) created rpc-tests.py
2) deleted rpc-tests.sh
3) travis.yml points to rpc-tests.py
4) Modified Makefile.am
5) Updated README.md
6) Added tests_config.py and deleted tests-config.sh
7) Modified configure.ac with script to set correct path in tests_config.py

Zcash: Migrated our test list over, and other necessary modifications.
The UI changes were not migrated.
2020-11-20 03:32:53 +00:00
Casey Rodarmor 4761c9f2bc Add p2p-fullblocktest.py
Zcash: Includes pyflakes changes.
2020-11-20 03:32:53 +00:00
Daniel Kraft 95d1f887ca Fix crash when mining with empty keypool.
Since the introduction of the ScriptForMining callback, the mining
functions (setgenerate and generate) crash with an assertion failure
(due to a NULL pointer script returned) if the keypool is empty.  Fix
this by giving a proper error.

Zcash: Adapted to our MinerAddress type.

Co-authored-by: Jack Grigg <jack@electriccoin.co>
2020-11-20 03:30:37 +00:00
Homu 10b09d483e Auto merge of #4793 - str4d:faster-rpc-tests, r=str4d
test: Use default shielded address in RPC tests where the type is irrelevant

This means we are running these tests against the recommended functionality,
and that these RPC tests will now use the faster Sapling addresses.
2020-11-19 14:15:08 +00:00
Homu 949fdca3a6 Auto merge of #4873 - str4d:rpc-test-blocks, r=str4d
Enable mininodes to construct blocks in RPC tests

Includes changes to disable difficulty adjustment on regtest, cherry-picked from:
- bitcoin/bitcoin#6853
- bitcoin/bitcoin#13046

The ZIP 143/243 logic is ported from https://github.com/zcash-hackworks/zcash-test-vectors
2020-11-18 15:11:49 +00:00
Homu 2cc836680a Auto merge of #4843 - furszy:2020_improving_sendmany_redudant_loops_first_round, r=daira
Improving asyncOp_sendmany redundancies (first round)

Work over async sendmany operation flow. Specifically on the utxo finding, utxo selection, total amount calculation and the tx inputs appending process.  Removed several unneeded for loops over the entire utxo set, re structuring and cleaning the flow.
2020-11-18 13:54:13 +00:00
Jack Grigg a7f9365fb8 test: Set hashFinalSaplingRoot default correctly in create_block 2020-11-18 11:12:09 +00:00
Jack Grigg 030a685392 test: Fix SignatureHash RPC test helper
- Check validity of inIdx.
- ZIP 143 and 243 support.
2020-11-18 11:11:54 +00:00
Jack Grigg bbed3ecc3c test: Remove FindAndDelete from RPC test framework 2020-11-17 18:48:32 +00:00
Jack Grigg 8bc0a591a0 Revert "remove SignatureHash from python rpc tests"
This reverts commit 910ff448d6.
2020-11-17 18:48:32 +00:00
Jesse Cohen aead91419e [doc][trivial] no retargeting in regtest mode 2020-11-17 18:48:32 +00:00
mdr0id 4dc94c954f Update expected fails for Sprout txns flows on Canopy 2020-11-16 15:57:49 -08:00
furszy ca9fc49fc4
rpc_wallet_tests: changed "Insufficient funds" error message to a proper "Insufficient transparent funds". This is because we are now throwing the insufficient transparent balance rpc error inside load_utxo. 2020-11-13 10:23:59 -03:00
Jack Grigg 6c43a57e78 test: Use default shielded address in RPC tests where the type is irrelevant
This means we are running these tests against the recommended functionality,
and that these RPC tests will now use the faster Sapling addresses.
2020-11-11 22:43:26 +00:00
Homu 60331b9e83 Auto merge of #4574 - oxarbitrage:issue4475, r=daira
Add blockhash parameter to getrawtransaction

Code ported manually from https://github.com/bitcoin/bitcoin/pull/10275 in attempt to fix https://github.com/zcash/zcash/issues/4475
2020-11-10 15:52:11 +00:00
Jack Grigg dbfb3565d6 lint: Opt out of locale dependence in Zcash shell scripts 2020-11-09 23:06:21 +00:00
Jack Grigg 7c4b19fa4d lint: Use consistent shebangs 2020-11-09 23:06:21 +00:00
Jack Grigg 337e528d33 python: Explicitly set encoding to utf8 when opening text files 2020-11-09 23:06:21 +00:00
Wladimir J. van der Laan 5295079b2e test: Explicitly set encoding to utf8 when opening text files
These are text files but their encoding does not depend on the locale.
Not all of them require utf8 but it is better to fix it at something
to remove potential unpredictability.

This is necessary on FreeBSD where no locale is set by default,
and apparently Python defaults not only the terminal encoding to the locale
but that of every text file. So without LOCALE environment it defaults text
file encoding to ASCII. This causes problems with e.g. `bitcoin.conf`.

Luckily the locale doesn't affect the default encoding for str.encode() and
bytes.decode() on Python 3, so this is the only change necessary.
2020-11-09 22:59:04 +00:00
Daira Hopwood 90493623a0
Fix a comment about postponement of Clang upgrade 2020-11-03 09:55:02 +00:00
Sean Bowe 34c2a4b380
Postpone clang and libc++ dependency updates until after the 4.1.0 release. 2020-11-02 13:36:34 -07:00
Larry Ruane a91a7d37cf add z_gettreestate rpc 2020-10-26 23:08:47 -06:00
therealyingtong d8f20860de Postpone native_ccache 4.0 2020-10-21 23:15:19 +08:00
Homu de5c28b4b7 Auto merge of #4613 - str4d:claaaaaang, r=daira
Pin Clang for all compilation

fixes #2513, fixes #4697, fixes #4698, fixes #4699. refs #4607, #4292
2020-10-20 11:15:47 +00:00
Homu 8fa19fd50d Auto merge of #4794 - LarryRuane:allow-getaddressutxos, r=daira
allow getaddressutxos if -lightwalletd

The wallets need to be able to retrieve, via lightwalletd, the UTXO set for a given address, see https://github.com/zcash/lightwalletd/issues/312. The `getaddressutxos` rpc that was added for Insight Explorer is perfect for this, but this rpc is disabled when only `-lightwalletd` is enabled (that is, it requires `-insightexplorer`, which is a superset of `-lightwalletd`). However, the `DB_ADDRESSUNSPENTINDEX` index that supports this rpc is present when only `-lightwalletd` is enabled. This was probably a (fortuitous, as it turns out) oversight. So all that's required is to allow this rpc when `-lightwalletd` is enabled.
2020-10-20 09:31:39 +00:00
Larry Ruane 083a05b04f allow getaddressutxos if -lightwalletd 2020-10-19 11:26:16 -06:00
Homu ceb642550b Auto merge of #4789 - daira:4785-macos-tests, r=str4d
Test suite portability fixes for macOS

Fixes zcash/zcash#4785
2020-10-16 17:06:40 +00:00
Jack Grigg ee079a683b test: Fix various pyflakes warnings 2020-10-16 16:00:02 +01:00
Jack Grigg ea7d57aa47 QA: Add native_clang and libcxx to updatecheck.py 2020-10-16 15:43:50 +01:00
Daira Hopwood c06cc5854e qa/zcash/full_test_suite.py: print immediately if a test fails.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2020-10-16 03:58:31 +01:00
Daira Hopwood f85ed2741c qa/zcash/full_test_suite.py: changes needed for macOS. fixes #4785
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
2020-10-16 03:57:57 +01:00
Jack Grigg bf450b2607 qa: Disable FORTIFY_SOURCE checks
FORTIFY_SOURCE does mostly nothing for Clang before 10, which we don't
pin yet (and won't before we next upgrade Rust).
2020-10-14 21:52:32 +01:00
Jack Grigg cd68450174 depends: Postpone current and scheduled Rust releases until 2021
We don't currently have a need for newer releases, and will likely be
pinning Rust for longer periods of time once we are also pinning Clang.
Rust releases occur every six weeks, so we can pre-emptively postpone
releases through the end of this year.
2020-10-09 12:42:32 +01:00
Jack Grigg 7c9fa28274 depends: ZeroMQ 4.3.3 2020-10-06 00:03:21 +01:00
Jack Grigg a9f62bdda0 depends: Boost 1.74.0
- The old patch is no longer necessary because of this upstream fix:
    https://github.com/boostorg/build/pull/560

- Boost 1.72 removed a <deque> from an include, which exposed a missing
  include in src/httpserver.cpp.

- Boost 1.73 moved function placeholders into the boost::placeholders
  namespace.

- The new patch is a fix from just after Boost 1.74 was released, fixing
  a warning that was missed.
2020-10-05 19:48:46 +01:00
Homu ce44f60c26 Auto merge of #4758 - str4d:depends-updates, r=str4d
Update dependencies
2020-10-01 16:34:04 +00:00
Homu a983344931 Auto merge of #4752 - str4d:small-httpserver-backports, r=daira
Small httpserver.cpp backports

Also includes a change to the `uiInterface.NotifyBlockTip` signal API.
These remove merge conflicts from subsequent backports for `sync.h`.

Cherry-picked from the following upstream PRs:
- bitcoin/bitcoin#6859
- bitcoin/bitcoin#7112
  - Only the non-QT changes.
- bitcoin/bitcoin#7966
- bitcoin/bitcoin#8421
  - We already backported the second commit in zcash/zcash#2555
2020-10-01 12:48:58 +00:00
Jack Grigg e15eaf0343 QA: Fix backporting bugs in httpbasics.py 2020-10-01 13:44:21 +01:00
Jack Grigg 076c1af55c depends: utfcpp 3.1.2 2020-10-01 11:10:33 +01:00
Jack Grigg 78a7230f50 depends: googletest 1.8.1 2020-10-01 10:57:57 +01:00
Jack Grigg 7725f66051 depends: ccache 3.7.11 2020-10-01 10:57:26 +01:00
Jack Grigg 96da9c2c00 depends: libevent 2.1.12
This includes https://github.com/libevent/libevent/pull/637 which fixes
a problem we encounter during cross-compilation with clang.
2020-10-01 10:56:42 +01:00
Jack Grigg bab853748e QA: Remove OpenSSL from updatecheck.py 2020-09-30 00:40:12 +01:00
Wladimir J. van der Laan 23556e690e http: Restrict maximum size of request line + headers
Prevent memory exhaustion by sending lots of data.
Also add a test to `httpbasics.py`.

Closes #6425
2020-09-30 00:35:32 +01:00
Homu 9245388bbc Auto merge of #4749 - str4d:update-config-guess-and-sub, r=str4d
depends: Update to latest config.guess & config.sub
2020-09-29 02:17:39 +00:00
Jack Grigg f52793d0d0 QA: Switch to x86_64-pc-linux-gnu for hard-coded Linux HOST
The change to config.guess for x86_64:Linux was made here:
    http://git.savannah.gnu.org/cgit/config.git/commit/?id=ca9bfb8cc75a2be1819d89c664a867785c96c9ba
2020-09-28 21:28:43 +01:00
Homu 514d868179 Auto merge of #4739 - str4d:update-leveldb, r=str4d
Update LevelDB to upstream commit f545dfabf

Cherry-picked from the following upstream PRs:

- bitcoin/bitcoin#7911
- bitcoin/bitcoin#7982
- bitcoin/bitcoin#8133
- bitcoin/bitcoin#8784
  - Only the missing changes.
- bitcoin/bitcoin#8826
- bitcoin/bitcoin#8613
- bitcoin/bitcoin#10544
- bitcoin/bitcoin#10633
  - Only the changes to files and code we have.
- bitcoin/bitcoin#10806
- bitcoin/bitcoin#10958
- bitcoin/bitcoin#12451
- bitcoin/bitcoin#13925
- bitcoin/bitcoin#15270

This upgrades LevelDB in the exact same commit progression as upstream, up to January 2019.
2020-09-25 16:16:37 +00:00
Dimitris Apostolou 67ffb189b7
Fix typos 2020-09-25 10:49:57 +03:00
Homu 49fc20aced Auto merge of #4720 - ebfull:fix-perf-regressions, r=str4d
Address some performance regressions

@str4d identified and fixed some performance regressions in our elliptic curve and proving crates, and we also changed to complete formulas in `bls12_381` to get some better performance in some cases. The result:

```
v3.1.0
"runningtime": 0.59883

before upgrading rust:
"runningtime": 0.823932

after upgrading rust:
"runningtime": 0.83004

after upgrading rust packages:
"runningtime": 0.763065
```

which gets us some of the way back to where we were.
2020-09-23 01:36:00 +00:00
Dimitris Tsapakidis b892fe13f8 Fixed multiple typos
A few "a->an" and "an->a".
"Shows, if the supplied default SOCKS5 proxy" -> "Shows if the supplied default SOCKS5 proxy". Change made on 3 occurrences.
"without fully understanding the ramification of a command" -> "without fully understanding the ramifications of a command".
Removed duplicate words such as "the the".

Zcash: Only the changes to files and code that we have.
2020-09-23 00:07:48 +01:00
Sean Bowe 03ce24013a
Update Rust to 1.44.1. 2020-09-22 15:06:01 -06:00
Homu c9672a49c2 Auto merge of #4696 - str4d:3640-z_sendmany-any-taddr, r=str4d
wallet: Add ANY_TADDR special string to z_sendmany

When using this special string as the from address, non-coinbase UTXOs
from any transparent addresses within the wallet will be used to fund the
transaction. Change outputs will be sent to a new transparent address,
as with any other spend of transparent funds.

Closes zcash/zcash#3640.
2020-09-22 18:09:58 +00:00
Jack Grigg e33f530a65 wallet: Ignore coinbase UTXOs with z_sendmany ANY_TADDR
Coinbase UTXOs from multiple transparent addresses may be shielded
together using z_shieldcoinbase.
2020-09-22 17:45:09 +01:00