Commit Graph

16673 Commits

Author SHA1 Message Date
furszy bed3c9ee3e Use references instead of pointers where possible.
Also, rename WitnessNoteIfMine -> WitnessMyNoteIfNecessary
and apply other suggestions from code review

Co-authored-by: Daira Hopwood <daira@jacaranda.org>
2022-07-19 09:59:22 -06:00
furszy 8fb46c794e Rework Sprout and Sapling witnesses increment and cache workflow, so it does not loop over the entire wallet txs map indiscriminately.
From:
```
1) for-each tx in wallet : call CopyPreviousWitnesses.
2) for-each tx in block:
   a) for-each note in tx:
      a1) for-each tx in wallet : call AppendNoteCommitment.
      a2) if note is mine : call WitnessNoteIfMine.
3) for-each tx in wallet : call UpdateWitnessHeights.
```

To:

```
1) for-each tx in block:
   a) gather note commitments in vecComm.
   b) witness note if ours.
2) for-each shield tx in wallet:
   a) copy the previous witness.
   b) append vecComm notes commitment.
   c) Update witness last processed height.
```
2022-07-19 09:52:40 -06:00
furszy 54b55ebae6 Fix missing vector include and vector type definition 2022-07-19 07:22:24 -06:00
Kris Nuttycombe c1a18cab39
Merge pull request #6040 from nuttycom/test/persisted_sprout_chain
Add persistent test data to avoid needing to create new Sprout outputs in RPC tests.
2022-07-18 11:35:57 -06:00
Kris Nuttycombe 035a26b674 Replace setup_clean_chain with cache_behavior in rpc test init. 2022-07-18 09:00:34 -06:00
Kris Nuttycombe 6477c6165b Update getblocktemplate test to use persisted sprout chain 2022-07-18 09:00:34 -06:00
Kris Nuttycombe 91f62f4161 Update finalsaplingroot test to use persisted sprout chains. 2022-07-18 09:00:34 -06:00
Daira Hopwood dc7f3e74fb
Merge pull request #6084 from str4d/fix-connectblock-benchmarks
bench: Fix ConnectBlock large block benchmarks
2022-07-18 15:27:57 +01:00
Jack Grigg bd17ae588a bench: Fix ConnectBlock large block benchmarks
`zcbenchmark` internally loops within the same process to run the same
benchmark multiple times. This meant it was being caught up in the
global validity cache, giving faster results for every iteration except
the first. This was not noticeable for the historic slow transparent
block, but became noticeable once we started caching Sapling and Orchard
bundle validity in zcash/zcash#6073.

As the intention of the benchmarks is to measure the worst case where
the block in question has not had any of its transactions observed
before (as is the case for IBD), we now disable cache storage if calling
`ConnectBlock` from a slow block benchmark.
2022-07-16 14:04:03 +00:00
str4d 3213527141
Merge pull request #6079 from str4d/fastrandom-backports
Use FastRandomContext for all tests
2022-07-16 11:09:47 +01:00
Pieter Wuille 101d19fe17 scripted-diff: Use new naming style for insecure_rand* functions
-BEGIN VERIFY SCRIPT-
sed -i 's/\<insecure_randbits(/InsecureRandBits(/g' src/test/*.cpp src/test/*.h src/wallet/test/*.cpp
sed -i 's/\<insecure_randbool(/InsecureRandBool(/g' src/test/*.cpp src/test/*.h src/gtest/*.cpp src/wallet/test/*.cpp
sed -i 's/\<insecure_randrange(/InsecureRandRange(/g' src/test/*.cpp src/test/*.h src/gtest/*.cpp src/wallet/test/*.cpp
sed -i 's/\<insecure_randbytes(/InsecureRandBytes(/g' src/test/*.cpp src/test/*.h src/wallet/test/*.cpp
sed -i 's/\<insecure_rand256(/InsecureRand256(/g' src/test/*.cpp src/test/*.h src/wallet/test/*.cpp
sed -i 's/\<insecure_rand(/InsecureRand32(/g' src/test/*.cpp src/test/*.h src/wallet/test/*.cpp
sed -i 's/\<seed_insecure_rand(/SeedInsecureRand(/g' src/test/*.cpp src/test/*.h src/wallet/test/*.cpp
-END VERIFY SCRIPT-

(cherry picked from commit bitcoin/bitcoin@e945848582)
2022-07-15 22:10:23 +00:00
Pieter Wuille 3e6b2edc74 scripted-diff: Use randbits/bool instead of randrange where possible
-BEGIN VERIFY SCRIPT-
sed -i 's/insecure_randbits(1)/insecure_randbool()/g' src/test/*_tests.cpp
sed -i 's/insecure_randrange(2)/insecure_randbool()/g' src/test/*_tests.cpp
sed -i 's/insecure_randrange(2)/insecure_randbool()/g' src/gtest/test_*.cpp
sed -i 's/insecure_randrange(4)/insecure_randbits(2)/g' src/test/*_tests.cpp
sed -i 's/insecure_randrange(32)/insecure_randbits(5)/g' src/test/*_tests.cpp
sed -i 's/insecure_randrange(256)/insecure_randbits(8)/g' src/test/*_tests.cpp
-END VERIFY SCRIPT-

Zcash: Added gtests to script.

(cherry picked from commit bitcoin/bitcoin@2fcd9cc86b)
2022-07-15 22:10:23 +00:00
Pieter Wuille 008fb745c5 Use randbits instead of ad-hoc emulation in prevector tests
(cherry picked from commit bitcoin/bitcoin@2ada678521)
2022-07-15 22:10:23 +00:00
Pieter Wuille 722d1c4c64 Replace rand() & ((1 << N) - 1) with randbits(N)
Zcash: Excluding changes to tests we don't have.

(cherry picked from commit bitcoin/bitcoin@5f0b04eedc)
2022-07-15 22:10:23 +00:00
Pieter Wuille 05c60b2b7e Replace more rand() % NUM by randranges
Zcash: Excluding changes to tests we don't have.

(cherry picked from commit bitcoin/bitcoin@3ecabae363)
2022-07-15 22:10:23 +00:00
Pieter Wuille 5a1c450496 scripted-diff: use insecure_rand256/randrange more
-BEGIN VERIFY SCRIPT-
sed -i "s/\<GetRandHash(/insecure_rand256(/" src/test/*_tests.cpp
sed -i "s/\<GetRand(/insecure_randrange(/" src/test/*_tests.cpp src/test/test_bitcoin.cpp
sed -i 's/\<insecure_rand() % \([0-9]\+\)/insecure_randrange(\1)/g' src/test/*_tests.cpp
-END VERIFY SCRIPT-

Zcash: Excludes changes to files we don't have.

(cherry picked from commit bitcoin/bitcoin@efee1db21a)
2022-07-15 22:10:23 +00:00
Pieter Wuille eab53622f4 Add various insecure_rand wrappers for tests
(cherry picked from commit bitcoin/bitcoin@1119927df0)
2022-07-15 22:10:23 +00:00
Pieter Wuille f7a23218d1 Merge test_random.h into test_bitcoin.h
Zcash: Excludes changes to files we don't have, and removes the
duplicate definitions in `src/gtest/utils.h`.

(cherry picked from commit bitcoin/bitcoin@124d13a58c)
2022-07-15 22:10:23 +00:00
Pieter Wuille 0ef7ce2c61 scripted-diff: Rename cuckoo tests' local rand context
-BEGIN VERIFY SCRIPT-
sed -i 's/insecure_rand/local_rand_ctx/g' src/test/cuckoocache_tests.cpp
-END VERIFY SCRIPT-

(cherry picked from commit bitcoin/bitcoin@90620d66c9)
2022-07-15 22:10:23 +00:00
Pieter Wuille c9a9691a2f Add FastRandomContext::rand256() and ::randbytes()
FastRandomContext now provides all functionality that the real Rand* functions
provide.

(cherry picked from commit bitcoin/bitcoin@37e864eb9f)
2022-07-15 22:10:23 +00:00
Kris Nuttycombe 420f8dfe38
Merge pull request #6078 from nuttycom/bug/osx_missing_chrono_header
Add missing <chrono> header to util/time.h
2022-07-15 14:05:27 -06:00
Kris Nuttycombe 33d2dce183 Add missing <chrono> header to util/time.h 2022-07-15 12:54:37 -06:00
Kris Nuttycombe 95bbfd09aa Update feature_zip239 test to use persisted sprout chains. 2022-07-15 08:23:57 -06:00
Kris Nuttycombe b2fc56c24a Update sprout_sapling_migration test to use persisted sprout chains. 2022-07-15 08:23:57 -06:00
Kris Nuttycombe afea5c386d Add RPC test initialization using the persisted Sprout chains. 2022-07-15 08:23:57 -06:00
Kris Nuttycombe 532017e74e Remove the temporary test that was used for setup of the cached Sprout fixtures. 2022-07-15 08:23:57 -06:00
Kris Nuttycombe 5d44ce318e Add persistent Sprout test data.
This is in preparation for removing the ability to generate
Sprout outputs from z_shieldcoinbase. Once that is complete,
we will no longer be able to use `z_shieldcoinbase` for test
setup for uses of Sprout funds; instead, the persisted blockchain
state created in this commit will be used for tests that require
the use of Sprout funds.

This brings the test framework into line with how Sprout funds
are now used on mainnet and testnet; existing Sprout funds may
be spent, and Sprout change may be created, but no funds may
be transfered into the Sprout pool, since the activation of
ZIP 211.
2022-07-15 08:23:57 -06:00
Kris Nuttycombe 40df3d402e
Merge pull request #6042 from nuttycom/feature/clock_capability
Add a clock for testing with an offset from the system clock.
2022-07-15 08:23:23 -06:00
Kris Nuttycombe 0a4f3a3df0 Apply suggestions from code review
Co-authored-by: str4d <thestr4d@gmail.com>
Co-authored-by: Daira Hopwood <daira@jacaranda.org>
2022-07-14 19:59:52 -06:00
str4d 0260b7e40b
Merge pull request #6073 from str4d/cache-proof-and-sig-validity
Cache Sapling and Orchard bundle validation
2022-07-15 01:34:27 +01:00
str4d 4689c59f3e
Improve bundlecache documentation
Co-authored-by: Daira Hopwood <daira@jacaranda.org>
2022-07-15 00:22:31 +01:00
Kris Nuttycombe e7922af588 Add a clock for testing with an offset from the system clock.
This change improves clock management for zcashd by ensuring
that all clock methods (obtaining seconds, milliseconds, and
microseconds since the epoch) agree under testing conditions
using `-mocktime`, and also adds a feature that allows tests
to specify an offset to the system clock; this is useful to
allow comprehensive testing of the "timejacking attack mitigation"
consensus rules.
2022-07-14 16:33:54 -06:00
Jack Grigg ebb37770a8 Move explicit instantiations for `BundleValidityCache` into `zcash/cache.cpp`
Also cleans them up a bit using the typedef.
2022-07-14 19:35:50 +00:00
str4d e75213ff65
Merge pull request #6069 from str4d/prune-witnesses-for-definitely-spent-notes
wallet: Prune witnesses for notes spent more than 100 blocks ago
2022-07-14 20:19:49 +01:00
Kris Nuttycombe aa6dbba5d2
Merge pull request #6074 from nuttycom/NO_zmq
Update documentation to refer to the actual two-hyphen `--nozmq` flag
2022-07-14 13:13:03 -06:00
Kris Nuttycombe d1dabd5a04
Merge pull request #6019 from zcash/add_docker_tromp_miner
Add environment variables for Equihash solver and number of threads to entrypoint.sh
2022-07-14 11:38:18 -06:00
Kris Nuttycombe 054334bbb4
Merge pull request #6029 from turrisxyz/Pinned-Dependencies-GitHub
chore: Set permissions for GitHub actions
2022-07-14 11:02:36 -06:00
Jack Grigg a5d2b8cd0c wallet: Improve documentation of `SproutNotData` and `SaplingNoteData` 2022-07-14 16:11:10 +00:00
Jack Grigg 847ce5375f wallet: Throw error if `ReadBlockFromDisk` fails 2022-07-14 16:06:13 +00:00
zancas a4a0fbb3c0 match the actual two hyphen flag 2022-07-14 07:44:17 -06:00
Kris Nuttycombe 4be354aad5
Merge pull request #6067 from daira/dont-rename-main-thread
Don't rename main thread
2022-07-14 07:37:23 -06:00
Kris Nuttycombe 51cc19d492
Merge pull request #5996 from adityapk00/gettx_version
Add tx version
2022-07-14 07:13:07 -06:00
Jack Grigg d5a271c7a4 Add bundle kind to `BundleValidityCache` initialization log message 2022-07-13 22:41:56 +00:00
Jack Grigg 087c85ec03 Cache Sapling and Orchard bundle validation
This adds two new CuckooCaches in validation, each caching whether all
of a transaction bundle's proofs and signatures were valid.

Bundles which match the validation cache never have proofs or signatures
added to the batch validators. For blocks where all transactions have
been previously observed in the mempool, the final validation of the
batches should be a no-op.

Part of zcash/zcash#6049.
2022-07-13 15:49:37 +00:00
Kris Nuttycombe 882935e35e
Merge pull request #5065 from str4d/merkle-backports
Bitcoin 0.12 Merkle tree PRs
2022-07-12 16:54:28 -06:00
Jack Grigg 12c3cfd339 wallet: Make `{Increment, Decrement}NoteWitnesses`-internal helpers static 2022-07-12 21:50:29 +00:00
Jack Grigg 82361aebc3 wallet: Prune witnesses for notes spent more than 100 blocks ago
Once a note has been spent for at least 100 blocks, the wallet will not
see that spend be rolled back, as the node itself will reject rollbacks
of that magnitude. We can therefore stop tracking witnesses for these
notes, improving the performance of block scanning.

Part of zcash/zcash#6052.

Co-authored-by: Daira Hopwood <daira@jacaranda.org>
Co-authored-by: Kris Nuttycombe <kris@nutty.land>
2022-07-12 21:50:29 +00:00
Kris Nuttycombe cc9449f905
Merge pull request #5066 from str4d/cuckoocache
Replace SigCache implementation with CuckooCache
2022-07-12 15:41:05 -06:00
str4d 4ae67b01f6
Merge pull request #6070 from str4d/wallet-useful-atmp-error
Return useful error message on ATMP failure
2022-07-12 02:59:31 +01:00
instagibbs 52ae7dbb90 Return useful error message on ATMP failure
Zcash: Excludes changes to QT wallet, includes our own uses of
CommitTransaction.

(cherry picked from commit bitcoin/bitcoin@169bdabe14)
2022-07-11 23:34:48 +00:00