Without this fix we will get:
AttributeError: 'SpendDescriptionV5' object has no attribute 'anchor'
On certain V5 transactions rehash / calc_sha256, for example:
ca6abd8ef7d6ef158a4a35ea2c2c0cf122f2f664a88f8fa5b6fd79e48c5bed59
We must not lock on cs_main during wallet synchronization, to avoid
locking side channels between the wallet and the networking stack.
However, the wallet code that is called on every block to update tree
states etc. was also checking whether it should run Sprout-to-Sapling
migrations, and that was being gated on IBD completion (to avoid stale
transactions being generated). IsInitialBlockDownload locks on cs_main
before IBD completion :)
We now rely on the fact that IBD does not end until the chain tip is
within nMaxTipAge of the current time, so we can check that as a proxy.
We were already applying a similar check to avoid hibernation-related
issues; these are now merged.
1758a92ffd Merge #950: ci: Add ppc64le build
c58c4ea470 ci: Add ppc64le build
7973576f6e Merge #662: Add ecmult_gen, ecmult_const and ecmult to benchmark
8f879c2887 Fix array size in bench_ecmult
2fe1b50df1 Add ecmult_gen, ecmult_const and ecmult to benchmark
593e6bad9c Clean up ecmult_bench to make space for more benchmarks
50f3367712 Merge #947: ci: Run PRs on merge result even for i686
a35fdd3478 ci: Run PRs on merge result even for i686
3dc8c072b6 Merge #846: ci: Run ASan/LSan and reorganize sanitizer and Valgrind jobs
02dcea1ad9 ci: Make test iterations configurable and tweak for sanitizer builds
489ff5c20a tests: Treat empty SECP2561_TEST_ITERS as if it was unset
fcfcb97e74 ci: Simplify to use generic wrapper for QEMU, Valgrind, etc
de4157f13a ci: Run ASan/LSan and reorganize sanitizer and Valgrind jobs
399722a63a Merge #941: Clean up git tree
09b3bb8648 Clean up git tree
bf0ac46066 Merge #930: Add ARM32/ARM64 CI
202a030f7d Merge #850: add `secp256k1_ec_pubkey_cmp` method
1e78c18d5b Merge bitcoin-core/secp256k1#940: contrib: Explain explicit header guards
69394879b6 Merge #926: secp256k1.h: clarify that by default arguments must be != NULL
6eceec6d56 add `secp256k1_xonly_pubkey_cmp` method
0d9561ae87 add `secp256k1_ec_pubkey_cmp` method
22a9ea154a contrib: Explain explicit header guards
6c52ae8724 Merge #937: Have ge_set_gej_var, gej_double_var and ge_set_all_gej_var initialize all fields of their outputs.
185a6af227 Merge #925: changed include statements without prefix 'include/'
14c9739a1f tests: Improve secp256k1_ge_set_all_gej_var for some infinity inputs
4a19668c37 tests: Test secp256k1_ge_set_all_gej_var for all infinity inputs
3c90bdda95 change local lib headers to be relative for those pointing at "include/" dir
45b6468d7e Have secp256k1_ge_set_all_gej_var initialize all fields. Previous behaviour would not initialize r->y values in the case where infinity is passed in. Furthermore, the previous behaviour wouldn't initialize anything in the case where all inputs were infinity.
31c0f6de41 Have secp256k1_gej_double_var initialize all fields. Previous behaviour would not initialize r->x and r->y values in the case where infinity is passed in.
dd6c3de322 Have secp256k1_ge_set_gej_var initialize all fields. Previous behaviour would not initialize r->x and r->y values in the case where infinity is passed in.
d0bd2693e3 Merge bitcoin-core/secp256k1#936: Fix gen_context/ASM build on ARM
8bbad7a18e Add asm build to ARM32 CI
7d65ed5214 Add ARM32/ARM64 CI
c8483520c9 Makefile.am: Don't pass a variable twice
2161f31785 Makefile.am: Honor config when building gen_context
99f47c20ec gen_context: Don't use external ASM because it complicates the build
98e0358d29 Merge #933: Avoids a missing brace warning in schnorrsig/tests_impl.h on old compilers
99e2d5be0d Avoids a missing brace warning in schnorrsig/tests_impl.h on old compilers.
34388af6b6 Merge #922: Add mingw32-w64/wine CI build
7012a188e6 Merge #928: Define SECP256K1_BUILD in secp256k1.c directly.
ed5a199bed tests: fopen /dev/urandom in binary mode
ae9e648526 Define SECP256K1_BUILD in secp256k1.c directly.
4dc37bf81b Add mingw32-w64/wine CI build
0881633dfd secp256k1.h: clarify that by default arguments must be != NULL
git-subtree-dir: src/secp256k1
git-subtree-split: 1758a92ffd896af533b142707e9892ea6e15e5db
efad3506a8 Merge #906: Use modified divsteps with initial delta=1/2 for constant-time
cc2c09e3a7 Merge #918: Clean up configuration in gen_context
07067967ee add ECMULT_GEN_PREC_BITS to basic_config.h
a3aa2628c7 gen_context: Don't include basic-config.h
be0609fd54 Add unit tests for edge cases with delta=1/2 variant of divsteps
cd393ce228 Optimization: only do 59 hddivsteps per iteration instead of 62
277b224b6a Use modified divsteps with initial delta=1/2 for constant-time
376ca366db Fix typo in explanation
1e5d50fa93 Merge #889: fix uninitialized read in tests
c083cc6e52 Merge #903: Make argument of fe_normalizes_to_zero{_var} const
6e898534ff Merge #907: changed import to use brackets <> for openssl
4504472269 changed import to use brackets <> for openssl as they are not local to the project
26de4dfeb1 Merge #831: Safegcd inverses, drop Jacobi symbols, remove libgmp
23c3fb629b Make argument of fe_normalizes_to_zero{_var} const
24ad04fc06 Make scalar_inverse{,_var} benchmark scale with SECP256K1_BENCH_ITERS
ebc1af700f Optimization: track f,g limb count and pass to new variable-time update_fg_var
b306935ac1 Optimization: use formulas instead of lookup tables for cancelling g bits
9164a1b658 Optimization: special-case zero modulus limbs in modinv64
1f233b3fa0 Remove num/gmp support
20448b8d09 Remove unused Jacobi symbol support
5437e7bdfb Remove unused scalar_sqr
aa9cc52180 Improve field/scalar inverse tests
1e0e885c8a Make field/scalar code use the new modinv modules for inverses
436281afdc Move secp256k1_fe_inverse{_var} to per-impl files
aa404d53be Move secp256k1_scalar_{inverse{_var},is_even} to per-impl files
08d54964e5 Improve bounds checks in modinv modules
151aac00d3 Add tests for modinv modules
d8a92fcc4c Add extensive comments on the safegcd algorithm and implementation
8e415acba2 Add safegcd based modular inverse modules
de0a643c3d Add secp256k1_ctz{32,64}_var functions
4c3ba88c3a Merge #901: ci: Switch all Linux builds to Debian and more improvements
9361f360bb ci: Select number of parallel make jobs depending on CI environment
28eccdf806 ci: Split output of logs into multiple sections
c7f754fe4d ci: Run PRs on merge result instead of on the source branch
b994a8be3c ci: Print information about binaries using "file"
f24e122d13 ci: Switch all Linux builds to Debian
ebdba03cb5 Merge #891: build: Add workaround for automake 1.13 and older
3a8b47bc6d Merge #894: ctime_test: move context randomization test to the end
7d3497cdc4 ctime_test: move context randomization test to the end
99a1cfec17 print warnings for conditional-uninitialized
3d2cf6c5bd initialize variable in tests
f329bba244 build: Add workaround for automake 1.13 and older
24d1656c32 Merge #882: Use bit ops instead of int mult for constant-time logic in gej_add_ge
e491d06b98 Use bit ops instead of int mult for constant-time logic in gej_add_ge
f8c0b57e6b Merge #864: Add support for Cirrus CI
cc2a5451dc ci: Refactor Nix shell files
2480e55c8f ci: Remove support for Travis CI
2b359f1c1d ci: Enable simple cache for brewing valgrind on macOS
8c02e465c5 ci: Add support for Cirrus CI
659d0d4798 Merge #880: Add parens around ROUND_TO_ALIGN's parameter.
b6f649889a Add parens around ROUND_TO_ALIGN's parameter. This makes the macro robust against a hypothetical ROUND_TO_ALIGN(foo ? sizeA : size B) invocation.
git-subtree-dir: src/secp256k1
git-subtree-split: efad3506a8937162e8010f5839fdf3771dfcf516
As of zcash/librustzcash#633, `SaplingDomain::IncomingViewingKey` now
allocates memory internally, and this memory persists as long as the
`BatchRunner` is alive. Now that we have decoupled the measurement of
heap usage for batch tasks from their internals, we can add bounds to
all of the generic parameters of `Batch` to enable correctly measuring
their actual heap usage.
Ported from zcash/librustzcash@913aa0a988.
This enables the heap usage measurements to be conditionally enabled by
the `BatchRunner` user. Importantly, when heap usage measurements are
not enabled, the `DynamicUsage` bound on `Batch` is not required.
This refactor also fixes a bug in the prior implementation. We were
counting the heap usage of a task when it started to run, but the item
may have been in the `rayon` work-stealing queues for a non-negligible
period before then. We now count the heap usage immediately before
spawning the task into the `rayon` thread pool.
Ported from zcash/librustzcash@c98f04330d.
Previously, `finalorchardroot` and `orchard.anchor` were serialized differently
in the RPC API, which made it difficult to correlate them. This changes the
serialization for `finalorchardroot` to match `orchard.anchor`.
Previously we were sending an `Option<DecryptedNote>` from each `Batch`
back to its parent `BatchRunner`. However, this requires allocating
sufficient space in the channel to handle the case where every output
can be decrypted. In general this will not be the case, and we can
instead signal "nothing decrypted" by just dropping the channel sender.
This reduces the post-batch-scanning memory usage of `BatchRunner` from
being linear in the number of on-chain outputs, to being linear in the
number of outputs for the wallet.
ThreadNotifyWallets was created in order to decouple wallet scanning of
transactions from the main chain updates, avoiding timing leaks to
network peers. To further ensure that not even lock contention could be
used to extract timing information, ThreadNotifyWallets collects update
information from the main chain on integer second boundaries.
In general this means that the wallet is processing the last second's
worth of blocks between each synchronization point. However, when there
are sequences of blocks that are costly for the wallet to scan, it may
take the wallet longer than a second to process a second's worth of
blocks connected to the main chain. This means that ThreadNotifyWallets
needs to wait until the next integer second boundary before collecting
the next set of updates, which means it will be processing at least two
seconds' worth of blocks. For extended periods where the chain contains
many outputs, the wallet will get progressively further behind the main
chain.
At the time that ThreadNotifyWallets was created, the above behaviour
was fine, because while the wallet scanning process consumed a lot of
CPU time, it did not consume much memory (as blocks are stored on disk).
However, we recently added batch scanning, which requires allocating
memory for each output that is being scanned. For a wallet with a
growing gap between its scanned-to height and the chain tip height, the
size of the necessary allocation will grow with each integer second
boundary crossed, until the node reaches OOM.
The solution is to implement backpressure: if we reach a memory limit
before we've finished adding blocks to the batch scanners, then we start
consuming the results from the batch scanner to free up memory space for
subsequent batches.
This commit implements the logic necessary to interleave batch creation
and batch result consumption. It does not apply any batch memory limits,
and as such should be effectively a no-op refactor.
Co-authored-by: Daira Hopwood <daira@jacaranda.org>
We were (most likely) copying potentially large structures in places
where we only required references to the data, for the same reason as
the bug fixed in zcash/zcash#6169: in most cases, `auto` does not infer
a reference, causing assignments to `auto` variables to invoke the copy
constructor.
Co-authored-by: Daira Hopwood <daira@jacaranda.org>
C++ `auto`, as opposed to `auto&`, does not infer reference types (except when it does:
https://blog.petrzemek.net/2017/12/08/when-auto-seemingly-deduces-a-reference-in-cpp/
but that doesn't apply here). The inferred type of `orchard_bundle` was therefore
`const OrchardBundle`. This made the assignment a call to `OrchardBundle::operator=`
which calls `orchard_bundle_clone`. A reference assignment is sufficient, because `tx`
is in scope throughout `ContextualCheckTransaction`.
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
Greg Pfeil
Daira Hopwood
Kris Nuttycombe
DeckerSU
str4d
Jack Grigg
sasha
Pieter Wuille
Greg Pfeil
Alex