* add grpc mempool test research
* add a config flag for mempool injection of transactions in test
* Only copy the inner state directory in the send transactions test
* Preload Zcash parameters in some transaction verification tests
* Add a block and transaction Hash method to convert from display order bytes
* Update test coverage docs
* Add debugging output for mempool transaction verification
* Test fetching sent mempool transactions using gRPC
* Add extra log checks to the send transaction test
* Wait for zebrad mempool activation before running gRPC tests
* Update send transaction test for lightwalletd not returning mempool transactions
* Check zebrad logs instead of disabled lightwalletd gRPCs
* Add a debug option that makes RPCs pretend the sync is finished
* Remove an unused debug option
* Remove unused test code and downgrade some logs
* Fix test log checks
* Fix some rustdoc warnings
* Fix a compilation error due to new function arguments
* Make zebrad sync timeouts consistent and remove outdated code
* Document how to increase temporary directory space for tests
* Stop checking for a log that doesn't always happen
* Remove some commented-out code
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
* Update a comment about run time
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
* Add new config to new tests from the `main` branch
* Add transactions to the list, rather than replacing the list with each new block
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
* update zebra to 1.0.0-beta.13
* add changelog
* update the release date
* Update changelog for cryptographic batch PRs
* Add `bitvec` to the changelog
* Update CHANGELOG.md
* Add zebra-network fix and tokio version bump
* Recommend that users update their rust compiler
* Update disk and network usage for recent large blocks
* Fix spacing
* Add disk and network usage update to README
* apply size suggestions
Co-authored-by: teor <teor@riseup.net>
* add known performance issues
* update changelog
* change release date
Co-authored-by: teor <teor@riseup.net>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* Split disk reads from CPU-heavy Sprout interstitial tree cryptography
* Improve anchor validation debugging and error messages
* Work around a test data bug, and save some CPU
* Remove redundant checks for empty shielded data
* Skip generating unused interstitial treestates
* Do disk fetches and quick checks, then CPU-heavy cryptography
* Wrap HistoryTree in an Arc in the state
* Run CPU-intensive chain validation and updates in parallel rayon threads
* Refactor to prepare for parallel tree root calculations
* Run finalized state note commitment tree root updates in parallel rayon threads
* Update finalized state note commitment trees using parallel rayon threads
* Fix a comment typo and add a TODO
* Split sprout treestate fetch into its own function
* Move parallel note commitment trees to zebra-chain
* Re-calculate the tree roots in the same parallel batches
* Do non-finalized note commitment tree updates in parallel threads
* Update comments about note commitment tree rebuilds
* Do post-fork tree updates in parallel threads
* Add a TODO for parallel tree updates in tests
* Fix broken intra-doc links
* Clarify documentation for sprout treestates
* Spawn large database reads into blocking tokio threads
* Concurrently read all blocks, headers, and transactions from disk
* Run zebra-state transaction deserialization on a rayon thread
* Only fetch block headers from the database to answer headers requests
* Move writing to the database to a blocking thread
* Add blocking threads to tests that need them
* Remove mempool downloader requirement for a populated state
And stop populating states that don't need it,
so we can use tokio::time::pause() in those tests.
* Improve debugging for an intermittent test failure
* Try to avoid a race condition populating the mempool in tests
* Initialize the rayon threadpool with a new config for CPU-bound threads
* Verify proofs and signatures on the rayon thread pool
* Only spawn one concurrent batch per verifier, for now
* Allow tower-batch to queue multiple batches
* Fix up a potentially incorrect comment
* Rename some variables for concurrent batches
* Spawn multiple batches concurrently, without any limits
* Simplify batch worker loop using OptionFuture
* Clear pending batches once they finish
* Stop accepting new items when we're at the concurrent batch limit
* Fail queued requests on drop
* Move pending_items and the batch timer into the worker struct
* Add worker fields to batch trace logs
* Run docker tests on PR series
* During full verification, process 20 blocks concurrently
* Remove an outdated comment about yielding to other tasks
* Use a new channel for each batch
* Prefer the batch timer if there are also new batch requests
* Allow other tasks to run after each batch
* Label each batch worker with the verifier's type
* Rename Handle to ErrorHandle, and fix up some docs
* Check batch worker tasks for panics and task termination
* Use tokio's PollSemaphore instead of an outdated Semaphore impl
* Run all verifier cryptography on a blocking thread
Also use a new verifier channel for each batch.
* Make flush and drop behaviour consistent for all verifiers
* Partly fix an incorrect NU5 test
* Switch batch tests to the multi-threaded runtime
* Export all verifier primitive modules from zebra-consensus
* Remove outdated test code in tower-batch
* Use a watch channel to send batch verifier results
* Use spawn_blocking for batch fallback verifiers
* Spawn cryptography batches onto blocking tokio threads
* Use smaller batches for halo2
* Minor tower-batch cleanups
* Fix doc link in zebra-test
* Drop previous permit before acquiring another to avoid a deadlock edge case
* Return the maximum checkpoint height from the chain verifier
* Return the verified block height from the sync downloader
* Track the verified height in the syncer
* Use a lower concurrency limit during full verification
* Get the tip from the state before the first verified block
* Limit the number of submitted download and verify blocks in a batch
* Adjust lookahead limits when transitioning to full verification
* Keep unused extra hashes and submit them to the downloader later
* Remove redundant verified_height and state_tip()
* Split the checkpoint and full verify concurrency configs
* Decrease full verification concurrency to 5 blocks
10 concurrent blocks causes 3 minute stalls on some blocks on my machine.
(And it has about 4x as many cores as a standard machine.)
* cargo +stable fmt --all
* Remove a log that's verbose with smaller lookahead limits
* Apply the full verify concurrency limit to the inbound service
* Add a summary of the config changes to the CHANGELOG
* Increase the default full verify concurrency limit to 30
* Only verify halo2 proofs once per transaction (rather than once per action)
* Update comments on how there is one aggregate Halo2 proof instead of one per Action
Co-authored-by: Marek <mail@marek.onl>
* cargo +stable fmt --all
Co-authored-by: Deirdre Connolly <durumcrustulum@gmail.com>
Co-authored-by: Marek <mail@marek.onl>
* Update Zebra to 1.0.0-beta.12
* Simplify tower-* versioning by updating both to 0.2.27
* Simplify version search and replace by removing an old version from the docs
* Update Cargo.lock
* Add draft changelog as of PR #4693
* Update CHANGELOG to PR #4680
* Improve time logging using humantime
* Only log full seconds, ignore the fractional part
* Move humantime_seconds to tracing::fmt
* Move the progress task to its own module
* Add missing humantime dependency
* Log the network upgrade in progress logs
* Log when Zebra verifies the final checkpoint
* Always activate tokio/tracing feature
And always build tests with all tokio features.
* Refactor tracing-subscriber init to simplify it
* Add the tokio-console feature and dependencies
* Add optional tokio-console support, and log the installed tracing layers at info level
Uses a tracing Registry for tokio-console, and a fmt::Subscriber otherwise.
* Add some TODOs based on tracing-subscriber features
* Fix up some spans
* Add a TODO for fixing a log filter bug in tokio-console mode
* Fix the syntax of links in comments
* Fix a mistake in the docs
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
* Remove unnecessary angle brackets from a link
* Revert the changes for links that serve as references
* Revert "Revert the changes for links that serve as references"
This reverts commit 8b091aa9fa.
* Remove `<` `>` from links that serve as references
This reverts commit 046ef25620.
* Don't use `<` `>` in normal comments
* Don't use `<` `>` for normal comments
* Revert changes for comments starting with `//`
* Fix some warnings produced by `cargo doc`
* Fix some rustdoc warnings
* Fix some warnings
* Refactor some changes
* Fix some rustdoc warnings
* Fix some rustdoc warnings
* Resolve various TODOs
Co-authored-by: teor <teor@riseup.net>
* Fix some unresolved links
* Allow links to private items
* Fix some unresolved links
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
Co-authored-by: teor <teor@riseup.net>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* Fix the syntax of links in comments
* Fix a mistake in the docs
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
* Remove unnecessary angle brackets from a link
* Revert the changes for links that serve as references
* Revert "Revert the changes for links that serve as references"
This reverts commit 8b091aa9fa.
* Remove `<` `>` from links that serve as references
This reverts commit 046ef25620.
* Don't use `<` `>` in normal comments
* Don't use `<` `>` for normal comments
* Revert changes for comments starting with `//`
* Fix some warnings produced by `cargo doc`
* Fix some rustdoc warnings
* Fix some warnings
* Refactor some changes
* Fix some rustdoc warnings
* Fix some rustdoc warnings
* Resolve various TODOs
Co-authored-by: teor <teor@riseup.net>
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
Co-authored-by: teor <teor@riseup.net>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
dependabot[bot]
teor
Conrado Gouvea
Marek
Alfredo Garcia