* Patch equihash to use the solver branch
* Add an internal-miner feature and set up its dependencies
* Remove 'Experimental' from mining RPC docs
* Fix a nightly clippy::question_mark lint
* Move a byte array utility function to zebra-chain
* fixup! Add an internal-miner feature and set up its dependencies
* Add an equihash::Solution::solve() method with difficulty checks
* Check solution is valid before returning it
* Add a TODO to check for peers before mining
* Move config validation into GetBlockTemplateRpcImpl::new()
* fixup! fixup! Add an internal-miner feature and set up its dependencies
* Use the same generic constraints for GetBlockTemplateRpcImpl struct and impls
* Start adding an internal miner component
* Add the miner task to the start command
* Add basic miner code
* Split out a method to mine one block
* Spawn to a blocking thread
* Wait until a valid template is available
* Handle shutdown
* Run mining on low priority threads
* Ignore some invalid solutions
* Use a difference nonce for each solver thread
* Update TODOs
* Change the patch into a renamed dependency to simplify crate releases
* Clean up instrumentation and TODOs
* Make RPC instances cloneable and clean up generics
* Make LongPollId Copy so it's easier to use
* Add API to restart mining if there's a new block template
* Actually restart mining if there's a new block template
* Tidy instrumentation
* fixup! Move config validation into GetBlockTemplateRpcImpl::new()
* fixup! Make RPC instances cloneable and clean up generics
* Run the template generator and one miner concurrently
* Reduce logging
* Fix a bug in getblocktemplate RPC tip change detection
* Work around some watch channel change bugs
* Rate-limit template changes in the receiver
* Run one mining solver per available core
* Use updated C code with double-free protection
* Update to the latest solver branch
* Return and submit all valid solutions
* Document what INPUT_LENGTH means
* Fix watch channel change detection
* Don't return early when a mining task fails
* Spawn async miner tasks to avoid cooperative blocking, deadlocks, and improve shutdown responsiveness
* Make existing parallelism docs and configs consistent
* Add a mining parallelism config
* Use the minimum of the configured or available threads for mining
* Ignore optional feature fields in tests
* Downgrade some frequent logs to debug
* Document new zebrad features and tasks
* Describe the internal-miner feature in the CHANGELOG
* Update dependency to de-duplicate equihash solutions
* Use futures::StreamExt instead of TryStreamExt
* Fix a panic message typo
* impl TryFrom<zcash_primitives::BlockHeight> for Height
* Add type-safe read and write database methods
* Only allow typed access to the scanner DB
* Update docs
* Implement a common method as a trait
* Fix imports
* Tidy state imports
* Activate tracing logging macros in the whole scanner crate
* Fix dead code warnings
* Use a more sensible export order
* Remove a 1.72 lint exception now 1.74 is stable
* Switch history trees over to TypedColumnFamily, and remove redundant code
* Add typed batch creation methods, and switch history trees to them
* Convert ValueBalance to typed column families
* Make the APIs compatible after a merge
* Use `ZebraDb` instead of `DiskDb` where needed
---------
Co-authored-by: Marek <mail@marek.onl>
* impl TryFrom<zcash_primitives::BlockHeight> for Height
* Add type-safe read and write database methods
* Only allow typed access to the scanner DB
* Update docs
* Implement a common method as a trait
* Fix imports
* Tidy state imports
* Activate tracing logging macros in the whole scanner crate
* Fix dead code warnings
* Add a scanning results reader
* Simplify intro & titles in `zebra-utils/README.md`
* Add a tutorial for the scanning results reader
* Reformat `zebra-utils/Cargo.toml`
Co-authored-by: teor <teor@riseup.net>
* Update `zebra-utils/README.md`
Co-authored-by: teor <teor@riseup.net>
* standard grouping
Co-authored-by: teor <teor@riseup.net>
* Add a comment on a first empty memo
Co-authored-by: teor <teor@riseup.net>
* Use `exactly_one()` instead of `next()`
* Simplify various type conversions
---------
Co-authored-by: teor <teor@riseup.net>
* Refactor test data into functions
* Add a typed snapshot test for scanner storage
* Use standard hex serialization with SaplingScannedResult
* Simplify transaction::Hash hex serialization
* Sort HashMaps before snapshotting
* Add typed snapshot data
* Derive & impl helper traits from `std`
* Create `compact_to_v4` fn
* Create `fake_block` fn
* Refactor existing tests to use the new functions
* Cosmetics
* Refactor docs
* Put `Default` behind `cfg_attr(test)`
Rationale
---------
We avoid implementing `Default` on consensus-critical types because it's
easy to miss an incorrect use in a review. It's easy to hide a
`default()` in a call like `unwrap_or_default()` or even more subtle
methods.
---------
Co-authored-by: teor <teor@riseup.net>
* Wrap `zcash_client_backend::scanning::scan_block`
* Use the new `scan_block` fn
* Use full path for `zcash::primitives::Network`
* Add docs for `scan_block`
* Impl `From` for networks in `zcash_primitives`
* Update zebra-scan/src/tests.rs
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
* Move code from `tests.rs` to `scan.rs`
---------
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
* Refactor return type of poll_discover()
* Simplify poll_ready() by removing preselected peers
* Fix peer set readiness check
* Pass task context correctly to background tasks
* Make poll_discover() return Pending
* Make poll_inventory() return Pending
* Make poll_unready() return Poll::Pending
* Simplify with futures::ready!() and ?
* When there are no peers, wake on newly ready peers, or new peers, in that order
* Preserve the original waker when there are no unready peers
* Fix polling docs and remove unnecessary code
* Make sure we're ignoring Poll::Pending not Result
* Make panic checking method names clearer
* Fix connection Client task wakeups and error handling
* Cleanup connection panic handling and add wakeup docs
* Fix connection client task wakeups to prevent hangs
* Simplify error and pending handling
* Clarify inventory set behaviour
* Define peer set poll_* methods so they return Ok if they do something
* Clarify documentation
Co-authored-by: Arya <aryasolhi@gmail.com>
* Fix test that depended on preselected peers
* Check ready peers for errors before sending requests to them
* Fix a hanging test by not waiting for irrelevant actions
* Only remove cancel handles when they are required
* fix incorrect panic on termination setting
* Clarify method comments
Co-authored-by: Arya <aryasolhi@gmail.com>
---------
Co-authored-by: Arya <aryasolhi@gmail.com>
* Stop using `SemanticallyVerifiedBlockWithTrees`
* Apply suggestions from code review
Co-authored-by: teor <teor@riseup.net>
* Doc that default treestate corresponds to genesis
* Doc the diff between default values of tree roots
---------
Co-authored-by: teor <teor@riseup.net>
* Simple replacements of doc.zebra.zfnd.org with docs.rs
* Manual fixes for specific main/internal/external docs
* Point developer docs to doc-internal.zebra.zfnd.org
* fastmod --glob '\!.git' -- doc.zebra.zfnd.org/zebrad docs.rs/zebrad/latest/zebrad
* Manually remove any remaining doc.zfnd.zebra.org links
* Remove the external docs job
* Add changelog entry and fix links
* Fix links that were broken before this PR
* Fix the combining of Sapling subtrees
This commit fixes a bug due to which the function `sapling_subtrees`
used to always include all subtrees from the non-finalized state without
respecting the given limit.
* Fix the combining of Orchard subtrees
This commit fixes a bug due to which the function `orchard_subtrees`
used to always include all subtrees from the non-finalized state without
respecting the given limit.
* Add additional checks when retrieving subtrees
* Allow raw subtree insertions into `Chain` in tests
* Derive `Default` for `Chain`
* Derive `Default` for note commitment tree nodes
* Use `pub(super)` for `DiskWriteBatch`
Set the visibility of `DiskWriteBatch` to `pub(super)`.
* Add tests for retrieving subtrees
* Use `default()` for subtree roots in tests
This change is unrelated to the PR.
* Refactor docs for inserting subtrees
This change is unrelated to the PR.
* Handle negative and zero getnetworksolsps arguments correctly
* Simplify the solsps loop structure
* Simplify loop by avoiding manual iteration and peeking
* Avoid division by zero
* Use min and max times rather than first and last times
* Refactor block iterators so they are more efficient
* Make finding chains easier
* Simplify block iteration code
* Remove implemented TODO comments
* Simplify internal iterator state
* Implement iteration by any chain item
* Iterate block headers rather than full blocks
* Ignore code that is (sometimes) dead
* Fix a dead code warning
* Add a no blocks in state error constant
* Check result values in the RPC test
* Fix invalid calculation handling
* Avoid manual handling of previous sapling trees by using iterator windows instead
* Avoid manual sapling subtree index handling by comparing prev and current subtree indexes instead
* Simplify adding notes by using the exact number of remaining notes
* Simplify by skipping the first block, because it can't complete a subtree
* Re-use existing tree update code
* Apply the sapling changes to orchard subtree updates
* add a reverse database column family iterator function
* Make skipping the lowest tree independent of iteration order
* Move new subtree checks into the iterator, rename to end_height
* Split subtree calculation into a new method
* Split the calculate and write methods
* Quickly check the first subtree before running the full upgrade
* Do the quick checks every time Zebra runs, and refactor slow check error handling
* Do quick checks for orchard as well
* Make orchard tree upgrade match sapling upgrade code
* Upgrade subtrees in reverse height order
* Bump the database patch version so the upgrade runs again
* Reset previous subtree upgrade data before doing this one
* Add extra checks to subtree calculation to diagnose errors
* Use correct heights for subtrees completed at the end of a block
* Add even more checks to diagnose issues
* Instrument upgrade methods to improve diagnostics
* Prevent modification of re-used trees
* Debug with subtree positions as well
* Fix an off-by-one error with completed subtrees
* Fix typos and confusing comments
Co-authored-by: Marek <mail@marek.onl>
* Fix mistaken previous tree handling and end tree comments
* Remove unnecessary subtraction in remaining leaves calc
* Log heights when assertions fail
* Fix new subtree detection filter
* Move new subtree check into a method, cleanup unused code
* Remove redundant assertions
* Wait for subtree upgrade before testing RPCs
* Fix subtree search in quick check
* Temporarily upgrade subtrees in forward height order
* Clarify some comments
* Fix missing test imports
* Fix subtree logging
* Add a comment about a potential hang with future upgrades
* Fix zebrad var ownership
* Log more info when add_subtrees.rs fails
* cargo fmt --all
* Fix unrelated clippy::unnecessary_unwrap
* cargo clippy --fix --all-features --all-targets; cargo fmt --all
* Stop the quick check depending on tree de-duplication
* Refactor waiting for the upgrade into functions
* Wait for state upgrades whenever the cached state is updated
* Wait for the testnet upgrade in the right place
* Fix unused variable
* Fix a subtree detection bug and comments
* Remove an early reference to reverse direction
* Stop skipping subtrees completed at the end of blocks
* Actually fix new subtree code
---------
Co-authored-by: Marek <mail@marek.onl>
* Copy the add_subtrees upgrade from the original branch
* Copy the database write changes in shielded.rs from the original branch
* Copy the tree API changes from the original branch
* Simplify subtree APIs to avoid exposing frontiers
* Fix a dead code warning by re-using existing methods
* Use mpsc::Receiver<CancelFormatChange> in the subtree upgrade
* Run the subtree upgrade on startup
* Bump the database format version to 25.2.0
* Fix a confusing 'upgrade complete' log
* Clarify some comments and error messages
* Simplify prev_tree unwrap to avoid an (impossible?) concurrency bug
* Use separate subtree writing functions
* Use common note commitment list code
* Fix subtree completion condition and add asserts
* Simplify subtree API and avoid exposing Address
* Fix API compatibility when Arcs are removed
* Log when each subtree is added
* If a format change is cancelled, don't mark the database as upgraded or do format checks
* Log subtree progress about once every two minutes
* Adds a state validity check for subtrees upgrade
* Orchard is faster, decrease log interval
* Clarify subtree index docs
* Move a log to the correct location
* Refactor subtree upgrade to remove duplicate inverted loop conditions
* updates subtree state validity check
* Add a subtree format check when there is no upgrade
* Fix an off-by-one error with the final subtree check
* Use error-level logs for database format checks
* Skip format checks in tests that create invalid formats
* fix state validity test
* Add a concurrency comment to subtree by height methods
* Add individual subtree state methods: reverts removing these methods in an earlier PR
* fastmod "subtrees_by_index" "subtree_list_by_index_for_rpc"
---------
Co-authored-by: arya2 <aryasolhi@gmail.com>
* Make NoteCommitmentSubtreeIndex compatible with serde-based RPCs
* Add a stub for z_getsubtreesbyindex
* Define a GetSubtrees RPC response type
* Reject invalid shielded pool names
* Make limit optional
* Define state request and response types for subtrees
* Implement FromDisk for NoteCommitmentSubtreeIndex and add a round-trip test
* Make subtrees compatible with round-trip proptests
* Add finalized state subtree list methods and delete unused methods
* Remove Arc from subtrees in zebra-chain
* Remove Arc from subtrees in zebra-state and use BTreeMap
* Implement subtree list lookups in the non-finalized state and delete unused methods
* Implement consistent concurrent subtree read requests
* Implement ToHex for sapling::Node
* Implement ToHex for orchard::Node
* Implement z_get_subtrees_by_index RPC
* Check for the start_index from the non-finalized state
* Remove an unused mut
* Fix missing doc links
* Fix RPC comments
* Temporarily remove the z_get_subtrees_by_index RPC method
* zebra-chain changes from the subtree-boundaries branch
```sh
git checkout -b subtree-boundaries-zebra-chain main
git checkout origin/subtree-boundaries zebra-chain
git commit
```
* Temporarily populate new subtree fields with None - for revert
This temporary commit needs to be reverted in the next PR.
* Applies suggestions from code review
* removes from_repr_unchecked methods
* simplifies loop
* adds subtrees to zebra-state
* uses split_at, from_repr, & updates state-db-upgrades.md
* Update book/src/dev/state-db-upgrades.md
Co-authored-by: teor <teor@riseup.net>
* renames partial_subtree to subtree_data
* tests that subtree serialization format
* adds raw data format serialization round-trip test
* decrements minor version and skips inserting subtrees in db
---------
Co-authored-by: teor <teor@riseup.net>
* zebra-chain changes from the subtree-boundaries branch
```sh
git checkout -b subtree-boundaries-zebra-chain main
git checkout origin/subtree-boundaries zebra-chain
git commit
```
* Temporarily populate new subtree fields with None - for revert
This temporary commit needs to be reverted in the next PR.
* Applies suggestions from code review
* removes from_repr_unchecked methods
* simplifies loop
---------
Co-authored-by: arya2 <aryasolhi@gmail.com>
* fix clippy and rustfmt after last releases today
* Ignore clippy::arc_with_non_send_sync false positives in proptest code only
* Remove .cargo/config.toml changes that didn't work
---------
Co-authored-by: teor <teor@riseup.net>
* Add support for deleting the trees
* Prune the trees
* Remove `Network` from `DiskWriteBatch`
Removing the `Network` from `DiskWriteBatch` makes it easy to
instantiate `DiskWriteBatch`es in `ZebraDb` that remove individual note
commitment trees.
The `Network` from `DiskWriteBatch` was used only for transparent
addresses, so the refactor isn't large. After removing it from
`DiskWriteBatch`, I passed it as a function argument instead. However,
we should simplify the parameter lists because at least two functions
have more than seven parameters now.
* Support individual tree removal in `ZebraDb`
* Refactor the tree removal task
* Prune old comments
* Remove redundant code
* Batch the removals
* delete ranges before relevant network upgrades
* moves prev_tree inits
* add iterator methods for reading note commitment trees
* Sets up skeleton of sapling pipeline
* Replaces .filter with .take_while
Fills in pipeline
Reuses zs_range_iter instead of repeating that code
Updates logic to stop at initial tip height
* uses std threads
* delete_range excludes end key
* fixes off by one bugs
* Log warning when a send fails
* Removes progress logs
* Log join errors instead of panicking
* Revert: Make the `db` field of `ZebraDb` private
* Move `delete_range_sapling_tree`
* Remove a redundant `else if` branch
Rationale:
The condition `n == 1` for the removed branch is true for a subset of
values of `n` in the preceding condition `n >= 1`.
* Use more specific error messages
* Revert: Remove redundant methods for tree removal
* Suggestions for Deduplicate note commitment trees stored in the finalized state (#7330)
* Add TODOs to some `Height` methods
* Add methods for deleting individual trees
* Refactor the tasks for deleting trees
---------
Co-authored-by: arya2 <aryasolhi@gmail.com>
* Pass ZebraDB to batch preparation
* Dedup the insertion of Sapling trees into database
* Dedup the insertion of Orchard trees into database
* Update snapshots
* Rename batch preparation of trees
* Simplify the naming of note commitment trees
* Correctly retrieve Sapling trees from fin state
* Correctly retrieve Orchard trees from fin state
* Simplify the naming of methods for Sprout trees
* Simplify the naming of methods for Sapling trees
* Simplify the naming of methods for Orchard trees
* Reduce disk reads by caching trees. (#7276)
* Bump the state minor version
* Reset the state patch version
* Simplify the preparation of genesis trees
* Store the roots of the trees of the genesis block
* Add the genesis roots to snapshots
* fix(test): Don't include shielded data in genesis blocks (#7302)
* fix(state): Fix marking format upgrades (#7304)
---------
Co-authored-by: Arya <aryasolhi@gmail.com>
* Remove duplicate asserts
There are the same two asserts above the two removed ones.
* Remove workarounds for inserting trees into NFS
NFS = non finalized state
* Use correct height for constructing new chain
We were using the height of the last block instead of the initial block
to construct a new chain.
* Don't push the 0th block into a chain
* Don't commit two blocks at the same height
* Add helpers for heights
* Support the retrieval of deduped Sprout trees
* Dedup Sprout trees
* Refactor docs for adding & removing Sprout trees
* Support the retrieval of deduped Sapling trees
* Dedup Sapling trees
* Refactor docs for adding & removing Sapling trees
* Support the retrieval of deduped Orchard trees
* Dedup Orchard trees
* Refactor docs for adding & removing Orchard trees
* Make the docs for storing trees clearer
* Remove duplicate asserts
There are the same two asserts above the two removed ones.
* Remove workarounds for inserting trees into NFS
NFS = non finalized state
* Use correct height for constructing new chain
We were using the height of the last block instead of the initial block
to construct a new chain.
* Don't push the 0th block into a chain
* Don't commit two blocks at the same height
* Fix typo
* Generate chains with at least two blocks
---------
Co-authored-by: teor <teor@riseup.net>
* Add an async-error feature and an initial module structure
* Implement checking for panics in OS threads and async tasks
* Implement waiting for panics in OS threads and async tasks
* Add a TODO to simplify some state request error handling
* Use the new panic-checking methods in zebra-state
* Use new panic-checking methods in zebra-network
* fixup! Implement waiting for panics in OS threads and async tasks
* Replace existing async code with generic panic-checking methods
* Simplify trait to a single method
* Move thread panic code into generic trait impls
* Simplify option handling
Co-authored-by: Arya <aryasolhi@gmail.com>
* Fix comment
Co-authored-by: Arya <aryasolhi@gmail.com>
* Add missing track_caller
---------
Co-authored-by: Arya <aryasolhi@gmail.com>
* update ecc dependencies no serialization
* use zcash_primitives merke tree functions for serialization/deserialization in orchard and sapling trees
* some more work in sapling/orchard serialization/deserialization
* clippy
* fix doc links
* fix missing doc
* make orchard trees serializa/deserialize as they were before upgrade
* make sapling trees serialize/deserialize as they were before upgrade
* use legacy for sprout
* remove unused code
* readd snapshot tests
* repalce some code
* upgrade zcash_proofs
* remove legacy code for sprout
* fix the count method
* add root to serialize
* fixes in as_bytes and from_bytes
* use legacy code
* add todo about pow2 tests
* remove unused sprout code
* fix doc typos
* Add a recalculate_root() method to trees for tests
* Rename test tree types to make them easier to change
* Add TODOs for tests for old and new serialization formats
* fix doc typos
* add more test to note commitment trees
* fix comment
* fix leaf serializatiuon, reverse to old hashes in pow2 tests
* fix serialization
* put sapling SerializedTree code back
* put orchard SerializedTree code back
* clippy
* add duplicated dependencies until zebra_script updates
* fix a doc link
* minor cleanup
* remove todo comment from tests
* add one more check to tests
* update zebra_script
* update deny.toml
* replace custom function with library
* fix some tests
* update docs
* Remove duplicate dependencies from deny.toml
Based on the list at:
https://github.com/ZcashFoundation/zebra/actions/runs/5557139662/jobs/10150543673?pr=7053#step:5:14
---------
Co-authored-by: teor <teor@riseup.net>
* Add a recalculate_root() method to trees for tests
* Rename test tree types to make them easier to change
* Add TODOs for tests for old and new serialization formats
* fix doc typos
---------
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
* Add an assert_frontier_eq() method to note commitment trees for tests
* Check round-trip serialization for note commitment trees
* fix typos
---------
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
* Update license description in README for MIT-only crates
* Draft changelog with trivial issues
* Remove trivial issues
* Update changelog entries as of commit 2a31972 and PR #7103
* Update mainnet and testnet checkpoints as of 2023-06-30
* chore: Release
* Estimate release height for Zebra v1.0.1
Block height 2139118 at 2023-06-30 01:55:38 UTC
Release is likely to be 2023-07-01
2139118 + 1152 * 3 = 2142574
Then round up to the nearest 1000.
* Add methods for loading entire column families from the database
* Add a method that loads all the sprout trees from the database
* Add snapshot tests for sprout note commitment trees
* Add round-trip proptests for tree root database serialization
* Add a manual sprout note commitment tree database serialization snapshot test
* Add tests for 1,2,4,8 note commitments in a tree
* Remove redundant "rand" package rename in dependencies
* Randomly cache roots rather than only caching even roots
---------
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* Use Arc::into_inner() to avoid potential concurrency issues
* Remove some outdated clippy lint workarounds (fixed in Rust 1.66)
* Update the required Rust version to 1.70
* De-duplicate app_version and user_agent code, rename to build_version
* Make RPC testnet flag forward-compatible with additional testnets
* Fix RPC tests with new argument
* Use "modified" rather than "dirty" for uncommitted changes in build metadata
* Split the vergen version into its own function
* change(crates): Publish tower-batch-control
* Cargo fields for zebra-test
* Cargo fields for tower-fallback
* Cargo fields for zebra-chain
* Cargo fields for zebra-consensus
* Cargo fields for zebra-network
* Cargo fields for zebra-node-services
* Cargo fields for zebra-rpc
* Cargo fields for zebra-script
* Cargo fields for zebra-state
* Cargo fields for zebrad
* Cargo fields for zebra-test
* Include zebra-test version and path for zebra-chain
* Include zebra-chain version and path for zebra-node-services
* Include zebra-chain version and path for zebra-script
* Include zebra-chain and zebra-test versions and paths for zebra-state
* Include zebra-* versions and paths for zebra-consensus
* Include zebra-* and tower-fallback versions and paths for tower-batch-control
* Include tower-* versions and paths for zebra-consensus
* Fixed tower-fallback license string
* Remove versions for dev-dependencise
* Update zebra-rpc/Cargo.toml
Co-authored-by: teor <teor@riseup.net>
* tower-batch-control license MUST remain MIT only
* Align zebra-network with upstream tower's MIT license
* Update zebra-test/Cargo.toml
Co-authored-by: teor <teor@riseup.net>
* Update zebra-utils/Cargo.toml
Co-authored-by: teor <teor@riseup.net>
* fix(legal): Use correct licenses for tower-batch-control and zebra-network (#6927)
* Don't modify whitespace
* Resolve rustdoc and cargo-release warnings
* Add missing zebra internal dependency versions
* Workaround missing zebra-utils test dependency in zebrad test-only feature
* Fix missing zebra internal dependency versions
* Add README, home page, and search terms to every crate
* Fix README publish warnings
* Change tower crates to beta versions
* Add zebra-utils to the list of allowed unused dependencies
* Update tower-batch-control/Cargo.toml
* Update tower-fallback/Cargo.toml
---------
Co-authored-by: teor <teor@riseup.net>
* build: Upgrade to ed5519-zebra 4.0.0
* skip-tree hashbrown =0.13.2 because ed25519-zebra is 0.14.0 and a dep of metrics-exporter-prometheus is 0.13.2
* Use correct versions in deny.toml
* Oops turns out we need both exceptions
---------
Co-authored-by: teor <teor@riseup.net>
* Rewrite some state cache docs to clarify
* Add a zebra_network::Config.cache_dir for peer address caches
* Add new config test files and fix config test failure message
* Create some zebra-chain and zebra-network convenience functions
* Add methods for reading and writing the peer address cache
* Add cached disk peers to the initial peers list
* Add metrics and logging for loading and storing the peer cache
* Replace log of useless redacted peer IP addresses
* Limit the peer cache minimum and maximum size, don't write empty caches
* Add a cacheable_peers() method to the address book
* Add a peer disk cache updater task to the peer set tasks
* Document that the peer cache is shared by multiple instances unless configured otherwise
* Disable peer cache read/write in disconnected tests
* Make initial peer cache updater sleep shorter for tests
* Add unit tests for reading and writing the peer cache
* Update the task list in the start command docs
* Modify the existing persistent acceptance test to check for peer caches
* Update the peer cache directory when writing test configs
* Add a CacheDir type so the default config can be enabled, but tests can disable it
* Update tests to use the CacheDir config type
* Rename some CacheDir internals
* Add config file test cases for each kind of CacheDir config
* Panic if the config contains invalid socket addresses, rather than continuing
* Add a network directory to state cache directory contents tests
* Add new network.cache_dir config to the config parsing tests
* ZIPs were updated to remove ambiguity, this was tracked in #1267.
* #2105 was fixed by #3039 and #2379 was closed by #3069
* #2230 was a duplicate of #2231 which was closed by #2511
* #3235 was obsoleted by #2156 which was fixed by #3505
* #1850 was fixed by #2944, #1851 was fixed by #2961 and #2902 was fixed by #2969
* We migrated to Rust 2021 edition in Jan 2022 with #3332
* #1631 was closed as not needed
* #338 was fixed by #3040 and #1162 was fixed by #3067
* #2079 was fixed by #2445
* #4794 was fixed by #6122
* #1678 stopped being an issue
* #3151 was fixed by #3934
* #3204 was closed as not needed
* #1213 was fixed by #4586
* #1774 was closed as not needed
* #4633 was closed as not needed
* Clarify behaviour of difficulty spacing
Co-authored-by: teor <teor@riseup.net>
* Update comment to reflect implemented behaviour
Co-authored-by: teor <teor@riseup.net>
* Update comment to reflect implemented behaviour when retrying block downloads
Co-authored-by: teor <teor@riseup.net>
* Update `TODO` to remove closed issue and clarify when we might want to fix
Co-authored-by: teor <teor@riseup.net>
* Update `TODO` to remove closed issue and clarify what we might want to change in future
Co-authored-by: teor <teor@riseup.net>
* Clarify benefits of how we do block verification
Co-authored-by: teor <teor@riseup.net>
* Fix rustfmt errors
---------
Co-authored-by: teor <teor@riseup.net>
* Rename variables so it's clearer what they do
* Fully document zebra-checkpoints arguments
* Remove some outdated references to zcashd
* Add a json-conversion feature for converting JSON to valid Heights
* Simplify zebra-checkpoints code using conversion methods
* Track the last checkpoint height rather than the height gap
* Move all the CLI-specific code into a single function
* Remove cfg(feature) from the test RPC client API
* Move the RpcRequestClient into zebra-node-services
* Fix the case of RpcRequestClient
* Add transport and addr arguments to zebra-checkpoints
* Make zebra-checkpoints support both CLI and direct JSON-RPC connections
* Fix RpcRequestClient compilation
* Add a suggestion for zcashd authentication failures
* Document required features
* Handle differences in CLI & direct parameter and response formats
* Replace a custom function with an existing dependency function
* Add a checkpoint list test for minimum height gaps
* add error handling
* change error name
* Error types oriented around the primary types we expose in the zebra-chain API
* Fix Ok spelling
* orchard::note::new(): return NoteError if randomness produces invalid Pallas point
---------
Co-authored-by: Deirdre Connolly <durumcrustulum@gmail.com>
Co-authored-by: teor <teor@riseup.net>
* Adds `maturity_height` to VerifiedUnminedTx
Filters out transactions that are invalid at next_block_height in getblocktemplate method
* Adds unit testing
* rustfmt
* rejects txs with immature coinbase spends from mempool
* Condenses fns for transparent coinbase spend check
* Updates calls to VerifiedUnminedTx::new()
* Update zebra-chain/src/transparent/utxo.rs
* Applies suggestions from code review
* Implement Display and to_string() for NetworkUpgrade
* Add a progress-bar feature to zebrad
* Add the progress bar writer to the tracing component
* Add a block progress bar transmitter
* Correctly shut down the progress bar, and shut it down on an interrupt
* Make it clearer that the progress task never exits
* Add a config for writing logs to a file
* Add a progress-bar feature to zebra-network
* Add a progress bar for the address book size
* Add progress bars for never attempted and failed peers
* Add an optional limit and label to connection counters
* Add open connection progress bars
* Improve CheckpointList API and CheckpointVerifier debugging
* Add checkpoint index and checkpoint queue progress bars
* Security: Limit the number of non-finalized chains tracked by Zebra
* Make some NonFinalizedState methods available with proptest-impl
* Add a non-finalized chain count progress bar
* Track the last fork height for newly forked chains
* Add a should_count_metrics to Chain
* Add a display method for PartialCumulativeWork
* Add a progress bar for each chain fork
* Add a NonFinalizedState::disable_metrics() method and switch to using it
* Move metrics out of Chain because we can't update Arc<Chain>
* Fix: consistently use best chain order when searching chains
* Track Chain progress bars in NonFinalizedState
* Display work as bits, not a multiple of the target difficulty
* Handle negative fork lengths by reporting "No fork"
* Correctly disable unused fork bars
* clippy: rewrite using `match _.cmp(_) { ... }`
* Initial mempool progress bar implementation
* Update Cargo.lock
* Add the actual transaction size as a description to the cost bar
* Only show mempool progress bars after first activation
* Add queued and rejected mempool progress bars
* Clarify cost note is actual size
* Add tracing.log_file config and progress-bar feature to zebrad docs
* Derive Clone for Chain
* Upgrade to howudoin 0.1.2 and remove some bug workarounds
* Directly call the debug formatter to Display a Network
Co-authored-by: Arya <aryasolhi@gmail.com>
* Rename the address count metric to num_addresses
Co-authored-by: Arya <aryasolhi@gmail.com>
* Simplify reverse checkpoint lookup
Co-authored-by: Arya <aryasolhi@gmail.com>
* Simplify progress bar shutdown code
Co-authored-by: Arya <aryasolhi@gmail.com>
* Remove unused MIN_TRANSPARENT_TX_MEMPOOL_SIZE
* Document that the progress task runs forever
* Fix progress log formatting
* If progress-bar is on, log to a file by default
* Create missing directories for log files
* Add file security docs for running Zebra with elevated permissions
* Document automatic log file, spell progress-bar correctly
---------
Co-authored-by: Arya <aryasolhi@gmail.com>
* Unify the `impl`s of `Sub` and `Add` for `Height`
* Adjust tests for `Height` subtraction
* Use `Height` instead of `i32`
* Use `block:Height` in RPC tests
* Use `let .. else` statement
Co-authored-by: Arya <aryasolhi@gmail.com>
* Update zebra-consensus/src/block/subsidy/general.rs
* Refactor the handling of height differences
* Remove a redundant comment
* Update zebrad/src/components/sync/progress.rs
Co-authored-by: Arya <aryasolhi@gmail.com>
* Update progress.rs
* impl TryFrom<u32> for Height
* Make some test assertions clearer
* Refactor estimate_up_to()
* Restore a comment that was accidentally removed
* Document when estimate_distance_to_network_chain_tip() returns None
* Change HeightDiff to i64 and make Height.sub(Height) return HeightDiff (no Option)
* Update chain tip estimates for HeightDiff i64
* Update subsidy for HeightDiff i64
* Fix some height calculation test edge cases
* Fix the funding stream interval calculation
---------
Co-authored-by: Arya <aryasolhi@gmail.com>
Co-authored-by: teor <teor@riseup.net>
teor
dependabot[bot]
Alfredo Garcia
Marek
Arya
Pili Guerra
Deirdre Connolly