* impl TryFrom<zcash_primitives::BlockHeight> for Height
* Add type-safe read and write database methods
* Only allow typed access to the scanner DB
* Update docs
* Implement a common method as a trait
* Fix imports
* Tidy state imports
* Activate tracing logging macros in the whole scanner crate
* Fix dead code warnings
* Use a more sensible export order
* Remove a 1.72 lint exception now 1.74 is stable
* Switch history trees over to TypedColumnFamily, and remove redundant code
* Add typed batch creation methods, and switch history trees to them
* Convert ValueBalance to typed column families
* Make the APIs compatible after a merge
* Use `ZebraDb` instead of `DiskDb` where needed
---------
Co-authored-by: Marek <mail@marek.onl>
* Create an empty storage/db module
* Use ephemeral storage in tests
* Populate storage inside new() method
* Move scanner setup into an init() method
* Pass the network to scanner init
* Create a database but don't actually use it
* Skip shutdown format checks when skipping format upgrades
* Allow the scanner to skip launching format upgrades in production
* Refactor skipping format upgrades so it is consistent
* Allow checking configs for equality
* Restore Network import
* Update Connection debug impl missed in previous PRs
* Add an initial_cached_addrs argument to Connection::new()
* Stop sending version and remote IP peers directly to the address book
* Remove the NeverAttemptedAlternate peer state
* Remove the NewAlternate MetaAddrChange
* Remove the never_attempted_alternate AddressBookMetrics
* Update CandidateSet docs and add a Debug impl
* Update Connection debug impl missed in previous PRs
* Add an initial_cached_addrs argument to Connection::new()
* Stop sending version and remote IP peers directly to the address book
* Update zebra-network/src/peer/connection.rs
Co-authored-by: teor <teor@riseup.net>
---------
Co-authored-by: Marek <mail@marek.onl>
* Updates ADDR_RESPONSE_LIMIT_DENOMINATOR to 4
* Moves logic getting a fraction of Zebra's peers to a method in the address book
* Adds and uses CachedPeerAddrs struct in inbound service
* moves and documents constant
* fixes test
* Apply suggestions from code review
Co-authored-by: teor <teor@riseup.net>
* updates docs
* renames sanitized_window method
* renames CachedPeerAddrs to CachedPeerAddrResponse
* updates test
* moves try_refresh to per request
* Make unused sanitization method pub(crate)
* Apply suggestions from code review
Co-authored-by: teor <teor@riseup.net>
* moves CachedPeerAddrResponse to a module
* updates unit test
* fixes unit test
* removes unnecessary condition
* clears cached getaddr response if it can't refresh for over a minute after the refresh time
* tests that inbound service gives out the same addresses for every Peers request before the refresh interval
* Applies suggestion from code review
* fixes doc link
* renames constant
* Fix docs on new constant
* applies suggestion from code review
* uses longer cache expiry time
* Adds code comments
---------
Co-authored-by: teor <teor@riseup.net>
* Reformat a method to make related code clearer
* Limit how many addresses we take from each peer response
* Only put extra addresses back in the cache if there are actually any extras
* Refactor address caching into a single method
* Use the same method for updating the peer cache and taking peers from it
* Choose response peers at random, and partially randomly drop excess peers
* Update docs for this fix
* Fix an index bug and clarify a comment
* Add a proptest for cache updates
* Refactor return type of poll_discover()
* Simplify poll_ready() by removing preselected peers
* Fix peer set readiness check
* Pass task context correctly to background tasks
* Make poll_discover() return Pending
* Make poll_inventory() return Pending
* Make poll_unready() return Poll::Pending
* Simplify with futures::ready!() and ?
* When there are no peers, wake on newly ready peers, or new peers, in that order
* Preserve the original waker when there are no unready peers
* Fix polling docs and remove unnecessary code
* Make sure we're ignoring Poll::Pending not Result
* Make panic checking method names clearer
* Fix connection Client task wakeups and error handling
* Cleanup connection panic handling and add wakeup docs
* Fix connection client task wakeups to prevent hangs
* Simplify error and pending handling
* Clarify inventory set behaviour
* Define peer set poll_* methods so they return Ok if they do something
* Clarify documentation
Co-authored-by: Arya <aryasolhi@gmail.com>
* Fix test that depended on preselected peers
* Check ready peers for errors before sending requests to them
* Fix a hanging test by not waiting for irrelevant actions
* Only remove cancel handles when they are required
* fix incorrect panic on termination setting
* Clarify method comments
Co-authored-by: Arya <aryasolhi@gmail.com>
---------
Co-authored-by: Arya <aryasolhi@gmail.com>
* Fix a potential hang accessing the address book directly
* Remove unused connection shutdown MetaAddr arguments
* Add an UpdateConnected MetaAddrChange, that sends initial connection info
* Fix some tests
* Fix a panic with a zero channel size
* Refactor out try_to_sync_once()
* Add outer timeouts for obtaining and extending tips
* Refactor out request_genesis_once()
* Wrap genesis download once in a timeout
* Increase the genesis timeout to avoid denial of service from old nodes
* Add an outer timeout to mempool crawls
* Add an outer timeout to mempool download/verify
* Remove threaded mutex blocking from the inbound service
* Explain why inbound readiness never hangs
* Fix whitespace that cargo fmt doesn't
* Avoid hangs by always resetting the past lookahead limit flag
* Document block-specific and syncer-wide errors
* Update zebrad/src/components/sync.rs
Co-authored-by: Marek <mail@marek.onl>
* Use correct condition for log messages
Co-authored-by: Marek <mail@marek.onl>
* Keep lookahead reset metric
---------
Co-authored-by: Arya <aryasolhi@gmail.com>
Co-authored-by: Marek <mail@marek.onl>
* Simple replacements of doc.zebra.zfnd.org with docs.rs
* Manual fixes for specific main/internal/external docs
* Point developer docs to doc-internal.zebra.zfnd.org
* fastmod --glob '\!.git' -- doc.zebra.zfnd.org/zebrad docs.rs/zebrad/latest/zebrad
* Manually remove any remaining doc.zfnd.zebra.org links
* Remove the external docs job
* Add changelog entry and fix links
* Fix links that were broken before this PR
* Fixes bug where Zebra won't reconnect to peers after brief loss of network connectivity
* only dial on timercrawl when theres a new address or zero active outbound conns
* fix clippy and rustfmt after last releases today
* Ignore clippy::arc_with_non_send_sync false positives in proptest code only
* Remove .cargo/config.toml changes that didn't work
---------
Co-authored-by: teor <teor@riseup.net>
* Add an async-error feature and an initial module structure
* Implement checking for panics in OS threads and async tasks
* Implement waiting for panics in OS threads and async tasks
* Add a TODO to simplify some state request error handling
* Use the new panic-checking methods in zebra-state
* Use new panic-checking methods in zebra-network
* fixup! Implement waiting for panics in OS threads and async tasks
* Replace existing async code with generic panic-checking methods
* Simplify trait to a single method
* Move thread panic code into generic trait impls
* Simplify option handling
Co-authored-by: Arya <aryasolhi@gmail.com>
* Fix comment
Co-authored-by: Arya <aryasolhi@gmail.com>
* Add missing track_caller
---------
Co-authored-by: Arya <aryasolhi@gmail.com>
* Adds most_recent_by_ip field to address book
* adds test
* Apply suggestions from code review
* fixes lint
* Updates most_recent_by_ip in .take()
Updates should_update_most_recent_by_ip() and has_active_peer_with_ip to check last_attempt and last_failure times
Renames has_active_peer_with_ip
* Documents that Zebra will not initiate more than 1 outbound connections per IP
* Fixes is_ready_for_connection_attempt_with_ip
Adds test coverage for AttemptPending and Failed
Fixes new_outbound_peer_connections_are_rate_limited proptest
* Applies suggestions from code review.
* Applies suggestions from code review
* Always return true from `is_ready_for_connection_attempt_with_ip` if max_connections_per_ip != 0
* Update max_connections_per_ip config docs
* Warn about invalid config fields and use default values
* Ignores last_attempt and last_failure in is_ready_for_connection_attempt_with_ip
updates test
* Only update most_recent_by_ip if update.last_conn_state is responded.
* Apply suggestions from code review
Co-authored-by: teor <teor@riseup.net>
* fixes lint
* Update zebra-network/src/address_book.rs
Co-authored-by: teor <teor@riseup.net>
* Apply suggestions from code review
Co-authored-by: teor <teor@riseup.net>
* Fix Rust syntax
* Fix whitespace
---------
Co-authored-by: teor <teor@riseup.net>
* Remove duplicate "running" message, send log file opening logs to stderr
* Add missing block progress bar desc, make them consistent
* Put progress bars in a specific order in each section
* Actually make block progress bar desc consistent
* Make progress bar order even more consistent
* Fix fork blocks plural for 1 block
* Use the correct number of chain fork bars
* Disable confusing partial work display
* Add struct field category comments
* Silence a verbose inventory log
* Use Arc::into_inner() to avoid potential concurrency issues
* Remove some outdated clippy lint workarounds (fixed in Rust 1.66)
* Update the required Rust version to 1.70
* Limits num peer conns per ip
* Update zebra-network/src/peer_set/set.rs
* Update zebra-network/src/constants.rs
* Apply suggestions from code review
Co-authored-by: teor <teor@riseup.net>
* Keep old peer connections, rather than replacing them with new connections
* Adds max_conns_per_ip field
Configures the max to usize::MAX for some tests.
* Adds a test to check that max_conns_per_ip is enforced
---------
Co-authored-by: teor <teor@riseup.net>
* Add a missing timeout to outbound TCP connections
* Move inbound handshakes into their own function, replacing the manual span
* Delete a useless manual span in zebra_network::config
* Add an extra timeout to the spawned inbound handshake task
* Increase concurrency limit, reduce peer broadcast
* Fix a div_ceil() TODO
* Document security requirements of inbound peer overload handling
* Reduce drop probability and fix its formatting
* Put a 5 second timeout on inbound service requests
* Update drop probability tests
* Add error types, metrics, and logging for InboundTimeout errors
* fix: prevent progress bar from panicking using workaround
* remove more progress bar lengths
* re-add set_len() calls commented out; restore one that shouldn't cause panics
* Update zebra-state/src/service/non_finalized_state.rs
Co-authored-by: teor <teor@riseup.net>
---------
Co-authored-by: teor <teor@riseup.net>
* change(crates): Publish tower-batch-control
* Cargo fields for zebra-test
* Cargo fields for tower-fallback
* Cargo fields for zebra-chain
* Cargo fields for zebra-consensus
* Cargo fields for zebra-network
* Cargo fields for zebra-node-services
* Cargo fields for zebra-rpc
* Cargo fields for zebra-script
* Cargo fields for zebra-state
* Cargo fields for zebrad
* Cargo fields for zebra-test
* Include zebra-test version and path for zebra-chain
* Include zebra-chain version and path for zebra-node-services
* Include zebra-chain version and path for zebra-script
* Include zebra-chain and zebra-test versions and paths for zebra-state
* Include zebra-* versions and paths for zebra-consensus
* Include zebra-* and tower-fallback versions and paths for tower-batch-control
* Include tower-* versions and paths for zebra-consensus
* Fixed tower-fallback license string
* Remove versions for dev-dependencise
* Update zebra-rpc/Cargo.toml
Co-authored-by: teor <teor@riseup.net>
* tower-batch-control license MUST remain MIT only
* Align zebra-network with upstream tower's MIT license
* Update zebra-test/Cargo.toml
Co-authored-by: teor <teor@riseup.net>
* Update zebra-utils/Cargo.toml
Co-authored-by: teor <teor@riseup.net>
* fix(legal): Use correct licenses for tower-batch-control and zebra-network (#6927)
* Don't modify whitespace
* Resolve rustdoc and cargo-release warnings
* Add missing zebra internal dependency versions
* Workaround missing zebra-utils test dependency in zebrad test-only feature
* Fix missing zebra internal dependency versions
* Add README, home page, and search terms to every crate
* Fix README publish warnings
* Change tower crates to beta versions
* Add zebra-utils to the list of allowed unused dependencies
* Update tower-batch-control/Cargo.toml
* Update tower-fallback/Cargo.toml
---------
Co-authored-by: teor <teor@riseup.net>
* Stop sending peer errors on the PeerSet channel, to respect send limits
* Move locking out of the cralwer select!, potential deadlock or hang risk
* Move report_failed() out of the CandidateSet, reducing concurrency risks
* Make CandidateSet Send
* Make all CandidateSet operations concurrent, previous hand/deadlock bug
* Reduce the gap between handshakes and peer set updates, and exit the task on shutdown
* Implement minor and patch database format versions
* Log and update database format versions when opening database
* Refactor the current list of column families into a constant
* Open all available column families, including from future Zebra versions
* Refactor note commitment tree lookups to go through the height methods
* Make Sapling/Orchard note commitment tree lookup forwards compatible
* Ignore errors reading column family lists from disk
* Update format version comments and TODOs
* Correctly log newly created database formats
* Fix a new cargo lint about resolver versions
* cargo clippy --fix --all-features --all-targets
* cargo fmt --all
* Add missing tokio feature in the state, revealed by the new resolver
* Add missing dev dependencies in zebra-node-services
* Add a missing `tokio` feature from PR #6813
---------
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* Force Client tasks to shut down when it is dropped
* Try to close the peer sender sink on drop
* Reliably shut down the peer sender when the Connection is shut down
* Add a TODO for closing peer_rx
* Add logging for duplicate peer connections and IP addresses
* Rewrite some state cache docs to clarify
* Add a zebra_network::Config.cache_dir for peer address caches
* Add new config test files and fix config test failure message
* Create some zebra-chain and zebra-network convenience functions
* Add methods for reading and writing the peer address cache
* Add cached disk peers to the initial peers list
* Add metrics and logging for loading and storing the peer cache
* Replace log of useless redacted peer IP addresses
* Limit the peer cache minimum and maximum size, don't write empty caches
* Add a cacheable_peers() method to the address book
* Add a peer disk cache updater task to the peer set tasks
* Document that the peer cache is shared by multiple instances unless configured otherwise
* Disable peer cache read/write in disconnected tests
* Make initial peer cache updater sleep shorter for tests
* Add unit tests for reading and writing the peer cache
* Update the task list in the start command docs
* Modify the existing persistent acceptance test to check for peer caches
* Update the peer cache directory when writing test configs
* Add a CacheDir type so the default config can be enabled, but tests can disable it
* Update tests to use the CacheDir config type
* Rename some CacheDir internals
* Add config file test cases for each kind of CacheDir config
* Panic if the config contains invalid socket addresses, rather than continuing
* Add a network directory to state cache directory contents tests
* Add new network.cache_dir config to the config parsing tests
* fix(security): Randomly drop connections when inbound service is overloaded
* Uses progressively higher drop probabilities
* Replaces Error::Overloaded with Fatal when internal services shutdown
* Applies suggestions from code review.
* Quickens initial drop probability decay and updates comment
* Applies suggestions from code review.
* Fixes drop connection probablity calc
* Update connection state metrics for different overload/error outcomes
* Split overload handler into separate methods
* Add unit test for drop probability function properties
* Add respond_error methods to zebra-test to help with type resolution
* Initial test that Overloaded errors cause some continues and some closes
* Tune the number of test runs and test timing
* Fix doctests and replace some confusing example requests
---------
Co-authored-by: arya2 <aryasolhi@gmail.com>
* Ignore out of order Address Book changes, and restructure the function
* Handle concurrent changes using the connection state machine order
* Handle out of order changes correctly
* Pass times through the call stack so they are consistent in tests
* Add time arguments to tests
* Fix tests that were broken by the address order checks
* fastmod wall_ local_ zebra*
* cargo fmt --all
* Fix a bug in the concurrent change check
* Test all the new apply and skip checks for address changes
* Document more edge cases and increase the concurrency time slightly
* Simplify enum ordering matches
* Fix comment typos
Co-authored-by: Arya <aryasolhi@gmail.com>
---------
Co-authored-by: Arya <aryasolhi@gmail.com>
dependabot[bot]
teor
Arya
Alfredo Garcia
Conrado Gouvea
Deirdre Connolly