* Fixes bug where Zebra won't reconnect to peers after brief loss of network connectivity
* only dial on timercrawl when theres a new address or zero active outbound conns
* fix clippy and rustfmt after last releases today
* Ignore clippy::arc_with_non_send_sync false positives in proptest code only
* Remove .cargo/config.toml changes that didn't work
---------
Co-authored-by: teor <teor@riseup.net>
* Add an async-error feature and an initial module structure
* Implement checking for panics in OS threads and async tasks
* Implement waiting for panics in OS threads and async tasks
* Add a TODO to simplify some state request error handling
* Use the new panic-checking methods in zebra-state
* Use new panic-checking methods in zebra-network
* fixup! Implement waiting for panics in OS threads and async tasks
* Replace existing async code with generic panic-checking methods
* Simplify trait to a single method
* Move thread panic code into generic trait impls
* Simplify option handling
Co-authored-by: Arya <aryasolhi@gmail.com>
* Fix comment
Co-authored-by: Arya <aryasolhi@gmail.com>
* Add missing track_caller
---------
Co-authored-by: Arya <aryasolhi@gmail.com>
* Adds most_recent_by_ip field to address book
* adds test
* Apply suggestions from code review
* fixes lint
* Updates most_recent_by_ip in .take()
Updates should_update_most_recent_by_ip() and has_active_peer_with_ip to check last_attempt and last_failure times
Renames has_active_peer_with_ip
* Documents that Zebra will not initiate more than 1 outbound connections per IP
* Fixes is_ready_for_connection_attempt_with_ip
Adds test coverage for AttemptPending and Failed
Fixes new_outbound_peer_connections_are_rate_limited proptest
* Applies suggestions from code review.
* Applies suggestions from code review
* Always return true from `is_ready_for_connection_attempt_with_ip` if max_connections_per_ip != 0
* Update max_connections_per_ip config docs
* Warn about invalid config fields and use default values
* Ignores last_attempt and last_failure in is_ready_for_connection_attempt_with_ip
updates test
* Only update most_recent_by_ip if update.last_conn_state is responded.
* Apply suggestions from code review
Co-authored-by: teor <teor@riseup.net>
* fixes lint
* Update zebra-network/src/address_book.rs
Co-authored-by: teor <teor@riseup.net>
* Apply suggestions from code review
Co-authored-by: teor <teor@riseup.net>
* Fix Rust syntax
* Fix whitespace
---------
Co-authored-by: teor <teor@riseup.net>
* Remove duplicate "running" message, send log file opening logs to stderr
* Add missing block progress bar desc, make them consistent
* Put progress bars in a specific order in each section
* Actually make block progress bar desc consistent
* Make progress bar order even more consistent
* Fix fork blocks plural for 1 block
* Use the correct number of chain fork bars
* Disable confusing partial work display
* Add struct field category comments
* Silence a verbose inventory log
* Use Arc::into_inner() to avoid potential concurrency issues
* Remove some outdated clippy lint workarounds (fixed in Rust 1.66)
* Update the required Rust version to 1.70
* Limits num peer conns per ip
* Update zebra-network/src/peer_set/set.rs
* Update zebra-network/src/constants.rs
* Apply suggestions from code review
Co-authored-by: teor <teor@riseup.net>
* Keep old peer connections, rather than replacing them with new connections
* Adds max_conns_per_ip field
Configures the max to usize::MAX for some tests.
* Adds a test to check that max_conns_per_ip is enforced
---------
Co-authored-by: teor <teor@riseup.net>
* Add a missing timeout to outbound TCP connections
* Move inbound handshakes into their own function, replacing the manual span
* Delete a useless manual span in zebra_network::config
* Add an extra timeout to the spawned inbound handshake task
* Increase concurrency limit, reduce peer broadcast
* Fix a div_ceil() TODO
* Document security requirements of inbound peer overload handling
* Reduce drop probability and fix its formatting
* Put a 5 second timeout on inbound service requests
* Update drop probability tests
* Add error types, metrics, and logging for InboundTimeout errors
* fix: prevent progress bar from panicking using workaround
* remove more progress bar lengths
* re-add set_len() calls commented out; restore one that shouldn't cause panics
* Update zebra-state/src/service/non_finalized_state.rs
Co-authored-by: teor <teor@riseup.net>
---------
Co-authored-by: teor <teor@riseup.net>
* change(crates): Publish tower-batch-control
* Cargo fields for zebra-test
* Cargo fields for tower-fallback
* Cargo fields for zebra-chain
* Cargo fields for zebra-consensus
* Cargo fields for zebra-network
* Cargo fields for zebra-node-services
* Cargo fields for zebra-rpc
* Cargo fields for zebra-script
* Cargo fields for zebra-state
* Cargo fields for zebrad
* Cargo fields for zebra-test
* Include zebra-test version and path for zebra-chain
* Include zebra-chain version and path for zebra-node-services
* Include zebra-chain version and path for zebra-script
* Include zebra-chain and zebra-test versions and paths for zebra-state
* Include zebra-* versions and paths for zebra-consensus
* Include zebra-* and tower-fallback versions and paths for tower-batch-control
* Include tower-* versions and paths for zebra-consensus
* Fixed tower-fallback license string
* Remove versions for dev-dependencise
* Update zebra-rpc/Cargo.toml
Co-authored-by: teor <teor@riseup.net>
* tower-batch-control license MUST remain MIT only
* Align zebra-network with upstream tower's MIT license
* Update zebra-test/Cargo.toml
Co-authored-by: teor <teor@riseup.net>
* Update zebra-utils/Cargo.toml
Co-authored-by: teor <teor@riseup.net>
* fix(legal): Use correct licenses for tower-batch-control and zebra-network (#6927)
* Don't modify whitespace
* Resolve rustdoc and cargo-release warnings
* Add missing zebra internal dependency versions
* Workaround missing zebra-utils test dependency in zebrad test-only feature
* Fix missing zebra internal dependency versions
* Add README, home page, and search terms to every crate
* Fix README publish warnings
* Change tower crates to beta versions
* Add zebra-utils to the list of allowed unused dependencies
* Update tower-batch-control/Cargo.toml
* Update tower-fallback/Cargo.toml
---------
Co-authored-by: teor <teor@riseup.net>
* Stop sending peer errors on the PeerSet channel, to respect send limits
* Move locking out of the cralwer select!, potential deadlock or hang risk
* Move report_failed() out of the CandidateSet, reducing concurrency risks
* Make CandidateSet Send
* Make all CandidateSet operations concurrent, previous hand/deadlock bug
* Reduce the gap between handshakes and peer set updates, and exit the task on shutdown
* Implement minor and patch database format versions
* Log and update database format versions when opening database
* Refactor the current list of column families into a constant
* Open all available column families, including from future Zebra versions
* Refactor note commitment tree lookups to go through the height methods
* Make Sapling/Orchard note commitment tree lookup forwards compatible
* Ignore errors reading column family lists from disk
* Update format version comments and TODOs
* Correctly log newly created database formats
* Fix a new cargo lint about resolver versions
* cargo clippy --fix --all-features --all-targets
* cargo fmt --all
* Add missing tokio feature in the state, revealed by the new resolver
* Add missing dev dependencies in zebra-node-services
* Add a missing `tokio` feature from PR #6813
---------
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* Force Client tasks to shut down when it is dropped
* Try to close the peer sender sink on drop
* Reliably shut down the peer sender when the Connection is shut down
* Add a TODO for closing peer_rx
* Add logging for duplicate peer connections and IP addresses
* Rewrite some state cache docs to clarify
* Add a zebra_network::Config.cache_dir for peer address caches
* Add new config test files and fix config test failure message
* Create some zebra-chain and zebra-network convenience functions
* Add methods for reading and writing the peer address cache
* Add cached disk peers to the initial peers list
* Add metrics and logging for loading and storing the peer cache
* Replace log of useless redacted peer IP addresses
* Limit the peer cache minimum and maximum size, don't write empty caches
* Add a cacheable_peers() method to the address book
* Add a peer disk cache updater task to the peer set tasks
* Document that the peer cache is shared by multiple instances unless configured otherwise
* Disable peer cache read/write in disconnected tests
* Make initial peer cache updater sleep shorter for tests
* Add unit tests for reading and writing the peer cache
* Update the task list in the start command docs
* Modify the existing persistent acceptance test to check for peer caches
* Update the peer cache directory when writing test configs
* Add a CacheDir type so the default config can be enabled, but tests can disable it
* Update tests to use the CacheDir config type
* Rename some CacheDir internals
* Add config file test cases for each kind of CacheDir config
* Panic if the config contains invalid socket addresses, rather than continuing
* Add a network directory to state cache directory contents tests
* Add new network.cache_dir config to the config parsing tests
* fix(security): Randomly drop connections when inbound service is overloaded
* Uses progressively higher drop probabilities
* Replaces Error::Overloaded with Fatal when internal services shutdown
* Applies suggestions from code review.
* Quickens initial drop probability decay and updates comment
* Applies suggestions from code review.
* Fixes drop connection probablity calc
* Update connection state metrics for different overload/error outcomes
* Split overload handler into separate methods
* Add unit test for drop probability function properties
* Add respond_error methods to zebra-test to help with type resolution
* Initial test that Overloaded errors cause some continues and some closes
* Tune the number of test runs and test timing
* Fix doctests and replace some confusing example requests
---------
Co-authored-by: arya2 <aryasolhi@gmail.com>
* Ignore out of order Address Book changes, and restructure the function
* Handle concurrent changes using the connection state machine order
* Handle out of order changes correctly
* Pass times through the call stack so they are consistent in tests
* Add time arguments to tests
* Fix tests that were broken by the address order checks
* fastmod wall_ local_ zebra*
* cargo fmt --all
* Fix a bug in the concurrent change check
* Test all the new apply and skip checks for address changes
* Document more edge cases and increase the concurrency time slightly
* Simplify enum ordering matches
* Fix comment typos
Co-authored-by: Arya <aryasolhi@gmail.com>
---------
Co-authored-by: Arya <aryasolhi@gmail.com>
* ZIPs were updated to remove ambiguity, this was tracked in #1267.
* #2105 was fixed by #3039 and #2379 was closed by #3069
* #2230 was a duplicate of #2231 which was closed by #2511
* #3235 was obsoleted by #2156 which was fixed by #3505
* #1850 was fixed by #2944, #1851 was fixed by #2961 and #2902 was fixed by #2969
* We migrated to Rust 2021 edition in Jan 2022 with #3332
* #1631 was closed as not needed
* #338 was fixed by #3040 and #1162 was fixed by #3067
* #2079 was fixed by #2445
* #4794 was fixed by #6122
* #1678 stopped being an issue
* #3151 was fixed by #3934
* #3204 was closed as not needed
* #1213 was fixed by #4586
* #1774 was closed as not needed
* #4633 was closed as not needed
* Clarify behaviour of difficulty spacing
Co-authored-by: teor <teor@riseup.net>
* Update comment to reflect implemented behaviour
Co-authored-by: teor <teor@riseup.net>
* Update comment to reflect implemented behaviour when retrying block downloads
Co-authored-by: teor <teor@riseup.net>
* Update `TODO` to remove closed issue and clarify when we might want to fix
Co-authored-by: teor <teor@riseup.net>
* Update `TODO` to remove closed issue and clarify what we might want to change in future
Co-authored-by: teor <teor@riseup.net>
* Clarify benefits of how we do block verification
Co-authored-by: teor <teor@riseup.net>
* Fix rustfmt errors
---------
Co-authored-by: teor <teor@riseup.net>
* Update MAX_TX_INV_IN_MESSAGE for ZIP-239 WTX IDs
* Combine multiple transaction updates into a single gossip & rate-limit gossips
* Rate-limit block gossips
* Fix mempool_transaction_expiration gossip test timings
* Enforce MAX_TX_INV_IN_MESSAGE in the network layer, rather than each service
* Fix documentation for `Message::Tx`
* Split MAX_INV_IN_RECEIVED_MESSAGE and MAX_TX_INV_IN_SENT_MESSAGE
* Fix log message typo
* Move some docs to/from another PR
---------
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* Add a PeerSocketAddr type which hides its IP address, but shows the port
* Manually replace SocketAddr with PeerSocketAddr where needed
```sh
fastmod SocketAddr PeerSocketAddr zebra-network
```
* Add missing imports
* Make converting into PeerSocketAddr easier
* Fix some unused imports
* Add a canonical_peer_addr() function
* Fix connection handling for PeerSocketAddr
* Fix serialization for PeerSocketAddr
* Fix tests for PeerSocketAddr
* Remove some unused imports
* Fix address book listener handling
* Remove redundant imports and conversions
* Update outdated IPv4-mapped IPv6 address code
* Make addresses canonical when deserializing
* Stop logging peer addresses in RPC code
* Update zebrad tests with new PeerSocketAddr type
* Update zebra-rpc tests with new PeerSocketAddr type
---------
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* add user agent as argument, use git to auto build zebra user agent
* try to fix test
* fix typo
* change expect text
* remove newline
* fix some docs
Co-authored-by: Marek <mail@marek.onl>
---------
Co-authored-by: Marek <mail@marek.onl>
* refuse to run Zebra if it is too old
* update the release checklist to consider the constants
* bring newline back
* apply new end of support code
* attempt to add tests (not working yet)
* move eos to progress task
* move tests
* add acceptance test (not working)
* fix tests
* change to block height checks (ugly code)
* change warn days
* refactor estimated blocks per day, etc
* move end of support code to its own task
* change test
* fix some docs
* move constants
* remove uneeded conversions
* downgrade tracing
* reduce end of support time, fix ci changing debugs to info again
* update instructions
* add failure messages
* cargo lock update
* unify releaase name constant
* change info msg
* clippy fixes
* add a block explorer
* ignore testnet in end of support task
* change panic to 16 weeks
* add some documentation about end of support
* Tweak docs wording
---------
Co-authored-by: teor <teor@riseup.net>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* Reject nodes using the ZClassic default ports
* Always check regtest and other coin ports, even if no network is supplied
* Warn if Zebra is configured with ports from other coins
* Allow unspecified addresses and ports for inbound listeners
---------
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* Close the new connection if Zebra unexpectedly generates a duplicate random nonce
* Add a missing test module comment
* Avoid peer attacks that replay self-connection nonces to manipulate the nonce set (needs tests)
* Add a test that makes sure network self-connections fail
* Log an info level when self-connections fail (this should be rare)
* Just use plain blocks for mutex critical sections
* Add a missing space
* Silence an extremely verbose error in zebra-consensus tests
This disables around 10,000 logs like:
2023-04-18T02:46:28.441662Z WARN init{config=Config { checkpoint_sync: true, debug_skip_parameter_preload: false } network=Mainnet debug_skip_parameter_preload=true}: unexpected error: Closed in state request while verifying previous state checkpoints
* Increase the outbound connection interval to 100ms
* Start the inbound service as soon as possible, and the syncer last
* Increase acceptance test time limits to get more debug info
* Add more debug info to inbound service overload tracing messages
* Use a stricter connection rate limit for successful inbound peer connections
* Limit the number of nonces in the self-connection nonce set
* Rate-limit failed inbound connections as well
* Justify the sleep and the yield_now
* Use the configured connection limit rather than a constant
* Tests that the number of nonces is limited (#37)
* Tests that the number of nonces is limited
* removes unused constant
* test that it reaches the nonce limit
---------
Co-authored-by: Arya <aryasolhi@gmail.com>
teor
Alfredo Garcia
Arya
Conrado Gouvea
Deirdre Connolly
Pili Guerra