Security: Zebra should stop gossiping unreachable addresses to other nodes, Action: re-deploy all nodes (#2392)

* Rename some methods and constants for clarity

Using the following commands:

```
fastmod '\bis_ready_for_attempt\b' is_ready_for_connection_attempt
  # One instance required a tweak, because of the ASCII diagram.
fastmod '\bwas_recently_live\b' has_connection_recently_responded
fastmod '\bwas_recently_attempted\b' was_connection_recently_attempted
fastmod '\bwas_recently_failed\b' has_connection_recently_failed
fastmod '\bLIVE_PEER_DURATION\b' MIN_PEER_RECONNECTION_DELAY
```

* Use `Instant::elapsed` for conciseness

Instead of `Instant::now().saturating_duration_since`. They're both
equivalent, and `elapsed` only panics if the `Instant` is somehow
synthetically generated.

* Allow `Duration32` to be created in other crates

Export the `Duration32` from the `zebra_chain::serialization` module.

* Add some new `Duration32` constructors

Create some helper `const` constructors to make it easy to create
constant durations. Add methods to create a `Duration32` from seconds,
minutes and hours.

* Avoid gossiping unreachable peers

When sanitizing the list of peers to gossip, remove those that we
haven't seen in more than three hours.

* Test if unreachable addresses aren't gossiped

Create a property test with random addreses inserted into an
`AddressBook`, and verify that the sanitized list of addresses does not
contain any addresses considered unreachable.

* Test if new alternate address isn't gossipable

Create a new alternate peer, because that type of `MetaAddr` does not
have `last_response` or `untrusted_last_seen` times. Verify that the
peer is not considered gossipable.

* Test if local listener is gossipable

The `MetaAddr` representing the local peer's listening address should
always be considered gossipable.

* Test if gossiped peer recently seen is gossipable

Create a `MetaAddr` representing a gossiped peer that was reported to be
seen recently. Check that the peer is considered gossipable.

* Test peer reportedly last seen in the future

Create a `MetaAddr` representing a peer gossiped and reported to have
been last seen in a time that's in the future. Check that the peer is
considered gossipable, to check that the fallback calculation is working
as intended.

* Test gossiped peer reportedly seen long ago

Create a `MetaAddr` representing a gossiped peer that was reported to
last have been seen a long time ago. Check that the peer is not
considered gossipable.

* Test if just responded peer is gossipable

Create a `MetaAddr` representing a peer that has just responded and
check that it is considered gossipable.

* Test if recently responded peer is gossipable

Create a `MetaAddr` representing a peer that last responded within the
duration a peer is considered reachable. Verify that the peer is
considered gossipable.

* Test peer that responded long ago isn't gossipable

Create a `MetaAddr` representing a peer that last responded outside the
duration a peer is considered reachable. Verify that the peer is not
considered gossipable.
This commit is contained in:
Janito Vaqueiro Ferreira Filho 2021-06-29 02:12:27 -03:00 committed by GitHub
parent 1624377da7
commit b68202c68a
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
10 changed files with 316 additions and 32 deletions

View File

@ -22,7 +22,7 @@ pub mod sha256d;
pub mod arbitrary;
pub use constraint::AtLeastOne;
pub use date_time::DateTime32;
pub use date_time::{DateTime32, Duration32};
pub use error::SerializationError;
pub use read_zcash::{canonical_socket_addr, ReadZcashExt};
pub use write_zcash::WriteZcashExt;

View File

@ -117,6 +117,27 @@ impl Duration32 {
/// The latest possible `Duration32` value.
pub const MAX: Duration32 = Duration32 { seconds: u32::MAX };
/// Creates a new [`Duration32`] to represent the given amount of seconds.
pub const fn from_seconds(seconds: u32) -> Self {
Duration32 { seconds }
}
/// Creates a new [`Duration32`] to represent the given amount of minutes.
///
/// If the resulting number of seconds does not fit in a [`u32`], [`Duration32::MAX`] is
/// returned.
pub const fn from_minutes(minutes: u32) -> Self {
Duration32::from_seconds(minutes.saturating_mul(60))
}
/// Creates a new [`Duration32`] to represent the given amount of hours.
///
/// If the resulting number of seconds does not fit in a [`u32`], [`Duration32::MAX`] is
/// returned.
pub const fn from_hours(hours: u32) -> Self {
Duration32::from_minutes(hours.saturating_mul(60))
}
/// Returns the number of seconds in this duration.
pub fn seconds(&self) -> u32 {
self.seconds

View File

@ -14,6 +14,9 @@ use zebra_chain::serialization::canonical_socket_addr;
use crate::{meta_addr::MetaAddrChange, types::MetaAddr, PeerAddrState};
#[cfg(test)]
mod tests;
/// A database of peer listener addresses, their advertised services, and
/// information on when they were last seen.
///
@ -159,6 +162,14 @@ impl AddressBook {
let mut peers = peers
.values()
.filter_map(MetaAddr::sanitize)
// Security: remove peers that:
// - last responded more than three hours ago, or
// - haven't responded yet but were reported last seen more than three hours ago
//
// This prevents Zebra from gossiping nodes that are likely unreachable. Gossiping such
// nodes impacts the network health, because connection attempts end up being wasted on
// peers that are less likely to respond.
.filter(MetaAddr::is_active_for_gossip)
.collect::<Vec<_>>();
peers.shuffle(&mut rand::thread_rng());
peers
@ -256,7 +267,8 @@ impl AddressBook {
None => false,
// NeverAttempted, Failed, and AttemptPending peers should never be live
Some(peer) => {
peer.last_connection_state == PeerAddrState::Responded && peer.was_recently_live()
peer.last_connection_state == PeerAddrState::Responded
&& peer.has_connection_recently_responded()
}
}
}
@ -291,7 +303,7 @@ impl AddressBook {
// Skip live peers, and peers pending a reconnect attempt, then sort using BTreeSet
self.by_addr
.values()
.filter(|peer| peer.is_ready_for_attempt())
.filter(|peer| peer.is_ready_for_connection_attempt())
.collect::<BTreeSet<_>>()
.into_iter()
.cloned()
@ -314,7 +326,7 @@ impl AddressBook {
self.by_addr
.values()
.filter(|peer| !peer.is_ready_for_attempt())
.filter(|peer| !peer.is_ready_for_connection_attempt())
.cloned()
}

View File

@ -0,0 +1 @@
mod prop;

View File

@ -0,0 +1,36 @@
use std::net::SocketAddr;
use proptest::{collection::vec, prelude::*};
use tracing::Span;
use zebra_chain::serialization::Duration32;
use super::super::AddressBook;
use crate::{
constants::MAX_PEER_ACTIVE_FOR_GOSSIP,
meta_addr::{arbitrary::MAX_META_ADDR, MetaAddr},
};
const TIME_ERROR_MARGIN: Duration32 = Duration32::from_seconds(1);
proptest! {
#[test]
fn only_recently_reachable_are_gossiped(
local_listener in any::<SocketAddr>(),
addresses in vec(any::<MetaAddr>(), 0..MAX_META_ADDR),
) {
zebra_test::init();
let address_book = AddressBook::new_with_addrs(local_listener, Span::none(), addresses);
for gossiped_address in address_book.sanitized() {
let duration_since_last_seen = gossiped_address
.last_seen()
.expect("Peer that was never seen before is being gossiped")
.saturating_elapsed()
.saturating_sub(TIME_ERROR_MARGIN);
prop_assert!(duration_since_last_seen <= MAX_PEER_ACTIVE_FOR_GOSSIP);
}
}
}

View File

@ -8,7 +8,7 @@ use regex::Regex;
// XXX should these constants be split into protocol also?
use crate::protocol::external::types::*;
use zebra_chain::parameters::NetworkUpgrade;
use zebra_chain::{parameters::NetworkUpgrade, serialization::Duration32};
/// The buffer size for the peer set.
///
@ -43,7 +43,19 @@ pub const HANDSHAKE_TIMEOUT: Duration = Duration::from_secs(4);
/// This avoids explicit synchronization, but relies on the peer
/// connector actually setting up channels and these heartbeats in a
/// specific manner that matches up with this math.
pub const LIVE_PEER_DURATION: Duration = Duration::from_secs(60 + 20 + 20 + 20);
pub const MIN_PEER_RECONNECTION_DELAY: Duration = Duration::from_secs(60 + 20 + 20 + 20);
/// The maximum duration since a peer was last seen to consider it reachable.
///
/// This is used to prevent Zebra from gossiping addresses that are likely unreachable. Peers that
/// have last been seen more than this duration ago will not be gossiped.
///
/// This is determined as a tradeoff between network health and network view leakage. From the
/// [Bitcoin protocol documentation](https://en.bitcoin.it/wiki/Protocol_documentation#getaddr):
///
/// "The typical presumption is that a node is likely to be active if it has been sending a message
/// within the last three hours."
pub const MAX_PEER_ACTIVE_FOR_GOSSIP: Duration32 = Duration32::from_hours(3);
/// Regular interval for sending keepalive `Ping` messages to each
/// connected peer.
@ -170,7 +182,7 @@ mod tests {
use super::*;
/// This assures that the `Duration` value we are computing for
/// LIVE_PEER_DURATION actually matches the other const values it
/// MIN_PEER_RECONNECTION_DELAY actually matches the other const values it
/// relies on.
#[test]
fn ensure_live_peer_duration_value_matches_others() {
@ -179,7 +191,7 @@ mod tests {
let constructed_live_peer_duration =
HEARTBEAT_INTERVAL + REQUEST_TIMEOUT + REQUEST_TIMEOUT + REQUEST_TIMEOUT;
assert_eq!(LIVE_PEER_DURATION, constructed_live_peer_duration);
assert_eq!(MIN_PEER_RECONNECTION_DELAY, constructed_live_peer_duration);
}
/// Make sure that the timeout values are consistent with each other.

View File

@ -28,7 +28,7 @@ use proptest_derive::Arbitrary;
#[cfg(any(test, feature = "proptest-impl"))]
use zebra_chain::serialization::arbitrary::canonical_socket_addr_strategy;
#[cfg(any(test, feature = "proptest-impl"))]
mod arbitrary;
pub(crate) mod arbitrary;
#[cfg(test)]
mod tests;
@ -411,16 +411,16 @@ impl MetaAddr {
/// Returns `true` if the peer is likely connected and responsive in the peer
/// set.
///
/// [`constants::LIVE_PEER_DURATION`] represents the time interval in which
/// [`constants::MIN_PEER_RECONNECTION_DELAY`] represents the time interval in which
/// we should receive at least one message from a peer, or close the
/// connection. Therefore, if the last-seen timestamp is older than
/// [`constants::LIVE_PEER_DURATION`] ago, we know we should have
/// [`constants::MIN_PEER_RECONNECTION_DELAY`] ago, we know we should have
/// disconnected from it. Otherwise, we could potentially be connected to it.
pub fn was_recently_live(&self) -> bool {
pub fn has_connection_recently_responded(&self) -> bool {
if let Some(last_response) = self.last_response {
// Recent times and future times are considered live
last_response.saturating_elapsed()
<= constants::LIVE_PEER_DURATION
<= constants::MIN_PEER_RECONNECTION_DELAY
.try_into()
.expect("unexpectedly large constant")
} else {
@ -433,12 +433,12 @@ impl MetaAddr {
///
/// Returns `true` if this peer was recently attempted, or has a connection
/// attempt in progress.
pub fn was_recently_attempted(&self) -> bool {
pub fn was_connection_recently_attempted(&self) -> bool {
if let Some(last_attempt) = self.last_attempt {
// Recent times and future times are considered live.
// Instants are monotonic, so `now` should always be later than `last_attempt`,
// except for synthetic data in tests.
Instant::now().saturating_duration_since(last_attempt) <= constants::LIVE_PEER_DURATION
last_attempt.elapsed() <= constants::MIN_PEER_RECONNECTION_DELAY
} else {
// If there has never been any attempt, it can't possibly be live
false
@ -448,22 +448,41 @@ impl MetaAddr {
/// Have we recently had a failed connection to this peer?
///
/// Returns `true` if this peer has recently failed.
pub fn was_recently_failed(&self) -> bool {
pub fn has_connection_recently_failed(&self) -> bool {
if let Some(last_failure) = self.last_failure {
// Recent times and future times are considered live
Instant::now().saturating_duration_since(last_failure) <= constants::LIVE_PEER_DURATION
last_failure.elapsed() <= constants::MIN_PEER_RECONNECTION_DELAY
} else {
// If there has never been any failure, it can't possibly be recent
false
}
}
/// Has this peer been seen recently?
///
/// Returns `true` if this peer has responded recently or if the peer was gossiped with a
/// recent reported last seen time.
///
/// [`constants::MAX_PEER_ACTIVE_FOR_GOSSIP`] represents the maximum time since a peer was seen
/// to still be considered reachable.
pub fn is_active_for_gossip(&self) -> bool {
if let Some(last_seen) = self.last_seen() {
// Correctness: `last_seen` shouldn't ever be in the future, either because we set the
// time or because another peer's future time was sanitized when it was added to the
// address book
last_seen.saturating_elapsed() <= constants::MAX_PEER_ACTIVE_FOR_GOSSIP
} else {
// Peer has never responded and does not have a gossiped last seen time
false
}
}
/// Is this address ready for a new outbound connection attempt?
pub fn is_ready_for_attempt(&self) -> bool {
pub fn is_ready_for_connection_attempt(&self) -> bool {
self.last_known_info_is_valid_for_outbound()
&& !self.was_recently_live()
&& !self.was_recently_attempted()
&& !self.was_recently_failed()
&& !self.has_connection_recently_responded()
&& !self.was_connection_recently_attempted()
&& !self.has_connection_recently_failed()
}
/// Is the [`SocketAddr`] we have for this peer valid for outbound
@ -520,6 +539,16 @@ impl MetaAddr {
}
}
#[cfg(test)]
impl MetaAddr {
/// Forcefully change the time this peer last responded.
///
/// This method is for test-purposes only.
pub(crate) fn set_last_response(&mut self, last_response: DateTime32) {
self.last_response = Some(last_response);
}
}
impl MetaAddrChange {
/// Return the address for this change.
pub fn addr(&self) -> SocketAddr {

View File

@ -19,7 +19,7 @@ use zebra_chain::serialization::{canonical_socket_addr, ZcashDeserialize, ZcashS
use super::check;
use crate::{
constants::LIVE_PEER_DURATION,
constants::MIN_PEER_RECONNECTION_DELAY,
meta_addr::{
arbitrary::{MAX_ADDR_CHANGE, MAX_META_ADDR},
MetaAddr, MetaAddrChange,
@ -228,7 +228,7 @@ proptest! {
}
/// Make sure that [`MetaAddr`]s do not get retried more than once per
/// [`LIVE_PEER_DURATION`], regardless of the [`MetaAddrChange`]s that are
/// [`MIN_PEER_RECONNECTION_DELAY`], regardless of the [`MetaAddrChange`]s that are
/// applied to them.
///
/// This is the simple version of the test, which checks [`MetaAddr`]s by
@ -243,9 +243,9 @@ proptest! {
let mut attempt_count: usize = 0;
for change in changes {
while addr.is_ready_for_attempt() {
while addr.is_ready_for_connection_attempt() {
attempt_count += 1;
// Assume that this test doesn't last longer than LIVE_PEER_DURATION
// Assume that this test doesn't last longer than MIN_PEER_RECONNECTION_DELAY
prop_assert!(attempt_count <= 1);
// Simulate an attempt
@ -309,7 +309,7 @@ proptest! {
.unwrap_or(DEFAULT_VERBOSE_TEST_PROPTEST_CASES)))]
/// Make sure that [`MetaAddr`]s do not get retried more than once per
/// [`LIVE_PEER_DURATION`], regardless of the [`MetaAddrChange`]s that are
/// [`MIN_PEER_RECONNECTION_DELAY`], regardless of the [`MetaAddrChange`]s that are
/// applied to a single peer's entries in the [`AddressBook`].
///
/// This is the complex version of the test, which checks [`MetaAddr`],
@ -323,7 +323,7 @@ proptest! {
// Run the test for this many simulated live peer durations
const LIVE_PEER_INTERVALS: u32 = 3;
// Run the test for this much simulated time
let overall_test_time: Duration = LIVE_PEER_DURATION * LIVE_PEER_INTERVALS;
let overall_test_time: Duration = MIN_PEER_RECONNECTION_DELAY * LIVE_PEER_INTERVALS;
// Advance the clock by this much for every peer change
let peer_change_interval: Duration = overall_test_time / MAX_ADDR_CHANGE.try_into().unwrap();
@ -355,9 +355,9 @@ proptest! {
tokio::time::pause();
// The earliest time we can have a valid next attempt for this peer
let earliest_next_attempt = Instant::now() + LIVE_PEER_DURATION;
let earliest_next_attempt = Instant::now() + MIN_PEER_RECONNECTION_DELAY;
// The number of attempts for this peer in the last LIVE_PEER_DURATION
// The number of attempts for this peer in the last MIN_PEER_RECONNECTION_DELAY
let mut attempt_count: usize = 0;
for (i, change) in changes.into_iter().enumerate() {
@ -415,7 +415,7 @@ proptest! {
// Run the test for this many simulated live peer durations
const LIVE_PEER_INTERVALS: u32 = 3;
// Run the test for this much simulated time
let overall_test_time: Duration = LIVE_PEER_DURATION * LIVE_PEER_INTERVALS;
let overall_test_time: Duration = MIN_PEER_RECONNECTION_DELAY * LIVE_PEER_INTERVALS;
// Advance the clock by this much for every peer change
let peer_change_interval: Duration = overall_test_time / MAX_ADDR_CHANGE.try_into().unwrap();
@ -441,7 +441,7 @@ proptest! {
let addr = addrs.entry(addr.addr).or_insert(*addr);
let change = changes.get(change_index);
while addr.is_ready_for_attempt() {
while addr.is_ready_for_connection_attempt() {
*attempt_counts.entry(addr.addr).or_default() += 1;
assert!(*attempt_counts.get(&addr.addr).unwrap() <= LIVE_PEER_INTERVALS + 1);

View File

@ -1,6 +1,17 @@
//! Test vectors for MetaAddr.
use std::net::SocketAddr;
use zebra_chain::serialization::{DateTime32, Duration32};
use super::{super::MetaAddr, check};
use crate::{constants::MAX_PEER_ACTIVE_FOR_GOSSIP, protocol::types::PeerServices};
/// Margin of error for time-based tests.
///
/// This is a short duration to consider as error due to a test's execution time when comparing
/// [`DateTime32`]s.
const TEST_TIME_ERROR_MARGIN: Duration32 = Duration32::from_seconds(1);
/// Make sure that the sanitize function handles minimum and maximum times.
#[test]
@ -34,3 +45,165 @@ fn sanitize_extremes() {
check::sanitize_avoids_leaks(&max_time_entry, &max_sanitized);
}
}
/// Test if a newly created local listening address is gossipable.
///
/// The local listener [`MetaAddr`] is always considered gossipable.
#[test]
fn new_local_listener_is_gossipable() {
zebra_test::init();
let address = SocketAddr::from(([192, 168, 180, 9], 10_000));
let peer = MetaAddr::new_local_listener_change(&address)
.into_new_meta_addr()
.expect("MetaAddrChange can't create a new MetaAddr");
assert!(peer.is_active_for_gossip());
}
/// Test if a recently received alternate peer address is not gossipable.
///
/// Such [`MetaAddr`] is only considered gossipable after Zebra has tried to connect to it and
/// confirmed that the address is reachable.
#[test]
fn new_alternate_peer_address_is_not_gossipable() {
zebra_test::init();
let address = SocketAddr::from(([192, 168, 180, 9], 10_000));
let peer = MetaAddr::new_alternate(&address, &PeerServices::NODE_NETWORK)
.into_new_meta_addr()
.expect("MetaAddrChange can't create a new MetaAddr");
assert!(!peer.is_active_for_gossip());
}
/// Test if recently received gossiped peer is gossipable.
#[test]
fn gossiped_peer_reportedly_to_be_seen_recently_is_gossipable() {
zebra_test::init();
let address = SocketAddr::from(([192, 168, 180, 9], 10_000));
// Report last seen within the reachable interval.
let offset = MAX_PEER_ACTIVE_FOR_GOSSIP
.checked_sub(TEST_TIME_ERROR_MARGIN)
.expect("Test margin is too large");
let last_seen = DateTime32::now()
.checked_sub(offset)
.expect("Offset is too large");
let peer = MetaAddr::new_gossiped_meta_addr(address, PeerServices::NODE_NETWORK, last_seen);
assert!(peer.is_active_for_gossip());
}
/// Test if received gossiped peer that was reportedly last seen in the future is gossipable.
#[test]
fn gossiped_peer_reportedly_seen_in_the_future_is_gossipable() {
zebra_test::init();
let address = SocketAddr::from(([192, 168, 180, 9], 10_000));
// Report last seen in the future
let last_seen = DateTime32::now()
.checked_add(MAX_PEER_ACTIVE_FOR_GOSSIP)
.expect("Reachable peer duration is too large");
let peer = MetaAddr::new_gossiped_meta_addr(address, PeerServices::NODE_NETWORK, last_seen);
assert!(peer.is_active_for_gossip());
}
/// Test if gossiped peer that was reported last seen a long time ago is not gossipable.
#[test]
fn gossiped_peer_reportedly_seen_long_ago_is_not_gossipable() {
zebra_test::init();
let address = SocketAddr::from(([192, 168, 180, 9], 10_000));
// Report last seen just outside the reachable interval.
let offset = MAX_PEER_ACTIVE_FOR_GOSSIP
.checked_add(TEST_TIME_ERROR_MARGIN)
.expect("Test margin is too large");
let last_seen = DateTime32::now()
.checked_sub(offset)
.expect("Offset is too large");
let peer = MetaAddr::new_gossiped_meta_addr(address, PeerServices::NODE_NETWORK, last_seen);
assert!(!peer.is_active_for_gossip());
}
/// Test that peer that has just responded is gossipable.
#[test]
fn recently_responded_peer_is_gossipable() {
zebra_test::init();
let address = SocketAddr::from(([192, 168, 180, 9], 10_000));
let peer_seed = MetaAddr::new_alternate(&address, &PeerServices::NODE_NETWORK)
.into_new_meta_addr()
.expect("MetaAddrChange can't create a new MetaAddr");
// Create a peer that has responded
let peer = MetaAddr::new_responded(&address, &PeerServices::NODE_NETWORK)
.apply_to_meta_addr(peer_seed)
.expect("Failed to create MetaAddr for responded peer");
assert!(peer.is_active_for_gossip());
}
/// Test that peer that last responded in the reachable interval is gossipable.
#[test]
fn not_so_recently_responded_peer_is_still_gossipable() {
zebra_test::init();
let address = SocketAddr::from(([192, 168, 180, 9], 10_000));
let peer_seed = MetaAddr::new_alternate(&address, &PeerServices::NODE_NETWORK)
.into_new_meta_addr()
.expect("MetaAddrChange can't create a new MetaAddr");
// Create a peer that has responded
let mut peer = MetaAddr::new_responded(&address, &PeerServices::NODE_NETWORK)
.apply_to_meta_addr(peer_seed)
.expect("Failed to create MetaAddr for responded peer");
// Tweak the peer's last response time to be within the limits of the reachable duration
let offset = MAX_PEER_ACTIVE_FOR_GOSSIP
.checked_sub(TEST_TIME_ERROR_MARGIN)
.expect("Test margin is too large");
let last_response = DateTime32::now()
.checked_sub(offset)
.expect("Offset is too large");
peer.set_last_response(last_response);
assert!(peer.is_active_for_gossip());
}
/// Test that peer that responded long ago is not gossipable.
#[test]
fn responded_long_ago_peer_is_not_gossipable() {
zebra_test::init();
let address = SocketAddr::from(([192, 168, 180, 9], 10_000));
let peer_seed = MetaAddr::new_alternate(&address, &PeerServices::NODE_NETWORK)
.into_new_meta_addr()
.expect("MetaAddrChange can't create a new MetaAddr");
// Create a peer that has responded
let mut peer = MetaAddr::new_responded(&address, &PeerServices::NODE_NETWORK)
.apply_to_meta_addr(peer_seed)
.expect("Failed to create MetaAddr for responded peer");
// Tweak the peer's last response time to be outside the limits of the reachable duration
let offset = MAX_PEER_ACTIVE_FOR_GOSSIP
.checked_add(TEST_TIME_ERROR_MARGIN)
.expect("Test margin is too large");
let last_response = DateTime32::now()
.checked_sub(offset)
.expect("Offset is too large");
peer.set_last_response(last_response);
assert!(!peer.is_active_for_gossip());
}

View File

@ -70,7 +70,7 @@ mod tests;
/// ││ ▼ ││
/// ││ Λ ││
/// ││ ╲ filter by ││
/// ││ ▕ ▏ is_ready_for_attempt ││
/// ││ ▕ ▏ is_ready_for_connection_attempt ││
/// ││ ╲ to remove recent `Responded`, ││
/// ││ V `AttemptPending`, and `Failed` peers ││
/// ││ │ ││