Commit Graph

198 Commits

Author SHA1 Message Date
Henry de Valence d170608c13 Remove outdated note.
The MetaAddr type is used not just in the Bitcoin code so it doesn't make sense
to keep it in the Bitcoin protocol implementation.
2019-11-13 18:43:18 -05:00
Henry de Valence 9a0bffecb8 Sanitize outbound address responses.
This aims to prevent a remote peer from inspecting timings of all messages
received by this node.
2019-11-13 18:43:18 -05:00
Deirdre Connolly 910268a45b Remove to-fix comment 2019-11-12 22:39:47 -05:00
Deirdre Connolly 4d3ab201e6 seed command seems to be functional
Moved SeedService out of the command closure Command currently spawns
a tokio task to DOS the seed service with `Request::GetPeers` every
second.

Pertains to #54
2019-11-12 22:39:47 -05:00
Deirdre Connolly 0ac1b663fe Keep sets of initial peers as Strings in config file 2019-11-12 22:39:47 -05:00
Deirdre Connolly b5bbef5c47 Default init seed nodes based on network choice
And more flushed out but incomplete
2019-11-12 22:39:47 -05:00
Henry de Valence 47ec2e2689 Remove stub discover module. 2019-10-22 19:06:08 -07:00
Henry de Valence c3ec235a5b Suppress unused import warnings. 2019-10-22 19:06:08 -07:00
Henry de Valence ed2ee9d42f Add a PeerConnector wrapper around PeerHandshake 2019-10-22 19:06:08 -07:00
Henry de Valence 9e2678d76c Rename PeerConnector to PeerHandshake.
It's only responsible for doing the handshakes, so it should be named that way,
and then we can have a Connector responsible for actually opening the TCP
connection.
2019-10-22 19:06:08 -07:00
Henry de Valence 121cea610b Unlink peer spans from their creation details. 2019-10-22 19:06:08 -07:00
Henry de Valence 6faebe844c Make zebra-network::Config toml-able.
The toml serializer function we are using -- maybe because of to_string_pretty
(?) barfs on structs that mix ordering of simple values and "tables", so just
keep all the Durations to the end.
2019-10-22 19:06:08 -07:00
Henry de Valence 0833d31ec7 Use a timer to add peers by interval. 2019-10-22 19:06:08 -07:00
Henry de Valence 9a779a639f Add a timeout for peer handshakes. 2019-10-22 19:06:08 -07:00
Henry de Valence 027bdc8465 Rework initial crawler logic.
This splits out the connection handling code into a try_connect closure, which
could be refactored into a Service of its own.

On creation, when we are likely to have very few peers, launch many concurrent
connections to the first few candidates in the initial candidate set, before
continuing to grow the peer set according to demand signals.
2019-10-22 19:06:08 -07:00
Henry de Valence e1a35490af Move the CandidateSet to its own file.
Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>
2019-10-22 19:06:08 -07:00
Henry de Valence 2f3292759f Add an explicit tracing span to each address book.
Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>
2019-10-22 19:06:08 -07:00
Henry de Valence e0e17a4719 Send an initial request for peers on launching crawler.
Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>
2019-10-22 19:06:08 -07:00
Henry de Valence b1832ce593 Initial work to add a crawl-and-dial task.
This responds to peerset demand by connecting to additional peers.

Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>
2019-10-22 19:06:08 -07:00
Henry de Valence 4055eb8889 bugfix: ensure the PeerServer always calls fail_with before exit
This caused a panic in the PeerSet when remote peers disconnected from us.
2019-10-22 17:55:09 -04:00
Deirdre Connolly 32aea0cbf9 Add useful docstrings 2019-10-21 15:55:18 -04:00
Deirdre Connolly 3de34290e6 Construct LIVE_PEER_DURATION from other timeout and interval constants
Use constants::HEARTBEAT_INTERVAL in our ping generator, add a test to check that LIVE_PEER_DURATION
is consistent with the other constants.
2019-10-21 15:55:18 -04:00
Deirdre Connolly 8588c44bcf Add a comment about not cloning the server channel any more than we are 2019-10-21 15:55:18 -04:00
Deirdre Connolly 61a07c67ef Inside tokio::spawn, loop over Iterator stream and send ClientRequest
msgs on the channel instead

Related to #49
2019-10-21 15:55:18 -04:00
Deirdre Connolly e65f5a05ea Broken: I can't seem to return either an impl Future or Result to satisfy for_each 2019-10-21 15:55:18 -04:00
Deirdre Connolly 3548998980 Set server state to Failed if a response to a heartbeat Ping never comes 2019-10-21 15:55:18 -04:00
Deirdre Connolly adffc4239d Partially complete heartbeats to peer 2019-10-21 15:55:18 -04:00
Henry de Valence 62e423bad8 Update zebra-network/src/address_book.rs
Co-Authored-By: Deirdre Connolly <deirdre@zfnd.org>
2019-10-21 14:40:03 -04:00
Henry de Valence a8ef02c826 Refactor AddressBook::update, add contains, get.
This also makes the quadratic `assert_consistency` check run only in
test configs.
2019-10-21 14:40:03 -04:00
Henry de Valence 501db9fcc8 Add AddressBook::is_potentially_connected()
This allows checking whether a SocketAddr could potentially be
connected, based on the contents of the address book.
2019-10-21 14:40:03 -04:00
Henry de Valence fe9cef261d Allow draining AddressBook entries oldest-first. 2019-10-21 14:40:03 -04:00
Henry de Valence 39d38a8647 Rewrite AddressBook to use a BTreeSet.
The previous implementation failed when timestamps were duplicated between
peers, because there was not a 1-1 relationship between timestamps and peers.
2019-10-21 14:40:03 -04:00
Henry de Valence ecd57f43ed Implement Extend and Drain for AddressBook. 2019-10-21 14:40:03 -04:00
Henry de Valence 0bfd57def2 Add iteration functions to `AddressBook`.
The disconnected_peers() function allows us to prevent duplicate
connections without maintaining shared state between the peerset and the
dial-additional-peers task.
2019-10-21 14:40:03 -04:00
Henry de Valence b03a83fa86 Simplify TimestampCollector.
Previously, the TimestampCollector was intended to own the address book
data, so it was intended to be cloneable and hold shared state among all
of its handles.  This is now modeled more directly by an
`Arc<Mutex<AddressBook>>`, so the only functionality left in the
`TimestampCollector` is setting up the inital worker, which is better
called `spawn` than `new`.

This also fixes a problem introduced in the previous commit where the
`TimestampCollector` was dropped, causing the worker task to shut down
early.
2019-10-21 14:40:03 -04:00
Henry de Valence 53be838d51 Extract `TimestampData` to `AddressBook`.
This allows us to hide the `TimestampCollector` and to expose only the
address book data required by the inbound request service.  It also lets
us have a common data structure (the `AddressBook`) for collecting peer
information that can be used to manage information that other peers
report to us.
2019-10-21 14:40:03 -04:00
Henry de Valence 15a698b23c Use MetaAddr in the timestamp collector.
We will need service bits information to report on peer addresses, so we
need to collect it in the timestamp collector.
2019-10-21 14:40:03 -04:00
Henry de Valence ad43a61fb4 Ensure that all types appearing in public types are exported. 2019-10-18 16:11:01 -07:00
Henry de Valence 1f0a7e5e73 Remove Codec::builder doctest.
Doctests can only test public API, so now that the Codec is private, we can't
have a doctest.  Since this test was only a code example (no behaviour test),
there's no value in replacing it with a unit test.
2019-10-18 16:11:01 -07:00
Henry de Valence c7e0d63bed Export the BoxedZebraService type alias.
This gives API consumers a convenient name, and makes the Rustdoc output
significantly cleaner (because `init` can return a `BoxedZebraService`, not a
`Box<dyn ...ManyTypeConstraints.......>`.
2019-10-18 16:11:01 -07:00
Henry de Valence 1c6c4140ed Add a nice comment to zebra-network. 2019-10-18 16:11:01 -07:00
Henry de Valence f0bb2bff77 Do not export zebra-network internals.
Until we finish outbound peer dialing, this still contains
one module that re-exports the `PeerConnector`.
2019-10-18 16:11:01 -07:00
Henry de Valence d4dc4f0d04 Add tracing output on unhandled peer requests 2019-10-18 16:11:01 -07:00
Henry de Valence 54948b07e2 Add a peer listener task. 2019-10-18 16:11:01 -07:00
Henry de Valence 5847b490da Move PeerSet setup logic into a peer_set::init() 2019-10-18 16:11:01 -07:00
Henry de Valence 63cf340ab4 Add fields to zebra-network Config. 2019-10-18 16:11:01 -07:00
Henry de Valence 0a56830eab Ascii-fy the command string in message header trace 2019-10-17 13:22:33 -04:00
Henry de Valence ff27334e81 Make PeerConnector tower::Buffer'able 2019-10-17 09:34:18 -07:00
Henry de Valence db7ac53f3b Add a Mutex<HashSet<Nonce>> to detect self-conns. 2019-10-17 09:34:18 -07:00
Henry de Valence ed335e68f4 Remove outdated comment
Now that the `PeerConnector` handles both incoming and outgoing
handshakes, determining the next peer address is definitely out of scope
-- it takes a pre-existing tcp connection.
2019-10-17 09:34:18 -07:00
Henry de Valence 31651cf87c Fix copy-paste bug in Codec
Co-authored-by: George Tankersley <george@zfnd.org>
2019-10-17 09:34:18 -07:00
Henry de Valence 8a1aa71736 Modify PeerConnector to also handle inbound conns.
Because the Bitcoin handshake is symmetric, we can reuse the same logic
for both incoming and outgoing connections.
2019-10-17 09:34:18 -07:00
Deirdre Connolly 980ce4593b Fix merge artifacts in Cargo.toml 2019-10-17 09:33:10 -07:00
Deirdre Connolly 0db69addec Add custom Debug impl for DecodeState
Relates to #63
2019-10-17 09:33:10 -07:00
Deirdre Connolly 051dc2f039 Implement custom Debug for Magic types
Related to #63
2019-10-17 09:33:10 -07:00
Henry de Valence f6e62b0f5e Remove failure from zebra-chain, zebra-network.
Failure uses a distinct Fail trait rather than the standard library's
Error trait, which causes a lot of interoperability problems with tower
and other Error-using crates.  Since failure was created, the standard
library's Error trait was improved, and its conveniences are now
available without the custom Fail trait using `thiserror` (for easy
error derives) and `anyhow` (for a better boxed Error).
2019-10-16 13:16:52 -04:00
Deirdre Connolly 199038e6b8 Rename bound vars in match arms for PeerServer state machine
Co-Authored-By: Henry de Valence <hdevalence@hdevalence.ca>
2019-10-15 14:49:11 -04:00
Henry de Valence fc872ea03f Remove async-await feature of futures. 2019-10-15 14:49:11 -04:00
Henry de Valence 373a8fbcfd Refactor PeerServer event loop to avoid select! 2019-10-15 14:49:11 -04:00
Henry de Valence 16f51e4d48 Add a timeout to the `PeerServer` event loop.
I think this code could be cleaned up significantly (e.g., removing the
other use of select!) but that's potentially a larger change than this
PR.
2019-10-15 14:49:11 -04:00
Henry de Valence b45efbdaf2
Add peer count data to TimestampData::update trace (#66)
* Add peer count data to TimestampData::update trace

* Update docstring typo
2019-10-11 12:41:37 -07:00
Henry de Valence ae1a164ff8
Beginning of peerset implementation. (#62)
* Don't expose submodules of zebra_network::peer.

* PeerSet, PeerDiscover stubs.

Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>

* Initial work on PeerSet.

This is adapted from the MIT-licensed tower-balance implementation.

* Use PeerSet in the connect stub.
2019-10-10 18:15:24 -07:00
Deirdre Connolly dc18e8f24c Remove .expect()s for block and transaction, they might fail for writer reasons 2019-10-09 22:25:37 -04:00
Deirdre Connolly 4ec7590b42 Add placeholders for remaining messages in write_body 2019-10-09 22:25:37 -04:00
Deirdre Connolly d8986098c6 Add write_body implementations for GetData, NotFound, and a placeholder for Reject 2019-10-09 22:25:37 -04:00
Deirdre Connolly 483d6584f9 Add Block msg reader and writers, and Tx msg writer 2019-10-09 22:25:37 -04:00
Deirdre Connolly 8dc9181610 Fill out write_body for GetBlocks, GetHeaders, Headers messages 2019-10-09 22:25:37 -04:00
Deirdre Connolly a0302a5507 Implement read_getaddr 2019-10-09 22:25:37 -04:00
Deirdre Connolly cfbe8dfdbf Add comments about the 500 blocks /2000 headers max numbers are speculative based on Bitcoin docs 2019-10-09 22:25:37 -04:00
Deirdre Connolly 5acdaa6d2f Remove some defunct XXXs 2019-10-09 22:25:37 -04:00
Deirdre Connolly dd1b9166b0 Implement Zcash(De)Serialize for BlockHeaderHash, use general read_list for getheaders/headers 2019-10-09 22:25:37 -04:00
Deirdre Connolly eed69063f6 Add read_list to ReadZcashExt 2019-10-09 22:25:37 -04:00
Deirdre Connolly d470dd9709 Parse messages 2019-10-09 22:25:37 -04:00
Deirdre Connolly b991c413cd Parse GetBlock messages 2019-10-09 22:25:37 -04:00
Deirdre Connolly 7632d5b8cc Abstract the common case of a message with a Vec<InventoryHash> 2019-10-09 22:25:37 -04:00
Deirdre Connolly 9699ef2fa1 Codec::read_tx() 2019-10-09 22:25:37 -04:00
Deirdre Connolly 565dc92afe Support a response message 2019-10-09 22:25:37 -04:00
Henry de Valence 8a3b4f48fc Move all git dependencies to published crates. (#58)
The earlier issues with version resolution incompatibilities are now
resolved, and all the upstream changes we made to tracing are released.
2019-10-09 00:10:01 -04:00
Henry de Valence 6f79f2843d Use emojis in user-agent to detect Unicode support. 2019-10-08 23:34:16 -04:00
Henry de Valence f284f6d6cf Tweak debug output in PeerConnector handshake. 2019-10-08 23:34:16 -04:00
Henry de Valence fb2b502eb9 Add a `Config` struct to zebra-network.
This struct is pulled into the main abscissa config as a subsection.
2019-10-08 23:34:16 -04:00
Henry de Valence 1266653be2
Handle error conversions properly. (#56)
This adds a type alias, BoxedStdError, for a boxed std::error::Error
trait object, and uses it in the where bounds for the generic service
code.  In the future, we may want to standardize on using
std::error::Error exclusively, but we would then possibly lose out on
backtrace information.
2019-10-08 13:49:12 -07:00
Henry de Valence 92dc7a5ea1
Fix authorship, license information. (#55)
* Fix authorship, license information.

I *thought* I had done a sed pass over the Cargo defaults when doing
repository initialization, but I guess I missed it or something.
Anyways, fixed now.
2019-10-08 09:25:59 -07:00
Henry de Valence ed608f7231
Initial tower-based peer implementation. (#17)
Add a tower-based peer implementation.  

Tower provides middleware for request-response oriented protocols, while Bitcoin/Zcash just send messages which could be interpreted either as requests or responses, depending on context.  To bridge this mismatch we define our own internal request/response protocol, and implement a per-peer event loop that scans incoming messages and interprets them either as requests from the remote peer to our node, or as responses to requests we made previously.  This is performed by the `PeerService` task, and a corresponding `PeerClient: tower::Service` can send it requests.  These tasks are themselves created by a `PeerConnector: tower::Service` which dials a remote peer and performs a handshake.
2019-10-07 15:36:16 -07:00
Henry de Valence f5dca597dd Replace PeerServices(u64) with a bitflags struct.
This gives considerably better ergonomics.
2019-10-01 01:07:56 -04:00
Henry de Valence 9603a29399 Rename `Services` to `PeerServices`.
This field is called `services` in Bitcoin and Zcash, but because we use
that word internally for other purposes, calling it `PeerServices`
disambiguates the meaning to "the services advertised by the peer",
rather than, e.g., a `tower::Service`.
2019-10-01 01:07:56 -04:00
Deirdre Connolly 2739970113 Fill out TransactionHash and rename
Resolves #35
2019-09-27 19:22:34 -07:00
Deirdre Connolly b21b09bf8e Moved stub TxHash into zebra_chain::transaction 2019-09-27 19:22:34 -07:00
Deirdre Connolly 29591df47e Use the BlockHeaderHash from zebra-chain in the Inv message parsing 2019-09-27 19:22:34 -07:00
Henry de Valence 0a85be285d Add addr, getaddr serialization. 2019-09-27 20:41:45 -04:00
Henry de Valence b426630613 Clarify comment in decoder state. 2019-09-27 20:41:45 -04:00
Henry de Valence 422c783a47 Rename Message::Inventory -> Message::Inv
I don't feel super strongly about this change, so I'm happy to drop it,
but it makes the parsing match statements line up nicely and aligns
naming with the naming used in Bitcoin.
2019-09-27 20:41:45 -04:00
Henry de Valence 958fca8e68 Parse inv messages, refactor inventory vectors.
This removes the inventory vector structs from `zebra-chain` (as they
are really part of the network protocol) and refactors them into a
single `InventoryHash` type.  This corresponds to Bitcoin's "inventory
vector" but with a different, better name (it's not a vector, it's just
a typed hash of some other item).
2019-09-27 20:41:45 -04:00
Henry de Valence ab06750db3 zebra-network: move types -> protocol::types
These types are used for protocol messages, so it makes more sense to
keep them scoped with the protocol handling, rather than other
networking logic.
2019-09-27 20:41:45 -04:00
Deirdre Connolly 35f03dc55d Make Message.Block just point at a Block type, in codec call block.zcash_serialize() 2019-09-26 23:41:25 -04:00
Deirdre Connolly 3f2a1b4f2c Move around MerkleTree* structs 2019-09-26 23:41:25 -04:00
Deirdre Connolly 677d53897f Use Vec<u8> for the equihash solution instead of [u8; 1344] for now 2019-09-26 23:41:25 -04:00
Deirdre Connolly cecbb1cc0a Fill out the Block Message type
Should we serialize out `Block` or leave explicit like so? ¯\_(ツ)_/¯
2019-09-26 23:41:25 -04:00
Henry de Valence 48a5054c87 Delete unused variable.
This is no longer required because the body reader methods have access
to the version via the codec state.
2019-09-25 14:59:47 -07:00
Henry de Valence 0196c2c4cd Place header encoding prior to body encoding. 2019-09-25 14:59:47 -07:00