Commit Graph

260 Commits

Author SHA1 Message Date
Henry de Valence 29f901add3 Rename Response::Ok to Response::Nil.
This is a better name because it signals "no data in response" rather
than "Ok", which is semantically mixed with `Ok/Err` of `Result`.
2020-02-10 09:03:56 -08:00
Henry de Valence 5929e05e52 Remove `PushPeers` and ignore unsolicited `addr` messages.
PushPeers is more complicated to thread into the rest of our
architecture (we would need to establish a data path connecting our
service handling inbound requests to the network layer's auto-crawler),
and since we crawl the network automatically anyways, we don't actually
need to accept them in order to get updated address information.

The only possible problem with this approach is that zcashd refuses to
answer multiple address requests from the same connection, ostensibly
for fingerprinting prevention (although it's totally happy to give
exactly the same information, as long as you hang up and reconnect
first, lol).  It's unclear how this will interact with our design -- on
the one hand, it could mean that we don't get new addr information when
we ask, but on the other hand, we may have enough churn in our
connection pool that this isn't a problem anyways.
2020-02-10 09:03:56 -08:00
Henry de Valence 2c0f48b587 Refactor connection logic and try a block request.
Attempting to implement requests for block data revealed a problem with
the previous connection logic.  Block data is requested by sending a
`getdata` message with hashes of the requested blocks; the peer responds
with a sequence of `block` messages with the blocks themselves.

However, this wasn't possible to handle with the previous connection
logic, which could only convert a single Bitcoin message into a
Response.  Instead, we factor out the message handling logic into a
Handler, which can statefully accumulate arbitrary data into a Response
and signal completion.  This is still pretty ugly but it does work.

As a side effect, the HeartbeatNonceMismatch error is removed; because
the Handler now tries to process messages until it comes to a Response,
it just ignores mismatched nonces (and will eventually time out).

The previous Mempool and Transaction requests were removed but could be
re-added in a different form later.  Also, the `Get` prefixes are
removed from `Request` to tidy the name.
2020-02-10 09:03:56 -08:00
Henry de Valence 972d16518f Make ZcashSerialize infallible mod its Writer.
Closes #158.

As discussed on the issue, this makes it possible to safely serialize
data into hashes, and encourages serializable data to make illegal
states unrepresentable.
2020-02-05 19:48:43 -05:00
Henry de Valence b0f61c4dd2 Remove outdated comment (we use tokio codecs now) 2020-02-05 19:44:35 -05:00
Henry de Valence ab94acf7da fmt 2020-02-05 19:44:35 -05:00
Henry de Valence eeb4a2470b Remove version fields from Block, Tx messages.
These are included in the Block, Transaction objects themselves, so the
previous code ended up trying to deserialize two version fields per
object.

Closes #226.
2020-02-05 19:44:35 -05:00
Henry de Valence 51c744b1ae Update network version number. 2020-02-05 14:06:35 -08:00
Henry de Valence 8d58dd804f Note that tracing causes clippy false positives
Thanks @hawkw for pointing this out.
2020-02-05 12:42:32 -08:00
Henry de Valence f04f4f0b98 Apply clippy fixes 2020-02-05 12:42:32 -08:00
dependabot-preview[bot] 979cf7ac6d Bump indexmap from 1.3.1 to 1.3.2
Bumps [indexmap](https://github.com/bluss/indexmap) from 1.3.1 to 1.3.2.
- [Release notes](https://github.com/bluss/indexmap/releases)
- [Commits](https://github.com/bluss/indexmap/compare/1.3.1...1.3.2)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-02-05 14:14:51 -05:00
Henry de Valence b24f53f4a1 Add From impls for InventoryHash 2020-02-04 17:48:36 -08:00
Deirdre Connolly 6d3d4c4f64 s/GetData/NotFound/ in read_notfound 2020-02-04 18:04:53 -05:00
Deirdre Connolly 1ca55846eb Little test to exercise sha256dWriter::flush() 2020-02-04 18:04:53 -05:00
Deirdre Connolly beb72080cb Delete out of date comment on incomplete Message variants 2020-01-30 13:39:13 -05:00
Deirdre Connolly 53a7af82a0 Add back a missing quotemark
Co-Authored-By: Henry de Valence <hdevalence@hdevalence.ca>
2020-01-28 03:48:23 -05:00
Deirdre Connolly 71d5571e39 Add roundtrip proptest for LockTime serialization/deserialization
Relates to #150
2020-01-28 03:48:23 -05:00
Deirdre Connolly d8ebeea08c Add proptest regressions file 2020-01-28 03:48:23 -05:00
Deirdre Connolly c2411f4315 Add a little proptest around Magic's Debug impl 2020-01-28 03:48:23 -05:00
Deirdre Connolly 9709b54c57 Fix proptest-regressions locations after reorganizations of files 2020-01-28 03:48:23 -05:00
Henry de Valence bea7b195ce Remove vestigial should_be_private module. 2020-01-16 19:27:08 -05:00
Henry de Valence 7cc44f4fa9 Move server.rs to connection.rs and change imports. 2020-01-16 13:20:03 -05:00
Henry de Valence 77ad61331c Rename `peer::Server` to `peer::Connection`.
This doesn't change the file path or edit imports so that the diff is easier to review.
2020-01-16 13:20:03 -05:00
Deirdre Connolly a3b09c3690
Target crates.io 0.3 versions of tower and tower-load (#172) 2020-01-14 17:18:14 -05:00
Henry de Valence 92ddf0542f Provide impl Zcash[De]Serialize for Vec<T: Zcash[De]Serialize>.
This replaces the read_list function and makes the code significantly cleaner.

The only downside is that it loses exact preallocation, but this is probably not a big deal.
2019-12-31 02:46:39 -05:00
Henry de Valence 2965187b91 Upgrade tokio, futures, hyper to released versions. 2019-12-13 17:42:15 -05:00
Deirdre Connolly 82e246d87b
Merge pull request #135 from ZcashFoundation/130
On receipt of a Filter(Load|Add|Clear) message, disconnect from peer
2019-12-05 14:06:05 -05:00
Deirdre Connolly d559a3fa52 Return an error result if the filterload msg size is out of bounds 2019-12-05 13:48:42 -05:00
Deirdre Connolly cbd95dfc1f Pass body_len to read_filterload instead of calculating it ourself
This lets us still be generic over the reader trait and we aren't recomputing  again.

Co-Authored-By: Henry de Valence <hdevalence@hdevalence.ca>
2019-12-05 13:48:42 -05:00
Deirdre Connolly 3fbfffeb44 Remove superfluous zcash_(de)serialize impls 2019-12-05 13:48:42 -05:00
Deirdre Connolly 9a0c2198aa Parse out exact/max possible lengths for filterload filters
Add some unit tests that make sure we obey max values.
2019-12-05 13:48:42 -05:00
Deirdre Connolly 78f0b8aab0 Remove NODE_BLOOM service bit
We do not support bloom filters.
2019-12-05 13:48:42 -05:00
Deirdre Connolly 3c26092b01 Read max of 520 bytes for data fields when parsing filteradd messages 2019-12-05 13:48:42 -05:00
Deirdre Connolly 1b8b4d0fac Encode and decode Filter* messages
Also remove stubs related to MerkleBlock and the catchall case for unknown types since we
are finally matching all variants of the Message enum.
2019-12-05 13:48:42 -05:00
Deirdre Connolly 72def27b78 Add FilterLoad, FilterAdd, FilterClear message variants
Also remove MerkleBlock stub, we are no longer supporting it (zcashd doesn't either).
2019-12-05 13:48:42 -05:00
Deirdre Connolly f5aa5f3794 Add Tweak and Filter types 2019-12-05 13:48:42 -05:00
Henry de Valence 36cd6d6e06 cargo fmt 2019-11-27 23:53:36 -05:00
Henry de Valence f58aaac1ae Privately re-export ErrorSlot, ClientRequest in peer.
This means that all sub-modules of `peer` can import everything they need from
the `peer` module itself, without having to be aware of the internal structure
of their sibling modules.
2019-11-27 23:53:36 -05:00
Henry de Valence ad6525574b Rename PeerConnector -> peer::Connector 2019-11-27 23:53:36 -05:00
Henry de Valence 778e49b127 Rename PeerHandshake -> peer::Handshake 2019-11-27 23:53:36 -05:00
Henry de Valence 9ff0fd90dc Rename ServerState -> State.
There's no need to write `Server` every time because it's only used inside of
the server code, and when the handshake service constructs a Server.
2019-11-27 23:53:36 -05:00
Henry de Valence d1b3e8fe6b Rename PeerServer -> peer::Server 2019-11-27 23:53:36 -05:00
Henry de Valence 77191e62f6 Remove outdated fixup note. 2019-11-27 23:53:36 -05:00
Henry de Valence da78603d3a Rename `PeerClient` to `peer::Client`. 2019-11-27 23:53:36 -05:00
Henry de Valence 4fbc8270a2 Move PeerSet initialization into a submodule. 2019-11-27 05:06:01 -05:00
Henry de Valence 6db852fab2 Refactor protocol into internal, external modules.
This commit just moves things around and patches import paths.
2019-11-27 05:06:01 -05:00
Deirdre Connolly 8a9a5ba29b Revert "Add some simple proptests using the Arbitray trait on Requests and Responses, gated to test only"
This reverts commit 5a123acf56.
2019-11-26 19:35:49 -05:00
Deirdre Connolly 6f52fc7773 Add 'Other' opcode matching when parsing Reject messages 2019-11-26 19:35:49 -05:00
Deirdre Connolly dd042cf4d8 Add some simple proptests using the Arbitray trait on Requests and Responses, gated to test only 2019-11-26 19:35:49 -05:00
Deirdre Connolly 6168cb51d7 Prefixed currently unused error variable with underscore 2019-11-26 19:35:49 -05:00
Deirdre Connolly d78ead4a1a Removed unused trait import 2019-11-26 19:35:49 -05:00
Deirdre Connolly b9c27e5683 Handle Response::Error and send Message::Reject generated from a PeerError::Rejected 2019-11-26 19:35:49 -05:00
Deirdre Connolly f5f1fe9bbc Handle incoming Reject messages when we expect a response 2019-11-26 19:35:49 -05:00
Deirdre Connolly efd37300b9 Impl From trait for Responses from generic Error impls
Also include new Response::Error variant.
2019-11-26 19:35:49 -05:00
Deirdre Connolly 9cdef4acf0 Implement From trait for generic Error impls
Also add 'Other' RejectCode variant.
2019-11-26 19:35:49 -05:00
Deirdre Connolly 49c5265d41 Add Rejected variant to PeerError enum, for now 2019-11-26 19:35:49 -05:00
Deirdre Connolly bae9347f6e Rustfmt 2019-11-26 19:35:49 -05:00
Deirdre Connolly ed77aaacd9 Read and write Reject messages
Also change  field type to  as it's unclear if it actually has
to match the set of enum variants we care about.
2019-11-26 19:35:49 -05:00
Deirdre Connolly 189d89a7fc Handle 'mempool' messages as 'GetMempool' requests
With a 'Transactions' response that gets turned into an 'Inv(Vec<InventoryHash::Tx>)' message.

We don't yet handle a response from our peer for a 'mempool', which will have to be
a more generic 'Inv' type because we might receive transaction hashes we don't know about yet.

Pertains to #26
2019-11-18 15:55:25 -05:00
Deirdre Connolly 98079c9d77 Support Mempool message
This does not yet push requests into services that actually respond with transaction
hashes in our node's mempool, which doesn't exist yet.

Pertains to #26
2019-11-18 15:55:25 -05:00
Henry de Valence e63f8b51b1 Fix default port setting. 2019-11-13 18:47:44 -05:00
Henry de Valence 2ac77ab704 fmt 2019-11-13 18:43:18 -05:00
Henry de Valence d170608c13 Remove outdated note.
The MetaAddr type is used not just in the Bitcoin code so it doesn't make sense
to keep it in the Bitcoin protocol implementation.
2019-11-13 18:43:18 -05:00
Henry de Valence 9a0bffecb8 Sanitize outbound address responses.
This aims to prevent a remote peer from inspecting timings of all messages
received by this node.
2019-11-13 18:43:18 -05:00
Deirdre Connolly 910268a45b Remove to-fix comment 2019-11-12 22:39:47 -05:00
Deirdre Connolly 4d3ab201e6 seed command seems to be functional
Moved SeedService out of the command closure Command currently spawns
a tokio task to DOS the seed service with `Request::GetPeers` every
second.

Pertains to #54
2019-11-12 22:39:47 -05:00
Deirdre Connolly 0ac1b663fe Keep sets of initial peers as Strings in config file 2019-11-12 22:39:47 -05:00
Deirdre Connolly b5bbef5c47 Default init seed nodes based on network choice
And more flushed out but incomplete
2019-11-12 22:39:47 -05:00
Henry de Valence 47ec2e2689 Remove stub discover module. 2019-10-22 19:06:08 -07:00
Henry de Valence c3ec235a5b Suppress unused import warnings. 2019-10-22 19:06:08 -07:00
Henry de Valence ed2ee9d42f Add a PeerConnector wrapper around PeerHandshake 2019-10-22 19:06:08 -07:00
Henry de Valence 9e2678d76c Rename PeerConnector to PeerHandshake.
It's only responsible for doing the handshakes, so it should be named that way,
and then we can have a Connector responsible for actually opening the TCP
connection.
2019-10-22 19:06:08 -07:00
Henry de Valence 121cea610b Unlink peer spans from their creation details. 2019-10-22 19:06:08 -07:00
Henry de Valence 6faebe844c Make zebra-network::Config toml-able.
The toml serializer function we are using -- maybe because of to_string_pretty
(?) barfs on structs that mix ordering of simple values and "tables", so just
keep all the Durations to the end.
2019-10-22 19:06:08 -07:00
Henry de Valence 0833d31ec7 Use a timer to add peers by interval. 2019-10-22 19:06:08 -07:00
Henry de Valence 9a779a639f Add a timeout for peer handshakes. 2019-10-22 19:06:08 -07:00
Henry de Valence 027bdc8465 Rework initial crawler logic.
This splits out the connection handling code into a try_connect closure, which
could be refactored into a Service of its own.

On creation, when we are likely to have very few peers, launch many concurrent
connections to the first few candidates in the initial candidate set, before
continuing to grow the peer set according to demand signals.
2019-10-22 19:06:08 -07:00
Henry de Valence e1a35490af Move the CandidateSet to its own file.
Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>
2019-10-22 19:06:08 -07:00
Henry de Valence 2f3292759f Add an explicit tracing span to each address book.
Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>
2019-10-22 19:06:08 -07:00
Henry de Valence e0e17a4719 Send an initial request for peers on launching crawler.
Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>
2019-10-22 19:06:08 -07:00
Henry de Valence b1832ce593 Initial work to add a crawl-and-dial task.
This responds to peerset demand by connecting to additional peers.

Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>
2019-10-22 19:06:08 -07:00
Henry de Valence 4055eb8889 bugfix: ensure the PeerServer always calls fail_with before exit
This caused a panic in the PeerSet when remote peers disconnected from us.
2019-10-22 17:55:09 -04:00
Deirdre Connolly 32aea0cbf9 Add useful docstrings 2019-10-21 15:55:18 -04:00
Deirdre Connolly 3de34290e6 Construct LIVE_PEER_DURATION from other timeout and interval constants
Use constants::HEARTBEAT_INTERVAL in our ping generator, add a test to check that LIVE_PEER_DURATION
is consistent with the other constants.
2019-10-21 15:55:18 -04:00
Deirdre Connolly 8588c44bcf Add a comment about not cloning the server channel any more than we are 2019-10-21 15:55:18 -04:00
Deirdre Connolly 61a07c67ef Inside tokio::spawn, loop over Iterator stream and send ClientRequest
msgs on the channel instead

Related to #49
2019-10-21 15:55:18 -04:00
Deirdre Connolly e65f5a05ea Broken: I can't seem to return either an impl Future or Result to satisfy for_each 2019-10-21 15:55:18 -04:00
Deirdre Connolly 3548998980 Set server state to Failed if a response to a heartbeat Ping never comes 2019-10-21 15:55:18 -04:00
Deirdre Connolly adffc4239d Partially complete heartbeats to peer 2019-10-21 15:55:18 -04:00
Henry de Valence 62e423bad8 Update zebra-network/src/address_book.rs
Co-Authored-By: Deirdre Connolly <deirdre@zfnd.org>
2019-10-21 14:40:03 -04:00
Henry de Valence a8ef02c826 Refactor AddressBook::update, add contains, get.
This also makes the quadratic `assert_consistency` check run only in
test configs.
2019-10-21 14:40:03 -04:00
Henry de Valence 501db9fcc8 Add AddressBook::is_potentially_connected()
This allows checking whether a SocketAddr could potentially be
connected, based on the contents of the address book.
2019-10-21 14:40:03 -04:00
Henry de Valence fe9cef261d Allow draining AddressBook entries oldest-first. 2019-10-21 14:40:03 -04:00
Henry de Valence 39d38a8647 Rewrite AddressBook to use a BTreeSet.
The previous implementation failed when timestamps were duplicated between
peers, because there was not a 1-1 relationship between timestamps and peers.
2019-10-21 14:40:03 -04:00
Henry de Valence ecd57f43ed Implement Extend and Drain for AddressBook. 2019-10-21 14:40:03 -04:00
Henry de Valence 0bfd57def2 Add iteration functions to `AddressBook`.
The disconnected_peers() function allows us to prevent duplicate
connections without maintaining shared state between the peerset and the
dial-additional-peers task.
2019-10-21 14:40:03 -04:00
Henry de Valence b03a83fa86 Simplify TimestampCollector.
Previously, the TimestampCollector was intended to own the address book
data, so it was intended to be cloneable and hold shared state among all
of its handles.  This is now modeled more directly by an
`Arc<Mutex<AddressBook>>`, so the only functionality left in the
`TimestampCollector` is setting up the inital worker, which is better
called `spawn` than `new`.

This also fixes a problem introduced in the previous commit where the
`TimestampCollector` was dropped, causing the worker task to shut down
early.
2019-10-21 14:40:03 -04:00
Henry de Valence 53be838d51 Extract `TimestampData` to `AddressBook`.
This allows us to hide the `TimestampCollector` and to expose only the
address book data required by the inbound request service.  It also lets
us have a common data structure (the `AddressBook`) for collecting peer
information that can be used to manage information that other peers
report to us.
2019-10-21 14:40:03 -04:00
Henry de Valence 15a698b23c Use MetaAddr in the timestamp collector.
We will need service bits information to report on peer addresses, so we
need to collect it in the timestamp collector.
2019-10-21 14:40:03 -04:00
Henry de Valence ad43a61fb4 Ensure that all types appearing in public types are exported. 2019-10-18 16:11:01 -07:00