Commit Graph

47 Commits

Author SHA1 Message Date
Jane Lusby 867dd0b475
Setup tracing-flame for use profiling zebrad (#436)
* Setup tracing-flame for use profiling zebrad

* start work on conditional flamegraph generation

* review time!

* update comments

* Update Cargo.toml

* disable default features for inferno

* reorganize

* missing one trait

* Apply suggestions from code review

* graceful shutdown!

* remove special case handling on ctrlc for cleanup

* rename signal fn to better represent its responsibility

* remove unused global hook for flushing flamegraph

* move tracing logic to the right file

* just copy linkerd's signal handling logic

* update book

* make zebrad app drop on shutdown normally

* Update zebrad/src/components/tokio.rs

Co-authored-by: teor <teor@riseup.net>

* Update zebrad/src/application.rs

Co-authored-by: teor <teor@riseup.net>

* Apply suggestions from code review

Co-authored-by: teor <teor@riseup.net>

* cleanup a little

* ooh yea there's an API for that

* setup env-filter for backup subscriber

* document env filter

* document return codes

* forgot to save

* Update book/src/applications/zebrad.md

Co-authored-by: teor <teor@riseup.net>

Co-authored-by: teor <teor@riseup.net>
2020-08-05 16:35:56 -07:00
dependabot[bot] 8e268150a7 build(deps): bump tracing-subscriber from 0.2.9 to 0.2.10
Bumps [tracing-subscriber](https://github.com/tokio-rs/tracing) from 0.2.9 to 0.2.10.
- [Release notes](https://github.com/tokio-rs/tracing/releases)
- [Commits](https://github.com/tokio-rs/tracing/compare/tracing-subscriber-0.2.9...tracing-subscriber-0.2.10)

Signed-off-by: dependabot[bot] <support@github.com>
2020-08-03 21:11:50 -04:00
Alfredo Garcia f2d7bb3177
Command execution tests (#690)
* add zebrad acceptance tests
* add custom command test helpers that work with kill
* add and use info event for start and seed commands
* combine conflicting tests into one test case

Co-authored-by: Jane Lusby <jane@zfnd.org>
2020-08-01 16:15:26 +10:00
Henry de Valence 4aa00ad216 Align crate versions and user-agent with NU numbers.
We had a brief discussion on discord and it seemed like we had consensus on the
following versioning policy:

* zebrad: match major version to NU version, so we will start by releasing
  zebrad 3.0.0;

* zebra-* libraries: start by matching zebrad's version, then increment major
  versions of each library as we need to make breaking changes (potentially
  faster than the zebrad version, always respecting semver but making no
  guarantees about the longevity of major releases).

This commit sets all of the crate versions to 3.0.0-alpha.0 -- the -alpha.0
marks it as a prerelease not subject to perfect adherence to compatibility
guarantees.
2020-07-24 11:46:37 -07:00
dependabot[bot] f7c59c99b5 build(deps): bump tracing-subscriber from 0.2.8 to 0.2.9
Bumps [tracing-subscriber](https://github.com/tokio-rs/tracing) from 0.2.8 to 0.2.9.
- [Release notes](https://github.com/tokio-rs/tracing/releases)
- [Commits](https://github.com/tokio-rs/tracing/compare/tracing-subscriber-0.2.8...tracing-subscriber-0.2.9)

Signed-off-by: dependabot[bot] <support@github.com>
2020-07-24 14:31:44 -04:00
Jane Lusby c1a1493159
use dirs crate for default location of state and config (#714)
* use dirs crate for default location of state and config
* panic if a path isn't specified for zebra-state
2020-07-23 21:12:20 +10:00
Jane Lusby a722cf33f7 enable new tracing instrumentation in tokio 2020-07-22 14:39:54 -04:00
dependabot[bot] 2208a6a22d build(deps): bump tracing-subscriber from 0.2.7 to 0.2.8
Bumps [tracing-subscriber](https://github.com/tokio-rs/tracing) from 0.2.7 to 0.2.8.
- [Release notes](https://github.com/tokio-rs/tracing/releases)
- [Commits](https://github.com/tokio-rs/tracing/compare/tracing-subscriber-0.2.7...tracing-subscriber-0.2.8)

Signed-off-by: dependabot[bot] <support@github.com>
2020-07-21 12:01:40 -04:00
dependabot[bot] c3fcac8a5c Bump hyper from 0.13.6 to 0.13.7
Bumps [hyper](https://github.com/hyperium/hyper) from 0.13.6 to 0.13.7.
- [Release notes](https://github.com/hyperium/hyper/releases)
- [Changelog](https://github.com/hyperium/hyper/blob/master/CHANGELOG.md)
- [Commits](https://github.com/hyperium/hyper/compare/v0.13.6...v0.13.7)

Signed-off-by: dependabot[bot] <support@github.com>
2020-07-14 10:40:53 -07:00
teor e1bb95603c
Put zebrad/Cargo.toml in a nicer order (#650) 2020-07-14 10:17:05 -07:00
dependabot[bot] a1d02c2606 Bump tracing-subscriber from 0.2.6 to 0.2.7
Bumps [tracing-subscriber](https://github.com/tokio-rs/tracing) from 0.2.6 to 0.2.7.
- [Release notes](https://github.com/tokio-rs/tracing/releases)
- [Commits](https://github.com/tokio-rs/tracing/compare/tracing-subscriber-0.2.6...tracing-subscriber-0.2.7)

Signed-off-by: dependabot[bot] <support@github.com>
2020-07-02 06:47:04 -04:00
Jane Lusby 7245d91fe9
fix block downloading to be parallelized and commited via the verifier (#540) 2020-06-30 09:42:09 -07:00
Henry de Valence 70241d3cad Fix broken git dependencies.
Pinning hashes means these won't break again in the future; they can always be updated.
2020-06-22 20:23:02 -07:00
Jane Lusby 246e7cd2a9
Start testing out new version of `eyre` and `color-eyre` in zebra (#526)
* port to new version of eyre without generics

* correctly setup color_eyre hooks

Co-authored-by: Jane Lusby <jane@zfnd.org>
2020-06-22 15:36:23 -07:00
dependabot[bot] f301de41fa Bump tracing-subscriber from 0.2.5 to 0.2.6
Bumps [tracing-subscriber](https://github.com/tokio-rs/tracing) from 0.2.5 to 0.2.6.
- [Release notes](https://github.com/tokio-rs/tracing/releases)
- [Commits](https://github.com/tokio-rs/tracing/compare/tracing-subscriber/-0.2.5...tracing-subscriber-0.2.6)

Signed-off-by: dependabot[bot] <support@github.com>
2020-06-22 12:01:47 -04:00
Jane Lusby 431f194c0f
propagate errors out of zebra_network::init (#435)
Prior to this change, the service returned by `zebra_network::init` would spawn background tasks that could silently fail, causing unexpected errors in the zebra_network service.

This change modifies the `PeerSet` that backs `zebra_network::init` to store all of the `JoinHandle`s for each background task it depends on. The `PeerSet` then checks this set of futures to see if any of them have exited with an error or a panic, and if they have it returns the error as part of `poll_ready`.
2020-06-09 12:24:28 -07:00
Deirdre Connolly 42cc55b0bb Remove testing tokio task
That fires 'GetPeers' requests at our running 'zebra seed'.
2020-06-08 19:26:23 -04:00
Deirdre Connolly 8f5e7c268b Request::Peers not GetPeers 2020-06-08 19:26:23 -04:00
Jane Lusby 18b4dbc16c
fix tracing configuration issues (#432) 2020-06-04 19:34:06 -07:00
dependabot-preview[bot] 07b7c711fb Bump color-eyre from 0.3.2 to 0.3.4
Bumps [color-eyre](https://github.com/yaahc/color-eyre) from 0.3.2 to 0.3.4.
- [Release notes](https://github.com/yaahc/color-eyre/releases)
- [Commits](https://github.com/yaahc/color-eyre/commits)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-06-02 22:55:02 -04:00
Jane Lusby e9af80b875
Add initial version of zebra-state (#414)
* rename zebra-storage to zebra-state

* Setup initial skeleton for zebra-state

* add test

* Apply suggestions from code review

Co-authored-by: Henry de Valence <hdevalence@hdevalence.ca>

* move shared test vectors to a common crate

Co-authored-by: Jane Lusby <jane@zfnd.org>
Co-authored-by: Henry de Valence <hdevalence@hdevalence.ca>
2020-06-02 16:16:17 -07:00
dependabot-preview[bot] 8e7d91b4a3 Bump eyre from 0.4.2 to 0.4.3
Bumps [eyre](https://github.com/yaahc/eyre) from 0.4.2 to 0.4.3.
- [Release notes](https://github.com/yaahc/eyre/releases)
- [Commits](https://github.com/yaahc/eyre/commits)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-06-02 18:48:50 -04:00
dependabot-preview[bot] a62b93d47c Bump hyper from 0.13.5 to 0.13.6
Bumps [hyper](https://github.com/hyperium/hyper) from 0.13.5 to 0.13.6.
- [Release notes](https://github.com/hyperium/hyper/releases)
- [Changelog](https://github.com/hyperium/hyper/blob/master/CHANGELOG.md)
- [Commits](https://github.com/hyperium/hyper/compare/v0.13.5...v0.13.6)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-05-30 00:29:05 -04:00
Jane Lusby da72c5a86a
switch from abscissa::Context to color-eyre (#409)
Co-authored-by: Jane Lusby <jane@zfnd.org>
2020-05-28 23:01:24 -04:00
dependabot-preview[bot] 46cb7c02f2 Bump once_cell from 1.3.1 to 1.4.0
Bumps [once_cell](https://github.com/matklad/once_cell) from 1.3.1 to 1.4.0.
- [Release notes](https://github.com/matklad/once_cell/releases)
- [Changelog](https://github.com/matklad/once_cell/blob/master/CHANGELOG.md)
- [Commits](https://github.com/matklad/once_cell/compare/v1.3.1...v1.4.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-05-13 22:28:18 -04:00
dependabot-preview[bot] 83ee4c2ca3 Bump hyper from 0.13.4 to 0.13.5
Bumps [hyper](https://github.com/hyperium/hyper) from 0.13.4 to 0.13.5.
- [Release notes](https://github.com/hyperium/hyper/releases)
- [Changelog](https://github.com/hyperium/hyper/blob/master/CHANGELOG.md)
- [Commits](https://github.com/hyperium/hyper/compare/v0.13.4...v0.13.5)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-04-18 08:43:40 +00:00
dependabot-preview[bot] 6867449ff4 Bump hyper from 0.13.3 to 0.13.4
Bumps [hyper](https://github.com/hyperium/hyper) from 0.13.3 to 0.13.4.
- [Release notes](https://github.com/hyperium/hyper/releases)
- [Changelog](https://github.com/hyperium/hyper/blob/master/CHANGELOG.md)
- [Commits](https://github.com/hyperium/hyper/compare/v0.13.3...v0.13.4)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-03-20 21:05:17 -04:00
dependabot-preview[bot] cf0348b4f5 Bump hyper from 0.13.2 to 0.13.3
Bumps [hyper](https://github.com/hyperium/hyper) from 0.13.2 to 0.13.3.
- [Release notes](https://github.com/hyperium/hyper/releases)
- [Changelog](https://github.com/hyperium/hyper/blob/master/CHANGELOG.md)
- [Commits](https://github.com/hyperium/hyper/compare/v0.13.2...v0.13.3)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-03-03 20:16:27 -05:00
Henry de Valence 75d3d44fb3 Metrics MVP: add two metrics and export them to Prometheus.
Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>
2020-02-14 20:14:05 -05:00
Henry de Valence 9273f83761 Remove tracing-subscriber.
We don't need to handle the subscriber directly since we upstreamed this
functionality into Abscissa.
2020-02-05 14:05:46 -08:00
dependabot-preview[bot] dd24dbece3 Bump hyper from 0.13.1 to 0.13.2
Bumps [hyper](https://github.com/hyperium/hyper) from 0.13.1 to 0.13.2.
- [Release notes](https://github.com/hyperium/hyper/releases)
- [Changelog](https://github.com/hyperium/hyper/blob/master/CHANGELOG.md)
- [Commits](https://github.com/hyperium/hyper/compare/v0.13.1...v0.13.2)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-01-29 17:25:34 -05:00
dependabot-preview[bot] dbe7427f58 Bump once_cell from 1.3.0 to 1.3.1
Bumps [once_cell](https://github.com/matklad/once_cell) from 1.3.0 to 1.3.1.
- [Release notes](https://github.com/matklad/once_cell/releases)
- [Changelog](https://github.com/matklad/once_cell/blob/master/CHANGELOG.md)
- [Commits](https://github.com/matklad/once_cell/compare/v1.3.0...v1.3.1)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-01-18 23:26:25 -05:00
Tony Arcieri 45eb81a204 Upgrade to Abscissa v0.5 2020-01-15 12:06:31 -08:00
Henry de Valence d3e954cd4a Remove vestigial tower git dep 2020-01-15 14:23:01 -05:00
dependabot-preview[bot] 3c77c6f685 Bump hyper from 0.13.0 to 0.13.1
Bumps [hyper](https://github.com/hyperium/hyper) from 0.13.0 to 0.13.1.
- [Release notes](https://github.com/hyperium/hyper/releases)
- [Changelog](https://github.com/hyperium/hyper/blob/master/CHANGELOG.md)
- [Commits](https://github.com/hyperium/hyper/compare/v0.13.0...v0.13.1)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-12-17 17:11:28 -05:00
Henry de Valence 2965187b91 Upgrade tokio, futures, hyper to released versions. 2019-12-13 17:42:15 -05:00
Henry de Valence 9a0bffecb8 Sanitize outbound address responses.
This aims to prevent a remote peer from inspecting timings of all messages
received by this node.
2019-11-13 18:43:18 -05:00
Henry de Valence fc872ea03f Remove async-await feature of futures. 2019-10-15 14:49:11 -04:00
Henry de Valence ae1a164ff8
Beginning of peerset implementation. (#62)
* Don't expose submodules of zebra_network::peer.

* PeerSet, PeerDiscover stubs.

Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>

* Initial work on PeerSet.

This is adapted from the MIT-licensed tower-balance implementation.

* Use PeerSet in the connect stub.
2019-10-10 18:15:24 -07:00
Henry de Valence 8a3b4f48fc Move all git dependencies to published crates. (#58)
The earlier issues with version resolution incompatibilities are now
resolved, and all the upstream changes we made to tracing are released.
2019-10-09 00:10:01 -04:00
Henry de Valence 92dc7a5ea1
Fix authorship, license information. (#55)
* Fix authorship, license information.

I *thought* I had done a sed pass over the Cargo defaults when doing
repository initialization, but I guess I missed it or something.
Anyways, fixed now.
2019-10-08 09:25:59 -07:00
Henry de Valence ed608f7231
Initial tower-based peer implementation. (#17)
Add a tower-based peer implementation.  

Tower provides middleware for request-response oriented protocols, while Bitcoin/Zcash just send messages which could be interpreted either as requests or responses, depending on context.  To bridge this mismatch we define our own internal request/response protocol, and implement a per-peer event loop that scans incoming messages and interprets them either as requests from the remote peer to our node, or as responses to requests we made previously.  This is performed by the `PeerService` task, and a corresponding `PeerClient: tower::Service` can send it requests.  These tasks are themselves created by a `PeerConnector: tower::Service` which dials a remote peer and performs a handshake.
2019-10-07 15:36:16 -07:00
Henry de Valence c8a3d47b56 Use tracing::instrument and monitor for messages. 2019-09-23 22:17:12 -04:00
Henry de Valence 15ca12a2f5 Add a `connect` command for testing.
With `./src/zcashd -debug=net -logips=1`:
```
2019-09-19 15:24:38 Added connection to 127.0.0.1:35932 peer=1
2019-09-19 15:24:38 connection from 127.0.0.1:35932 accepted
2019-09-19 15:24:38 socket closed
2019-09-19 15:24:38 disconnecting peer=1
```

With `RUST_LOG="trace"`, `cargo run connect`:
```
Sep 19 08:24:24.530  INFO zebrad::commands::connect: version=Version { version: Version(170007), services: Services(1), timestamp: 2019-09-19T15:24:24.530059300Z, address_recv: (Services(1), V4(127.0.0.1:8233)), address_from: (Services(1), V4(127.0.0.1:9000)), nonce: Nonce(1), user_agent: "Zebra Connect", start_height: BlockHeight(0), relay: false }
Sep 19 08:24:24.530 TRACE Task::run: tokio_executor::threadpool::task: state=Running
Sep 19 08:24:24.530 DEBUG Task::run: tokio_net::driver::reactor: adding I/O source token=0
Sep 19 08:24:24.530  INFO zebrad::commands::connect: version_bytes="24e9276476657273696f6e000000000063000000cb30ab03179802000100000000000000a89d835d00000000010000000000000000000000000000000000ffff7f0000012029010000000000000000000000000000000000ffff7f000001232801000000000000000d5a6562726120436f6e6e6563740000000000"
Sep 19 08:24:24.530 TRACE Task::run: log: registering with poller
```
2019-09-22 17:27:08 -04:00
Henry de Valence df7801d623 Temporarily change hyper to git version.
This avoids some crate selection conflicts, but makes some futures
extension traits fall out of order?  This seems to be an issue with
`pin-project` resolved in the git branch of `hyper` (but not yet
released).
2019-09-22 17:27:08 -04:00
Deirdre Connolly 162b37fe8d Tracing endpoint (#3)
* Add a TracingConfig and some components

Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>

* Restructure, use dependency injection, initialize tracing

* Start a placeholder loop in start command

* Add hyper alpha.1, bump tokio to alpha.4

* Hello world endpoint using async/await from hyper 0.13 alpha

Also cleaned up some linter messages.

Co-authored-by: Henry de Valence <hdevalence@hdevalence.ca>

* Update to tracing_subscriber 0.1

* fmt

* add rust-toolchain

* Remove hyper::Version import

* wip: start filter_handler impl

* Add .rustfmt.toml

* rustfmt

* Tidy up .rustfmt.toml

* Add filter reloading handling.

* bump toolchain

* Remove generated hello world acceptance tests.

These test the behaviour of the autogenerated binary and work as examples of
how to test the behaviour of abscissa binaries.  Since we don't print "Hello
World" any more, they fail, but we don't yet have replacement behaviour to add
tests for, so they're removed for now.

* Clean up config file handling with Option::and_then.
2019-09-09 13:05:42 -07:00
Henry de Valence ec363d2d41 Create workspace skeleton based on design.md 2019-08-29 14:46:54 -07:00