Commit Graph

1037 Commits

Author SHA1 Message Date
Henry de Valence 6d1a4b2218
Load config after initializing the Terminal (#848) 2020-08-06 17:22:40 -07:00
Alfredo Garcia c52481c041 fix logs 2020-08-07 09:21:57 +10:00
Henry de Valence bdf66387d6
Reorganize the book. (#843)
* Reorganize the book.

This PR has one unfortunate change, which is that the README.md and
CONTRIBUTING.md files in the book are symlinks to files in the parent
directory.  The motivation for this is to ensure that we don't maintain two
copies of the same data, and that the landing page of the website matches the
landing page of the Github repo, etc.   However, I'm not sure whether these
symlinks will work correctly on Windows.

The alternatives are:

- Duplicate the contents of the files and expect that people will know to keep
  them in sync;

- Use relative links `../../README.md` in the `SUMMARY.md`.  This seemed like
  it caused mdbook to dump the rendered files into the repository root rather
  than keeping them in the `book` directory.

- Use a symlink (chosen option).  This may not work on Windows but I think that
  the worst outcome would be that the book would be unbuildable unless someone
  used WSL or something.  This seems like the least bad option.

* Remove symlinks in favor of #include 

Turns out the symlinks aren't required!
2020-08-06 15:39:54 -07:00
Henry de Valence 3d46ab746a
Clean up options in network config section. (#839)
Closes #536.

This removes:

- the user-agent (we can add a mechanism to specify extra BIP14 components later, if any users ask us for that feature);
- the EWMA parameters (these were put in the config just to avoid making a choice);
- the peer connection timeout (we can change the default value if anyone ever has a problem with it);
- the peer set request buffer size (setting this too low can make the application deadlock);

The new peer interval is left in.
2020-08-06 11:29:00 -07:00
Jane Lusby 3e9c6f054b
fix log level default for server commands (#840)
* fix log level default for server commands

* remove dbg
2020-08-06 11:23:00 -07:00
Henry de Valence a77328ad7c
Refactor tracing components (#834)
* Split tracing component code into modules.

* Repatriate Tracing and simplify config handling.

We upstreamed our Tracing component, expecting not to have to exert fine
control over the tracing settings.  But this turned out not to be the case, and
now that we want to do other things (flamegraphs, journalctl, opentelemetry,
etc), we end up with really awkward code (as in the current flamegraph
handling).

This also makes use of the changes to `init()` to load the config early to pass
configuration data into the components, which avoids the need for the
refactoring in #775.

Finally, we restore support for the `-v` flag when the filter is unset.  Closes #831.

* Disable tracing and metrics endpoints by default.

Closes #660.

* Switch back to upstream Abscissa.

* Integrate flamegraph support into the new Tracing component.

* Pass -v in acceptance tests to get info-level output.

* Clean up acceptance test code.
2020-08-06 10:29:31 -07:00
dependabot[bot] c0caba401c build(deps): bump structopt from 0.3.15 to 0.3.16
Bumps [structopt](https://github.com/TeXitoi/structopt) from 0.3.15 to 0.3.16.
- [Release notes](https://github.com/TeXitoi/structopt/releases)
- [Changelog](https://github.com/TeXitoi/structopt/blob/master/CHANGELOG.md)
- [Commits](https://github.com/TeXitoi/structopt/commits)

Signed-off-by: dependabot[bot] <support@github.com>
2020-08-06 06:22:55 -04:00
Henry de Valence 6c379be7f2 consensus: temporarily allow dead code in zebra_consensus::primitives. 2020-08-05 21:28:55 -07:00
Henry de Valence 2efb4eb262 consensus: add a static redjubjub::VERIFIER with fallback 2020-08-05 21:28:55 -07:00
Henry de Valence 8d4f154c9f consensus: move redjubjub to new primitives module
This will let it sit alongside a groth16 verifier.
2020-08-05 21:28:55 -07:00
Henry de Valence a987394782
Add KaTeX to rendered docs. (#832)
This adds a single test formula to the `zebra-chain` docs.  We can remove it
after merging this and running it through CI to confirm that it works.
2020-08-05 17:34:30 -07:00
Jane Lusby 867dd0b475
Setup tracing-flame for use profiling zebrad (#436)
* Setup tracing-flame for use profiling zebrad

* start work on conditional flamegraph generation

* review time!

* update comments

* Update Cargo.toml

* disable default features for inferno

* reorganize

* missing one trait

* Apply suggestions from code review

* graceful shutdown!

* remove special case handling on ctrlc for cleanup

* rename signal fn to better represent its responsibility

* remove unused global hook for flushing flamegraph

* move tracing logic to the right file

* just copy linkerd's signal handling logic

* update book

* make zebrad app drop on shutdown normally

* Update zebrad/src/components/tokio.rs

Co-authored-by: teor <teor@riseup.net>

* Update zebrad/src/application.rs

Co-authored-by: teor <teor@riseup.net>

* Apply suggestions from code review

Co-authored-by: teor <teor@riseup.net>

* cleanup a little

* ooh yea there's an API for that

* setup env-filter for backup subscriber

* document env filter

* document return codes

* forgot to save

* Update book/src/applications/zebrad.md

Co-authored-by: teor <teor@riseup.net>

Co-authored-by: teor <teor@riseup.net>
2020-08-05 16:35:56 -07:00
Alfredo Garcia ded273413a
Use transcripts in block verify tests (#675)
* change consensus block tests to transcripts

* fix lints

Co-authored-by: Jane Lusby <jlusby42@gmail.com>

Co-authored-by: Jane Lusby <jlusby42@gmail.com>
2020-08-05 14:41:41 -07:00
Henry de Valence 4a03d76a41
Remove environment variables in favor of documented config options. (#827)
* Load tracing filter only from config and simplify logic.

* Configure the state storage in the config, not an environment variable.

This also changes the config so that the path is always set rather than being
optional, because Zebra always needs a place to store its config.
2020-08-05 11:48:08 -07:00
teor d49c11ee70 fix: Allow PROPTEST_CASES to override the lower block case default 2020-08-05 02:07:05 -07:00
Henry de Valence 82da4a5326 Remove connect command. 2020-08-04 23:34:45 -07:00
Deirdre Connolly 1f57b3a948 Update data-flow.md 2020-08-04 22:44:39 -07:00
Deirdre Connolly 6024ffdc97 Update data-flow.md 2020-08-04 22:44:39 -07:00
Deirdre Connolly f2f5641691 Move data-flow.md to design/ 2020-08-04 22:44:39 -07:00
Deirdre Connolly a99330f025 Create data-flow.md 2020-08-04 22:44:39 -07:00
teor ac7a4ae517 fix: Speed up the block round trip proptest
Reduce the number of cases run by the block round trip proptest, to
speed up the Zebra tests.
2020-08-04 22:35:01 -07:00
Alfredo Garcia 70597c1da3
Add testnet blocks to zebra_state (#770)
* add testnet blocks, use them in state tests

* remove mutex and just add more transcripts

Co-authored-by: Jane Lusby <jane@zfnd.org>
2020-08-04 13:39:32 -07:00
Alfredo Garcia e037466e26
Acceptance tests - check kill signal (#814)
* check kill signal exit code

* change names and add docs

* change exit_status() to was_killed()

* change assert calls
2020-08-04 13:38:39 -07:00
teor 78b5bf5e9a refactor: Move the difficulty tests into their own file 2020-08-04 15:24:52 -04:00
teor c95d980bc2 doc: Explain current and minimum network protocol versions 2020-08-04 15:11:16 -04:00
teor ffb6aacf33 feature: Check difficulty in BlockVerifier 2020-08-04 12:29:07 +10:00
teor efb9313460 fix: Resolve a type ambiguity in a proptest
For some reason, implementing
`PartialOrd<BlockHeaderHash> for ExpandedDifficulty` (or its converse)
makes this unrelated code ambiguous.
2020-08-04 12:29:07 +10:00
teor c8b502a5da feature: Compare ExpandedDifficulty and BlockHeaderHash 2020-08-04 12:29:07 +10:00
teor b7fac7b3bc feature: Make ExpandedDifficulty use U256 internally
Adds a dependency on the primitive-types crate.

Also adds custom hex debug formatting for compact and expanded
difficulties.
2020-08-04 12:29:07 +10:00
teor c04d1b7b8f doc: Consensus-critical difficulty encodings 2020-08-04 12:29:07 +10:00
teor 06157a7c8e
Update the BlockHeader fields for Heartwood (#767)
The Heartwood upgrade changes the meaning of the hashFinalSaplingRoot to
hashLightClientRoot. Since we don't know the network upgrade heights in
zebra-chain, we just use [u8; 32] for now.
2020-08-04 12:15:28 +10:00
teor 59eb23772d feature: Use the Canopy testnet network protocol version
Canopy will activate on testnet within the next 24 hours. To continue to
use testnet, we need to upgrade the Zebra network protocol version.
2020-08-04 12:13:58 +10:00
dependabot[bot] 8e268150a7 build(deps): bump tracing-subscriber from 0.2.9 to 0.2.10
Bumps [tracing-subscriber](https://github.com/tokio-rs/tracing) from 0.2.9 to 0.2.10.
- [Release notes](https://github.com/tokio-rs/tracing/releases)
- [Commits](https://github.com/tokio-rs/tracing/compare/tracing-subscriber-0.2.9...tracing-subscriber-0.2.10)

Signed-off-by: dependabot[bot] <support@github.com>
2020-08-03 21:11:50 -04:00
Alfredo Garcia deba84b96e rustfmt 2020-08-03 15:50:48 -04:00
Alfredo Garcia 5f23970377 move env variable creation to test_cmd 2020-08-03 15:50:48 -04:00
Alfredo Garcia 2dacd0a62b change default state path 2020-08-03 15:50:48 -04:00
dependabot[bot] f3b65f9609 build(deps): bump tracing from 0.1.17 to 0.1.18
Bumps [tracing](https://github.com/tokio-rs/tracing) from 0.1.17 to 0.1.18.
- [Release notes](https://github.com/tokio-rs/tracing/releases)
- [Commits](https://github.com/tokio-rs/tracing/compare/tracing-0.1.17...tracing-0.1.18)

Signed-off-by: dependabot[bot] <support@github.com>
2020-08-03 15:49:30 -04:00
teor 195948e5b1 feature: Add an ExpandedDifficulty type and conversion
Also add tests.
2020-08-03 10:28:31 +10:00
teor c4dec3fb36 feature: Make a CompactDifficulty wrapper
Wrap the compact difficulty "bits" field in a CompactDifficulty struct,
and rename the header field for clarity.
2020-08-03 10:28:31 +10:00
Alfredo Garcia f2d7bb3177
Command execution tests (#690)
* add zebrad acceptance tests
* add custom command test helpers that work with kill
* add and use info event for start and seed commands
* combine conflicting tests into one test case

Co-authored-by: Jane Lusby <jane@zfnd.org>
2020-08-01 16:15:26 +10:00
Jane Lusby e6b849568f
Add support for errors in zebra_test::Transcript (#678)
* Add support for errors in zebra_test::Transcript

* test transcript with an error checker

* switch to option instead of MockError

* update docs

* dont use verifier against ready_and

* cleanup exports and add docs

* handle todos

* fix doctest

* temp: use cleaner error handling example

* add ability to test only for presence of error
2020-07-31 11:54:18 -07:00
teor d4d1edad5a
fix: Use types to avoid ChainVerifier inconsistencies (#797) 2020-07-31 10:21:20 +10:00
Jane Lusby 1015db25a8
remap the state storage to only store blocks once (#741) 2020-07-31 09:21:54 +10:00
Alfredo Garcia 917a4fbdbe
Add last_checkpoint argument to zebra-checkpoints (#793) 2020-07-31 08:33:00 +10:00
teor e74ff18708 refactor: Move the checkpoint list tests to their own file 2020-07-31 07:21:43 +10:00
teor 5b9703f95a fix: Avoid creating a CheckpointVerifier if it is not needed
Also:
* make sure the checkpoints cover all pre-Sapling blocks
* do some minor refactoring to reduce duplicate code
2020-07-30 13:30:18 +10:00
teor fbeae99328 fix: Only warn on unexpected high blocks
High blocks are expected when we restart with most of the chain in our
state.

Also downgrade the message from warn to info.
2020-07-30 13:30:18 +10:00
Alfredo Garcia 89ef08793b delete file 2020-07-29 19:44:21 -07:00
Alfredo Garcia 617f1d80ef move docs to zebra book 2020-07-29 19:44:21 -07:00
Alfredo Garcia 6297a7cd19 document zebrad enviroment variables 2020-07-29 19:44:21 -07:00