Commit Graph

37 Commits

Author SHA1 Message Date
Ramana Venkata 991c70723a zebrad: Create zebrad.toml in acceptance tests from ZebradConfig
Fixes #929
2020-08-23 21:24:19 -04:00
Alfredo Garcia d349f2bbc2
Refactor acceptance serialized_tests (#920)
* add network listening address to default config
2020-08-20 07:48:22 +10:00
Jane Lusby 1d6183ef84
Make zebra-script wrap the C++ script implementation in zcashconsensus (#708)
* attempt to use zcashconsensus crate in zebra-script

* boop

* update verify fn to use zebra types

* a bit more cleanup

* cleanup

* more

* beep boop

* fix renamed member

* cleaning

* get a real branch id

* remove as of yet unneeded api

* Update zebra-chain/src/transaction.rs

* Update zebra-chain/src/transaction.rs

* more cleanup

* oops wrong dep section

* use a tuple to communicate arg association

* update to use published version of zcash_script

* fix new compiler error

* install llvm on windows

* fix bindgen bug????

* try to get docker file to win

* okay try everything

* fix windows build maybe

* always download choco

* fix paths for moved types

* try a different error message

* try convenience script

* try installing just llvm

* add back one more

* try installing some headers

* try a diff package

* try everything

* remove the minimum

* try newer docker builder image

* cleanup docker image

* cleanup extra ci step
2020-08-18 11:08:53 -07:00
Alfredo Garcia e73f976194
Valid generated config acceptance test (#859)
* add valid generated config test

* change to pathbuf

* use -c to make sure we are using the generated file

* add and use a ZebraTestDir type

* change approach to generate tempdir in top of each test

* pass tempdir to test_cmd and set current dir to it

* add and use a `generated_config_path` variable in tests
2020-08-13 13:31:13 -07:00
Henry de Valence a79ce97957
Fix sync algorithm. (#887)
* checkpoint: reject older of duplicate verification requests.

If we get a duplicate block verification request, we should drop the older one
in favor of the newer one, because the older request is likely to have been
canceled.  Previously, this code would accept up to four duplicate verification
requests, then fail all subsequent ones.

* sync: add a timeout layer to block requests.

Note that if this timeout is too short, we'll bring down the peer set in a
retry storm.

* sync: restart syncing on error

Restart the syncing process when an error occurs, rather than ignoring it.
Restarting means we discard all tips and start over with a new block locator,
so we can have another chance to "unstuck" ourselves.

* sync: additional debug info

* sync: handle lookahead limit correctly.

Instead of extracting all the completed task results, the previous code pulled
results out until there were fewer tasks than the lookahead limit, then
stopped.  This meant that completed tasks could be left until the limit was
exceeded again.  Instead, extract all completed results, and use the number of
pending tasks to decide whether to extend the tip or wait for blocks to finish.

* network: add debug instrumentation to retry policy

* sync: instrument the spawned task

* sync: streamline ObtainTips/ExtendTips logic & tracing

This change does three things:

1.  It aligns the implementation of ObtainTips and ExtendTips so that they use
the same deduplication method.  This means that when debugging we only have one
deduplication algorithm to focus on.

2.  It streamlines the tracing output to not include information already
included in spans. Both obtain_tips and extend_tips have their own spans
attached to the events, so it's not necessary to add Scope: prefixes in
messages.

3.  It changes the messages to be focused on reporting the actual
events rather than the interpretation of the events (e.g., "got genesis hash in
response" rather than "peer could not extend tip").  The motivation for this
change is that when debugging, the interpretation of events is already known to
be incorrect, in the sense that the mental model of the code (no bug) does not
match its behavior (has bug), so presenting minimally-interpreted events forces
interpretation relative to the actual code.

* sync: hack to work around zcashd behavior

* sync: localize debug statement in extend_tips

* sync: change algorithm to define tips as pairs of hashes.

This is different enough from the existing description that its comments no
longer apply, so I removed them.  A further chunk of work is to change the sync
RFC to document this algorithm.

* sync: reduce block timeout

* state: add resource limits for sled

Closes #888

* sync: add a restart timeout constant

* sync: de-pub constants
2020-08-12 16:48:01 -07:00
dependabot[bot] 945b019739
build(deps): bump tracing-subscriber from 0.2.10 to 0.2.11 (#873)
Bumps [tracing-subscriber](https://github.com/tokio-rs/tracing) from 0.2.10 to 0.2.11.
- [Release notes](https://github.com/tokio-rs/tracing/releases)
- [Commits](https://github.com/tokio-rs/tracing/compare/tracing-subscriber-0.2.10...tracing-subscriber-0.2.11)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2020-08-11 10:30:50 -07:00
dependabot[bot] 5cb61136c9
build(deps): bump tracing from 0.1.18 to 0.1.19 (#872)
Bumps [tracing](https://github.com/tokio-rs/tracing) from 0.1.18 to 0.1.19.
- [Release notes](https://github.com/tokio-rs/tracing/releases)
- [Commits](https://github.com/tokio-rs/tracing/compare/tracing-0.1.18...tracing-0.1.19)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2020-08-11 10:18:54 -07:00
Alfredo Garcia c9093e4d59
Make more checks in non server acceptance tests (#860)
* make sure no info is printed in non server tests

* check exact full output for validity instead of log msgs

* add end of output character to version regex

* use coercions, use equality operator

Co-authored-by: Jane Lusby <jlusby42@gmail.com>

Co-authored-by: Jane Lusby <jlusby42@gmail.com>
2020-08-10 12:50:48 -07:00
Jane Lusby 867dd0b475
Setup tracing-flame for use profiling zebrad (#436)
* Setup tracing-flame for use profiling zebrad

* start work on conditional flamegraph generation

* review time!

* update comments

* Update Cargo.toml

* disable default features for inferno

* reorganize

* missing one trait

* Apply suggestions from code review

* graceful shutdown!

* remove special case handling on ctrlc for cleanup

* rename signal fn to better represent its responsibility

* remove unused global hook for flushing flamegraph

* move tracing logic to the right file

* just copy linkerd's signal handling logic

* update book

* make zebrad app drop on shutdown normally

* Update zebrad/src/components/tokio.rs

Co-authored-by: teor <teor@riseup.net>

* Update zebrad/src/application.rs

Co-authored-by: teor <teor@riseup.net>

* Apply suggestions from code review

Co-authored-by: teor <teor@riseup.net>

* cleanup a little

* ooh yea there's an API for that

* setup env-filter for backup subscriber

* document env filter

* document return codes

* forgot to save

* Update book/src/applications/zebrad.md

Co-authored-by: teor <teor@riseup.net>

Co-authored-by: teor <teor@riseup.net>
2020-08-05 16:35:56 -07:00
Henry de Valence 4a03d76a41
Remove environment variables in favor of documented config options. (#827)
* Load tracing filter only from config and simplify logic.

* Configure the state storage in the config, not an environment variable.

This also changes the config so that the path is always set rather than being
optional, because Zebra always needs a place to store its config.
2020-08-05 11:48:08 -07:00
Alfredo Garcia 70597c1da3
Add testnet blocks to zebra_state (#770)
* add testnet blocks, use them in state tests

* remove mutex and just add more transcripts

Co-authored-by: Jane Lusby <jane@zfnd.org>
2020-08-04 13:39:32 -07:00
Alfredo Garcia e037466e26
Acceptance tests - check kill signal (#814)
* check kill signal exit code

* change names and add docs

* change exit_status() to was_killed()

* change assert calls
2020-08-04 13:38:39 -07:00
dependabot[bot] 8e268150a7 build(deps): bump tracing-subscriber from 0.2.9 to 0.2.10
Bumps [tracing-subscriber](https://github.com/tokio-rs/tracing) from 0.2.9 to 0.2.10.
- [Release notes](https://github.com/tokio-rs/tracing/releases)
- [Commits](https://github.com/tokio-rs/tracing/compare/tracing-subscriber-0.2.9...tracing-subscriber-0.2.10)

Signed-off-by: dependabot[bot] <support@github.com>
2020-08-03 21:11:50 -04:00
Alfredo Garcia deba84b96e rustfmt 2020-08-03 15:50:48 -04:00
Alfredo Garcia 5f23970377 move env variable creation to test_cmd 2020-08-03 15:50:48 -04:00
dependabot[bot] f3b65f9609 build(deps): bump tracing from 0.1.17 to 0.1.18
Bumps [tracing](https://github.com/tokio-rs/tracing) from 0.1.17 to 0.1.18.
- [Release notes](https://github.com/tokio-rs/tracing/releases)
- [Commits](https://github.com/tokio-rs/tracing/compare/tracing-0.1.17...tracing-0.1.18)

Signed-off-by: dependabot[bot] <support@github.com>
2020-08-03 15:49:30 -04:00
Alfredo Garcia f2d7bb3177
Command execution tests (#690)
* add zebrad acceptance tests
* add custom command test helpers that work with kill
* add and use info event for start and seed commands
* combine conflicting tests into one test case

Co-authored-by: Jane Lusby <jane@zfnd.org>
2020-08-01 16:15:26 +10:00
Jane Lusby e6b849568f
Add support for errors in zebra_test::Transcript (#678)
* Add support for errors in zebra_test::Transcript

* test transcript with an error checker

* switch to option instead of MockError

* update docs

* dont use verifier against ready_and

* cleanup exports and add docs

* handle todos

* fix doctest

* temp: use cleaner error handling example

* add ability to test only for presence of error
2020-07-31 11:54:18 -07:00
Jane Lusby 1015db25a8
remap the state storage to only store blocks once (#741) 2020-07-31 09:21:54 +10:00
teor 12cbe410f2 fix: Hide warn-level logs in the zebra-consensus tests
Unless RUST_LOG is set.
2020-07-27 13:51:33 -04:00
teor a60299d86c fix: Change the default test log level to warn 2020-07-27 13:51:33 -04:00
Henry de Valence 4aa00ad216 Align crate versions and user-agent with NU numbers.
We had a brief discussion on discord and it seemed like we had consensus on the
following versioning policy:

* zebrad: match major version to NU version, so we will start by releasing
  zebrad 3.0.0;

* zebra-* libraries: start by matching zebrad's version, then increment major
  versions of each library as we need to make breaking changes (potentially
  faster than the zebrad version, always respecting semver but making no
  guarantees about the longevity of major releases).

This commit sets all of the crate versions to 3.0.0-alpha.0 -- the -alpha.0
marks it as a prerelease not subject to perfect adherence to compatibility
guarantees.
2020-07-24 11:46:37 -07:00
dependabot[bot] f7c59c99b5 build(deps): bump tracing-subscriber from 0.2.8 to 0.2.9
Bumps [tracing-subscriber](https://github.com/tokio-rs/tracing) from 0.2.8 to 0.2.9.
- [Release notes](https://github.com/tokio-rs/tracing/releases)
- [Commits](https://github.com/tokio-rs/tracing/compare/tracing-subscriber-0.2.8...tracing-subscriber-0.2.9)

Signed-off-by: dependabot[bot] <support@github.com>
2020-07-24 14:31:44 -04:00
dependabot[bot] 909b74a613
build(deps): bump tracing from 0.1.16 to 0.1.17 (#734)
Bumps [tracing](https://github.com/tokio-rs/tracing) from 0.1.16 to 0.1.17.
- [Release notes](https://github.com/tokio-rs/tracing/releases)
- [Commits](https://github.com/tokio-rs/tracing/compare/tracing-0.1.16...tracing-0.1.17)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2020-07-23 10:54:56 -07:00
dependabot[bot] 2208a6a22d build(deps): bump tracing-subscriber from 0.2.7 to 0.2.8
Bumps [tracing-subscriber](https://github.com/tokio-rs/tracing) from 0.2.7 to 0.2.8.
- [Release notes](https://github.com/tokio-rs/tracing/releases)
- [Commits](https://github.com/tokio-rs/tracing/compare/tracing-subscriber-0.2.7...tracing-subscriber-0.2.8)

Signed-off-by: dependabot[bot] <support@github.com>
2020-07-21 12:01:40 -04:00
Jane Lusby f3dd24bb3c
bug cleanup related to error handling (#552)
* cleanup error handling fixes
2020-07-17 18:19:00 -04:00
Alfredo Garcia 04b2eeedd0
Longer checkpoint gap test case (#643)
* create longer checkpoint test

* use futures unordered to syncronize background futures

Co-authored-by: Jane Lusby <jane@zfnd.org>
2020-07-13 09:29:21 -07:00
Alfredo Garcia 5d6a5ca329
Add consensus coinbase checks (#593)
* add coinbase check and test case

Co-authored-by: Jane Lusby <jane@zfnd.org>
2020-07-10 09:13:44 +10:00
dependabot[bot] d627889104 Bump tracing from 0.1.15 to 0.1.16
Bumps [tracing](https://github.com/tokio-rs/tracing) from 0.1.15 to 0.1.16.
- [Release notes](https://github.com/tokio-rs/tracing/releases)
- [Commits](https://github.com/tokio-rs/tracing/compare/tracing-0.1.15...tracing-0.1.16)

Signed-off-by: dependabot[bot] <support@github.com>
2020-07-09 10:22:27 -07:00
Jane Lusby a733580fd8
add proptests for invalid forms of input (#550)
* add proptests for invalid forms of input

* apply proptests to each valid block we have

* start moving equihash logic to BlockHeader

* provide an error based interface

* cleanup logging in dependent crate

* use is_equihash_solution_valid everywhere

* switch to proper equihash crate branch
2020-07-07 11:01:02 -07:00
Alfredo Garcia db30e53470
Create a test for block size (#558)
* add block size limit test
* calculate max_transactions_in_block and max_inputs_in_tx
* abstract block generation
* move all test vectors to zebra-test
2020-07-07 10:37:32 +10:00
Alfredo Garcia d736355f9f
Add ZIP143 and ZIP243 test vectors (#585) 2020-07-03 09:45:40 +10:00
dependabot[bot] a1d02c2606 Bump tracing-subscriber from 0.2.6 to 0.2.7
Bumps [tracing-subscriber](https://github.com/tokio-rs/tracing) from 0.2.6 to 0.2.7.
- [Release notes](https://github.com/tokio-rs/tracing/releases)
- [Commits](https://github.com/tokio-rs/tracing/compare/tracing-subscriber-0.2.6...tracing-subscriber-0.2.7)

Signed-off-by: dependabot[bot] <support@github.com>
2020-07-02 06:47:04 -04:00
Jane Lusby 2abf4b93a8 consolidate test init functions into zebra-test (#541)
* consolidate test init logic into one place

* rustfmt

Co-authored-by: Jane Lusby <jane@zfnd.org>
2020-06-24 11:47:18 -07:00
Henry de Valence e8d42264e8 Add a check method to the Transcript. 2020-06-24 11:47:18 -07:00
Henry de Valence 5c9cd52baf Add a Transcript Service for testing. 2020-06-24 11:47:18 -07:00
Henry de Valence e8561d8f9e rename zebra-test-vectors to zebra-test 2020-06-22 20:23:02 -07:00