zebra/zebra-state/Cargo.toml

63 lines
1.7 KiB
TOML
Raw Normal View History

[package]
name = "zebra-state"
version = "1.0.0-beta.16"
authors = ["Zcash Foundation <zebra@zfnd.org>"]
license = "MIT OR Apache-2.0"
edition = "2021"
[features]
proptest-impl = ["proptest", "proptest-derive", "zebra-test", "zebra-chain/proptest-impl"]
[dependencies]
Upgrade dependencies (#3625) * Upgrade some dependencies * Upgrade some dependencies * Upgrade dependencies for zebrad * Upgrade tracing dependencies * Revert `tor` & `arti` * Upgrade `criterion` & `pin-project` in `deny.toml` * Remove some dependencies from `skip-tree` in `deny.toml` * Revert some the versions of dependencies because of duplicates * Revert proptest regressions * Upgrade dependencies, then ignore some more duplicates (#3716) * feat(actions)!: add full sync test (#3582) * add(tests): full sync test * fix(test): add build * fix(deploy): escape double dashes '--' correctly * fix(test): remove unexpected --no-capture arg error: Found argument '--nocapture' which wasn't expected, or isn't valid in this context * refactor(docker): use default executable as entrypoint * refactor(startup): add a custom entrypoint * fix(test): add missing TEST_FULL_SYNC variable * test(timeout): use the biggest machine * fix * fix(deploy): use latest successful image * typo * refactor(docker): generate config file at startup * revert(build): changes were made to docker * fix(docker): send variables correctly to the entrypoint * test different conf file approach * fix(env): add RUN_TEST env variable * ref: use previous approach * fix(color): use environment variable * fix(resources): use our normal machine size * fix(ci): double CPU and RAM for full sync test * fix(test): check for zebrad test output in the correct order The mempool is only activated once, so we must check for that log first. After mempool activation, the stop regex is logged at least once. (It might be logged before as well, but we can't rely on that.) When checking that the mempool didn't activate, wait for the `zebrad` command to exit, then check the entire log. * fix(ci): run full sync test with full compiler optimisations * fix(tests): reintroduce tests and run full sync on approval * fix(tests): reduce the changelog Co-authored-by: teor <teor@riseup.net> * fix(ci): update CI job path triggers (#3692) * ci(test): re-run tests when snapshot data changes * fix(ci): rebuild state when disk format changes * fix(ci): rebuild rust docs when code or dependencies change * doc(ci): explain why we run jobs when files change Co-authored-by: Gustavo Valverde <gustavo@iterativo.do> * fix(build): use the right multistage target (#3700) * fix(review): only assign one reviewer to general Rust reviews (#3708) If we assign two teams, GitHub assigns two reviewers. * fix(ci): change the color-eyre ignore to a tracing-subscriber ignore * fix(ci): ignore duplicate darling dependencies * doc(ci): remove an alternative resolution doc Co-authored-by: Gustavo Valverde <gustavo@iterativo.do> Co-authored-by: teor <teor@riseup.net> Co-authored-by: Gustavo Valverde <gustavo@iterativo.do>
2022-03-06 18:07:25 -08:00
bincode = "1.3.3"
chrono = { version = "0.4.22", default-features = false, features = ["clock", "std"] }
dirs = "4.0.0"
Upgrade dependencies (#3625) * Upgrade some dependencies * Upgrade some dependencies * Upgrade dependencies for zebrad * Upgrade tracing dependencies * Revert `tor` & `arti` * Upgrade `criterion` & `pin-project` in `deny.toml` * Remove some dependencies from `skip-tree` in `deny.toml` * Revert some the versions of dependencies because of duplicates * Revert proptest regressions * Upgrade dependencies, then ignore some more duplicates (#3716) * feat(actions)!: add full sync test (#3582) * add(tests): full sync test * fix(test): add build * fix(deploy): escape double dashes '--' correctly * fix(test): remove unexpected --no-capture arg error: Found argument '--nocapture' which wasn't expected, or isn't valid in this context * refactor(docker): use default executable as entrypoint * refactor(startup): add a custom entrypoint * fix(test): add missing TEST_FULL_SYNC variable * test(timeout): use the biggest machine * fix * fix(deploy): use latest successful image * typo * refactor(docker): generate config file at startup * revert(build): changes were made to docker * fix(docker): send variables correctly to the entrypoint * test different conf file approach * fix(env): add RUN_TEST env variable * ref: use previous approach * fix(color): use environment variable * fix(resources): use our normal machine size * fix(ci): double CPU and RAM for full sync test * fix(test): check for zebrad test output in the correct order The mempool is only activated once, so we must check for that log first. After mempool activation, the stop regex is logged at least once. (It might be logged before as well, but we can't rely on that.) When checking that the mempool didn't activate, wait for the `zebrad` command to exit, then check the entire log. * fix(ci): run full sync test with full compiler optimisations * fix(tests): reintroduce tests and run full sync on approval * fix(tests): reduce the changelog Co-authored-by: teor <teor@riseup.net> * fix(ci): update CI job path triggers (#3692) * ci(test): re-run tests when snapshot data changes * fix(ci): rebuild state when disk format changes * fix(ci): rebuild rust docs when code or dependencies change * doc(ci): explain why we run jobs when files change Co-authored-by: Gustavo Valverde <gustavo@iterativo.do> * fix(build): use the right multistage target (#3700) * fix(review): only assign one reviewer to general Rust reviews (#3708) If we assign two teams, GitHub assigns two reviewers. * fix(ci): change the color-eyre ignore to a tracing-subscriber ignore * fix(ci): ignore duplicate darling dependencies * doc(ci): remove an alternative resolution doc Co-authored-by: Gustavo Valverde <gustavo@iterativo.do> Co-authored-by: teor <teor@riseup.net> Co-authored-by: Gustavo Valverde <gustavo@iterativo.do>
2022-03-06 18:07:25 -08:00
displaydoc = "0.2.3"
futures = "0.3.24"
hex = "0.4.3"
change(state): Write non-finalized blocks to the state in a separate thread, to avoid network and RPC hangs (#5257) * Add a new block commit task and channels, that don't do anything yet * Add last_block_hash_sent to the state service, to avoid database accesses * Update last_block_hash_sent regardless of commit errors * Rename a field to StateService.max_queued_finalized_height * Commit finalized blocks to the state in a separate task * Check for panics in the block write task * Wait for the block commit task in tests, and check for errors * Always run a proptest that sleeps once * Add extra debugging to state shutdowns * Work around a RocksDB shutdown bug * Close the finalized block channel when we're finished with it * Only reset state queue once per error * Update some TODOs * Add a module doc comment * Drop channels and check for closed channels in the block commit task * Close state channels and tasks on drop * Remove some duplicate fields across StateService and ReadStateService * Try tweaking the shutdown steps * Update and clarify some comments * Clarify another comment * Don't try to cancel RocksDB background work on drop * Fix up some comments * Remove some duplicate code * Remove redundant workarounds for shutdown issues * Remode a redundant channel close in the block commit task * Remove a mistaken `!force` shutdown condition * Remove duplicate force-shutdown code and explain it better * Improve RPC error logging * Wait for chain tip updates in the RPC tests * Wait 2 seconds for chain tip updates before skipping them * Remove an unnecessary block_in_place() * Fix some test error messages that were changed by earlier fixes * Expand some comments, fix typos Co-authored-by: Marek <mail@marek.onl> * Actually drop children of failed blocks * Explain why we drop descendants of failed blocks * Clarify a comment * Wait for chain tip updates in a failing test on macOS * Clean duplicate finalized blocks when the non-finalized state activates * Send an error when receiving a duplicate finalized block * Update checkpoint block behaviour, document its consensus rule * Wait for chain tip changes in inbound_block_height_lookahead_limit test * Wait for the genesis block to commit in the fake peer set mempool tests * Disable unreliable mempool verification check in the send transaction test * Appease rustfmt * Use clear_finalized_block_queue() everywhere that blocks are dropped * Document how Finalized and NonFinalized clones are different * sends non-finalized blocks to the block write task * passes ZebraDb to commit_new_chain, commit_block, and no_duplicates_in_finalized_chain instead of FinalizedState * Update zebra-state/src/service/write.rs Co-authored-by: teor <teor@riseup.net> * updates comments, renames send_process_queued, other minor cleanup * update assert_block_can_be_validated comment * removes `mem` field from StateService * removes `disk` field from StateService and updates block_iter to use `ZebraDb` instead of the finalized state * updates tests that use the disk to use read_service.db instead * moves best_tip to a read fn and returns finalized & non-finalized states from setup instead of the state service * changes `contextual_validity` to get the network from the finalized_state instead of another param * swaps out StateService with FinalizedState and NonFinalizedState in tests * adds NotReadyToBeCommitted error and returns it from validate_and_commit when a blocks parent hash is not in any chain * removes NonFinalizedWriteCmd and calls, moves update_latest_channels above rsp_tx.send * makes parent_errors_map an indexmap * clears non-finalized block queue when the receiver is dropped and when the StateService is being dropped * sends non-finalized blocks to the block write task * passes ZebraDb to commit_new_chain, commit_block, and no_duplicates_in_finalized_chain instead of FinalizedState * updates comments, renames send_process_queued, other minor cleanup * Update zebra-state/src/service/write.rs Co-authored-by: teor <teor@riseup.net> * update assert_block_can_be_validated comment * removes `mem` field from StateService * removes `disk` field from StateService and updates block_iter to use `ZebraDb` instead of the finalized state * updates tests that use the disk to use read_service.db instead * moves best_tip to a read fn and returns finalized & non-finalized states from setup instead of the state service * changes `contextual_validity` to get the network from the finalized_state instead of another param * swaps out StateService with FinalizedState and NonFinalizedState in tests * adds NotReadyToBeCommitted error and returns it from validate_and_commit when a blocks parent hash is not in any chain * removes NonFinalizedWriteCmd and calls, moves update_latest_channels above rsp_tx.send * makes parent_errors_map an indexmap * clears non-finalized block queue when the receiver is dropped and when the StateService is being dropped * removes duplicate field definitions on StateService that were a result of a bad merge * update NotReadyToBeCommitted error message * Appear rustfmt * Fix doc links * Rename a function to initial_contextual_validity() * Do error tasks on Err, and success tasks on Ok * Simplify parent_error_map truncation * Rewrite best_tip() to use tip() * Rename latest_mem() to latest_non_finalized_state() ```sh fastmod latest_mem latest_non_finalized_state zebra* cargo fmt --all ``` * Simplify latest_non_finalized_state() using a new WatchReceiver API * Expand some error messages * Send the result after updating the channels, and document why * wait for chain_tip_update before cancelling download in mempool_cancel_mined * adds `sent_non_finalized_block_hashes` field to StateService * adds batched sent_hash insertions and checks sent hashes in queue_and_commit_non_finalized before adding a block to the queue * check that the `curr_buf` in SentHashes is not empty before pushing it to the `sent_bufs` * Apply suggestions from code review Co-authored-by: teor <teor@riseup.net> * Fix rustfmt * Check for finalized block heights using zs_contains() * adds known_utxos field to SentHashes * updates comment on SentHashes.add method * Apply suggestions from code review Co-authored-by: teor <teor@riseup.net> * return early when there's a duplicate hash in QueuedBlocks.queue instead of panicking * Make finalized UTXOs near the final checkpoint available for full block verification * Replace a checkpoint height literal with the actual config * Update mainnet and testnet checkpoints - 7 October 2022 * Fix some state service init arguments * Allow more lookahead in the downloader, but less lookahead in the syncer * Add the latest config to the tests, and fix the latest config check * Increase the number of finalized blocks checked for non-finalized block UTXO spends * fix(log): reduce verbose logs for block commits (#5348) * Remove some verbose block write channel logs * Only warn about tracing endpoint if the address is actually set * Use CloneError instead of formatting a non-cloneable error Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * Increase block verify timeout * Work around a known block timeout bug by using a shorter timeout Co-authored-by: teor <teor@riseup.net> Co-authored-by: Marek <mail@marek.onl> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-10-11 12:25:45 -07:00
indexmap = "1.9.1"
itertools = "0.10.5"
lazy_static = "1.4.0"
metrics = "0.20.1"
mset = "0.1.0"
regex = "1.6.0"
rlimit = "0.8.3"
rocksdb = { version = "0.19.0", default_features = false, features = ["lz4"] }
serde = { version = "1.0.145", features = ["serde_derive"] }
tempfile = "3.3.0"
thiserror = "1.0.37"
rayon = "1.5.3"
tokio = { version = "1.21.2", features = ["sync", "tracing"] }
tower = { version = "0.4.13", features = ["buffer", "util"] }
tracing = "0.1.37"
zebra-chain = { path = "../zebra-chain" }
zebra-test = { path = "../zebra-test/", optional = true }
proptest = { version = "0.10.1", optional = true }
proptest-derive = { version = "0.3.0", optional = true }
[dev-dependencies]
color-eyre = "0.6.2"
# This is a transitive dependency via color-eyre.
# Enable a feature that makes tinyvec compile much faster.
tinyvec = { version = "1.6.0", features = ["rustc_1_55"] }
once_cell = "1.15.0"
spandoc = "0.2.2"
hex = { version = "0.4.3", features = ["serde"] }
insta = { version = "1.21.0", features = ["ron"] }
2020-10-02 15:51:51 -07:00
proptest = "0.10.1"
4. Avoid repeated requests to peers after partial responses or errors (#3505) * fix(network): split synthetic NotFoundRegistry from message NotFoundResponse * docs(network): Improve `notfound` message documentation * refactor(network): Rename MustUseOneshotSender to MustUseClientResponseSender ``` fastmod MustUseOneshotSender MustUseClientResponseSender zebra* ``` * docs(network): fix a comment typo * refactor(network): remove generics from MustUseClientResponseSender * refactor(network): add an inventory collector to Client, but don't use it yet * feat(network): register missing peer responses as missing inventory We register this missing inventory based on peer responses, or connection errors or timeouts. Inbound message inventory tracking requires peers to send `notfound` messages. But `zcashd` skips `notfound` for blocks, so we can't rely on peer messages. This missing inventory tracking works regardless of peer `notfound` messages. * refactor(network): rename ResponseStatus to InventoryResponse ```sh fastmod ResponseStatus InventoryResponse zebra* ``` * refactor(network): rename InventoryStatus::inner() to to_inner() * fix(network): remove a redundant runtime.enter() in a test * doc(network): the exact time used to filter outbound peers doesn't matter * fix(network): handle block requests slightly more efficiently * doc(network): fix a typo * fmt(network): `cargo fmt` after rename ResponseStatus to InventoryResponse * doc(test): clarify some test comments * test(network): test synthetic notfound from connection errors and peer inventory routing * test(network): improve inbound test diagnostics * feat(network): add a proptest-impl feature to zebra-network * feat(network): add a test-only connect_isolated_with_inbound function * test(network): allow a response on the isolated peer test connection * test(network): fix failures in test synthetic notfound * test(network): Simplify SharedPeerError test assertions * test(network): test synthetic notfound from partially successful requests * test(network): MissingInventoryCollector ignores local NotFoundRegistry errors * fix(network): decrease the inventory rotation interval This stops us waiting 3-4 sync resets (4 minutes) before we retry a missing block. Now we wait 1-2 sync resets (2 minutes), which is still a reasonable rate limit. This should speed up syncing near the tip, and on testnet. * fmt(network): cargo fmt --all * cleanup(network): remove unnecessary allow(dead_code) * cleanup(network): stop importing the whole sync module into tests * doc(network): clarify syncer inventory retry constraint * doc(network): add a TODO for a fix to ensure API behaviour remains consistent * doc(network): fix a function doc typo * doc(network): clarify how we handle peers that don't send `notfound` * docs(network): clarify a test comment Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com> Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-02-14 17:44:33 -08:00
proptest-derive = "0.3.0"
2022-06-27 16:12:56 -07:00
halo2 = { package = "halo2_proofs", version = "0.2.0" }
jubjub = "0.9.0"
tokio = { version = "1.21.2", features = ["full", "tracing", "test-util"] }
zebra-chain = { path = "../zebra-chain", features = ["proptest-impl"] }
zebra-test = { path = "../zebra-test/" }