add(test): Integration test to send transactions using lightwalletd (#4068)
* Export the `zebra_state::Config::db_path` method Make it easier for tests to discover the sub-directory used to store the chain state data. * Generate code for interfacing with lightwalletd Use the `tonic-build` crate to generate Rust code for communicating with lightwalletd using gRPC. The `*.proto` files were obtained from the Zcash lightwalletd repository. * Use `block::Height` instead of `Height` Import the `block` instead to make it slightly clearer. * Add helper function to remove a file if it exists Try to remove it and ignore an error if it says that the file doesn't exist. This will be used later to remove the lock file from a copied chain state directory. * Add helper function to copy chain state dirs Copy an existing chain state directory into a new temporary directory. * Add a `BoxStateService` type alias Make it easier to write and read a boxed version of a state service. * Add a helper function to start the state service Make it easier to specify the state service to use an existing state cache directory. * Import `eyre!` macro at the module level Allow it to be used in different places without having to repeat the imports. * Add `load_tip_height_from_state_directory` helper A function to discover the current chain tip height stored in a state cache. * Add helper function to prepare partial sync. state Loads a partially synchronized cached state directory into a temporary directory that can be used by a zebrad instance, and also returns the chain tip block height of that state. * Add `perform_full_sync_starting_from` helper Runs a zebrad with an existing partially synchronized state, and finishes synchronizing it to the network chain tip. * Add function to load transactions from a block Use a provided state service to load all transactions from a block at a specified height. The state service is a generic type parameter, because `zebra_state::service::ReadStateService` is not exported publicly. Using a generic type parameter also allows the service to be wrapped in layers if needed in the future. * Add `load_transactions_from_block_after` helper A function to load transactions from a block stored in a cached state directory. The cached state must be synchronized to a chain tip higher than the requested height. * Add helper function to load some test transactions Given a partially synchronized chain state, it will extend that chain by performing a full synchronization, and obtain some transactions from one of the newly added blocks. * Update `spawn_zebrad_for_rpc_without_initial_peers` Wait until the mempool is activated. * Add method to start lightwalletd with RPC server Returns the lightwalletd instance and the port that it's listening for RPC connections. The instance can reuse an existing cached lightwalletd state if the `LIGHTWALLETD_DATA_DIR` environment variable is set. * Add a `LightwalletdRpcClient` type alias To make it easier to identify the type generated from the Protobuf files. * Add helper function to connect to lightwalletd Prepare an RPC client to send requests to a lightwalletd instance. * Add a `prepare_send_transaction_request` helper Creates a request message for lightwalletd to send a transaction. * Add test to send transactions using lightwalletd Obtain some valid transactions from future blocks and try to send them to a lightwalletd instance connected to a zebrad instance that hasn't seen those transactions yet. The transactions should be successfully queued in Zebra's mempool. * Make `zebra_directory` parameter generic Allow using a `TempDir` or a `PathBuf`. * Move lightwalletd protobuf files Place them closer to the module directory, so that it's clearer that they specify the RPC protocol for lightwalletd, and not Zebra itself. * Don't use coinbase transactions in the test Coinbase transactions are rejected by the mempool. * Don't remove state lock file It is removed automatically by Zebra when it shuts down, so if it exists it should be reported as a bug. * Force mempool to be enabled in Zebrad instance Speed up the initialization of the Zebrad instance used for lightwalletd to connect to. * Refactor to create `LIGHTWALLETD_DATA_DIR_VAR` Document how the environment variable can be used to speed up the test. * Check for process errors in spawned Zebra instance Enable checking for known process failure messages. * Add `FINISH_PARTIAL_SYNC_TIMEOUT` constant Document why it exists and how the choice of the value affects the test. * Add `LIGHTWALLETD_TEST_TIMEOUT` constant And use it for the Zebrad and the Lightwalletd instances used in the send transaction integration test. * Check `lightwalletd` process for errors Enable checking the lightwalletd process for known failure messages. * Update `tonic` and `prost` dependencies Use the latest version and fix CI failures because `rustfmt` isn't installed in the build environment. * Create `send_transaction_test` module Move the send transaction using lightwalletd test and its helper functions into a new module. * Move `LIGHTWALLETD_TEST_TIMEOUT` constant Place it in the parent `lightwalletd` module. * Move gRPC helper functions and types to `rpc` mod. Make them more accessible so that they can be used by other tests. * Create a `cached_state` module Move the test utility functions related to using a cached Zebra state into the module. * Move `perform_full_sync_starting_from` to `sync` Keep to closer to the synchronization utility functions. * Move Zebra cached state path variable constant Place it in the `cached_state` module. * Skip test if `ZEBRA_TEST_LIGHTWALLETD` is not set Make it part of the set of tests ignored as a whole if no lightwalletd tests should be executed. * Move `spawn_zebrad_for_rpc_without_initial_peers` Place it in the `launch` sub-module. * Rename `rpc` module into `wallet_grpc` Avoid any potential misunderstandings when the name is seen out of context. * Allow duplicate `heck` dependency At least until `structopt` is updated or `zebra-utils` is updated to use `clap` 3. * Fix a deny.toml typo * fix(build): CMake is required by `prost` crate Co-authored-by: teor <teor@riseup.net> Co-authored-by: Gustavo Valverde <gustavo@iterativo.do>
This commit is contained in:
Janito Vaqueiro Ferreira Filho
GitHub
parent
06dc5177f2
commit
5a94a09292
|
|
@ -293,6 +293,49 @@ version = "1.1.0"
|
|||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "d468802bab17cbc0cc575e9b053f41e72aa36bfa6b7f55e3529ffa43161b97fa"
|
||||
|
||||
[[package]]
|
||||
name = "axum"
|
||||
version = "0.5.3"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "f523b4e98ba6897ae90994bc18423d9877c54f9047b06a00ddc8122a957b1c70"
|
||||
dependencies = [
|
||||
"async-trait",
|
||||
"axum-core",
|
||||
"bitflags",
|
||||
"bytes",
|
||||
"futures-util",
|
||||
"http",
|
||||
"http-body",
|
||||
"hyper",
|
||||
"itoa 1.0.1",
|
||||
"matchit",
|
||||
"memchr",
|
||||
"mime",
|
||||
"percent-encoding",
|
||||
"pin-project-lite",
|
||||
"serde",
|
||||
"sync_wrapper",
|
||||
"tokio",
|
||||
"tower",
|
||||
"tower-http",
|
||||
"tower-layer",
|
||||
"tower-service",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "axum-core"
|
||||
version = "0.2.2"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "d3ddbd16eabff8b45f21b98671fddcc93daaa7ac4c84f8473693437226040de5"
|
||||
dependencies = [
|
||||
"async-trait",
|
||||
"bytes",
|
||||
"futures-util",
|
||||
"http",
|
||||
"http-body",
|
||||
"mime",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "backtrace"
|
||||
version = "0.3.64"
|
||||
|
|
@ -726,6 +769,15 @@ dependencies = [
|
|||
"vec_map",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "cmake"
|
||||
version = "0.1.48"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "e8ad8cef104ac57b68b89df3208164d228503abbdce70f6880ffa3d970e7443a"
|
||||
dependencies = [
|
||||
"cc",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "coarsetime"
|
||||
version = "0.1.21"
|
||||
|
|
@ -1475,6 +1527,12 @@ dependencies = [
|
|||
"subtle",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "fixedbitset"
|
||||
version = "0.4.1"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "279fb028e20b3c4c320317955b77c5e0c9701f05a1d309905d6fc702cdc5053e"
|
||||
|
||||
[[package]]
|
||||
name = "flate2"
|
||||
version = "1.0.22"
|
||||
|
|
@ -1872,6 +1930,12 @@ dependencies = [
|
|||
"unicode-segmentation",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "heck"
|
||||
version = "0.4.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "2540771e65fc8cb83cd6e8a237f70c319bd5c29f78ed1084ba5d50eeac86f7f9"
|
||||
|
||||
[[package]]
|
||||
name = "hermit-abi"
|
||||
version = "0.1.19"
|
||||
|
|
@ -1943,6 +2007,12 @@ dependencies = [
|
|||
"pin-project-lite",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "http-range-header"
|
||||
version = "0.3.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "0bfe8eed0a9285ef776bb792479ea3834e8b94e13d615c2f66d03dd50a435a29"
|
||||
|
||||
[[package]]
|
||||
name = "httparse"
|
||||
version = "1.6.0"
|
||||
|
|
@ -2008,6 +2078,18 @@ dependencies = [
|
|||
"tokio-rustls",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "hyper-timeout"
|
||||
version = "0.4.1"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "bbb958482e8c7be4bc3cf272a766a2b0bf1a6755e7a6ae777f017a31d11b13b1"
|
||||
dependencies = [
|
||||
"hyper",
|
||||
"pin-project-lite",
|
||||
"tokio",
|
||||
"tokio-io-timeout",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "hyper-tls"
|
||||
version = "0.5.0"
|
||||
|
|
@ -2381,6 +2463,12 @@ version = "0.1.9"
|
|||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "a3e378b66a060d48947b590737b30a1be76706c8dd7b8ba0f2fe3989c68a853f"
|
||||
|
||||
[[package]]
|
||||
name = "matchit"
|
||||
version = "0.5.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "73cbba799671b762df5a175adf59ce145165747bb891505c43d09aefbbf38beb"
|
||||
|
||||
[[package]]
|
||||
name = "maybe-uninit"
|
||||
version = "2.0.0"
|
||||
|
|
@ -2557,6 +2645,12 @@ version = "0.1.0"
|
|||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "05d1cf737c48148e9956a9ea00710258dcf2a7afdedc34a2c672a44866f618dc"
|
||||
|
||||
[[package]]
|
||||
name = "multimap"
|
||||
version = "0.8.3"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "e5ce46fe64a9d73be07dcbe690a38ce1b293be448fd8ce1e6c1b8062c9f72c6a"
|
||||
|
||||
[[package]]
|
||||
name = "native-tls"
|
||||
version = "0.2.8"
|
||||
|
|
@ -2957,6 +3051,16 @@ version = "2.1.0"
|
|||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "d4fd5641d01c8f18a23da7b6fe29298ff4b55afcccdf78973b24cf3175fee32e"
|
||||
|
||||
[[package]]
|
||||
name = "petgraph"
|
||||
version = "0.6.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "4a13a2fa9d0b63e5f22328828741e523766fff0ee9e779316902290dff3f824f"
|
||||
dependencies = [
|
||||
"fixedbitset",
|
||||
"indexmap",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "phf"
|
||||
version = "0.10.1"
|
||||
|
|
@ -3142,6 +3246,16 @@ version = "0.2.16"
|
|||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "eb9f9e6e233e5c4a35559a617bf40a4ec447db2e84c20b55a6f83167b7e57872"
|
||||
|
||||
[[package]]
|
||||
name = "prettyplease"
|
||||
version = "0.1.9"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "3b83ec2d0af5c5c556257ff52c9f98934e243b9fd39604bfb2a9b75ec2e97f18"
|
||||
dependencies = [
|
||||
"proc-macro2 1.0.36",
|
||||
"syn 1.0.86",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "proc-macro-crate"
|
||||
version = "0.1.5"
|
||||
|
|
@ -3230,6 +3344,61 @@ dependencies = [
|
|||
"syn 0.15.44",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "prost"
|
||||
version = "0.10.1"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "a07b0857a71a8cb765763950499cae2413c3f9cede1133478c43600d9e146890"
|
||||
dependencies = [
|
||||
"bytes",
|
||||
"prost-derive",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "prost-build"
|
||||
version = "0.10.1"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "120fbe7988713f39d780a58cf1a7ef0d7ef66c6d87e5aa3438940c05357929f4"
|
||||
dependencies = [
|
||||
"bytes",
|
||||
"cfg-if 1.0.0",
|
||||
"cmake",
|
||||
"heck 0.4.0",
|
||||
"itertools",
|
||||
"lazy_static",
|
||||
"log",
|
||||
"multimap",
|
||||
"petgraph",
|
||||
"prost",
|
||||
"prost-types",
|
||||
"regex",
|
||||
"tempfile",
|
||||
"which",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "prost-derive"
|
||||
version = "0.10.1"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "7b670f45da57fb8542ebdbb6105a925fe571b67f9e7ed9f47a06a84e72b4e7cc"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"itertools",
|
||||
"proc-macro2 1.0.36",
|
||||
"quote 1.0.15",
|
||||
"syn 1.0.86",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "prost-types"
|
||||
version = "0.10.1"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "2d0a014229361011dc8e69c8a1ec6c2e8d0f2af7c91e3ea3f5b2170298461e68"
|
||||
dependencies = [
|
||||
"bytes",
|
||||
"prost",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "quanta"
|
||||
version = "0.9.3"
|
||||
|
|
@ -4281,7 +4450,7 @@ version = "0.4.18"
|
|||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "dcb5ae327f9cc13b68763b5749770cb9e048a99bd9dfdfa58d0cf05d5f64afe0"
|
||||
dependencies = [
|
||||
"heck",
|
||||
"heck 0.3.3",
|
||||
"proc-macro-error",
|
||||
"proc-macro2 1.0.36",
|
||||
"quote 1.0.15",
|
||||
|
|
@ -4316,6 +4485,12 @@ dependencies = [
|
|||
"unicode-xid 0.2.2",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "sync_wrapper"
|
||||
version = "0.1.1"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "20518fe4a4c9acf048008599e464deb21beeae3d3578418951a189c235a7a9a8"
|
||||
|
||||
[[package]]
|
||||
name = "synstructure"
|
||||
version = "0.12.6"
|
||||
|
|
@ -4481,6 +4656,16 @@ dependencies = [
|
|||
"winapi",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "tokio-io-timeout"
|
||||
version = "1.2.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "30b74022ada614a1b4834de765f9bb43877f910cc8ce4be40e89042c9223a8bf"
|
||||
dependencies = [
|
||||
"pin-project-lite",
|
||||
"tokio",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "tokio-macros"
|
||||
version = "1.7.0"
|
||||
|
|
@ -4576,6 +4761,51 @@ dependencies = [
|
|||
"serde",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "tonic"
|
||||
version = "0.7.1"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "30fb54bf1e446f44d870d260d99957e7d11fb9d0a0f5bd1a662ad1411cc103f9"
|
||||
dependencies = [
|
||||
"async-stream",
|
||||
"async-trait",
|
||||
"axum",
|
||||
"base64",
|
||||
"bytes",
|
||||
"futures-core",
|
||||
"futures-util",
|
||||
"h2",
|
||||
"http",
|
||||
"http-body",
|
||||
"hyper",
|
||||
"hyper-timeout",
|
||||
"percent-encoding",
|
||||
"pin-project 1.0.10",
|
||||
"prost",
|
||||
"prost-derive",
|
||||
"tokio",
|
||||
"tokio-stream",
|
||||
"tokio-util 0.7.1",
|
||||
"tower",
|
||||
"tower-layer",
|
||||
"tower-service",
|
||||
"tracing",
|
||||
"tracing-futures",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "tonic-build"
|
||||
version = "0.7.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "4d17087af5c80e5d5fc8ba9878e60258065a0a757e35efe7a05b7904bece1943"
|
||||
dependencies = [
|
||||
"prettyplease",
|
||||
"proc-macro2 1.0.36",
|
||||
"prost-build",
|
||||
"quote 1.0.15",
|
||||
"syn 1.0.86",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "tor-bytes"
|
||||
version = "0.0.2"
|
||||
|
|
@ -4972,8 +5202,11 @@ dependencies = [
|
|||
"futures-core",
|
||||
"futures-util",
|
||||
"hdrhistogram",
|
||||
"indexmap",
|
||||
"pin-project 1.0.10",
|
||||
"pin-project-lite",
|
||||
"rand 0.8.5",
|
||||
"slab",
|
||||
"tokio",
|
||||
"tokio-util 0.7.1",
|
||||
"tower-layer",
|
||||
|
|
@ -5013,6 +5246,25 @@ dependencies = [
|
|||
"zebra-test",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "tower-http"
|
||||
version = "0.2.5"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "aba3f3efabf7fb41fae8534fc20a817013dd1c12cb45441efb6c82e6556b4cd8"
|
||||
dependencies = [
|
||||
"bitflags",
|
||||
"bytes",
|
||||
"futures-core",
|
||||
"futures-util",
|
||||
"http",
|
||||
"http-body",
|
||||
"http-range-header",
|
||||
"pin-project-lite",
|
||||
"tower",
|
||||
"tower-layer",
|
||||
"tower-service",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "tower-layer"
|
||||
version = "0.3.1"
|
||||
|
|
@ -6075,6 +6327,7 @@ dependencies = [
|
|||
"pin-project 1.0.10",
|
||||
"proptest",
|
||||
"proptest-derive",
|
||||
"prost",
|
||||
"rand 0.8.5",
|
||||
"regex",
|
||||
"reqwest",
|
||||
|
|
@ -6087,6 +6340,8 @@ dependencies = [
|
|||
"thiserror",
|
||||
"tokio",
|
||||
"toml",
|
||||
"tonic",
|
||||
"tonic-build",
|
||||
"tower",
|
||||
"tracing",
|
||||
"tracing-error 0.1.2",
|
||||
|
|
|
|||
|
|
@ -48,6 +48,9 @@ skip-tree = [
|
|||
# ticket #2984: owo-colors dependencies
|
||||
{ name = "color-eyre", version = "=0.5.11" },
|
||||
|
||||
# wait for structopt upgrade (or upgrade to clap 3)
|
||||
{ name = "heck", version = "=0.3.3" },
|
||||
|
||||
# wait for bellman to upgrade
|
||||
{ name = "blake2s_simd", version = "=0.5.11" },
|
||||
|
||||
|
|
|
|||
|
|
@ -24,6 +24,8 @@ RUN apt-get -qq update && \
|
|||
libclang-dev \
|
||||
clang \
|
||||
ca-certificates \
|
||||
cmake \
|
||||
protobuf-compiler \
|
||||
; \
|
||||
rm -rf /var/lib/apt/lists/* /tmp/*
|
||||
|
||||
|
|
|
|||
|
|
@ -56,7 +56,7 @@ fn gen_temp_path(prefix: &str) -> PathBuf {
|
|||
|
||||
impl Config {
|
||||
/// Returns the path for the finalized state database
|
||||
pub(crate) fn db_path(&self, network: Network) -> PathBuf {
|
||||
pub fn db_path(&self, network: Network) -> PathBuf {
|
||||
let net_dir = match network {
|
||||
Network::Mainnet => "mainnet",
|
||||
Network::Testnet => "testnet",
|
||||
|
|
|
|||
|
|
@ -67,12 +67,14 @@ proptest = { version = "0.10.1", optional = true }
|
|||
proptest-derive = { version = "0.3.0", optional = true }
|
||||
|
||||
[build-dependencies]
|
||||
tonic-build = "0.7.0"
|
||||
vergen = { version = "7.0.0", default-features = false, features = ["cargo", "git"] }
|
||||
|
||||
[dev-dependencies]
|
||||
abscissa_core = { version = "0.5", features = ["testing"] }
|
||||
hex = "0.4.3"
|
||||
once_cell = "1.10.0"
|
||||
prost = "0.10.1"
|
||||
regex = "1.5.5"
|
||||
reqwest = "0.11"
|
||||
semver = "1.0.7"
|
||||
|
|
@ -80,6 +82,7 @@ semver = "1.0.7"
|
|||
serde_json = { version = "1.0.79", features = ["preserve_order"] }
|
||||
tempfile = "3.3.0"
|
||||
tokio = { version = "1.17.0", features = ["full", "test-util"] }
|
||||
tonic = "0.7.0"
|
||||
|
||||
proptest = "0.10.1"
|
||||
proptest-derive = "0.3.0"
|
||||
|
|
|
|||
|
|
@ -55,4 +55,13 @@ fn main() {
|
|||
vergen(config).expect("non-git vergen should succeed");
|
||||
}
|
||||
}
|
||||
|
||||
tonic_build::configure()
|
||||
.build_client(true)
|
||||
.build_server(false)
|
||||
.compile(
|
||||
&["tests/common/lightwalletd/proto/service.proto"],
|
||||
&["tests/common/lightwalletd/proto"],
|
||||
)
|
||||
.expect("Failed to generate lightwalletd gRPC files");
|
||||
}
|
||||
|
|
|
|||
|
|
@ -21,17 +21,15 @@
|
|||
//! or you have poor network connectivity,
|
||||
//! skip all the network tests by setting the `ZEBRA_SKIP_NETWORK_TESTS` environmental variable.
|
||||
|
||||
use std::{
|
||||
collections::HashSet, convert::TryInto, env, net::SocketAddr, path::PathBuf, time::Duration,
|
||||
};
|
||||
use std::{collections::HashSet, convert::TryInto, env, path::PathBuf, time::Duration};
|
||||
|
||||
use color_eyre::{
|
||||
eyre::{Result, WrapErr},
|
||||
eyre::{eyre, Result, WrapErr},
|
||||
Help,
|
||||
};
|
||||
|
||||
use zebra_chain::{
|
||||
block::Height,
|
||||
block,
|
||||
parameters::Network::{self, *},
|
||||
};
|
||||
use zebra_network::constants::PORT_IN_USE_ERROR;
|
||||
|
|
@ -49,9 +47,13 @@ mod common;
|
|||
use common::{
|
||||
check::{is_zebrad_version, EphemeralCheck, EphemeralConfig},
|
||||
config::{default_test_config, persistent_test_config, testdir},
|
||||
launch::{ZebradTestDirExt, BETWEEN_NODES_DELAY, LAUNCH_DELAY, LIGHTWALLETD_DELAY},
|
||||
launch::{
|
||||
spawn_zebrad_for_rpc_without_initial_peers, ZebradTestDirExt, BETWEEN_NODES_DELAY,
|
||||
LAUNCH_DELAY, LIGHTWALLETD_DELAY,
|
||||
},
|
||||
lightwalletd::{
|
||||
random_known_rpc_port_config, zebra_skip_lightwalletd_tests, LightWalletdTestDirExt,
|
||||
LIGHTWALLETD_TEST_TIMEOUT,
|
||||
},
|
||||
sync::{
|
||||
create_cached_database_height, sync_until, MempoolBehavior, LARGE_CHECKPOINT_TEST_HEIGHT,
|
||||
|
|
@ -531,7 +533,7 @@ fn restart_stop_at_height() -> Result<()> {
|
|||
Ok(())
|
||||
}
|
||||
|
||||
fn restart_stop_at_height_for_network(network: Network, height: Height) -> Result<()> {
|
||||
fn restart_stop_at_height_for_network(network: Network, height: block::Height) -> Result<()> {
|
||||
let reuse_tempdir = sync_until(
|
||||
height,
|
||||
network,
|
||||
|
|
@ -566,7 +568,7 @@ fn restart_stop_at_height_for_network(network: Network, height: Height) -> Resul
|
|||
#[test]
|
||||
fn activate_mempool_mainnet() -> Result<()> {
|
||||
sync_until(
|
||||
Height(TINY_CHECKPOINT_TEST_HEIGHT.0 + 1),
|
||||
block::Height(TINY_CHECKPOINT_TEST_HEIGHT.0 + 1),
|
||||
Mainnet,
|
||||
STOP_AT_HEIGHT_REGEX,
|
||||
TINY_CHECKPOINT_TIMEOUT,
|
||||
|
|
@ -673,7 +675,7 @@ fn full_sync_test(network: Network, timeout_argument_name: &str) -> Result<()> {
|
|||
|
||||
if let Some(timeout_minutes) = timeout_argument {
|
||||
sync_until(
|
||||
Height::MAX,
|
||||
block::Height::MAX,
|
||||
network,
|
||||
SYNC_FINISHED_REGEX,
|
||||
Duration::from_secs(60 * timeout_minutes),
|
||||
|
|
@ -1445,7 +1447,6 @@ where
|
|||
// See #1781.
|
||||
#[cfg(target_os = "linux")]
|
||||
if node2.is_running() {
|
||||
use color_eyre::eyre::eyre;
|
||||
return node2
|
||||
.kill_on_error::<(), _>(Err(eyre!(
|
||||
"conflicted node2 was still running, but the test expected a panic"
|
||||
|
|
@ -1497,8 +1498,11 @@ async fn fully_synced_rpc_test() -> Result<()> {
|
|||
|
||||
let network = Network::Mainnet;
|
||||
|
||||
let (_zebrad, zebra_rpc_address) =
|
||||
spawn_zebrad_for_rpc_without_initial_peers(network, cached_state_path)?;
|
||||
let (_zebrad, zebra_rpc_address) = spawn_zebrad_for_rpc_without_initial_peers(
|
||||
network,
|
||||
cached_state_path,
|
||||
LIGHTWALLETD_TEST_TIMEOUT,
|
||||
)?;
|
||||
|
||||
// Make a getblock test that works only on synced node (high block number).
|
||||
// The block is before the mandatory checkpoint, so the checkpoint cached state can be used
|
||||
|
|
@ -1528,31 +1532,10 @@ async fn fully_synced_rpc_test() -> Result<()> {
|
|||
Ok(())
|
||||
}
|
||||
|
||||
/// Spawns a zebrad instance to interact with lightwalletd, but without an internet connection.
|
||||
/// Test sending transactions using a lightwalletd instance connected to a zebrad instance.
|
||||
///
|
||||
/// This prevents it from downloading blocks. Instead, the `zebra_directory` parameter allows
|
||||
/// providing an initial state to the zebrad instance.
|
||||
fn spawn_zebrad_for_rpc_without_initial_peers(
|
||||
network: Network,
|
||||
zebra_directory: PathBuf,
|
||||
) -> Result<(TestChild<PathBuf>, SocketAddr)> {
|
||||
let mut config = random_known_rpc_port_config()
|
||||
.expect("Failed to create a config file with a known RPC listener port");
|
||||
|
||||
config.state.ephemeral = false;
|
||||
config.network.initial_mainnet_peers = HashSet::new();
|
||||
config.network.initial_testnet_peers = HashSet::new();
|
||||
config.network.network = network;
|
||||
|
||||
let mut zebrad = zebra_directory
|
||||
.with_config(&mut config)?
|
||||
.spawn_child(args!["start"])?
|
||||
.with_timeout(Duration::from_secs(60 * 60))
|
||||
.bypass_test_capture(true);
|
||||
|
||||
let rpc_address = config.rpc.listen_addr.unwrap();
|
||||
|
||||
zebrad.expect_stdout_line_matches(&format!("Opened RPC endpoint at {}", rpc_address))?;
|
||||
|
||||
Ok((zebrad, rpc_address))
|
||||
/// See [`common::lightwalletd::send_transaction_test`] for more information.
|
||||
#[tokio::test]
|
||||
async fn sending_transactions_using_lightwalletd() -> Result<()> {
|
||||
common::lightwalletd::send_transaction_test::run().await
|
||||
}
|
||||
|
|
|
|||
|
|
@ -0,0 +1,111 @@
|
|||
//! Utility functions for tests that used cached Zebra state.
|
||||
|
||||
use std::path::{Path, PathBuf};
|
||||
|
||||
use color_eyre::eyre::{eyre, Result};
|
||||
use tempfile::TempDir;
|
||||
use tokio::fs;
|
||||
use tower::{util::BoxService, Service};
|
||||
|
||||
use zebra_chain::{block, chain_tip::ChainTip, parameters::Network};
|
||||
use zebra_state::{ChainTipChange, LatestChainTip};
|
||||
|
||||
use crate::common::config::testdir;
|
||||
|
||||
/// Path to a directory containing a cached Zebra state.
|
||||
pub const ZEBRA_CACHED_STATE_DIR_VAR: &str = "ZEBRA_CACHED_STATE_DIR";
|
||||
|
||||
/// Type alias for a boxed state service.
|
||||
pub type BoxStateService =
|
||||
BoxService<zebra_state::Request, zebra_state::Response, zebra_state::BoxError>;
|
||||
|
||||
/// Starts a state service using the provided `cache_dir` as the directory with the chain state.
|
||||
pub async fn start_state_service_with_cache_dir(
|
||||
network: Network,
|
||||
cache_dir: impl Into<PathBuf>,
|
||||
) -> Result<(
|
||||
BoxStateService,
|
||||
impl Service<
|
||||
zebra_state::ReadRequest,
|
||||
Response = zebra_state::ReadResponse,
|
||||
Error = zebra_state::BoxError,
|
||||
>,
|
||||
LatestChainTip,
|
||||
ChainTipChange,
|
||||
)> {
|
||||
let config = zebra_state::Config {
|
||||
cache_dir: cache_dir.into(),
|
||||
..zebra_state::Config::default()
|
||||
};
|
||||
|
||||
Ok(zebra_state::init(config, network))
|
||||
}
|
||||
|
||||
/// Loads the chain tip height from the state stored in a specified directory.
|
||||
pub async fn load_tip_height_from_state_directory(
|
||||
network: Network,
|
||||
state_path: &Path,
|
||||
) -> Result<block::Height> {
|
||||
let (_state_service, _read_state_service, latest_chain_tip, _chain_tip_change) =
|
||||
start_state_service_with_cache_dir(network, state_path).await?;
|
||||
|
||||
let chain_tip_height = latest_chain_tip
|
||||
.best_tip_height()
|
||||
.ok_or_else(|| eyre!("State directory doesn't have a chain tip block"))?;
|
||||
|
||||
Ok(chain_tip_height)
|
||||
}
|
||||
|
||||
/// Recursively copy a chain state directory into a new temporary directory.
|
||||
pub async fn copy_state_directory(source: impl AsRef<Path>) -> Result<TempDir> {
|
||||
let destination = testdir()?;
|
||||
|
||||
let mut remaining_directories = vec![PathBuf::from(source.as_ref())];
|
||||
|
||||
while let Some(directory) = remaining_directories.pop() {
|
||||
let sub_directories =
|
||||
copy_directory(&directory, source.as_ref(), destination.as_ref()).await?;
|
||||
|
||||
remaining_directories.extend(sub_directories);
|
||||
}
|
||||
|
||||
Ok(destination)
|
||||
}
|
||||
|
||||
/// Copy the contents of a directory, and return the sub-directories it contains.
|
||||
///
|
||||
/// Copies all files from the `directory` into the destination specified by the concatenation of
|
||||
/// the `base_destination_path` and `directory` stripped of its `prefix`.
|
||||
async fn copy_directory(
|
||||
directory: &Path,
|
||||
prefix: &Path,
|
||||
base_destination_path: &Path,
|
||||
) -> Result<Vec<PathBuf>> {
|
||||
let mut sub_directories = Vec::new();
|
||||
let mut entries = fs::read_dir(directory).await?;
|
||||
|
||||
let destination =
|
||||
base_destination_path.join(directory.strip_prefix(prefix).expect("Invalid path prefix"));
|
||||
|
||||
fs::create_dir_all(&destination).await?;
|
||||
|
||||
while let Some(entry) = entries.next_entry().await? {
|
||||
let entry_path = entry.path();
|
||||
let file_type = entry.file_type().await?;
|
||||
|
||||
if file_type.is_file() {
|
||||
let file_name = entry_path.file_name().expect("Missing file name");
|
||||
let destination_path = destination.join(file_name);
|
||||
|
||||
fs::copy(&entry_path, destination_path).await?;
|
||||
} else if file_type.is_dir() {
|
||||
sub_directories.push(entry_path);
|
||||
} else if file_type.is_symlink() {
|
||||
unimplemented!("Symbolic link support is currently not necessary");
|
||||
} else {
|
||||
panic!("Unknown file type");
|
||||
}
|
||||
}
|
||||
|
||||
Ok(sub_directories)
|
||||
}
|
||||
|
|
@ -6,18 +6,26 @@
|
|||
//! This file is only for test library code.
|
||||
|
||||
use std::{
|
||||
collections::HashSet,
|
||||
env,
|
||||
net::SocketAddr,
|
||||
path::{Path, PathBuf},
|
||||
time::Duration,
|
||||
};
|
||||
|
||||
use color_eyre::eyre::Result;
|
||||
|
||||
use zebra_chain::parameters::Network;
|
||||
use zebra_test::{
|
||||
args,
|
||||
command::{Arguments, TestDirExt, NO_MATCHES_REGEX_ITER},
|
||||
prelude::*,
|
||||
};
|
||||
use zebrad::config::ZebradConfig;
|
||||
|
||||
use zebra_test::{
|
||||
command::{Arguments, TestDirExt},
|
||||
prelude::*,
|
||||
use crate::{
|
||||
common::lightwalletd::random_known_rpc_port_config, PROCESS_FAILURE_MESSAGES,
|
||||
ZEBRA_FAILURE_MESSAGES,
|
||||
};
|
||||
|
||||
/// After we launch `zebrad`, wait this long for the command to start up,
|
||||
|
|
@ -173,6 +181,46 @@ where
|
|||
}
|
||||
}
|
||||
|
||||
/// Spawns a zebrad instance to interact with lightwalletd, but without an internet connection.
|
||||
///
|
||||
/// This prevents it from downloading blocks. Instead, the `zebra_directory` parameter allows
|
||||
/// providing an initial state to the zebrad instance.
|
||||
pub fn spawn_zebrad_for_rpc_without_initial_peers<P: ZebradTestDirExt>(
|
||||
network: Network,
|
||||
zebra_directory: P,
|
||||
timeout: Duration,
|
||||
) -> Result<(TestChild<P>, SocketAddr)> {
|
||||
let mut config = random_known_rpc_port_config()
|
||||
.expect("Failed to create a config file with a known RPC listener port");
|
||||
|
||||
config.state.ephemeral = false;
|
||||
config.network.initial_mainnet_peers = HashSet::new();
|
||||
config.network.initial_testnet_peers = HashSet::new();
|
||||
config.network.network = network;
|
||||
config.mempool.debug_enable_at_height = Some(0);
|
||||
|
||||
let mut zebrad = zebra_directory
|
||||
.with_config(&mut config)?
|
||||
.spawn_child(args!["start"])?
|
||||
.bypass_test_capture(true)
|
||||
.with_timeout(timeout)
|
||||
.with_failure_regex_iter(
|
||||
// TODO: replace with a function that returns the full list and correct return type
|
||||
ZEBRA_FAILURE_MESSAGES
|
||||
.iter()
|
||||
.chain(PROCESS_FAILURE_MESSAGES)
|
||||
.cloned(),
|
||||
NO_MATCHES_REGEX_ITER.iter().cloned(),
|
||||
);
|
||||
|
||||
let rpc_address = config.rpc.listen_addr.unwrap();
|
||||
|
||||
zebrad.expect_stdout_line_matches("activating mempool")?;
|
||||
zebrad.expect_stdout_line_matches(&format!("Opened RPC endpoint at {}", rpc_address))?;
|
||||
|
||||
Ok((zebrad, rpc_address))
|
||||
}
|
||||
|
||||
/// Panics if `$pred` is false, with an error report containing:
|
||||
/// * context from `$source`, and
|
||||
/// * an optional wrapper error, using `$fmt_arg`+ as a format string and
|
||||
|
|
|
|||
|
|
@ -5,7 +5,7 @@
|
|||
//! Test functions in this file will not be run.
|
||||
//! This file is only for test library code.
|
||||
|
||||
use std::{env, net::SocketAddr, path::Path};
|
||||
use std::{env, net::SocketAddr, path::Path, time::Duration};
|
||||
|
||||
use zebra_test::{
|
||||
command::{Arguments, TestChild, TestDirExt},
|
||||
|
|
@ -16,6 +16,9 @@ use zebrad::config::ZebradConfig;
|
|||
|
||||
use super::{config::default_test_config, launch::ZebradTestDirExt};
|
||||
|
||||
pub mod send_transaction_test;
|
||||
pub mod wallet_grpc;
|
||||
|
||||
/// The name of the env var that enables Zebra lightwalletd integration tests.
|
||||
/// These tests need a `lightwalletd` binary in the test machine's path.
|
||||
///
|
||||
|
|
@ -27,6 +30,9 @@ use super::{config::default_test_config, launch::ZebradTestDirExt};
|
|||
/// But the network tests are *disabled* by their environmental variables.
|
||||
const ZEBRA_TEST_LIGHTWALLETD: &str = "ZEBRA_TEST_LIGHTWALLETD";
|
||||
|
||||
/// The maximum time that a `lightwalletd` integration test is expected to run.
|
||||
pub const LIGHTWALLETD_TEST_TIMEOUT: Duration = Duration::from_secs(60 * 60);
|
||||
|
||||
/// Should we skip Zebra lightwalletd integration tests?
|
||||
#[allow(clippy::print_stderr)]
|
||||
pub fn zebra_skip_lightwalletd_tests() -> bool {
|
||||
|
|
|
|||
|
|
@ -0,0 +1,66 @@
|
|||
// Copyright (c) 2019-2020 The Zcash developers
|
||||
// Distributed under the MIT software license, see the accompanying
|
||||
// file COPYING or https://www.opensource.org/licenses/mit-license.php .
|
||||
|
||||
syntax = "proto3";
|
||||
package cash.z.wallet.sdk.rpc;
|
||||
option go_package = "lightwalletd/walletrpc";
|
||||
option swift_prefix = "";
|
||||
// Remember that proto3 fields are all optional. A field that is not present will be set to its zero value.
|
||||
// bytes fields of hashes are in canonical little-endian format.
|
||||
|
||||
// CompactBlock is a packaging of ONLY the data from a block that's needed to:
|
||||
// 1. Detect a payment to your shielded Sapling address
|
||||
// 2. Detect a spend of your shielded Sapling notes
|
||||
// 3. Update your witnesses to generate new Sapling spend proofs.
|
||||
message CompactBlock {
|
||||
uint32 protoVersion = 1; // the version of this wire format, for storage
|
||||
uint64 height = 2; // the height of this block
|
||||
bytes hash = 3; // the ID (hash) of this block, same as in block explorers
|
||||
bytes prevHash = 4; // the ID (hash) of this block's predecessor
|
||||
uint32 time = 5; // Unix epoch time when the block was mined
|
||||
bytes header = 6; // (hash, prevHash, and time) OR (full header)
|
||||
repeated CompactTx vtx = 7; // zero or more compact transactions from this block
|
||||
}
|
||||
|
||||
// CompactTx contains the minimum information for a wallet to know if this transaction
|
||||
// is relevant to it (either pays to it or spends from it) via shielded elements
|
||||
// only. This message will not encode a transparent-to-transparent transaction.
|
||||
message CompactTx {
|
||||
uint64 index = 1; // the index within the full block
|
||||
bytes hash = 2; // the ID (hash) of this transaction, same as in block explorers
|
||||
|
||||
// The transaction fee: present if server can provide. In the case of a
|
||||
// stateless server and a transaction with transparent inputs, this will be
|
||||
// unset because the calculation requires reference to prior transactions.
|
||||
// in a pure-Sapling context, the fee will be calculable as:
|
||||
// valueBalance + (sum(vPubNew) - sum(vPubOld) - sum(tOut))
|
||||
uint32 fee = 3;
|
||||
|
||||
repeated CompactSaplingSpend spends = 4; // inputs
|
||||
repeated CompactSaplingOutput outputs = 5; // outputs
|
||||
repeated CompactOrchardAction actions = 6;
|
||||
}
|
||||
|
||||
// CompactSaplingSpend is a Sapling Spend Description as described in 7.3 of the Zcash
|
||||
// protocol specification.
|
||||
message CompactSaplingSpend {
|
||||
bytes nf = 1; // nullifier (see the Zcash protocol specification)
|
||||
}
|
||||
|
||||
// output is a Sapling Output Description as described in section 7.4 of the
|
||||
// Zcash protocol spec. Total size is 948.
|
||||
message CompactSaplingOutput {
|
||||
bytes cmu = 1; // note commitment u-coordinate
|
||||
bytes epk = 2; // ephemeral public key
|
||||
bytes ciphertext = 3; // first 52 bytes of ciphertext
|
||||
}
|
||||
|
||||
// https://github.com/zcash/zips/blob/main/zip-0225.rst#orchard-action-description-orchardaction
|
||||
// (but not all fields are needed)
|
||||
message CompactOrchardAction {
|
||||
bytes nullifier = 1; // [32] The nullifier of the input note
|
||||
bytes cmx = 2; // [32] The x-coordinate of the note commitment for the output note
|
||||
bytes ephemeralKey = 3; // [32] An encoding of an ephemeral Pallas public key
|
||||
bytes ciphertext = 4; // [52] The note plaintext component of the encCiphertext field
|
||||
}
|
||||
|
|
@ -0,0 +1,185 @@
|
|||
// Copyright (c) 2019-2020 The Zcash developers
|
||||
// Distributed under the MIT software license, see the accompanying
|
||||
// file COPYING or https://www.opensource.org/licenses/mit-license.php .
|
||||
|
||||
syntax = "proto3";
|
||||
package cash.z.wallet.sdk.rpc;
|
||||
option go_package = "lightwalletd/walletrpc";
|
||||
option swift_prefix = "";
|
||||
import "compact_formats.proto";
|
||||
|
||||
// A BlockID message contains identifiers to select a block: a height or a
|
||||
// hash. Specification by hash is not implemented, but may be in the future.
|
||||
message BlockID {
|
||||
uint64 height = 1;
|
||||
bytes hash = 2;
|
||||
}
|
||||
|
||||
// BlockRange specifies a series of blocks from start to end inclusive.
|
||||
// Both BlockIDs must be heights; specification by hash is not yet supported.
|
||||
message BlockRange {
|
||||
BlockID start = 1;
|
||||
BlockID end = 2;
|
||||
}
|
||||
|
||||
// A TxFilter contains the information needed to identify a particular
|
||||
// transaction: either a block and an index, or a direct transaction hash.
|
||||
// Currently, only specification by hash is supported.
|
||||
message TxFilter {
|
||||
BlockID block = 1; // block identifier, height or hash
|
||||
uint64 index = 2; // index within the block
|
||||
bytes hash = 3; // transaction ID (hash, txid)
|
||||
}
|
||||
|
||||
// RawTransaction contains the complete transaction data. It also optionally includes
|
||||
// the block height in which the transaction was included, or, when returned
|
||||
// by GetMempoolStream(), the latest block height.
|
||||
message RawTransaction {
|
||||
bytes data = 1; // exact data returned by Zcash 'getrawtransaction'
|
||||
int64 height = 2; // height that the transaction was mined (or -1)
|
||||
}
|
||||
|
||||
// A SendResponse encodes an error code and a string. It is currently used
|
||||
// only by SendTransaction(). If error code is zero, the operation was
|
||||
// successful; if non-zero, it and the message specify the failure.
|
||||
message SendResponse {
|
||||
int32 errorCode = 1;
|
||||
string errorMessage = 2;
|
||||
}
|
||||
|
||||
// Chainspec is a placeholder to allow specification of a particular chain fork.
|
||||
message ChainSpec {}
|
||||
|
||||
// Empty is for gRPCs that take no arguments, currently only GetLightdInfo.
|
||||
message Empty {}
|
||||
|
||||
// LightdInfo returns various information about this lightwalletd instance
|
||||
// and the state of the blockchain.
|
||||
message LightdInfo {
|
||||
string version = 1;
|
||||
string vendor = 2;
|
||||
bool taddrSupport = 3; // true
|
||||
string chainName = 4; // either "main" or "test"
|
||||
uint64 saplingActivationHeight = 5; // depends on mainnet or testnet
|
||||
string consensusBranchId = 6; // protocol identifier, see consensus/upgrades.cpp
|
||||
uint64 blockHeight = 7; // latest block on the best chain
|
||||
string gitCommit = 8;
|
||||
string branch = 9;
|
||||
string buildDate = 10;
|
||||
string buildUser = 11;
|
||||
uint64 estimatedHeight = 12; // less than tip height if zcashd is syncing
|
||||
string zcashdBuild = 13; // example: "v4.1.1-877212414"
|
||||
string zcashdSubversion = 14; // example: "/MagicBean:4.1.1/"
|
||||
}
|
||||
|
||||
// TransparentAddressBlockFilter restricts the results to the given address
|
||||
// or block range.
|
||||
message TransparentAddressBlockFilter {
|
||||
string address = 1; // t-address
|
||||
BlockRange range = 2; // start, end heights
|
||||
}
|
||||
|
||||
// Duration is currently used only for testing, so that the Ping rpc
|
||||
// can simulate a delay, to create many simultaneous connections. Units
|
||||
// are microseconds.
|
||||
message Duration {
|
||||
int64 intervalUs = 1;
|
||||
}
|
||||
|
||||
// PingResponse is used to indicate concurrency, how many Ping rpcs
|
||||
// are executing upon entry and upon exit (after the delay).
|
||||
// This rpc is used for testing only.
|
||||
message PingResponse {
|
||||
int64 entry = 1;
|
||||
int64 exit = 2;
|
||||
}
|
||||
|
||||
message Address {
|
||||
string address = 1;
|
||||
}
|
||||
message AddressList {
|
||||
repeated string addresses = 1;
|
||||
}
|
||||
message Balance {
|
||||
int64 valueZat = 1;
|
||||
}
|
||||
|
||||
message Exclude {
|
||||
repeated bytes txid = 1;
|
||||
}
|
||||
|
||||
// The TreeState is derived from the Zcash z_gettreestate rpc.
|
||||
message TreeState {
|
||||
string network = 1; // "main" or "test"
|
||||
uint64 height = 2;
|
||||
string hash = 3; // block id
|
||||
uint32 time = 4; // Unix epoch time when the block was mined
|
||||
string tree = 5; // sapling commitment tree state
|
||||
}
|
||||
|
||||
// Results are sorted by height, which makes it easy to issue another
|
||||
// request that picks up from where the previous left off.
|
||||
message GetAddressUtxosArg {
|
||||
repeated string addresses = 1;
|
||||
uint64 startHeight = 2;
|
||||
uint32 maxEntries = 3; // zero means unlimited
|
||||
}
|
||||
message GetAddressUtxosReply {
|
||||
string address = 6;
|
||||
bytes txid = 1;
|
||||
int32 index = 2;
|
||||
bytes script = 3;
|
||||
int64 valueZat = 4;
|
||||
uint64 height = 5;
|
||||
}
|
||||
message GetAddressUtxosReplyList {
|
||||
repeated GetAddressUtxosReply addressUtxos = 1;
|
||||
}
|
||||
|
||||
service CompactTxStreamer {
|
||||
// Return the height of the tip of the best chain
|
||||
rpc GetLatestBlock(ChainSpec) returns (BlockID) {}
|
||||
// Return the compact block corresponding to the given block identifier
|
||||
rpc GetBlock(BlockID) returns (CompactBlock) {}
|
||||
// Return a list of consecutive compact blocks
|
||||
rpc GetBlockRange(BlockRange) returns (stream CompactBlock) {}
|
||||
|
||||
// Return the requested full (not compact) transaction (as from zcashd)
|
||||
rpc GetTransaction(TxFilter) returns (RawTransaction) {}
|
||||
// Submit the given transaction to the Zcash network
|
||||
rpc SendTransaction(RawTransaction) returns (SendResponse) {}
|
||||
|
||||
// Return the txids corresponding to the given t-address within the given block range
|
||||
rpc GetTaddressTxids(TransparentAddressBlockFilter) returns (stream RawTransaction) {}
|
||||
rpc GetTaddressBalance(AddressList) returns (Balance) {}
|
||||
rpc GetTaddressBalanceStream(stream Address) returns (Balance) {}
|
||||
|
||||
// Return the compact transactions currently in the mempool; the results
|
||||
// can be a few seconds out of date. If the Exclude list is empty, return
|
||||
// all transactions; otherwise return all *except* those in the Exclude list
|
||||
// (if any); this allows the client to avoid receiving transactions that it
|
||||
// already has (from an earlier call to this rpc). The transaction IDs in the
|
||||
// Exclude list can be shortened to any number of bytes to make the request
|
||||
// more bandwidth-efficient; if two or more transactions in the mempool
|
||||
// match a shortened txid, they are all sent (none is excluded). Transactions
|
||||
// in the exclude list that don't exist in the mempool are ignored.
|
||||
rpc GetMempoolTx(Exclude) returns (stream CompactTx) {}
|
||||
|
||||
// Return a stream of current Mempool transactions. This will keep the output stream open while
|
||||
// there are mempool transactions. It will close the returned stream when a new block is mined.
|
||||
rpc GetMempoolStream(Empty) returns (stream RawTransaction) {}
|
||||
|
||||
// GetTreeState returns the note commitment tree state corresponding to the given block.
|
||||
// See section 3.7 of the Zcash protocol specification. It returns several other useful
|
||||
// values also (even though they can be obtained using GetBlock).
|
||||
// The block can be specified by either height or hash.
|
||||
rpc GetTreeState(BlockID) returns (TreeState) {}
|
||||
|
||||
rpc GetAddressUtxos(GetAddressUtxosArg) returns (GetAddressUtxosReplyList) {}
|
||||
rpc GetAddressUtxosStream(GetAddressUtxosArg) returns (stream GetAddressUtxosReply) {}
|
||||
|
||||
// Return information about this lightwalletd instance and the blockchain
|
||||
rpc GetLightdInfo(Empty) returns (LightdInfo) {}
|
||||
// Testing-only, requires lightwalletd --ping-very-insecure (do not enable in production)
|
||||
rpc Ping(Duration) returns (PingResponse) {}
|
||||
}
|
||||
|
|
@ -0,0 +1,221 @@
|
|||
//! Test sending transactions using a lightwalletd instance connected to a zebrad instance.
|
||||
//!
|
||||
//! This test requires a cached chain state that is partially synchronized, i.e., it should be a
|
||||
//! few blocks below the network chain tip height.
|
||||
//!
|
||||
//! The transactions to use to send are obtained from the blocks synchronized by a temporary zebrad
|
||||
//! instance that are higher than the chain tip of the cached state.
|
||||
//!
|
||||
//! The zebrad instance connected to lightwalletd uses the cached state and does not connect to any
|
||||
//! external peers, which prevents it from downloading the blocks from where the test transactions
|
||||
//! were obtained. This is to ensure that zebra does not reject the transactions because they have
|
||||
//! already been seen in a block.
|
||||
|
||||
use std::{
|
||||
env,
|
||||
path::{Path, PathBuf},
|
||||
sync::Arc,
|
||||
};
|
||||
|
||||
use color_eyre::eyre::{eyre, Result};
|
||||
use futures::TryFutureExt;
|
||||
use tempfile::TempDir;
|
||||
use tower::{Service, ServiceExt};
|
||||
|
||||
use zebra_chain::{
|
||||
block, chain_tip::ChainTip, parameters::Network, serialization::ZcashSerialize,
|
||||
transaction::Transaction,
|
||||
};
|
||||
use zebra_state::HashOrHeight;
|
||||
|
||||
use crate::common::{
|
||||
cached_state::{
|
||||
copy_state_directory, load_tip_height_from_state_directory,
|
||||
start_state_service_with_cache_dir, ZEBRA_CACHED_STATE_DIR_VAR,
|
||||
},
|
||||
launch::spawn_zebrad_for_rpc_without_initial_peers,
|
||||
lightwalletd::{
|
||||
wallet_grpc::{self, connect_to_lightwalletd, spawn_lightwalletd_with_rpc_server},
|
||||
zebra_skip_lightwalletd_tests, LIGHTWALLETD_TEST_TIMEOUT,
|
||||
},
|
||||
sync::perform_full_sync_starting_from,
|
||||
};
|
||||
|
||||
/// The test entry point.
|
||||
pub async fn run() -> Result<()> {
|
||||
zebra_test::init();
|
||||
|
||||
// Skip the test unless the user specifically asked for it
|
||||
if zebra_skip_lightwalletd_tests() {
|
||||
return Ok(());
|
||||
}
|
||||
|
||||
let cached_state_path = match env::var_os(ZEBRA_CACHED_STATE_DIR_VAR) {
|
||||
Some(argument) => PathBuf::from(argument),
|
||||
None => {
|
||||
tracing::info!(
|
||||
"skipped send transactions using lightwalletd test, \
|
||||
set the {ZEBRA_CACHED_STATE_DIR_VAR:?} environment variable to run the test",
|
||||
);
|
||||
return Ok(());
|
||||
}
|
||||
};
|
||||
|
||||
let network = Network::Mainnet;
|
||||
|
||||
let (transactions, partial_sync_path) =
|
||||
load_transactions_from_a_future_block(network, cached_state_path).await?;
|
||||
|
||||
let (_zebrad, zebra_rpc_address) = spawn_zebrad_for_rpc_without_initial_peers(
|
||||
Network::Mainnet,
|
||||
partial_sync_path,
|
||||
LIGHTWALLETD_TEST_TIMEOUT,
|
||||
)?;
|
||||
|
||||
let (_lightwalletd, lightwalletd_rpc_port) =
|
||||
spawn_lightwalletd_with_rpc_server(zebra_rpc_address)?;
|
||||
|
||||
let mut rpc_client = connect_to_lightwalletd(lightwalletd_rpc_port).await?;
|
||||
|
||||
for transaction in transactions {
|
||||
let expected_response = wallet_grpc::SendResponse {
|
||||
error_code: 0,
|
||||
error_message: format!("\"{}\"", transaction.hash()),
|
||||
};
|
||||
|
||||
let request = prepare_send_transaction_request(transaction);
|
||||
|
||||
let response = rpc_client.send_transaction(request).await?.into_inner();
|
||||
|
||||
assert_eq!(response, expected_response);
|
||||
}
|
||||
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Loads transactions from a block that's after the chain tip of the cached state.
|
||||
///
|
||||
/// This copies the cached state into a temporary directory when it is needed to avoid overwriting
|
||||
/// anything. Two copies are made of the cached state.
|
||||
///
|
||||
/// The first copy is used by a zebrad instance connected to the network that finishes
|
||||
/// synchronizing the chain. The transactions are loaded from this updated state.
|
||||
///
|
||||
/// The second copy of the state is returned together with the transactions. This means that the
|
||||
/// returned tuple contains the temporary directory with the partially synchronized chain, and a
|
||||
/// list of valid transactions that are not in any of the blocks present in that partially
|
||||
/// synchronized chain.
|
||||
async fn load_transactions_from_a_future_block(
|
||||
network: Network,
|
||||
cached_state_path: PathBuf,
|
||||
) -> Result<(Vec<Arc<Transaction>>, TempDir)> {
|
||||
let (partial_sync_path, partial_sync_height) =
|
||||
prepare_partial_sync(network, cached_state_path).await?;
|
||||
|
||||
let full_sync_path =
|
||||
perform_full_sync_starting_from(network, partial_sync_path.as_ref()).await?;
|
||||
|
||||
let transactions =
|
||||
load_transactions_from_block_after(partial_sync_height, network, full_sync_path.as_ref())
|
||||
.await?;
|
||||
|
||||
Ok((transactions, partial_sync_path))
|
||||
}
|
||||
|
||||
/// Prepares the temporary directory of the partially synchronized chain.
|
||||
///
|
||||
/// Returns a temporary directory that can be used by a Zebra instance, as well as the chain tip
|
||||
/// height of the partially synchronized chain.
|
||||
async fn prepare_partial_sync(
|
||||
network: Network,
|
||||
cached_zebra_state: PathBuf,
|
||||
) -> Result<(TempDir, block::Height)> {
|
||||
let partial_sync_path = copy_state_directory(cached_zebra_state).await?;
|
||||
let partial_sync_state_dir = partial_sync_path.as_ref().join("state");
|
||||
let tip_height = load_tip_height_from_state_directory(network, &partial_sync_state_dir).await?;
|
||||
|
||||
Ok((partial_sync_path, tip_height))
|
||||
}
|
||||
|
||||
/// Loads transactions from a block that's after the specified `height`.
|
||||
///
|
||||
/// Starts at the block after the block at the specified `height`, and stops when it finds a block
|
||||
/// from where it can load at least one non-coinbase transaction.
|
||||
///
|
||||
/// # Panics
|
||||
///
|
||||
/// If the specified `state_path` contains a chain state that's not synchronized to a tip that's
|
||||
/// after `height`.
|
||||
async fn load_transactions_from_block_after(
|
||||
height: block::Height,
|
||||
network: Network,
|
||||
state_path: &Path,
|
||||
) -> Result<Vec<Arc<Transaction>>> {
|
||||
let (_read_write_state_service, mut state, latest_chain_tip, _chain_tip_change) =
|
||||
start_state_service_with_cache_dir(network, state_path.join("state")).await?;
|
||||
|
||||
let tip_height = latest_chain_tip
|
||||
.best_tip_height()
|
||||
.ok_or_else(|| eyre!("State directory doesn't have a chain tip block"))?;
|
||||
|
||||
assert!(
|
||||
tip_height > height,
|
||||
"Chain not synchronized to a block after the specified height"
|
||||
);
|
||||
|
||||
let mut target_height = height.0;
|
||||
let mut transactions = Vec::new();
|
||||
|
||||
while transactions.is_empty() {
|
||||
transactions =
|
||||
load_transactions_from_block(block::Height(target_height), &mut state).await?;
|
||||
|
||||
transactions.retain(|transaction| !transaction.is_coinbase());
|
||||
|
||||
target_height += 1;
|
||||
}
|
||||
|
||||
Ok(transactions)
|
||||
}
|
||||
|
||||
/// Performs a request to the provided read-only `state` service to fetch all transactions from a
|
||||
/// block at the specified `height`.
|
||||
async fn load_transactions_from_block<ReadStateService>(
|
||||
height: block::Height,
|
||||
state: &mut ReadStateService,
|
||||
) -> Result<Vec<Arc<Transaction>>>
|
||||
where
|
||||
ReadStateService: Service<
|
||||
zebra_state::ReadRequest,
|
||||
Response = zebra_state::ReadResponse,
|
||||
Error = zebra_state::BoxError,
|
||||
>,
|
||||
{
|
||||
let request = zebra_state::ReadRequest::Block(HashOrHeight::Height(height));
|
||||
|
||||
let response = state
|
||||
.ready()
|
||||
.and_then(|ready_service| ready_service.call(request))
|
||||
.map_err(|error| eyre!(error))
|
||||
.await?;
|
||||
|
||||
let block = match response {
|
||||
zebra_state::ReadResponse::Block(Some(block)) => block,
|
||||
zebra_state::ReadResponse::Block(None) => {
|
||||
panic!("Missing block at {height:?} from state")
|
||||
}
|
||||
_ => unreachable!("Incorrect response from state service: {response:?}"),
|
||||
};
|
||||
|
||||
Ok(block.transactions.to_vec())
|
||||
}
|
||||
|
||||
/// Prepare a request to send to lightwalletd that contains a transaction to be sent.
|
||||
fn prepare_send_transaction_request(transaction: Arc<Transaction>) -> wallet_grpc::RawTransaction {
|
||||
let transaction_bytes = transaction.zcash_serialize_to_vec().unwrap();
|
||||
|
||||
wallet_grpc::RawTransaction {
|
||||
data: transaction_bytes,
|
||||
height: -1,
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,72 @@
|
|||
//! Lightwalletd gRPC interface and utility functions.
|
||||
|
||||
use std::{env, net::SocketAddr};
|
||||
|
||||
use tempfile::TempDir;
|
||||
|
||||
use zebra_test::{args, net::random_known_port, prelude::*};
|
||||
|
||||
use crate::{
|
||||
common::{
|
||||
config::testdir,
|
||||
lightwalletd::{LightWalletdTestDirExt, LIGHTWALLETD_TEST_TIMEOUT},
|
||||
},
|
||||
LIGHTWALLETD_FAILURE_MESSAGES, LIGHTWALLETD_IGNORE_MESSAGES, PROCESS_FAILURE_MESSAGES,
|
||||
};
|
||||
|
||||
tonic::include_proto!("cash.z.wallet.sdk.rpc");
|
||||
|
||||
/// Optional environment variable with the cached state for lightwalletd.
|
||||
///
|
||||
/// Can be used to speed up the [`sending_transactions_using_lightwalletd`] test, by allowing the
|
||||
/// test to reuse the cached lightwalletd synchronization data.
|
||||
const LIGHTWALLETD_DATA_DIR_VAR: &str = "LIGHTWALLETD_DATA_DIR";
|
||||
|
||||
/// Type alias for the RPC client to communicate with a lightwalletd instance.
|
||||
pub type LightwalletdRpcClient =
|
||||
compact_tx_streamer_client::CompactTxStreamerClient<tonic::transport::Channel>;
|
||||
|
||||
/// Start a lightwalletd instance with its RPC server functionality enabled.
|
||||
///
|
||||
/// Returns the lightwalletd instance and the port number that it is listening for RPC connections.
|
||||
pub fn spawn_lightwalletd_with_rpc_server(
|
||||
zebrad_rpc_address: SocketAddr,
|
||||
) -> Result<(TestChild<TempDir>, u16)> {
|
||||
let lightwalletd_dir = testdir()?.with_lightwalletd_config(zebrad_rpc_address)?;
|
||||
|
||||
let lightwalletd_rpc_port = random_known_port();
|
||||
let lightwalletd_rpc_address = format!("127.0.0.1:{lightwalletd_rpc_port}");
|
||||
|
||||
let mut arguments = args!["--grpc-bind-addr": lightwalletd_rpc_address];
|
||||
|
||||
if let Ok(data_dir) = env::var(LIGHTWALLETD_DATA_DIR_VAR) {
|
||||
arguments.set_parameter("--data-dir", data_dir);
|
||||
}
|
||||
|
||||
let mut lightwalletd = lightwalletd_dir
|
||||
.spawn_lightwalletd_child(arguments)?
|
||||
.with_timeout(LIGHTWALLETD_TEST_TIMEOUT)
|
||||
.with_failure_regex_iter(
|
||||
// TODO: replace with a function that returns the full list and correct return type
|
||||
LIGHTWALLETD_FAILURE_MESSAGES
|
||||
.iter()
|
||||
.chain(PROCESS_FAILURE_MESSAGES)
|
||||
.cloned(),
|
||||
// TODO: some exceptions do not apply to the cached state tests (#3511)
|
||||
LIGHTWALLETD_IGNORE_MESSAGES.iter().cloned(),
|
||||
);
|
||||
|
||||
lightwalletd.expect_stdout_line_matches("Starting gRPC server")?;
|
||||
lightwalletd.expect_stdout_line_matches("Waiting for block")?;
|
||||
|
||||
Ok((lightwalletd, lightwalletd_rpc_port))
|
||||
}
|
||||
|
||||
/// Connect to a lightwalletd RPC instance.
|
||||
pub async fn connect_to_lightwalletd(lightwalletd_rpc_port: u16) -> Result<LightwalletdRpcClient> {
|
||||
let lightwalletd_rpc_address = format!("http://127.0.0.1:{lightwalletd_rpc_port}");
|
||||
|
||||
let rpc_client = LightwalletdRpcClient::connect(lightwalletd_rpc_address).await?;
|
||||
|
||||
Ok(rpc_client)
|
||||
}
|
||||
|
|
@ -9,6 +9,7 @@
|
|||
//! to avoid compiling an empty "common" test binary:
|
||||
//! https://doc.rust-lang.org/book/ch11-03-test-organization.html#submodules-in-integration-tests
|
||||
|
||||
pub mod cached_state;
|
||||
pub mod check;
|
||||
pub mod config;
|
||||
pub mod launch;
|
||||
|
|
|
|||
|
|
@ -5,7 +5,10 @@
|
|||
//! Test functions in this file will not be run.
|
||||
//! This file is only for test library code.
|
||||
|
||||
use std::{path::PathBuf, time::Duration};
|
||||
use std::{
|
||||
path::{Path, PathBuf},
|
||||
time::Duration,
|
||||
};
|
||||
|
||||
use color_eyre::eyre::Result;
|
||||
use tempfile::TempDir;
|
||||
|
|
@ -16,6 +19,7 @@ use zebrad::{components::sync, config::ZebradConfig};
|
|||
use zebra_test::{args, prelude::*};
|
||||
|
||||
use super::{
|
||||
cached_state::copy_state_directory,
|
||||
config::{persistent_test_config, testdir},
|
||||
launch::ZebradTestDirExt,
|
||||
};
|
||||
|
|
@ -50,6 +54,14 @@ pub const TINY_CHECKPOINT_TIMEOUT: Duration = Duration::from_secs(120);
|
|||
/// The maximum amount of time Zebra should take to sync a thousand blocks.
|
||||
pub const LARGE_CHECKPOINT_TIMEOUT: Duration = Duration::from_secs(180);
|
||||
|
||||
/// The maximum time to wait for Zebrad to synchronize up to the chain tip starting from a
|
||||
/// partially synchronized state.
|
||||
///
|
||||
/// The partially synchronized state is expected to be close to the tip, so this timeout can be
|
||||
/// lower than what's expected for a full synchronization. However, a value that's too short may
|
||||
/// cause the test to fail.
|
||||
pub const FINISH_PARTIAL_SYNC_TIMEOUT: Duration = Duration::from_secs(60 * 60);
|
||||
|
||||
/// The test sync height where we switch to using the default lookahead limit.
|
||||
///
|
||||
/// Most tests only download a few blocks. So tests default to the minimum lookahead limit,
|
||||
|
|
@ -265,6 +277,28 @@ pub fn sync_until(
|
|||
}
|
||||
}
|
||||
|
||||
/// Runs a zebrad instance to synchronize the chain to the network tip.
|
||||
///
|
||||
/// The zebrad instance is executed on a copy of the partially synchronized chain state. This copy
|
||||
/// is returned afterwards, containing the fully synchronized chain state.
|
||||
pub async fn perform_full_sync_starting_from(
|
||||
network: Network,
|
||||
partial_sync_path: &Path,
|
||||
) -> Result<TempDir> {
|
||||
let fully_synced_path = copy_state_directory(&partial_sync_path).await?;
|
||||
|
||||
sync_until(
|
||||
Height::MAX,
|
||||
network,
|
||||
SYNC_FINISHED_REGEX,
|
||||
FINISH_PARTIAL_SYNC_TIMEOUT,
|
||||
fully_synced_path,
|
||||
MempoolBehavior::ShouldAutomaticallyActivate,
|
||||
true,
|
||||
false,
|
||||
)
|
||||
}
|
||||
|
||||
/// Returns a test config for caching Zebra's state up to the mandatory checkpoint.
|
||||
pub fn cached_mandatory_checkpoint_test_config() -> Result<ZebradConfig> {
|
||||
let mut config = persistent_test_config()?;
|
||||
|
|
|
|||
Loading…
Reference in New Issue