zebra/docker/entrypoint.sh

126 lines
10 KiB
Bash
Raw Normal View History

#!/usr/bin/env bash
feat(actions)!: add full sync test (#3582) * add(tests): full sync test * fix(test): add build * fix(deploy): escape double dashes '--' correctly * fix(test): remove unexpected --no-capture arg error: Found argument '--nocapture' which wasn't expected, or isn't valid in this context * refactor(docker): use default executable as entrypoint * refactor(startup): add a custom entrypoint * fix(test): add missing TEST_FULL_SYNC variable * test(timeout): use the biggest machine * fix * fix(deploy): use latest successful image * typo * refactor(docker): generate config file at startup * revert(build): changes were made to docker * fix(docker): send variables correctly to the entrypoint * test different conf file approach * fix(env): add RUN_TEST env variable * ref: use previous approach * fix(color): use environment variable * fix(resources): use our normal machine size * fix(ci): double CPU and RAM for full sync test * fix(test): check for zebrad test output in the correct order The mempool is only activated once, so we must check for that log first. After mempool activation, the stop regex is logged at least once. (It might be logged before as well, but we can't rely on that.) When checking that the mempool didn't activate, wait for the `zebrad` command to exit, then check the entire log. * fix(ci): run full sync test with full compiler optimisations * fix(tests): reintroduce tests and run full sync on approval * fix(tests): reduce the changelog Co-authored-by: teor <teor@riseup.net>
2022-03-02 06:15:24 -08:00
# show the commands we are executing
feat(actions)!: add full sync test (#3582) * add(tests): full sync test * fix(test): add build * fix(deploy): escape double dashes '--' correctly * fix(test): remove unexpected --no-capture arg error: Found argument '--nocapture' which wasn't expected, or isn't valid in this context * refactor(docker): use default executable as entrypoint * refactor(startup): add a custom entrypoint * fix(test): add missing TEST_FULL_SYNC variable * test(timeout): use the biggest machine * fix * fix(deploy): use latest successful image * typo * refactor(docker): generate config file at startup * revert(build): changes were made to docker * fix(docker): send variables correctly to the entrypoint * test different conf file approach * fix(env): add RUN_TEST env variable * ref: use previous approach * fix(color): use environment variable * fix(resources): use our normal machine size * fix(ci): double CPU and RAM for full sync test * fix(test): check for zebrad test output in the correct order The mempool is only activated once, so we must check for that log first. After mempool activation, the stop regex is logged at least once. (It might be logged before as well, but we can't rely on that.) When checking that the mempool didn't activate, wait for the `zebrad` command to exit, then check the entire log. * fix(ci): run full sync test with full compiler optimisations * fix(tests): reintroduce tests and run full sync on approval * fix(tests): reduce the changelog Co-authored-by: teor <teor@riseup.net>
2022-03-02 06:15:24 -08:00
set -x
# exit if a command fails
set -e
# exit if any command in a pipeline fails
set -o pipefail
feat(actions)!: add full sync test (#3582) * add(tests): full sync test * fix(test): add build * fix(deploy): escape double dashes '--' correctly * fix(test): remove unexpected --no-capture arg error: Found argument '--nocapture' which wasn't expected, or isn't valid in this context * refactor(docker): use default executable as entrypoint * refactor(startup): add a custom entrypoint * fix(test): add missing TEST_FULL_SYNC variable * test(timeout): use the biggest machine * fix * fix(deploy): use latest successful image * typo * refactor(docker): generate config file at startup * revert(build): changes were made to docker * fix(docker): send variables correctly to the entrypoint * test different conf file approach * fix(env): add RUN_TEST env variable * ref: use previous approach * fix(color): use environment variable * fix(resources): use our normal machine size * fix(ci): double CPU and RAM for full sync test * fix(test): check for zebrad test output in the correct order The mempool is only activated once, so we must check for that log first. After mempool activation, the stop regex is logged at least once. (It might be logged before as well, but we can't rely on that.) When checking that the mempool didn't activate, wait for the `zebrad` command to exit, then check the entire log. * fix(ci): run full sync test with full compiler optimisations * fix(tests): reintroduce tests and run full sync on approval * fix(tests): reduce the changelog Co-authored-by: teor <teor@riseup.net>
2022-03-02 06:15:24 -08:00
# TODO: expand this section if needed (#4363)
echo "Test variables:"
echo "ZEBRA_TEST_LIGHTWALLETD=$ZEBRA_TEST_LIGHTWALLETD"
echo "Hard-coded Zebra full sync directory: /zebrad-cache"
echo "ZEBRA_CACHED_STATE_DIR=$ZEBRA_CACHED_STATE_DIR"
echo "LIGHTWALLETD_DATA_DIR=$LIGHTWALLETD_DATA_DIR"
change(ci): Generate mainnet checkpoints in CI (#6550) * Add extra test type modes to support zebra-checkpoints * Add Mainnet and Testnet zebra-checkpoints test harnesses * Add zebra-checkpoints to test docker images * Add zebra-checkpoints test entrypoints * Add Mainnet CI workflow for zebra-checkpoints * Enable zebra-checkpoints feature in the test image * Use the same features for (almost) all the docker tests * Make workflow features match Docker features * Add a feature note * Add a zebra-checkpoints test feature to zebrad * Remove the "no cached state" testnet code * Log a startup message to standard error when launching zebra-checkpoints * Rename tests to avoid partial name conflicts * Fix log formatting * Add sentry feature to experimental docker image build * Explain what ENTRYPOINT_FEATURES is used for * Use the correct zebra-checkpoints path * Silence zebrad logs while generating checkpoints * Fix zebra-checkpoints log handling * Re-enable waiting for zebrad to fully sync * Add documentation for how to run these tests individually * Start generating checkpoints from the last compiled-in checkpoint * Fix clippy lints * Revert changes to TestType * Wait for all the checkpoints before finishing * Add more stderr debugging to zebra-checkpoints * Fix an outdated module comment * Add a workaround for zebra-checkpoints launch/run issues * Use temp dir and log what it is * Log extra metadata about the zebra-checkpoints binary * Add note about unstable feature -Z bindeps * Temporarily make the test run faster and with debug info * Log the original test command name when showing stdout and stderr * Try zebra-checkpoints in the system path first, then the cargo path * Fix slow thread close bug in dual process test harness * If the logs are shown, don't say they are hidden * Run `zebra-checkpoints --help` to work out what's going on in CI * Build `zebra-utils` binaries for `zebrad` integration tests * Revert temporary debugging changes * Revert changes that were moved to another PR
2023-04-26 21:39:43 -07:00
echo "ENTRYPOINT_FEATURES=$ENTRYPOINT_FEATURES"
feat(actions)!: add full sync test (#3582) * add(tests): full sync test * fix(test): add build * fix(deploy): escape double dashes '--' correctly * fix(test): remove unexpected --no-capture arg error: Found argument '--nocapture' which wasn't expected, or isn't valid in this context * refactor(docker): use default executable as entrypoint * refactor(startup): add a custom entrypoint * fix(test): add missing TEST_FULL_SYNC variable * test(timeout): use the biggest machine * fix * fix(deploy): use latest successful image * typo * refactor(docker): generate config file at startup * revert(build): changes were made to docker * fix(docker): send variables correctly to the entrypoint * test different conf file approach * fix(env): add RUN_TEST env variable * ref: use previous approach * fix(color): use environment variable * fix(resources): use our normal machine size * fix(ci): double CPU and RAM for full sync test * fix(test): check for zebrad test output in the correct order The mempool is only activated once, so we must check for that log first. After mempool activation, the stop regex is logged at least once. (It might be logged before as well, but we can't rely on that.) When checking that the mempool didn't activate, wait for the `zebrad` command to exit, then check the entire log. * fix(ci): run full sync test with full compiler optimisations * fix(tests): reintroduce tests and run full sync on approval * fix(tests): reduce the changelog Co-authored-by: teor <teor@riseup.net>
2022-03-02 06:15:24 -08:00
case "$1" in
--* | -*)
exec zebrad "$@"
;;
*)
change(ci): Generate mainnet checkpoints in CI (#6550) * Add extra test type modes to support zebra-checkpoints * Add Mainnet and Testnet zebra-checkpoints test harnesses * Add zebra-checkpoints to test docker images * Add zebra-checkpoints test entrypoints * Add Mainnet CI workflow for zebra-checkpoints * Enable zebra-checkpoints feature in the test image * Use the same features for (almost) all the docker tests * Make workflow features match Docker features * Add a feature note * Add a zebra-checkpoints test feature to zebrad * Remove the "no cached state" testnet code * Log a startup message to standard error when launching zebra-checkpoints * Rename tests to avoid partial name conflicts * Fix log formatting * Add sentry feature to experimental docker image build * Explain what ENTRYPOINT_FEATURES is used for * Use the correct zebra-checkpoints path * Silence zebrad logs while generating checkpoints * Fix zebra-checkpoints log handling * Re-enable waiting for zebrad to fully sync * Add documentation for how to run these tests individually * Start generating checkpoints from the last compiled-in checkpoint * Fix clippy lints * Revert changes to TestType * Wait for all the checkpoints before finishing * Add more stderr debugging to zebra-checkpoints * Fix an outdated module comment * Add a workaround for zebra-checkpoints launch/run issues * Use temp dir and log what it is * Log extra metadata about the zebra-checkpoints binary * Add note about unstable feature -Z bindeps * Temporarily make the test run faster and with debug info * Log the original test command name when showing stdout and stderr * Try zebra-checkpoints in the system path first, then the cargo path * Fix slow thread close bug in dual process test harness * If the logs are shown, don't say they are hidden * Run `zebra-checkpoints --help` to work out what's going on in CI * Build `zebra-utils` binaries for `zebrad` integration tests * Revert temporary debugging changes * Revert changes that were moved to another PR
2023-04-26 21:39:43 -07:00
# For these tests, we activate the test features to avoid recompiling `zebrad`,
# but we don't actually run any gRPC tests.
if [[ "$RUN_ALL_TESTS" -eq "1" ]]; then
# Run all the available tests for the current environment.
# If the lightwalletd environmental variables are set, we will also run those tests.
change(ci): Generate mainnet checkpoints in CI (#6550) * Add extra test type modes to support zebra-checkpoints * Add Mainnet and Testnet zebra-checkpoints test harnesses * Add zebra-checkpoints to test docker images * Add zebra-checkpoints test entrypoints * Add Mainnet CI workflow for zebra-checkpoints * Enable zebra-checkpoints feature in the test image * Use the same features for (almost) all the docker tests * Make workflow features match Docker features * Add a feature note * Add a zebra-checkpoints test feature to zebrad * Remove the "no cached state" testnet code * Log a startup message to standard error when launching zebra-checkpoints * Rename tests to avoid partial name conflicts * Fix log formatting * Add sentry feature to experimental docker image build * Explain what ENTRYPOINT_FEATURES is used for * Use the correct zebra-checkpoints path * Silence zebrad logs while generating checkpoints * Fix zebra-checkpoints log handling * Re-enable waiting for zebrad to fully sync * Add documentation for how to run these tests individually * Start generating checkpoints from the last compiled-in checkpoint * Fix clippy lints * Revert changes to TestType * Wait for all the checkpoints before finishing * Add more stderr debugging to zebra-checkpoints * Fix an outdated module comment * Add a workaround for zebra-checkpoints launch/run issues * Use temp dir and log what it is * Log extra metadata about the zebra-checkpoints binary * Add note about unstable feature -Z bindeps * Temporarily make the test run faster and with debug info * Log the original test command name when showing stdout and stderr * Try zebra-checkpoints in the system path first, then the cargo path * Fix slow thread close bug in dual process test harness * If the logs are shown, don't say they are hidden * Run `zebra-checkpoints --help` to work out what's going on in CI * Build `zebra-utils` binaries for `zebrad` integration tests * Revert temporary debugging changes * Revert changes that were moved to another PR
2023-04-26 21:39:43 -07:00
cargo test --locked --release --features "$ENTRYPOINT_FEATURES" --workspace -- --nocapture --include-ignored
# For these tests, we activate the gRPC feature to avoid recompiling `zebrad`,
# but we don't actually run any gRPC tests.
elif [[ -n "$FULL_SYNC_MAINNET_TIMEOUT_MINUTES" ]]; then
# Run a Zebra full sync test on mainnet.
change(ci): Generate mainnet checkpoints in CI (#6550) * Add extra test type modes to support zebra-checkpoints * Add Mainnet and Testnet zebra-checkpoints test harnesses * Add zebra-checkpoints to test docker images * Add zebra-checkpoints test entrypoints * Add Mainnet CI workflow for zebra-checkpoints * Enable zebra-checkpoints feature in the test image * Use the same features for (almost) all the docker tests * Make workflow features match Docker features * Add a feature note * Add a zebra-checkpoints test feature to zebrad * Remove the "no cached state" testnet code * Log a startup message to standard error when launching zebra-checkpoints * Rename tests to avoid partial name conflicts * Fix log formatting * Add sentry feature to experimental docker image build * Explain what ENTRYPOINT_FEATURES is used for * Use the correct zebra-checkpoints path * Silence zebrad logs while generating checkpoints * Fix zebra-checkpoints log handling * Re-enable waiting for zebrad to fully sync * Add documentation for how to run these tests individually * Start generating checkpoints from the last compiled-in checkpoint * Fix clippy lints * Revert changes to TestType * Wait for all the checkpoints before finishing * Add more stderr debugging to zebra-checkpoints * Fix an outdated module comment * Add a workaround for zebra-checkpoints launch/run issues * Use temp dir and log what it is * Log extra metadata about the zebra-checkpoints binary * Add note about unstable feature -Z bindeps * Temporarily make the test run faster and with debug info * Log the original test command name when showing stdout and stderr * Try zebra-checkpoints in the system path first, then the cargo path * Fix slow thread close bug in dual process test harness * If the logs are shown, don't say they are hidden * Run `zebra-checkpoints --help` to work out what's going on in CI * Build `zebra-utils` binaries for `zebrad` integration tests * Revert temporary debugging changes * Revert changes that were moved to another PR
2023-04-26 21:39:43 -07:00
cargo test --locked --release --features "$ENTRYPOINT_FEATURES" --package zebrad --test acceptance -- --nocapture --include-ignored full_sync_mainnet
# List directory generated by test
# TODO: replace with $ZEBRA_CACHED_STATE_DIR in Rust and workflows
ls -lh "/zebrad-cache"/*/* || (echo "No /zebrad-cache/*/*"; ls -lhR "/zebrad-cache" | head -50 || echo "No /zebrad-cache directory")
elif [[ -n "$FULL_SYNC_TESTNET_TIMEOUT_MINUTES" ]]; then
# Run a Zebra full sync test on testnet.
cargo test --locked --release --features "$ENTRYPOINT_FEATURES" --package zebrad --test acceptance -- --nocapture --include-ignored full_sync_testnet
# List directory generated by test
# TODO: replace with $ZEBRA_CACHED_STATE_DIR in Rust and workflows
ls -lh "/zebrad-cache"/*/* || (echo "No /zebrad-cache/*/*"; ls -lhR "/zebrad-cache" | head -50 || echo "No /zebrad-cache directory")
elif [[ "$TEST_DISK_REBUILD" -eq "1" ]]; then
# Run a Zebra sync up to the mandatory checkpoint.
#
# TODO: use environmental variables instead of Rust features (part of #2995)
change(ci): Generate mainnet checkpoints in CI (#6550) * Add extra test type modes to support zebra-checkpoints * Add Mainnet and Testnet zebra-checkpoints test harnesses * Add zebra-checkpoints to test docker images * Add zebra-checkpoints test entrypoints * Add Mainnet CI workflow for zebra-checkpoints * Enable zebra-checkpoints feature in the test image * Use the same features for (almost) all the docker tests * Make workflow features match Docker features * Add a feature note * Add a zebra-checkpoints test feature to zebrad * Remove the "no cached state" testnet code * Log a startup message to standard error when launching zebra-checkpoints * Rename tests to avoid partial name conflicts * Fix log formatting * Add sentry feature to experimental docker image build * Explain what ENTRYPOINT_FEATURES is used for * Use the correct zebra-checkpoints path * Silence zebrad logs while generating checkpoints * Fix zebra-checkpoints log handling * Re-enable waiting for zebrad to fully sync * Add documentation for how to run these tests individually * Start generating checkpoints from the last compiled-in checkpoint * Fix clippy lints * Revert changes to TestType * Wait for all the checkpoints before finishing * Add more stderr debugging to zebra-checkpoints * Fix an outdated module comment * Add a workaround for zebra-checkpoints launch/run issues * Use temp dir and log what it is * Log extra metadata about the zebra-checkpoints binary * Add note about unstable feature -Z bindeps * Temporarily make the test run faster and with debug info * Log the original test command name when showing stdout and stderr * Try zebra-checkpoints in the system path first, then the cargo path * Fix slow thread close bug in dual process test harness * If the logs are shown, don't say they are hidden * Run `zebra-checkpoints --help` to work out what's going on in CI * Build `zebra-utils` binaries for `zebrad` integration tests * Revert temporary debugging changes * Revert changes that were moved to another PR
2023-04-26 21:39:43 -07:00
cargo test --locked --release --features "test_sync_to_mandatory_checkpoint_${NETWORK,,},$ENTRYPOINT_FEATURES" --package zebrad --test acceptance -- --nocapture --include-ignored "sync_to_mandatory_checkpoint_${NETWORK,,}"
# TODO: replace with $ZEBRA_CACHED_STATE_DIR in Rust and workflows
ls -lh "/zebrad-cache"/*/* || (echo "No /zebrad-cache/*/*"; ls -lhR "/zebrad-cache" | head -50 || echo "No /zebrad-cache directory")
elif [[ "$TEST_UPDATE_SYNC" -eq "1" ]]; then
# Run a Zebra sync starting at the cached tip, and syncing to the latest tip.
#
# List directory used by test
ls -lh "$ZEBRA_CACHED_STATE_DIR"/*/* || (echo "No $ZEBRA_CACHED_STATE_DIR/*/*"; ls -lhR "$ZEBRA_CACHED_STATE_DIR" | head -50 || echo "No $ZEBRA_CACHED_STATE_DIR directory")
change(ci): Generate mainnet checkpoints in CI (#6550) * Add extra test type modes to support zebra-checkpoints * Add Mainnet and Testnet zebra-checkpoints test harnesses * Add zebra-checkpoints to test docker images * Add zebra-checkpoints test entrypoints * Add Mainnet CI workflow for zebra-checkpoints * Enable zebra-checkpoints feature in the test image * Use the same features for (almost) all the docker tests * Make workflow features match Docker features * Add a feature note * Add a zebra-checkpoints test feature to zebrad * Remove the "no cached state" testnet code * Log a startup message to standard error when launching zebra-checkpoints * Rename tests to avoid partial name conflicts * Fix log formatting * Add sentry feature to experimental docker image build * Explain what ENTRYPOINT_FEATURES is used for * Use the correct zebra-checkpoints path * Silence zebrad logs while generating checkpoints * Fix zebra-checkpoints log handling * Re-enable waiting for zebrad to fully sync * Add documentation for how to run these tests individually * Start generating checkpoints from the last compiled-in checkpoint * Fix clippy lints * Revert changes to TestType * Wait for all the checkpoints before finishing * Add more stderr debugging to zebra-checkpoints * Fix an outdated module comment * Add a workaround for zebra-checkpoints launch/run issues * Use temp dir and log what it is * Log extra metadata about the zebra-checkpoints binary * Add note about unstable feature -Z bindeps * Temporarily make the test run faster and with debug info * Log the original test command name when showing stdout and stderr * Try zebra-checkpoints in the system path first, then the cargo path * Fix slow thread close bug in dual process test harness * If the logs are shown, don't say they are hidden * Run `zebra-checkpoints --help` to work out what's going on in CI * Build `zebra-utils` binaries for `zebrad` integration tests * Revert temporary debugging changes * Revert changes that were moved to another PR
2023-04-26 21:39:43 -07:00
cargo test --locked --release --features "$ENTRYPOINT_FEATURES" --package zebrad --test acceptance -- --nocapture --include-ignored zebrad_update_sync
elif [[ "$TEST_CHECKPOINT_SYNC" -eq "1" ]]; then
# Run a Zebra sync starting at the cached mandatory checkpoint, and syncing past it.
#
# List directory used by test
# TODO: replace with $ZEBRA_CACHED_STATE_DIR in Rust and workflows
ls -lh "/zebrad-cache"/*/* || (echo "No /zebrad-cache/*/*"; ls -lhR "/zebrad-cache" | head -50 || echo "No /zebrad-cache directory")
# TODO: use environmental variables instead of Rust features (part of #2995)
change(ci): Generate mainnet checkpoints in CI (#6550) * Add extra test type modes to support zebra-checkpoints * Add Mainnet and Testnet zebra-checkpoints test harnesses * Add zebra-checkpoints to test docker images * Add zebra-checkpoints test entrypoints * Add Mainnet CI workflow for zebra-checkpoints * Enable zebra-checkpoints feature in the test image * Use the same features for (almost) all the docker tests * Make workflow features match Docker features * Add a feature note * Add a zebra-checkpoints test feature to zebrad * Remove the "no cached state" testnet code * Log a startup message to standard error when launching zebra-checkpoints * Rename tests to avoid partial name conflicts * Fix log formatting * Add sentry feature to experimental docker image build * Explain what ENTRYPOINT_FEATURES is used for * Use the correct zebra-checkpoints path * Silence zebrad logs while generating checkpoints * Fix zebra-checkpoints log handling * Re-enable waiting for zebrad to fully sync * Add documentation for how to run these tests individually * Start generating checkpoints from the last compiled-in checkpoint * Fix clippy lints * Revert changes to TestType * Wait for all the checkpoints before finishing * Add more stderr debugging to zebra-checkpoints * Fix an outdated module comment * Add a workaround for zebra-checkpoints launch/run issues * Use temp dir and log what it is * Log extra metadata about the zebra-checkpoints binary * Add note about unstable feature -Z bindeps * Temporarily make the test run faster and with debug info * Log the original test command name when showing stdout and stderr * Try zebra-checkpoints in the system path first, then the cargo path * Fix slow thread close bug in dual process test harness * If the logs are shown, don't say they are hidden * Run `zebra-checkpoints --help` to work out what's going on in CI * Build `zebra-utils` binaries for `zebrad` integration tests * Revert temporary debugging changes * Revert changes that were moved to another PR
2023-04-26 21:39:43 -07:00
cargo test --locked --release --features "test_sync_past_mandatory_checkpoint_${NETWORK,,},$ENTRYPOINT_FEATURES" --package zebrad --test acceptance -- --nocapture --include-ignored "sync_past_mandatory_checkpoint_${NETWORK,,}"
elif [[ "$GENERATE_CHECKPOINTS_MAINNET" -eq "1" ]]; then
# Generate checkpoints after syncing Zebra from a cached state on mainnet.
#
# TODO: disable or filter out logs like:
# test generate_checkpoints_mainnet has been running for over 60 seconds
#
# List directory used by test
ls -lh "$ZEBRA_CACHED_STATE_DIR"/*/* || (echo "No $ZEBRA_CACHED_STATE_DIR/*/*"; ls -lhR "$ZEBRA_CACHED_STATE_DIR" | head -50 || echo "No $ZEBRA_CACHED_STATE_DIR directory")
cargo test --locked --release --features "$ENTRYPOINT_FEATURES" --package zebrad --test acceptance -- --nocapture --include-ignored generate_checkpoints_mainnet
elif [[ "$GENERATE_CHECKPOINTS_TESTNET" -eq "1" ]]; then
# Generate checkpoints after syncing Zebra on testnet.
#
# This test might fail if testnet is unstable.
#
# List directory used by test
ls -lh "$ZEBRA_CACHED_STATE_DIR"/*/* || (echo "No $ZEBRA_CACHED_STATE_DIR/*/*"; ls -lhR "$ZEBRA_CACHED_STATE_DIR" | head -50 || echo "No $ZEBRA_CACHED_STATE_DIR directory")
cargo test --locked --release --features "$ENTRYPOINT_FEATURES" --package zebrad --test acceptance -- --nocapture --include-ignored generate_checkpoints_testnet
elif [[ "$TEST_LWD_RPC_CALL" -eq "1" ]]; then
# Starting at a cached Zebra tip, test a JSON-RPC call to Zebra.
ls -lh "$ZEBRA_CACHED_STATE_DIR"/*/* || (echo "No $ZEBRA_CACHED_STATE_DIR/*/*"; ls -lhR "$ZEBRA_CACHED_STATE_DIR" | head -50 || echo "No $ZEBRA_CACHED_STATE_DIR directory")
change(ci): Generate mainnet checkpoints in CI (#6550) * Add extra test type modes to support zebra-checkpoints * Add Mainnet and Testnet zebra-checkpoints test harnesses * Add zebra-checkpoints to test docker images * Add zebra-checkpoints test entrypoints * Add Mainnet CI workflow for zebra-checkpoints * Enable zebra-checkpoints feature in the test image * Use the same features for (almost) all the docker tests * Make workflow features match Docker features * Add a feature note * Add a zebra-checkpoints test feature to zebrad * Remove the "no cached state" testnet code * Log a startup message to standard error when launching zebra-checkpoints * Rename tests to avoid partial name conflicts * Fix log formatting * Add sentry feature to experimental docker image build * Explain what ENTRYPOINT_FEATURES is used for * Use the correct zebra-checkpoints path * Silence zebrad logs while generating checkpoints * Fix zebra-checkpoints log handling * Re-enable waiting for zebrad to fully sync * Add documentation for how to run these tests individually * Start generating checkpoints from the last compiled-in checkpoint * Fix clippy lints * Revert changes to TestType * Wait for all the checkpoints before finishing * Add more stderr debugging to zebra-checkpoints * Fix an outdated module comment * Add a workaround for zebra-checkpoints launch/run issues * Use temp dir and log what it is * Log extra metadata about the zebra-checkpoints binary * Add note about unstable feature -Z bindeps * Temporarily make the test run faster and with debug info * Log the original test command name when showing stdout and stderr * Try zebra-checkpoints in the system path first, then the cargo path * Fix slow thread close bug in dual process test harness * If the logs are shown, don't say they are hidden * Run `zebra-checkpoints --help` to work out what's going on in CI * Build `zebra-utils` binaries for `zebrad` integration tests * Revert temporary debugging changes * Revert changes that were moved to another PR
2023-04-26 21:39:43 -07:00
cargo test --locked --release --features "$ENTRYPOINT_FEATURES" --package zebrad --test acceptance -- --nocapture --include-ignored fully_synced_rpc_test
elif [[ "$TEST_LWD_FULL_SYNC" -eq "1" ]]; then
# Starting at a cached Zebra tip, run a lightwalletd sync to tip.
ls -lh "$ZEBRA_CACHED_STATE_DIR"/*/* || (echo "No $ZEBRA_CACHED_STATE_DIR/*/*"; ls -lhR "$ZEBRA_CACHED_STATE_DIR" | head -50 || echo "No $ZEBRA_CACHED_STATE_DIR directory")
change(ci): Generate mainnet checkpoints in CI (#6550) * Add extra test type modes to support zebra-checkpoints * Add Mainnet and Testnet zebra-checkpoints test harnesses * Add zebra-checkpoints to test docker images * Add zebra-checkpoints test entrypoints * Add Mainnet CI workflow for zebra-checkpoints * Enable zebra-checkpoints feature in the test image * Use the same features for (almost) all the docker tests * Make workflow features match Docker features * Add a feature note * Add a zebra-checkpoints test feature to zebrad * Remove the "no cached state" testnet code * Log a startup message to standard error when launching zebra-checkpoints * Rename tests to avoid partial name conflicts * Fix log formatting * Add sentry feature to experimental docker image build * Explain what ENTRYPOINT_FEATURES is used for * Use the correct zebra-checkpoints path * Silence zebrad logs while generating checkpoints * Fix zebra-checkpoints log handling * Re-enable waiting for zebrad to fully sync * Add documentation for how to run these tests individually * Start generating checkpoints from the last compiled-in checkpoint * Fix clippy lints * Revert changes to TestType * Wait for all the checkpoints before finishing * Add more stderr debugging to zebra-checkpoints * Fix an outdated module comment * Add a workaround for zebra-checkpoints launch/run issues * Use temp dir and log what it is * Log extra metadata about the zebra-checkpoints binary * Add note about unstable feature -Z bindeps * Temporarily make the test run faster and with debug info * Log the original test command name when showing stdout and stderr * Try zebra-checkpoints in the system path first, then the cargo path * Fix slow thread close bug in dual process test harness * If the logs are shown, don't say they are hidden * Run `zebra-checkpoints --help` to work out what's going on in CI * Build `zebra-utils` binaries for `zebrad` integration tests * Revert temporary debugging changes * Revert changes that were moved to another PR
2023-04-26 21:39:43 -07:00
cargo test --locked --release --features "$ENTRYPOINT_FEATURES" --package zebrad --test acceptance -- --nocapture --include-ignored lightwalletd_full_sync
ls -lhR "$LIGHTWALLETD_DATA_DIR/db" || (echo "No $LIGHTWALLETD_DATA_DIR/db"; ls -lhR "$LIGHTWALLETD_DATA_DIR" | head -50 || echo "No $LIGHTWALLETD_DATA_DIR directory")
elif [[ "$TEST_LWD_UPDATE_SYNC" -eq "1" ]]; then
# Starting with a cached Zebra and lightwalletd tip, run a quick update sync.
ls -lh "$ZEBRA_CACHED_STATE_DIR"/*/* || (echo "No $ZEBRA_CACHED_STATE_DIR/*/*"; ls -lhR "$ZEBRA_CACHED_STATE_DIR" | head -50 || echo "No $ZEBRA_CACHED_STATE_DIR directory")
ls -lhR "$LIGHTWALLETD_DATA_DIR/db" || (echo "No $LIGHTWALLETD_DATA_DIR/db"; ls -lhR "$LIGHTWALLETD_DATA_DIR" | head -50 || echo "No $LIGHTWALLETD_DATA_DIR directory")
change(ci): Generate mainnet checkpoints in CI (#6550) * Add extra test type modes to support zebra-checkpoints * Add Mainnet and Testnet zebra-checkpoints test harnesses * Add zebra-checkpoints to test docker images * Add zebra-checkpoints test entrypoints * Add Mainnet CI workflow for zebra-checkpoints * Enable zebra-checkpoints feature in the test image * Use the same features for (almost) all the docker tests * Make workflow features match Docker features * Add a feature note * Add a zebra-checkpoints test feature to zebrad * Remove the "no cached state" testnet code * Log a startup message to standard error when launching zebra-checkpoints * Rename tests to avoid partial name conflicts * Fix log formatting * Add sentry feature to experimental docker image build * Explain what ENTRYPOINT_FEATURES is used for * Use the correct zebra-checkpoints path * Silence zebrad logs while generating checkpoints * Fix zebra-checkpoints log handling * Re-enable waiting for zebrad to fully sync * Add documentation for how to run these tests individually * Start generating checkpoints from the last compiled-in checkpoint * Fix clippy lints * Revert changes to TestType * Wait for all the checkpoints before finishing * Add more stderr debugging to zebra-checkpoints * Fix an outdated module comment * Add a workaround for zebra-checkpoints launch/run issues * Use temp dir and log what it is * Log extra metadata about the zebra-checkpoints binary * Add note about unstable feature -Z bindeps * Temporarily make the test run faster and with debug info * Log the original test command name when showing stdout and stderr * Try zebra-checkpoints in the system path first, then the cargo path * Fix slow thread close bug in dual process test harness * If the logs are shown, don't say they are hidden * Run `zebra-checkpoints --help` to work out what's going on in CI * Build `zebra-utils` binaries for `zebrad` integration tests * Revert temporary debugging changes * Revert changes that were moved to another PR
2023-04-26 21:39:43 -07:00
cargo test --locked --release --features "$ENTRYPOINT_FEATURES" --package zebrad --test acceptance -- --nocapture --include-ignored lightwalletd_update_sync
# These tests actually use gRPC.
elif [[ "$TEST_LWD_GRPC" -eq "1" ]]; then
# Starting with a cached Zebra and lightwalletd tip, test all gRPC calls to lightwalletd, which calls Zebra.
ls -lh "$ZEBRA_CACHED_STATE_DIR"/*/* || (echo "No $ZEBRA_CACHED_STATE_DIR/*/*"; ls -lhR "$ZEBRA_CACHED_STATE_DIR" | head -50 || echo "No $ZEBRA_CACHED_STATE_DIR directory")
ls -lhR "$LIGHTWALLETD_DATA_DIR/db" || (echo "No $LIGHTWALLETD_DATA_DIR/db"; ls -lhR "$LIGHTWALLETD_DATA_DIR" | head -50 || echo "No $LIGHTWALLETD_DATA_DIR directory")
change(ci): Generate mainnet checkpoints in CI (#6550) * Add extra test type modes to support zebra-checkpoints * Add Mainnet and Testnet zebra-checkpoints test harnesses * Add zebra-checkpoints to test docker images * Add zebra-checkpoints test entrypoints * Add Mainnet CI workflow for zebra-checkpoints * Enable zebra-checkpoints feature in the test image * Use the same features for (almost) all the docker tests * Make workflow features match Docker features * Add a feature note * Add a zebra-checkpoints test feature to zebrad * Remove the "no cached state" testnet code * Log a startup message to standard error when launching zebra-checkpoints * Rename tests to avoid partial name conflicts * Fix log formatting * Add sentry feature to experimental docker image build * Explain what ENTRYPOINT_FEATURES is used for * Use the correct zebra-checkpoints path * Silence zebrad logs while generating checkpoints * Fix zebra-checkpoints log handling * Re-enable waiting for zebrad to fully sync * Add documentation for how to run these tests individually * Start generating checkpoints from the last compiled-in checkpoint * Fix clippy lints * Revert changes to TestType * Wait for all the checkpoints before finishing * Add more stderr debugging to zebra-checkpoints * Fix an outdated module comment * Add a workaround for zebra-checkpoints launch/run issues * Use temp dir and log what it is * Log extra metadata about the zebra-checkpoints binary * Add note about unstable feature -Z bindeps * Temporarily make the test run faster and with debug info * Log the original test command name when showing stdout and stderr * Try zebra-checkpoints in the system path first, then the cargo path * Fix slow thread close bug in dual process test harness * If the logs are shown, don't say they are hidden * Run `zebra-checkpoints --help` to work out what's going on in CI * Build `zebra-utils` binaries for `zebrad` integration tests * Revert temporary debugging changes * Revert changes that were moved to another PR
2023-04-26 21:39:43 -07:00
cargo test --locked --release --features "$ENTRYPOINT_FEATURES" --package zebrad --test acceptance -- --nocapture --include-ignored lightwalletd_wallet_grpc_tests
feat(ci): add `sending_transactions_using_lightwalletd` test to CI (#4267) * feat(ci): add lightwalletd_*_sync tests to CI * feat(ci): add lightwalletd RPC call test * feat(ci): add send transactions test with lwd to CI * fix(ci): create a variable to run transactions test * refactor(ci): use docker in docker This is a workaround for an issue related to disk partitioning, caused by a GCP service called Konlet, while mounting the cached disks to the VM and then to the container * fix(build): persist docker login credentials * fix(ci): get sync height from docker logs instead of gcp * try: use gha cache for faster building * fix(ci): mount disk in container to make it available in vm * fix(build): do not invalidate cache between images * try(docker): invalidate cache as less as possible * fix(ci): GHA terminal is not a TTY * fix(build): do not ignore entrypoint.sh * fix * fix(ci): mount using root priveleges * fix(ci): use existing disk as cached state * fix(ci): wait for disks to get mounted * force rebuild * fix failed force * fix failed commit * WIP * fix(ci): some tests does not use a cached state * wip * refactor(ci): disk names and job segregation * fix(ci): do not name boot and attached disk the same * fix(ci): attach a disk to full sync, to snapshot the state * fix(ci): use correct disk implementations * fix(ci): use different disk name to allow test concurrency * feat(ci): add lightwalledt send transaction test * cleanup(ci): remove extra tests * fix(ci): allow disk concurrency with tests * fix(ci): add considerations for different tests * fix(reusable): last fixes * feat(ci): use reusable workflow for tests * fix(rw): remove nested worflow * fix(rw): minor fixes * force rebuild * fix(rw): do not use an input as job name * fix(rw): remove variable id * fix(ci): remove explicit conditions and id * fix(ci): docker does not need the variable sign ($) to work * fix(ci): mount typo * fix(ci): if a sync fails, always delete the instance This also reduces the amount of jobs needed. * refactor(ci): make all test depend on the same build * fix(ci): some tests require multiple variables * fix(docker): variable substitution * fix(ci): allow to run multiple commits from a PR at once * fix(docker): lower the NETWORK env var for test names * reduce uneeded diff * imp(keys): use better naming for builds_disks * imp(ci): use input defaults * imp(ci): remove test_name in favor of test_id * fix(ci): better key naming * fix(ci): long disk names breaks GCP naming convention * feat(ci): validate local state version with cached state * fix(ci): add condition to run tests * fix: typo * fix: app_name should not be required * fix: zebra_state_path shouldn't be required * fix: reduce diff * fix(ci): checkout to grep local state version * Update .github/workflows/test.yml Co-authored-by: teor <teor@riseup.net> * revert: merge all tests into a single workflow * Remove unused STATE_VERSION env var * fix: minor fixes * fix(ci): make test.patch the same as test * fix(ci): negate the input value * imp(ci): better cached state conditional handling * imp(ci): exit code is captured by `docker run` * fix(deploy): mount disks with better write performance * fix(ci): change sync id to a broader id name * fix(ci): use correct input validation * fix(ci): do not make test with cached state dependant on other * imp(ci): organiza keys better * fix(ci): use appropiate naming * fix(ci): create docker volume before mounting * fix(lint): do not fail on all new changes * imp(ci): do not report in pr review * fix(ci): partition clean disks * fix: typo * fix: test called the wrong way * fix(build): stop using gha cache * ref(ci): validate run condition before calling reusable workflow * fix(ci): use a better filesystem dir and fix other values * fix: linting errors * fix(ci): typo * Revert "fix(build): stop using gha cache" This reverts commit a8fbc5f416df561e58b388e065d1dc9696983508. Cache expiration is a lesser evil than not using caching at all and then failing with a 401 * imp(ci): do not set a default for needs_zebra_state * Update .github/workflows/test.yml Co-authored-by: teor <teor@riseup.net> * fix(deps): remove dependencies * force build * Update .github/workflows/test.yml Co-authored-by: teor <teor@riseup.net> * fix(docker): add RUST_LOG as an ARG and ENV * fix(test): add `#[ignore]` to send transactions test This test needs state then it should be marked as #[ignore] * fix(ci): differentiate between root cache path and its dir * Remove extra `state` directory That was a workaround for an issue that has been fixed. * imp(docs): use better test descriptions Co-authored-by: teor <teor@riseup.net> * fix: reduce unwanted diff with main * fix(ci): make lwd conditions consistent * Remove another extra `state` directory Was also part of a workaround for an issue that has been fixed. * fix(ci): use better conditionals to run test jobs Co-authored-by: teor <teor@riseup.net> * Tweak to support different lightwalletd versions Some versions print `Waiting for block`, and some versions print `Ingestor waiting for block`. Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: teor <teor@riseup.net> Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>
2022-05-05 22:30:38 -07:00
elif [[ "$TEST_LWD_TRANSACTIONS" -eq "1" ]]; then
# Starting with a cached Zebra and lightwalletd tip, test sending transactions gRPC call to lightwalletd, which calls Zebra.
ls -lh "$ZEBRA_CACHED_STATE_DIR"/*/* || (echo "No $ZEBRA_CACHED_STATE_DIR/*/*"; ls -lhR "$ZEBRA_CACHED_STATE_DIR" | head -50 || echo "No $ZEBRA_CACHED_STATE_DIR directory")
ls -lhR "$LIGHTWALLETD_DATA_DIR/db" || (echo "No $LIGHTWALLETD_DATA_DIR/db"; ls -lhR "$LIGHTWALLETD_DATA_DIR" | head -50 || echo "No $LIGHTWALLETD_DATA_DIR directory")
change(ci): Generate mainnet checkpoints in CI (#6550) * Add extra test type modes to support zebra-checkpoints * Add Mainnet and Testnet zebra-checkpoints test harnesses * Add zebra-checkpoints to test docker images * Add zebra-checkpoints test entrypoints * Add Mainnet CI workflow for zebra-checkpoints * Enable zebra-checkpoints feature in the test image * Use the same features for (almost) all the docker tests * Make workflow features match Docker features * Add a feature note * Add a zebra-checkpoints test feature to zebrad * Remove the "no cached state" testnet code * Log a startup message to standard error when launching zebra-checkpoints * Rename tests to avoid partial name conflicts * Fix log formatting * Add sentry feature to experimental docker image build * Explain what ENTRYPOINT_FEATURES is used for * Use the correct zebra-checkpoints path * Silence zebrad logs while generating checkpoints * Fix zebra-checkpoints log handling * Re-enable waiting for zebrad to fully sync * Add documentation for how to run these tests individually * Start generating checkpoints from the last compiled-in checkpoint * Fix clippy lints * Revert changes to TestType * Wait for all the checkpoints before finishing * Add more stderr debugging to zebra-checkpoints * Fix an outdated module comment * Add a workaround for zebra-checkpoints launch/run issues * Use temp dir and log what it is * Log extra metadata about the zebra-checkpoints binary * Add note about unstable feature -Z bindeps * Temporarily make the test run faster and with debug info * Log the original test command name when showing stdout and stderr * Try zebra-checkpoints in the system path first, then the cargo path * Fix slow thread close bug in dual process test harness * If the logs are shown, don't say they are hidden * Run `zebra-checkpoints --help` to work out what's going on in CI * Build `zebra-utils` binaries for `zebrad` integration tests * Revert temporary debugging changes * Revert changes that were moved to another PR
2023-04-26 21:39:43 -07:00
cargo test --locked --release --features "$ENTRYPOINT_FEATURES" --package zebrad --test acceptance -- --nocapture --include-ignored sending_transactions_using_lightwalletd
# These tests use mining code, but don't use gRPC.
# We add the mining feature here because our other code needs to pass tests without it.
elif [[ "$TEST_GET_BLOCK_TEMPLATE" -eq "1" ]]; then
# Starting with a cached Zebra tip, test getting a block template from Zebra's RPC server.
ls -lh "$ZEBRA_CACHED_STATE_DIR"/*/* || (echo "No $ZEBRA_CACHED_STATE_DIR/*/*"; ls -lhR "$ZEBRA_CACHED_STATE_DIR" | head -50 || echo "No $ZEBRA_CACHED_STATE_DIR directory")
change(ci): Generate mainnet checkpoints in CI (#6550) * Add extra test type modes to support zebra-checkpoints * Add Mainnet and Testnet zebra-checkpoints test harnesses * Add zebra-checkpoints to test docker images * Add zebra-checkpoints test entrypoints * Add Mainnet CI workflow for zebra-checkpoints * Enable zebra-checkpoints feature in the test image * Use the same features for (almost) all the docker tests * Make workflow features match Docker features * Add a feature note * Add a zebra-checkpoints test feature to zebrad * Remove the "no cached state" testnet code * Log a startup message to standard error when launching zebra-checkpoints * Rename tests to avoid partial name conflicts * Fix log formatting * Add sentry feature to experimental docker image build * Explain what ENTRYPOINT_FEATURES is used for * Use the correct zebra-checkpoints path * Silence zebrad logs while generating checkpoints * Fix zebra-checkpoints log handling * Re-enable waiting for zebrad to fully sync * Add documentation for how to run these tests individually * Start generating checkpoints from the last compiled-in checkpoint * Fix clippy lints * Revert changes to TestType * Wait for all the checkpoints before finishing * Add more stderr debugging to zebra-checkpoints * Fix an outdated module comment * Add a workaround for zebra-checkpoints launch/run issues * Use temp dir and log what it is * Log extra metadata about the zebra-checkpoints binary * Add note about unstable feature -Z bindeps * Temporarily make the test run faster and with debug info * Log the original test command name when showing stdout and stderr * Try zebra-checkpoints in the system path first, then the cargo path * Fix slow thread close bug in dual process test harness * If the logs are shown, don't say they are hidden * Run `zebra-checkpoints --help` to work out what's going on in CI * Build `zebra-utils` binaries for `zebrad` integration tests * Revert temporary debugging changes * Revert changes that were moved to another PR
2023-04-26 21:39:43 -07:00
cargo test --locked --release --features "getblocktemplate-rpcs,$ENTRYPOINT_FEATURES" --package zebrad --test acceptance -- --nocapture --include-ignored get_block_template
elif [[ "$TEST_SUBMIT_BLOCK" -eq "1" ]]; then
# Starting with a cached Zebra tip, test sending a block to Zebra's RPC port.
ls -lh "$ZEBRA_CACHED_STATE_DIR"/*/* || (echo "No $ZEBRA_CACHED_STATE_DIR/*/*"; ls -lhR "$ZEBRA_CACHED_STATE_DIR" | head -50 || echo "No $ZEBRA_CACHED_STATE_DIR directory")
change(ci): Generate mainnet checkpoints in CI (#6550) * Add extra test type modes to support zebra-checkpoints * Add Mainnet and Testnet zebra-checkpoints test harnesses * Add zebra-checkpoints to test docker images * Add zebra-checkpoints test entrypoints * Add Mainnet CI workflow for zebra-checkpoints * Enable zebra-checkpoints feature in the test image * Use the same features for (almost) all the docker tests * Make workflow features match Docker features * Add a feature note * Add a zebra-checkpoints test feature to zebrad * Remove the "no cached state" testnet code * Log a startup message to standard error when launching zebra-checkpoints * Rename tests to avoid partial name conflicts * Fix log formatting * Add sentry feature to experimental docker image build * Explain what ENTRYPOINT_FEATURES is used for * Use the correct zebra-checkpoints path * Silence zebrad logs while generating checkpoints * Fix zebra-checkpoints log handling * Re-enable waiting for zebrad to fully sync * Add documentation for how to run these tests individually * Start generating checkpoints from the last compiled-in checkpoint * Fix clippy lints * Revert changes to TestType * Wait for all the checkpoints before finishing * Add more stderr debugging to zebra-checkpoints * Fix an outdated module comment * Add a workaround for zebra-checkpoints launch/run issues * Use temp dir and log what it is * Log extra metadata about the zebra-checkpoints binary * Add note about unstable feature -Z bindeps * Temporarily make the test run faster and with debug info * Log the original test command name when showing stdout and stderr * Try zebra-checkpoints in the system path first, then the cargo path * Fix slow thread close bug in dual process test harness * If the logs are shown, don't say they are hidden * Run `zebra-checkpoints --help` to work out what's going on in CI * Build `zebra-utils` binaries for `zebrad` integration tests * Revert temporary debugging changes * Revert changes that were moved to another PR
2023-04-26 21:39:43 -07:00
cargo test --locked --release --features "getblocktemplate-rpcs,$ENTRYPOINT_FEATURES" --package zebrad --test acceptance -- --nocapture --include-ignored submit_block
else
exec "$@"
feat(actions)!: add full sync test (#3582) * add(tests): full sync test * fix(test): add build * fix(deploy): escape double dashes '--' correctly * fix(test): remove unexpected --no-capture arg error: Found argument '--nocapture' which wasn't expected, or isn't valid in this context * refactor(docker): use default executable as entrypoint * refactor(startup): add a custom entrypoint * fix(test): add missing TEST_FULL_SYNC variable * test(timeout): use the biggest machine * fix * fix(deploy): use latest successful image * typo * refactor(docker): generate config file at startup * revert(build): changes were made to docker * fix(docker): send variables correctly to the entrypoint * test different conf file approach * fix(env): add RUN_TEST env variable * ref: use previous approach * fix(color): use environment variable * fix(resources): use our normal machine size * fix(ci): double CPU and RAM for full sync test * fix(test): check for zebrad test output in the correct order The mempool is only activated once, so we must check for that log first. After mempool activation, the stop regex is logged at least once. (It might be logged before as well, but we can't rely on that.) When checking that the mempool didn't activate, wait for the `zebrad` command to exit, then check the entire log. * fix(ci): run full sync test with full compiler optimisations * fix(tests): reintroduce tests and run full sync on approval * fix(tests): reduce the changelog Co-authored-by: teor <teor@riseup.net>
2022-03-02 06:15:24 -08:00
fi
esac