zecfoundation
/
zebra
mirror of https://github.com/ZcashFoundation/zebra.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity

ref(docker): combine test and prod entrypoints into one (#7660) 

* ref(docker): combine test and prod entrypoints into one

This makes the following changes
- Uses `-x` instead of echoing the variables values
- Sets default values where required
- Create a function to list directories
- Create a function to run cargo tests
- Use a better approach to handle different options in the `case` manegement for tests and production
- Replaces all instances of `runtime-entrypoint.sh` with `entrypoint.sh`

* chore: use 2 spaces insted of 4 for indentation

* fix: add missing `test-threads` to `fully_synced_rpc_`

* fix: allow 3 arguments instead of 2 to handle `fully_synced_rpc_` correctly

* chore: remove extra file

* chore: add comments to main parts of the file

* fix(ci): just create the `$ZEBRA_CONF_PATH` if `$ENTRYPOINT_FEATURES` is not set

* fix(entrypoint): just print ${ZEBRA_CONF_PATH} if exists

* fix: missing condition

* ref: handle tests better if `$ENTRYPOINT_FEATURES` is set

* fix(ci): We just want `ZEBRA_CONF_PATH` to be set in the `release` image

* fix(entrypoint): fix the overall `case` logic

* fix(ci): allos to run with custom config in CI image

* fix(ci): more edgecases

* fix: we don't need to find files, but subdirectories

* fix(ci): handle Signal Forwarding and exit codes for `cargo`

* fix(ci): parse `fully_synced_rpc_` test correctly

* chore: add missing cache dir variable for LWD

* fix(entrypoint): handle an arbitrary number of arguments

* fix(entrypoint): handle features list

* fix(entrypoint): typo

* chore: typo
This commit is contained in:
Gustavo Valverde 2023-10-11 20:04:45 +01:00 committed by GitHub
parent 758eb6e0ea
commit fbd6b0f8b3
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
6 changed files with 338 additions and 240 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
.dockerignore
View File

@ -21,4 +21,3 @@
!zebra-* !zebra-*
!zebrad !zebrad
!docker/entrypoint.sh !docker/entrypoint.sh
!docker/runtime-entrypoint.sh

2
.github/workflows/continous-integration-docker.yml vendored
View File

@ -316,7 +316,7 @@ jobs:
run: | run: |
set -ex set -ex
docker pull ${{ vars.GAR_BASE }}/${{ vars.CI_IMAGE_NAME }}:sha-${{ env.GITHUB_SHA_SHORT }} docker pull ${{ vars.GAR_BASE }}/${{ vars.CI_IMAGE_NAME }}:sha-${{ env.GITHUB_SHA_SHORT }}
docker run -e NETWORK --detach -e ZEBRA_CONF_PATH --name variable-conf-tests -t ${{ vars.GAR_BASE }}/${{ vars.CI_IMAGE_NAME }}:sha-${{ env.GITHUB_SHA_SHORT }} -c $ZEBRA_CONF_PATH start docker run -e NETWORK --detach -e ZEBRA_CONF_PATH --name variable-conf-tests -t ${{ vars.GAR_BASE }}/${{ vars.CI_IMAGE_NAME }}:sha-${{ env.GITHUB_SHA_SHORT }} zebrad start
EXIT_STATUS=$(docker logs --tail all --follow variable-conf-tests 2>&1 | grep -q --extended-regexp --max-count=1 -e 'v1.0.0-rc.2.toml'; echo $?; ) EXIT_STATUS=$(docker logs --tail all --follow variable-conf-tests 2>&1 | grep -q --extended-regexp --max-count=1 -e 'v1.0.0-rc.2.toml'; echo $?; )
docker stop variable-conf-tests docker stop variable-conf-tests
docker logs variable-conf-tests docker logs variable-conf-tests

4
book/src/user/docker.md
View File

@ -51,7 +51,7 @@ And after our image has been built, we can run it on `Mainnet` with the followin
docker run --env LOG_COLOR="true" -p 9999:9999 local/zebra.mining docker run --env LOG_COLOR="true" -p 9999:9999 local/zebra.mining
``` ```
Based on our actual `runtime-entrypoint.sh` script, the following configuration file will be generated (on the fly, at startup) and used by Zebra: Based on our actual `entrypoint.sh` script, the following configuration file will be generated (on the fly, at startup) and used by Zebra:
```toml ```toml
[network] [network]
@ -80,7 +80,7 @@ endpoint_addr = "127.0.0.1:9999"
- `TEST_FEATURES`: Specifies the features for tests. Example: `"lightwalletd-grpc-tests zebra-checkpoints"` - `TEST_FEATURES`: Specifies the features for tests. Example: `"lightwalletd-grpc-tests zebra-checkpoints"`
- `ZEBRA_SKIP_IPV6_TESTS`: Skips IPv6 tests. Example: `1` - `ZEBRA_SKIP_IPV6_TESTS`: Skips IPv6 tests. Example: `1`
- `ENTRYPOINT_FEATURES`: Overrides the specific features used to run tests in `runtime-entrypoint.sh`. Example: `"default-release-binaries lightwalletd-grpc-tests"` - `ENTRYPOINT_FEATURES`: Overrides the specific features used to run tests in `entrypoint.sh`. Example: `"default-release-binaries lightwalletd-grpc-tests"`
#### CI/CD #### CI/CD

13
docker/Dockerfile
View File

@ -169,8 +169,8 @@ RUN rm -r zebra-original
# Build zebrad # Build zebrad
RUN cargo build --locked --release --features "${FEATURES}" --package zebrad --bin zebrad RUN cargo build --locked --release --features "${FEATURES}" --package zebrad --bin zebrad
COPY ./docker/runtime-entrypoint.sh / COPY ./docker/entrypoint.sh /
RUN chmod u+x /runtime-entrypoint.sh RUN chmod u+x /entrypoint.sh
# This stage is only used when deploying nodes or when only the resulting zebrad binary is needed # This stage is only used when deploying nodes or when only the resulting zebrad binary is needed
# #
@ -178,7 +178,7 @@ RUN chmod u+x /runtime-entrypoint.sh
# binary from the `release` stage, and the Zcash Sprout & Sapling parameters from ZCash # binary from the `release` stage, and the Zcash Sprout & Sapling parameters from ZCash
FROM debian:bullseye-slim AS runtime FROM debian:bullseye-slim AS runtime
COPY --from=release /opt/zebrad/target/release/zebrad /usr/local/bin COPY --from=release /opt/zebrad/target/release/zebrad /usr/local/bin
COPY --from=release /runtime-entrypoint.sh / COPY --from=release /entrypoint.sh /
COPY --from=us-docker.pkg.dev/zfnd-dev-zebra/zebra/zcash-params:edge /root/.zcash-params /root/.zcash-params COPY --from=us-docker.pkg.dev/zfnd-dev-zebra/zebra/zcash-params:edge /root/.zcash-params /root/.zcash-params
RUN apt-get update && \ RUN apt-get update && \
@ -189,9 +189,14 @@ RUN apt-get update && \
ARG FEATURES ARG FEATURES
ENV FEATURES=${FEATURES} ENV FEATURES=${FEATURES}
# Path and name of the config file
ENV ZEBRA_CONF_DIR=/etc/zebrad
ENV ZEBRA_CONF_FILE=zebrad.toml
# Expose configured ports # Expose configured ports
EXPOSE 8233 18233 EXPOSE 8233 18233
# Update the config file based on the Docker run variables, # Update the config file based on the Docker run variables,
# and launch zebrad with it # and launch zebrad with it
ENTRYPOINT [ "/runtime-entrypoint.sh" ] ENTRYPOINT [ "/entrypoint.sh" ]
CMD ["zebrad"]

438
docker/entrypoint.sh
View File

@ -1,128 +1,342 @@
#!/usr/bin/env bash #!/usr/bin/env bash
# show the commands we are executing # This script serves as the entrypoint for the Zebra Docker container.
#
# Description:
# This script serves as the primary entrypoint for the Docker container. Its main responsibilities include:
# 1. Environment Setup: Prepares the environment by setting various flags and parameters.
# 2. Configuration Management: Dynamically generates the `zebrad.toml` configuration file based on environment variables, ensuring the node starts with the desired settings.
# 3. Test Execution: Can run a series of tests to validate functionality based on specified environment variables.
# 4. Node Startup: Starts the node, allowing it to begin its operations.
#
# Show the commands we are executing
set -x set -x
# exit if a command fails # Exit if a command fails
set -e set -e
# exit if any command in a pipeline fails # Exit if any command in a pipeline fails
set -o pipefail set -o pipefail
####
# General Variables
# These variables are used to run the Zebra node.
####
# Path and name of the config file. These two have defaults set in the Dockerfile.
: "${ZEBRA_CONF_DIR:=}"
: "${ZEBRA_CONF_FILE:=}"
# [network]
: "${NETWORK:=Mainnet}" : "${NETWORK:=Mainnet}"
echo "Test variables:" : "${ZEBRA_LISTEN_ADDR:=0.0.0.0}"
echo "NETWORK=$NETWORK" # [consensus]
echo "ZEBRA_TEST_LIGHTWALLETD=$ZEBRA_TEST_LIGHTWALLETD" : "${ZEBRA_CHECKPOINT_SYNC:=true}"
echo "Hard-coded Zebra full sync directory: /zebrad-cache" # [state]
echo "ZEBRA_CACHED_STATE_DIR=$ZEBRA_CACHED_STATE_DIR" # Set this to change the default cached state directory
echo "LIGHTWALLETD_DATA_DIR=$LIGHTWALLETD_DATA_DIR" : "${ZEBRA_CACHED_STATE_DIR:=/var/cache/zebrad-cache}"
echo "ENTRYPOINT_FEATURES=$ENTRYPOINT_FEATURES" : "${LIGHTWALLETD_DATA_DIR:=/var/cache/lwd-cache}"
# [metrics]
: "${METRICS_ENDPOINT_ADDR:=0.0.0.0}"
: "${METRICS_ENDPOINT_PORT:=9999}"
# [tracing]
: "${LOG_COLOR:=false}"
: "${TRACING_ENDPOINT_ADDR:=0.0.0.0}"
: "${TRACING_ENDPOINT_PORT:=3000}"
# [rpc]
: "${RPC_LISTEN_ADDR:=0.0.0.0}"
# if ${RPC_PORT} is not set and ${FEATURES} contains getblocktemplate-rpcs,
# set ${RPC_PORT} to the default value for the current network
if [[ -z "${RPC_PORT}" ]]; then
if [[ " ${FEATURES} " =~ " getblocktemplate-rpcs " ]]; then
if [[ "${NETWORK}" = "Mainnet" ]]; then
: "${RPC_PORT:=8232}"
elif [[ "${NETWORK}" = "Testnet" ]]; then
: "${RPC_PORT:=18232}"
fi
fi
fi
####
# Test Variables
# These variables are used to run tests in the Dockerfile.
####
: "${RUN_ALL_TESTS:=}"
: "${ZEBRA_TEST_LIGHTWALLETD:=}"
: "${FULL_SYNC_MAINNET_TIMEOUT_MINUTES:=}"
: "${FULL_SYNC_TESTNET_TIMEOUT_MINUTES:=}"
: "${TEST_DISK_REBUILD:=}"
: "${TEST_UPDATE_SYNC:=}"
: "${TEST_CHECKPOINT_SYNC:=}"
: "${GENERATE_CHECKPOINTS_MAINNET:=}"
: "${GENERATE_CHECKPOINTS_TESTNET:=}"
: "${TEST_LWD_RPC_CALL:=}"
: "${TEST_LWD_FULL_SYNC:=}"
: "${TEST_LWD_UPDATE_SYNC:=}"
: "${TEST_LWD_GRPC:=}"
: "${TEST_LWD_TRANSACTIONS:=}"
: "${TEST_GET_BLOCK_TEMPLATE:=}"
: "${TEST_SUBMIT_BLOCK:=}"
: "${ENTRYPOINT_FEATURES:=}"
# Configuration file path
if [[ -n "${ZEBRA_CONF_DIR}" ]] && [[ -n "${ZEBRA_CONF_FILE}" ]]; then
ZEBRA_CONF_PATH="${ZEBRA_CONF_DIR}/${ZEBRA_CONF_FILE}"
fi
# Populate `zebrad.toml` before starting zebrad, using the environmental
# variables set by the Dockerfile or the user. If the user has already created a config, don't replace it.
#
# We disable most ports by default, so the default config is secure.
# Users have to opt-in to additional functionality by setting environmental variables.
if [[ -n "${ZEBRA_CONF_PATH}" ]] && [[ ! -f "${ZEBRA_CONF_PATH}" ]] && [[ -z "${ENTRYPOINT_FEATURES}" ]]; then
# Create the conf path and file
mkdir -p "${ZEBRA_CONF_DIR}" || { echo "Error creating directory ${ZEBRA_CONF_DIR}"; exit 1; }
touch "${ZEBRA_CONF_PATH}" || { echo "Error creating file ${ZEBRA_CONF_PATH}"; exit 1; }
# Populate the conf file
cat <<EOF > "${ZEBRA_CONF_PATH}"
[network]
network = "${NETWORK}"
listen_addr = "${ZEBRA_LISTEN_ADDR}"
[state]
cache_dir = "${ZEBRA_CACHED_STATE_DIR}"
EOF
if [[ " ${FEATURES} " =~ " prometheus " ]]; then # spaces are important here to avoid partial matches
cat <<EOF >> "${ZEBRA_CONF_PATH}"
[metrics]
endpoint_addr = "${METRICS_ENDPOINT_ADDR}:${METRICS_ENDPOINT_PORT}"
EOF
fi
if [[ -n "${RPC_PORT}" ]]; then
cat <<EOF >> "${ZEBRA_CONF_PATH}"
[rpc]
listen_addr = "${RPC_LISTEN_ADDR}:${RPC_PORT}"
EOF
fi
if [[ -n "${LOG_FILE}" ]] || [[ -n "${LOG_COLOR}" ]] || [[ -n "${TRACING_ENDPOINT_ADDR}" ]]; then
cat <<EOF >> "${ZEBRA_CONF_PATH}"
[tracing]
EOF
if [[ " ${FEATURES} " =~ " filter-reload " ]]; then # spaces are important here to avoid partial matches
cat <<EOF >> "${ZEBRA_CONF_PATH}"
endpoint_addr = "${TRACING_ENDPOINT_ADDR}:${TRACING_ENDPOINT_PORT}"
EOF
fi
# Set this to log to a file, if not set, logs to standard output
if [[ -n "${LOG_FILE}" ]]; then
mkdir -p "$(dirname "${LOG_FILE}")"
cat <<EOF >> "${ZEBRA_CONF_PATH}"
log_file = "${LOG_FILE}"
EOF
fi
# Zebra automatically detects if it is attached to a terminal, and uses colored output.
# Set this to 'true' to force using color even if the output is not a terminal.
# Set this to 'false' to disable using color even if the output is a terminal.
if [[ "${LOG_COLOR}" = "true" ]]; then
cat <<EOF >> "${ZEBRA_CONF_PATH}"
force_use_color = true
EOF
elif [[ "${LOG_COLOR}" = "false" ]]; then
cat <<EOF >> "${ZEBRA_CONF_PATH}"
use_color = false
EOF
fi
fi
if [[ -n "${MINER_ADDRESS}" ]]; then
cat <<EOF >> "${ZEBRA_CONF_PATH}"
[mining]
miner_address = "${MINER_ADDRESS}"
EOF
fi
fi
if [[ -n "${ZEBRA_CONF_PATH}" ]] && [[ -z "${ENTRYPOINT_FEATURES}" ]]; then
# Print the config file
echo "Using zebrad.toml:"
cat "${ZEBRA_CONF_PATH}"
fi
# Function to list directory
check_directory_files() {
local dir="$1"
# Check if the directory exists
if [[ -d "${dir}" ]]; then
# Check if there are any subdirectories
if find "${dir}" -mindepth 1 -type d | read -r; then
# Subdirectories exist, so we continue
:
else
# No subdirectories, print message and exit with status 1
echo "No subdirectories found in ${dir}."
exit 1
fi
else
# Directory doesn't exist, print message and exit with status 1
echo "Directory ${dir} does not exist."
exit 1
fi
}
# Function to run cargo test with an arbitrary number of arguments
run_cargo_test() {
# Start constructing the command, ensuring that $1 is enclosed in single quotes as it's a feature list
local cmd="exec cargo test --locked --release --features '$1' --package zebrad --test acceptance -- --nocapture --include-ignored"
# Shift the first argument, as it's already included in the cmd
shift
# Loop through the remaining arguments
for arg in "$@"; do
if [[ -n ${arg} ]]; then
# If the argument is non-empty, add it to the command
cmd+=" ${arg}"
fi
done
# Run the command using eval, this will replace the current process with the cargo command
eval "${cmd}" || { echo "Cargo test failed"; exit 1; }
}
# Main Execution Logic:
# This script orchestrates the execution flow based on the provided arguments and environment variables.
# - If "$1" is '--', '-', or 'zebrad', the script processes the subsequent arguments for the 'zebrad' command.
# - If ENTRYPOINT_FEATURES is unset, it checks for ZEBRA_CONF_PATH. If set, 'zebrad' runs with this custom configuration; otherwise, it runs with the provided arguments.
# - If "$1" is an empty string and ENTRYPOINT_FEATURES is set, the script enters the testing phase, checking various environment variables to determine the specific tests to run.
# - Different tests or operations are triggered based on the respective conditions being met.
# - If "$1" doesn't match any of the above, it's assumed to be a command, which is executed directly.
# This structure ensures a flexible execution strategy, accommodating various scenarios such as custom configurations, different testing phases, or direct command execution.
case "$1" in case "$1" in
--* | -*) --* | -* | zebrad)
exec zebrad "$@" shift
;; if [[ -n "${ZEBRA_CONF_PATH}" ]]; then
*) exec zebrad -c "${ZEBRA_CONF_PATH}" "$@" || { echo "Execution with custom configuration failed"; exit 1; }
# For these tests, we activate the test features to avoid recompiling `zebrad`, else
# but we don't actually run any gRPC tests. exec zebrad "$@" || { echo "Execution failed"; exit 1; }
if [[ "$RUN_ALL_TESTS" -eq "1" ]]; then fi
# Run all the available tests for the current environment. ;;
# If the lightwalletd environmental variables are set, we will also run those tests. "")
cargo test --locked --release --features "$ENTRYPOINT_FEATURES" --workspace -- --nocapture --include-ignored if [[ -n "${ENTRYPOINT_FEATURES}" ]]; then
# Validate the test variables
# For these tests, we activate the test features to avoid recompiling `zebrad`,
# but we don't actually run any gRPC tests.
if [[ "${RUN_ALL_TESTS}" -eq "1" ]]; then
# Run all the available tests for the current environment.
# If the lightwalletd environmental variables are set, we will also run those tests.
exec cargo test --locked --release --features "${ENTRYPOINT_FEATURES}" --workspace -- --nocapture --include-ignored
# For these tests, we activate the gRPC feature to avoid recompiling `zebrad`, # For these tests, we activate the gRPC feature to avoid recompiling `zebrad`,
# but we don't actually run any gRPC tests. # but we don't actually run any gRPC tests.
elif [[ -n "$FULL_SYNC_MAINNET_TIMEOUT_MINUTES" ]]; then elif [[ -n "${FULL_SYNC_MAINNET_TIMEOUT_MINUTES}" ]]; then
# Run a Zebra full sync test on mainnet. # Run a Zebra full sync test on mainnet.
cargo test --locked --release --features "$ENTRYPOINT_FEATURES" --package zebrad --test acceptance -- --nocapture --include-ignored full_sync_mainnet run_cargo_test "${ENTRYPOINT_FEATURES}" "full_sync_mainnet"
# List directory generated by test # List directory generated by test
# TODO: replace with $ZEBRA_CACHED_STATE_DIR in Rust and workflows # TODO: replace with ${ZEBRA_CACHED_STATE_DIR} in Rust and workflows
ls -lh "/zebrad-cache"/*/* || (echo "No /zebrad-cache/*/*"; ls -lhR "/zebrad-cache" | head -50 || echo "No /zebrad-cache directory") check_directory_files "/zebrad-cache"
elif [[ -n "$FULL_SYNC_TESTNET_TIMEOUT_MINUTES" ]]; then
# Run a Zebra full sync test on testnet.
cargo test --locked --release --features "$ENTRYPOINT_FEATURES" --package zebrad --test acceptance -- --nocapture --include-ignored full_sync_testnet
# List directory generated by test
# TODO: replace with $ZEBRA_CACHED_STATE_DIR in Rust and workflows
ls -lh "/zebrad-cache"/*/* || (echo "No /zebrad-cache/*/*"; ls -lhR "/zebrad-cache" | head -50 || echo "No /zebrad-cache directory")
elif [[ "$TEST_DISK_REBUILD" -eq "1" ]]; then
# Run a Zebra sync up to the mandatory checkpoint.
#
# TODO: use environmental variables instead of Rust features (part of #2995)
cargo test --locked --release --features "test_sync_to_mandatory_checkpoint_${NETWORK,,},$ENTRYPOINT_FEATURES" --package zebrad --test acceptance -- --nocapture --include-ignored "sync_to_mandatory_checkpoint_${NETWORK,,}"
# TODO: replace with $ZEBRA_CACHED_STATE_DIR in Rust and workflows
ls -lh "/zebrad-cache"/*/* || (echo "No /zebrad-cache/*/*"; ls -lhR "/zebrad-cache" | head -50 || echo "No /zebrad-cache directory")
elif [[ "$TEST_UPDATE_SYNC" -eq "1" ]]; then
# Run a Zebra sync starting at the cached tip, and syncing to the latest tip.
#
# List directory used by test
ls -lh "$ZEBRA_CACHED_STATE_DIR"/*/* || (echo "No $ZEBRA_CACHED_STATE_DIR/*/*"; ls -lhR "$ZEBRA_CACHED_STATE_DIR" | head -50 || echo "No $ZEBRA_CACHED_STATE_DIR directory")
cargo test --locked --release --features "$ENTRYPOINT_FEATURES" --package zebrad --test acceptance -- --nocapture --include-ignored zebrad_update_sync
elif [[ "$TEST_CHECKPOINT_SYNC" -eq "1" ]]; then
# Run a Zebra sync starting at the cached mandatory checkpoint, and syncing past it.
#
# List directory used by test
# TODO: replace with $ZEBRA_CACHED_STATE_DIR in Rust and workflows
ls -lh "/zebrad-cache"/*/* || (echo "No /zebrad-cache/*/*"; ls -lhR "/zebrad-cache" | head -50 || echo "No /zebrad-cache directory")
# TODO: use environmental variables instead of Rust features (part of #2995)
cargo test --locked --release --features "test_sync_past_mandatory_checkpoint_${NETWORK,,},$ENTRYPOINT_FEATURES" --package zebrad --test acceptance -- --nocapture --include-ignored "sync_past_mandatory_checkpoint_${NETWORK,,}"
elif [[ "$GENERATE_CHECKPOINTS_MAINNET" -eq "1" ]]; then elif [[ -n "${FULL_SYNC_TESTNET_TIMEOUT_MINUTES}" ]]; then
# Generate checkpoints after syncing Zebra from a cached state on mainnet. # Run a Zebra full sync test on testnet.
# run_cargo_test "${ENTRYPOINT_FEATURES}" "full_sync_testnet"
# TODO: disable or filter out logs like: # List directory generated by test
# test generate_checkpoints_mainnet has been running for over 60 seconds # TODO: replace with ${ZEBRA_CACHED_STATE_DIR} in Rust and workflows
# check_directory_files "/zebrad-cache"
# List directory used by test
ls -lh "$ZEBRA_CACHED_STATE_DIR"/*/* || (echo "No $ZEBRA_CACHED_STATE_DIR/*/*"; ls -lhR "$ZEBRA_CACHED_STATE_DIR" | head -50 || echo "No $ZEBRA_CACHED_STATE_DIR directory")
cargo test --locked --release --features "$ENTRYPOINT_FEATURES" --package zebrad --test acceptance -- --nocapture --include-ignored generate_checkpoints_mainnet
elif [[ "$GENERATE_CHECKPOINTS_TESTNET" -eq "1" ]]; then
# Generate checkpoints after syncing Zebra on testnet.
#
# This test might fail if testnet is unstable.
#
# List directory used by test
ls -lh "$ZEBRA_CACHED_STATE_DIR"/*/* || (echo "No $ZEBRA_CACHED_STATE_DIR/*/*"; ls -lhR "$ZEBRA_CACHED_STATE_DIR" | head -50 || echo "No $ZEBRA_CACHED_STATE_DIR directory")
cargo test --locked --release --features "$ENTRYPOINT_FEATURES" --package zebrad --test acceptance -- --nocapture --include-ignored generate_checkpoints_testnet
elif [[ "$TEST_LWD_RPC_CALL" -eq "1" ]]; then elif [[ "${TEST_DISK_REBUILD}" -eq "1" ]]; then
# Starting at a cached Zebra tip, test a JSON-RPC call to Zebra. # Run a Zebra sync up to the mandatory checkpoint.
ls -lh "$ZEBRA_CACHED_STATE_DIR"/*/* || (echo "No $ZEBRA_CACHED_STATE_DIR/*/*"; ls -lhR "$ZEBRA_CACHED_STATE_DIR" | head -50 || echo "No $ZEBRA_CACHED_STATE_DIR directory") #
# Run both the fully synced RPC test and the subtree snapshot test, one test at a time. # TODO: use environmental variables instead of Rust features (part of #2995)
# Since these tests use the same cached state, a state problem in the first test can fail the second test. run_cargo_test "test_sync_to_mandatory_checkpoint_${NETWORK,,},${ENTRYPOINT_FEATURES}" "sync_to_mandatory_checkpoint_${NETWORK,,}"
cargo test --locked --release --features "$ENTRYPOINT_FEATURES" --package zebrad --test acceptance -- --nocapture --include-ignored --test-threads 1 fully_synced_rpc_ # TODO: replace with ${ZEBRA_CACHED_STATE_DIR} in Rust and workflows
elif [[ "$TEST_LWD_FULL_SYNC" -eq "1" ]]; then check_directory_files "/zebrad-cache"
# Starting at a cached Zebra tip, run a lightwalletd sync to tip.
ls -lh "$ZEBRA_CACHED_STATE_DIR"/*/* || (echo "No $ZEBRA_CACHED_STATE_DIR/*/*"; ls -lhR "$ZEBRA_CACHED_STATE_DIR" | head -50 || echo "No $ZEBRA_CACHED_STATE_DIR directory")
cargo test --locked --release --features "$ENTRYPOINT_FEATURES" --package zebrad --test acceptance -- --nocapture --include-ignored lightwalletd_full_sync
ls -lhR "$LIGHTWALLETD_DATA_DIR/db" || (echo "No $LIGHTWALLETD_DATA_DIR/db"; ls -lhR "$LIGHTWALLETD_DATA_DIR" | head -50 || echo "No $LIGHTWALLETD_DATA_DIR directory")
elif [[ "$TEST_LWD_UPDATE_SYNC" -eq "1" ]]; then
# Starting with a cached Zebra and lightwalletd tip, run a quick update sync.
ls -lh "$ZEBRA_CACHED_STATE_DIR"/*/* || (echo "No $ZEBRA_CACHED_STATE_DIR/*/*"; ls -lhR "$ZEBRA_CACHED_STATE_DIR" | head -50 || echo "No $ZEBRA_CACHED_STATE_DIR directory")
ls -lhR "$LIGHTWALLETD_DATA_DIR/db" || (echo "No $LIGHTWALLETD_DATA_DIR/db"; ls -lhR "$LIGHTWALLETD_DATA_DIR" | head -50 || echo "No $LIGHTWALLETD_DATA_DIR directory")
cargo test --locked --release --features "$ENTRYPOINT_FEATURES" --package zebrad --test acceptance -- --nocapture --include-ignored lightwalletd_update_sync
# These tests actually use gRPC. elif [[ "${TEST_UPDATE_SYNC}" -eq "1" ]]; then
elif [[ "$TEST_LWD_GRPC" -eq "1" ]]; then # Run a Zebra sync starting at the cached tip, and syncing to the latest tip.
# Starting with a cached Zebra and lightwalletd tip, test all gRPC calls to lightwalletd, which calls Zebra. #
ls -lh "$ZEBRA_CACHED_STATE_DIR"/*/* || (echo "No $ZEBRA_CACHED_STATE_DIR/*/*"; ls -lhR "$ZEBRA_CACHED_STATE_DIR" | head -50 || echo "No $ZEBRA_CACHED_STATE_DIR directory") # List directory used by test
ls -lhR "$LIGHTWALLETD_DATA_DIR/db" || (echo "No $LIGHTWALLETD_DATA_DIR/db"; ls -lhR "$LIGHTWALLETD_DATA_DIR" | head -50 || echo "No $LIGHTWALLETD_DATA_DIR directory") check_directory_files "${ZEBRA_CACHED_STATE_DIR}"
cargo test --locked --release --features "$ENTRYPOINT_FEATURES" --package zebrad --test acceptance -- --nocapture --include-ignored lightwalletd_wallet_grpc_tests run_cargo_test "${ENTRYPOINT_FEATURES}" "zebrad_update_sync"
elif [[ "$TEST_LWD_TRANSACTIONS" -eq "1" ]]; then
# Starting with a cached Zebra and lightwalletd tip, test sending transactions gRPC call to lightwalletd, which calls Zebra.
ls -lh "$ZEBRA_CACHED_STATE_DIR"/*/* || (echo "No $ZEBRA_CACHED_STATE_DIR/*/*"; ls -lhR "$ZEBRA_CACHED_STATE_DIR" | head -50 || echo "No $ZEBRA_CACHED_STATE_DIR directory")
ls -lhR "$LIGHTWALLETD_DATA_DIR/db" || (echo "No $LIGHTWALLETD_DATA_DIR/db"; ls -lhR "$LIGHTWALLETD_DATA_DIR" | head -50 || echo "No $LIGHTWALLETD_DATA_DIR directory")
cargo test --locked --release --features "$ENTRYPOINT_FEATURES" --package zebrad --test acceptance -- --nocapture --include-ignored sending_transactions_using_lightwalletd
# These tests use mining code, but don't use gRPC. elif [[ "${TEST_CHECKPOINT_SYNC}" -eq "1" ]]; then
# We add the mining feature here because our other code needs to pass tests without it. # Run a Zebra sync starting at the cached mandatory checkpoint, and syncing past it.
elif [[ "$TEST_GET_BLOCK_TEMPLATE" -eq "1" ]]; then #
# Starting with a cached Zebra tip, test getting a block template from Zebra's RPC server. # List directory used by test
ls -lh "$ZEBRA_CACHED_STATE_DIR"/*/* || (echo "No $ZEBRA_CACHED_STATE_DIR/*/*"; ls -lhR "$ZEBRA_CACHED_STATE_DIR" | head -50 || echo "No $ZEBRA_CACHED_STATE_DIR directory") # TODO: replace with ${ZEBRA_CACHED_STATE_DIR} in Rust and workflows
cargo test --locked --release --features "getblocktemplate-rpcs,$ENTRYPOINT_FEATURES" --package zebrad --test acceptance -- --nocapture --include-ignored get_block_template check_directory_files "/zebrad-cache"
elif [[ "$TEST_SUBMIT_BLOCK" -eq "1" ]]; then # TODO: use environmental variables instead of Rust features (part of #2995)
# Starting with a cached Zebra tip, test sending a block to Zebra's RPC port. run_cargo_test "test_sync_past_mandatory_checkpoint_${NETWORK,,},${ENTRYPOINT_FEATURES}" "sync_past_mandatory_checkpoint_${NETWORK,,}"
ls -lh "$ZEBRA_CACHED_STATE_DIR"/*/* || (echo "No $ZEBRA_CACHED_STATE_DIR/*/*"; ls -lhR "$ZEBRA_CACHED_STATE_DIR" | head -50 || echo "No $ZEBRA_CACHED_STATE_DIR directory")
cargo test --locked --release --features "getblocktemplate-rpcs,$ENTRYPOINT_FEATURES" --package zebrad --test acceptance -- --nocapture --include-ignored submit_block
else elif [[ "${GENERATE_CHECKPOINTS_MAINNET}" -eq "1" ]]; then
exec "$@" # Generate checkpoints after syncing Zebra from a cached state on mainnet.
fi #
# TODO: disable or filter out logs like:
# test generate_checkpoints_mainnet has been running for over 60 seconds
#
# List directory used by test
check_directory_files "${ZEBRA_CACHED_STATE_DIR}"
run_cargo_test "${ENTRYPOINT_FEATURES}" "generate_checkpoints_mainnet"
elif [[ "${GENERATE_CHECKPOINTS_TESTNET}" -eq "1" ]]; then
# Generate checkpoints after syncing Zebra on testnet.
#
# This test might fail if testnet is unstable.
#
# List directory used by test
check_directory_files "${ZEBRA_CACHED_STATE_DIR}"
run_cargo_test "${ENTRYPOINT_FEATURES}" "generate_checkpoints_testnet"
elif [[ "${TEST_LWD_RPC_CALL}" -eq "1" ]]; then
# Starting at a cached Zebra tip, test a JSON-RPC call to Zebra.
check_directory_files "${ZEBRA_CACHED_STATE_DIR}"
# Run both the fully synced RPC test and the subtree snapshot test, one test at a time.
# Since these tests use the same cached state, a state problem in the first test can fail the second test.
run_cargo_test "${ENTRYPOINT_FEATURES}" "--test-threads" "1" "fully_synced_rpc_"
elif [[ "${TEST_LWD_FULL_SYNC}" -eq "1" ]]; then
# Starting at a cached Zebra tip, run a lightwalletd sync to tip.
check_directory_files "${ZEBRA_CACHED_STATE_DIR}"
run_cargo_test "${ENTRYPOINT_FEATURES}" "lightwalletd_full_sync"
check_directory_files "${LIGHTWALLETD_DATA_DIR}/db"
elif [[ "${TEST_LWD_UPDATE_SYNC}" -eq "1" ]]; then
# Starting with a cached Zebra and lightwalletd tip, run a quick update sync.
check_directory_files "${ZEBRA_CACHED_STATE_DIR}"
check_directory_files "${LIGHTWALLETD_DATA_DIR}/db"
run_cargo_test "${ENTRYPOINT_FEATURES}" "lightwalletd_update_sync"
# These tests actually use gRPC.
elif [[ "${TEST_LWD_GRPC}" -eq "1" ]]; then
# Starting with a cached Zebra and lightwalletd tip, test all gRPC calls to lightwalletd, which calls Zebra.
check_directory_files "${ZEBRA_CACHED_STATE_DIR}"
check_directory_files "${LIGHTWALLETD_DATA_DIR}/db"
run_cargo_test "${ENTRYPOINT_FEATURES}" "lightwalletd_wallet_grpc_tests"
elif [[ "${TEST_LWD_TRANSACTIONS}" -eq "1" ]]; then
# Starting with a cached Zebra and lightwalletd tip, test sending transactions gRPC call to lightwalletd, which calls Zebra.
check_directory_files "${ZEBRA_CACHED_STATE_DIR}"
check_directory_files "${LIGHTWALLETD_DATA_DIR}/db"
run_cargo_test "${ENTRYPOINT_FEATURES}" "sending_transactions_using_lightwalletd"
# These tests use mining code, but don't use gRPC.
# We add the mining feature here because our other code needs to pass tests without it.
elif [[ "${TEST_GET_BLOCK_TEMPLATE}" -eq "1" ]]; then
# Starting with a cached Zebra tip, test getting a block template from Zebra's RPC server.
check_directory_files "${ZEBRA_CACHED_STATE_DIR}"
run_cargo_test "getblocktemplate-rpcs,${ENTRYPOINT_FEATURES}" "get_block_template"
elif [[ "${TEST_SUBMIT_BLOCK}" -eq "1" ]]; then
# Starting with a cached Zebra tip, test sending a block to Zebra's RPC port.
check_directory_files "${ZEBRA_CACHED_STATE_DIR}"
run_cargo_test "getblocktemplate-rpcs,${ENTRYPOINT_FEATURES}" "submit_block"
else
exec "$@"
fi
fi
;;
*)
exec "$@"
;;
esac esac

120
docker/runtime-entrypoint.sh
View File

@ -1,120 +0,0 @@
#!/usr/bin/env bash
# Show the commands we are executing
set -x
# Exit if a command fails
set -e
# Exit if any command in a pipeline fails
set -o pipefail
# Set this to change the default cached state directory
# Path and name of the config file
: "${ZEBRA_CONF_DIR:=/etc/zebrad}"
: "${ZEBRA_CONF_FILE:=zebrad.toml}"
if [[ -n "$ZEBRA_CONF_DIR" ]] && [[ -n "$ZEBRA_CONF_FILE" ]]; then
ZEBRA_CONF_PATH="$ZEBRA_CONF_DIR/$ZEBRA_CONF_FILE"
fi
# [network]
: "${NETWORK:=Mainnet}"
: "${ZEBRA_LISTEN_ADDR:=0.0.0.0}"
# [consensus]
: "${ZEBRA_CHECKPOINT_SYNC:=true}"
# [state]
: "${ZEBRA_CACHED_STATE_DIR:=/var/cache/zebrad-cache}"
# [metrics]
: "${METRICS_ENDPOINT_ADDR:=0.0.0.0}"
: "${METRICS_ENDPOINT_PORT:=9999}"
# [tracing]
: "${LOG_COLOR:=false}"
: "${TRACING_ENDPOINT_ADDR:=0.0.0.0}"
: "${TRACING_ENDPOINT_PORT:=3000}"
# [rpc]
: "${RPC_LISTEN_ADDR:=0.0.0.0}"
if [[ -z "${RPC_PORT}" ]]; then
if [[ " ${FEATURES} " =~ " getblocktemplate-rpcs " ]]; then
if [[ "${NETWORK}" = "Mainnet" ]]; then
: "${RPC_PORT:=8232}"
elif [[ "${NETWORK}" = "Testnet" ]]; then
: "${RPC_PORT:=18232}"
fi
fi
fi
# Populate `zebrad.toml` before starting zebrad, using the environmental
# variables set by the Dockerfile or the user. If the user has already created a config, don't replace it.
#
# We disable most ports by default, so the default config is secure.
# Users have to opt-in to additional functionality by setting environmental variables.
if [[ -n "$ZEBRA_CONF_PATH" ]] && [[ ! -f "$ZEBRA_CONF_PATH" ]]; then
# Create the conf path and file
mkdir -p "$ZEBRA_CONF_DIR"
touch "$ZEBRA_CONF_PATH"
# Populate the conf file
cat <<EOF > "$ZEBRA_CONF_PATH"
[network]
network = "$NETWORK"
listen_addr = "$ZEBRA_LISTEN_ADDR"
[state]
cache_dir = "$ZEBRA_CACHED_STATE_DIR"
EOF
if [[ " $FEATURES " =~ " prometheus " ]]; then # spaces are important here to avoid partial matches
cat <<EOF >> "$ZEBRA_CONF_PATH"
[metrics]
endpoint_addr = "${METRICS_ENDPOINT_ADDR}:${METRICS_ENDPOINT_PORT}"
EOF
fi
if [[ -n "${RPC_PORT}" ]]; then
cat <<EOF >> "${ZEBRA_CONF_PATH}"
[rpc]
listen_addr = "${RPC_LISTEN_ADDR}:${RPC_PORT}"
EOF
fi
if [[ -n "$LOG_FILE" ]] || [[ -n "$LOG_COLOR" ]] || [[ -n "$TRACING_ENDPOINT_ADDR" ]]; then
cat <<EOF >> "$ZEBRA_CONF_PATH"
[tracing]
EOF
if [[ " $FEATURES " =~ " filter-reload " ]]; then # spaces are important here to avoid partial matches
cat <<EOF >> "$ZEBRA_CONF_PATH"
endpoint_addr = "${TRACING_ENDPOINT_ADDR}:${TRACING_ENDPOINT_PORT}"
EOF
fi
# Set this to log to a file, if not set, logs to standard output
if [[ -n "$LOG_FILE" ]]; then
mkdir -p "$(dirname "$LOG_FILE")"
cat <<EOF >> "$ZEBRA_CONF_PATH"
log_file = "${LOG_FILE}"
EOF
fi
# Zebra automatically detects if it is attached to a terminal, and uses colored output.
# Set this to 'true' to force using color even if the output is not a terminal.
# Set this to 'false' to disable using color even if the output is a terminal.
if [[ "$LOG_COLOR" = "true" ]]; then
cat <<EOF >> "$ZEBRA_CONF_PATH"
force_use_color = true
EOF
elif [[ "$LOG_COLOR" = "false" ]]; then
cat <<EOF >> "$ZEBRA_CONF_PATH"
use_color = false
EOF
fi
fi
if [[ -n "$MINER_ADDRESS" ]]; then
cat <<EOF >> "$ZEBRA_CONF_PATH"
[mining]
miner_address = "${MINER_ADDRESS}"
EOF
fi
fi
echo "Using zebrad.toml:"
cat "$ZEBRA_CONF_PATH"
exec zebrad -c "$ZEBRA_CONF_PATH" "$@"