Rename leader/validator to bootstrap-leader/fullnode

Only rsyncing the genesis ledger snuck in here as well
This commit is contained in:
Michael Vines 2018-12-06 13:38:45 -08:00
parent b34e197424
commit 70c149c7da
15 changed files with 135 additions and 144 deletions

9
.gitignore vendored
View File

@ -6,13 +6,10 @@
**/*.rs.bk
.cargo
# node configuration files
# node config that is rsynced
/config/
/config-private/
/config-drone/
/config-validator/
/config-client/
/multinode-demo/test/config-client/
# node config that remains local
/config-local/
# log files
*.log

View File

@ -41,7 +41,7 @@ $ git checkout $TAG
### Configuration Setup
The network is initialized with a genesis ledger and leader/validator configuration files.
The network is initialized with a genesis ledger and fullnode configuration files.
These files can be generated by running the following script.
```bash
@ -50,12 +50,12 @@ $ ./multinode-demo/setup.sh
### Drone
In order for the leader, client and validators to work, we'll need to
In order for the fullnodes and clients to work, we'll need to
spin up a drone to give out some test tokens. The drone delivers Milton
Friedman-style "air drops" (free tokens to requesting clients) to be used in
test transactions.
Start the drone on the leader node with:
Start the drone with:
```bash
$ ./multinode-demo/drone.sh
@ -64,13 +64,13 @@ $ ./multinode-demo/drone.sh
### Singlenode Testnet
Before you start a fullnode, make sure you know the IP address of the machine you
want to be the leader for the demo, and make sure that udp ports 8000-10000 are
want to be the bootstrap leader for the demo, and make sure that udp ports 8000-10000 are
open on all the machines you want to test with.
Now start the server in a separate shell:
Now start the bootstrap leader in a separate shell:
```bash
$ ./multinode-demo/leader.sh
$ ./multinode-demo/bootstrap-leader.sh
```
Wait a few seconds for the server to initialize. It will print "leader ready..." when it's ready to
@ -79,21 +79,21 @@ The drone does not need to be running for subsequent leader starts.
### Multinode Testnet
To run a multinode testnet, after starting a leader node, spin up some validator nodes in
separate shells:
To run a multinode testnet, after starting a leader node, spin up some
additional full nodes in separate shells:
```bash
$ ./multinode-demo/validator.sh
$ ./multinode-demo/fullnode-x.sh
```
To run a performance-enhanced leader or validator (on Linux),
To run a performance-enhanced full node on Linux,
[CUDA 10.0](https://developer.nvidia.com/cuda-downloads) must be installed on
your system:
```bash
$ ./fetch-perf-libs.sh
$ SOLANA_CUDA=1 ./multinode-demo/leader.sh
$ SOLANA_CUDA=1 ./multinode-demo/validator.sh
$ SOLANA_CUDA=1 ./multinode-demo/bootstrap-leader.sh
$ SOLANA_CUDA=1 ./multinode-demo/fullnode-x.sh
```
@ -153,14 +153,13 @@ $ sudo snap refresh solana --devmode
```
### Daemon Support
The snap supports running a leader, validator or leader+drone node as a system
daemon.
The snap supports running fullnodes and a drone as system daemons.
Run `sudo snap get solana` to view the current daemon configuration. To view
daemon logs:
1. Run `sudo snap logs -n=all solana` to view the daemon initialization log
2. Runtime logging can be found under `/var/snap/solana/current/leader/`,
`/var/snap/solana/current/validator/`, or `/var/snap/solana/current/drone/` depending
2. Runtime logging can be found under `/var/snap/solana/current/bootstrap-leader/`,
`/var/snap/solana/current/fullnode/`, or `/var/snap/solana/current/drone/` depending
on which `mode=` was selected. Within each log directory the file `current`
contains the latest log, and the files `*.s` (if present) contain older rotated
logs.
@ -177,7 +176,7 @@ Runtime configuration files for the daemon can be found in
#### Leader Daemon
```bash
$ sudo snap set solana mode=bootstrap-fullnode
$ sudo snap set solana mode=bootstrap-leader
```
`rsync` must be configured and running on the leader.
@ -200,7 +199,7 @@ to port tcp:873, tcp:9900 and the port range udp:8000-udp:10000**
To run both the Leader and Drone:
```bash
$ sudo snap set solana mode=bootstrap-fullnode+drone
$ sudo snap set solana mode=bootstrap-leader+drone
```

View File

@ -11,7 +11,7 @@ source scripts/configure-metrics.sh
multinode-demo/setup.sh
backgroundCommands="drone leader validator validator-x"
backgroundCommands="drone bootstrap-leader fullnode fullnode-x"
pids=()
for cmd in $backgroundCommands; do
@ -88,7 +88,7 @@ echo "--- Ledger verification"
(
source multinode-demo/common.sh
set -x
cp -R "$SOLANA_CONFIG_DIR"/ledger /tmp/ledger-$$
cp -R "$SOLANA_CONFIG_DIR"/bootstrap-leader-ledger /tmp/ledger-$$
$solana_ledger_tool --ledger /tmp/ledger-$$ verify || exit $?
rm -rf /tmp/ledger-$$
) || flag_error

View File

@ -0,0 +1 @@
leader.sh

View File

@ -8,8 +8,8 @@
#
rsync=rsync
leader_logger="tee leader.log"
validator_logger="tee validator.log"
bootstrap_leader_logger="tee bootstrap-leader.log"
fullnode_logger="tee fullnode.log"
drone_logger="tee drone.log"
if [[ $(uname) != Linux ]]; then
@ -32,12 +32,12 @@ if [[ -d $SNAP ]]; then # Running inside a Linux Snap?
}
rsync="$SNAP"/bin/rsync
multilog="$SNAP/bin/multilog t s16777215 n200"
leader_logger="$multilog $SNAP_DATA/leader"
validator_logger="$multilog t $SNAP_DATA/validator"
bootstrap_leader_logger="$multilog $SNAP_DATA/bootstrap-leader"
fullnode_logger="$multilog t $SNAP_DATA/fullnode"
drone_logger="$multilog $SNAP_DATA/drone"
# Create log directories manually to prevent multilog from creating them as
# 0700
mkdir -p "$SNAP_DATA"/{drone,leader,validator}
mkdir -p "$SNAP_DATA"/{drone,bootstrap-leader,fullnode}
elif [[ -n $USE_SNAP ]]; then # Use the Linux Snap binaries
solana_program() {
@ -130,7 +130,9 @@ tune_networking() {
fi
}
SOLANA_CONFIG_DIR=${SNAP_DATA:-$PWD}/config
SOLANA_CONFIG_PRIVATE_DIR=${SNAP_DATA:-$PWD}/config-private
SOLANA_CONFIG_VALIDATOR_DIR=${SNAP_DATA:-$PWD}/config-validator
# The directory on the bootstrap leader that is rsynced by other full nodes as
# they boot (TODO: Eventually this should go away)
SOLANA_RSYNC_CONFIG_DIR=${SNAP_DATA:-$PWD}/config
# Configuration that remains local
SOLANA_CONFIG_DIR=${SNAP_DATA:-$PWD}/config-local

View File

@ -19,10 +19,10 @@ usage() {
exit 1
}
[[ -f "$SOLANA_CONFIG_PRIVATE_DIR"/mint-id.json ]] || {
echo "$SOLANA_CONFIG_PRIVATE_DIR/mint-id.json not found, create it by running:"
[[ -f "$SOLANA_CONFIG_DIR"/mint-id.json ]] || {
echo "$SOLANA_CONFIG_DIR/mint-id.json not found, create it by running:"
echo
echo " ${here}/setup.sh -t leader"
echo " ${here}/setup.sh"
exit 1
}
@ -30,7 +30,7 @@ set -ex
trap 'kill "$pid" && wait "$pid"' INT TERM
$solana_drone \
--keypair "$SOLANA_CONFIG_PRIVATE_DIR"/mint-id.json \
--keypair "$SOLANA_CONFIG_DIR"/mint-id.json \
> >($drone_logger) 2>&1 &
pid=$!
wait "$pid"

View File

@ -0,0 +1 @@
validator-x.sh

1
multinode-demo/fullnode.sh Symbolic link
View File

@ -0,0 +1 @@
validator.sh

View File

@ -1,6 +1,6 @@
#!/usr/bin/env bash
#
# Starts a leader node
# Start the bootstrap leader node
#
here=$(dirname "$0")
@ -16,8 +16,8 @@ if [[ -d "$SNAP" ]]; then
[[ -n "$(snapctl get mode)" ]] || exit 0
fi
[[ -f "$SOLANA_CONFIG_DIR"/leader.json ]] || {
echo "$SOLANA_CONFIG_DIR/leader.json not found, create it by running:"
[[ -f "$SOLANA_CONFIG_DIR"/bootstrap-leader.json ]] || {
echo "$SOLANA_CONFIG_DIR/bootstrap-leader.json not found, create it by running:"
echo
echo " ${here}/setup.sh"
exit 1
@ -34,10 +34,10 @@ tune_networking
trap 'kill "$pid" && wait "$pid"' INT TERM
$program \
--no-leader-rotation \
--identity "$SOLANA_CONFIG_DIR"/leader.json \
--ledger "$SOLANA_CONFIG_DIR"/ledger \
--identity "$SOLANA_CONFIG_DIR"/bootstrap-leader.json \
--ledger "$SOLANA_CONFIG_DIR"/bootstrap-leader-ledger \
--rpc 8899 \
> >($leader_logger) 2>&1 &
> >($bootstrap_leader_logger) 2>&1 &
pid=$!
oom_score_adj "$pid" 1000
wait "$pid"

View File

@ -23,7 +23,7 @@ Creates a fullnode configuration
may be a private IP address unaccessible on the Intenet (default)
-p - Detect public address using public Internet servers
-t node_type - Create configuration files only for this kind of node. Valid
options are validator or leader. Creates configuration files
options are bootstrap_leader or fullnode. Creates configuration files
for both by default
EOF
@ -32,8 +32,8 @@ EOF
ip_address_arg=-l
num_tokens=1000000000
node_type_leader=true
node_type_validator=true
bootstrap_leader=true
fullnode=true
while getopts "h?n:lpt:" opt; do
case $opt in
h|\?)
@ -52,13 +52,13 @@ while getopts "h?n:lpt:" opt; do
t)
node_type="$OPTARG"
case $OPTARG in
leader)
node_type_leader=true
node_type_validator=false
bootstrap_leader|leader) # TODO: Remove legacy 'leader' option
bootstrap_leader=true
fullnode=false
;;
validator)
node_type_leader=false
node_type_validator=true
fullnode|validator) # TODO: Remove legacy 'validator' option
bootstrap_leader=false
fullnode=true
;;
*)
usage "Error: unknown node type: $node_type"
@ -74,49 +74,43 @@ done
set -e
for i in "$SOLANA_CONFIG_DIR" "$SOLANA_CONFIG_VALIDATOR_DIR" "$SOLANA_CONFIG_PRIVATE_DIR"; do
for i in "$SOLANA_RSYNC_CONFIG_DIR" "$SOLANA_CONFIG_DIR"; do
echo "Cleaning $i"
rm -rvf "$i"
mkdir -p "$i"
done
if $node_type_leader; then
leader_address_args=("$ip_address_arg")
leader_id_path="$SOLANA_CONFIG_PRIVATE_DIR"/leader-id.json
mint_id_path="$SOLANA_CONFIG_PRIVATE_DIR"/mint-id.json
if $bootstrap_leader; then
# Create genesis configuration
(
set -x
$solana_keygen -o "$SOLANA_CONFIG_DIR"/mint-id.json
$solana_keygen -o "$SOLANA_CONFIG_DIR"/bootstrap-leader-id.json
$solana_genesis \
--bootstrap-leader-keypair "$SOLANA_CONFIG_DIR"/bootstrap-leader-id.json \
--ledger "$SOLANA_RSYNC_CONFIG_DIR"/ledger \
--mint "$SOLANA_CONFIG_DIR"/mint-id.json \
--num_tokens "$num_tokens"
)
$solana_keygen -o "$leader_id_path"
# Create bootstrap leader configuration
(
set -x
$solana_fullnode_config \
--keypair="$SOLANA_CONFIG_DIR"/bootstrap-leader-id.json \
"$ip_address_arg" > "$SOLANA_CONFIG_DIR"/bootstrap-leader.json
echo "Creating $mint_id_path with $num_tokens tokens"
$solana_keygen -o "$mint_id_path"
echo "Creating $SOLANA_CONFIG_DIR/leader.json"
$solana_fullnode_config \
--keypair="$leader_id_path" \
"${leader_address_args[@]}" > "$SOLANA_CONFIG_DIR"/leader.json
echo "Creating $SOLANA_CONFIG_DIR/ledger"
$solana_genesis \
--num_tokens "$num_tokens" \
--mint "$mint_id_path" \
--bootstrap-leader-keypair "$leader_id_path" \
--ledger "$SOLANA_CONFIG_DIR"/ledger \
ls -lhR "$SOLANA_CONFIG_DIR"/
ls -lhR "$SOLANA_CONFIG_PRIVATE_DIR"/
cp -ra "$SOLANA_RSYNC_CONFIG_DIR"/ledger "$SOLANA_CONFIG_DIR"/bootstrap-leader-ledger
)
fi
if $node_type_validator; then
validator_address_args=("$ip_address_arg" -b 9000)
validator_id_path="$SOLANA_CONFIG_PRIVATE_DIR"/validator-id.json
$solana_keygen -o "$validator_id_path"
echo "Creating $SOLANA_CONFIG_VALIDATOR_DIR/validator.json"
$solana_fullnode_config \
--keypair="$validator_id_path" \
"${validator_address_args[@]}" > "$SOLANA_CONFIG_VALIDATOR_DIR"/validator.json
ls -lhR "$SOLANA_CONFIG_VALIDATOR_DIR"/
if $fullnode; then
(
set -x
$solana_keygen -o "$SOLANA_CONFIG_DIR"/fullnode-id.json
$solana_fullnode_config \
--keypair="$SOLANA_CONFIG_DIR"/fullnode-id.json \
"$ip_address_arg" -b 9000 > "$SOLANA_CONFIG_DIR"/fullnode.json
)
fi

View File

@ -1,8 +1,8 @@
#!/usr/bin/env bash
#
# Start a dynamically-configured validator node
# Start a dynamically-configured full node
#
here=$(dirname "$0")
exec "$here"/validator.sh -x "$@"
exec "$here"/fullnode.sh -x "$@"

View File

@ -1,6 +1,6 @@
#!/usr/bin/env bash
#
# Start a validator node
# Start a full node
#
here=$(dirname "$0")
# shellcheck source=multinode-demo/common.sh
@ -20,11 +20,11 @@ usage() {
echo "$*"
echo
fi
echo "usage: $0 [-x] [rsync network path to leader] [network entry point]"
echo "usage: $0 [-x] [rsync network path to bootstrap leader configuration] [network entry point]"
echo
echo " Start a validator on the specified network"
echo " Start a full node on the specified network"
echo
echo " -x: runs a new, dynamically-configured validator"
echo " -x: runs a new, dynamically-configured full node"
echo
exit 1
}
@ -100,47 +100,36 @@ else
fi
if ((!self_setup)); then
[[ -f $SOLANA_CONFIG_VALIDATOR_DIR/validator.json ]] || {
echo "$SOLANA_CONFIG_VALIDATOR_DIR/validator.json not found, create it by running:"
[[ -f $SOLANA_CONFIG_DIR/fullnode.json ]] || {
echo "$SOLANA_CONFIG_DIR/fullnode.json not found, create it by running:"
echo
echo " ${here}/setup.sh"
exit 1
}
validator_id_path=$SOLANA_CONFIG_PRIVATE_DIR/validator-id.json
validator_json_path=$SOLANA_CONFIG_VALIDATOR_DIR/validator.json
SOLANA_LEADER_CONFIG_DIR=$SOLANA_CONFIG_VALIDATOR_DIR/leader-config
fullnode_id_path=$SOLANA_CONFIG_DIR/fullnode-id.json
fullnode_json_path=$SOLANA_CONFIG_DIR/fullnode.json
ledger_config_dir=$SOLANA_CONFIG_DIR/fullnode-ledger
else
mkdir -p "$SOLANA_CONFIG_PRIVATE_DIR"
validator_id_path=$SOLANA_CONFIG_PRIVATE_DIR/validator-id-x$$.json
$solana_keygen -o "$validator_id_path"
mkdir -p "$SOLANA_CONFIG_DIR"
fullnode_id_path=$SOLANA_CONFIG_DIR/fullnode-id-x$$.json
$solana_keygen -o "$fullnode_id_path"
mkdir -p "$SOLANA_CONFIG_VALIDATOR_DIR"
validator_json_path=$SOLANA_CONFIG_VALIDATOR_DIR/validator-x$$.json
mkdir -p "$SOLANA_CONFIG_DIR"
fullnode_json_path=$SOLANA_CONFIG_DIR/fullnode-x$$.json
port=9000
(((port += ($$ % 1000)) && (port == 9000) && port++))
$solana_fullnode_config --keypair="$validator_id_path" -l -b "$port" > "$validator_json_path"
$solana_fullnode_config --keypair="$fullnode_id_path" -l -b "$port" > "$fullnode_json_path"
SOLANA_LEADER_CONFIG_DIR=$SOLANA_CONFIG_VALIDATOR_DIR/leader-config-x$$
ledger_config_dir=$SOLANA_CONFIG_DIR/fullnode-ledger-x$$
fi
[[ -r $validator_id_path ]] || {
echo "$validator_id_path does not exist"
[[ -r $fullnode_id_path ]] || {
echo "$fullnode_id_path does not exist"
exit 1
}
# A fullnode requires 2 tokens to function:
# - one token to create an instance of the vote_program with
# - one second token to keep the node identity public key valid.
(
set -x
$solana_wallet \
--keypair "$validator_id_path" \
--network "$leader_address" \
airdrop 2
)
rsync_url() { # adds the 'rsync://` prefix to URLs that need it
declare url="$1"
@ -165,19 +154,27 @@ rsync_leader_url=$(rsync_url "$leader")
tune_networking
set -ex
$rsync -vPr "$rsync_leader_url"/config/ "$SOLANA_LEADER_CONFIG_DIR"
[[ -d $SOLANA_LEADER_CONFIG_DIR/ledger ]] || {
$rsync -vPr "$rsync_leader_url"/config/ "$ledger_config_dir"
[[ -d $ledger_config_dir/ledger ]] || {
echo "Unable to retrieve ledger from $rsync_leader_url"
exit 1
}
# A fullnode requires 2 tokens to function:
# - one token to create an instance of the vote_program with
# - one second token to keep the node identity public key valid.
$solana_wallet \
--keypair "$fullnode_id_path" \
--network "$leader_address" \
airdrop 2
trap 'kill "$pid" && wait "$pid"' INT TERM
$program \
--no-leader-rotation \
--identity "$validator_json_path" \
--identity "$fullnode_json_path" \
--network "$leader_address" \
--ledger "$SOLANA_LEADER_CONFIG_DIR"/ledger \
> >($validator_logger) 2>&1 &
--ledger "$ledger_config_dir"/ledger \
> >($fullnode_logger) 2>&1 &
pid=$!
oom_score_adj "$pid" 1000
wait "$pid"

View File

@ -64,12 +64,12 @@ snap)
fi
if [[ $nodeType = bootstrap-fullnode ]]; then
nodeConfig="mode=bootstrap-fullnode+drone $commonNodeConfig"
ln -sf -T /var/snap/solana/current/leader/current fullnode.log
nodeConfig="mode=bootstrap-leader+drone $commonNodeConfig"
ln -sf -T /var/snap/solana/current/bootstrap-leader/current fullnode.log
ln -sf -T /var/snap/solana/current/drone/current drone.log
else
nodeConfig="mode=fullnode $commonNodeConfig"
ln -sf -T /var/snap/solana/current/validator/current fullnode.log
ln -sf -T /var/snap/solana/current/fullnode/current fullnode.log
fi
logmarker="solana deploy $(date)/$RANDOM"
@ -103,10 +103,10 @@ local|tar)
echo Selecting solana-fullnode-cuda
export SOLANA_CUDA=1
fi
./multinode-demo/setup.sh -t leader $setupArgs
./multinode-demo/setup.sh -t bootstrap_leader $setupArgs
./multinode-demo/drone.sh > drone.log 2>&1 &
./multinode-demo/leader.sh > leader.log 2>&1 &
ln -sTf leader.log fullnode.log
./multinode-demo/bootstrap-leader.sh > bootstrap-leader.log 2>&1 &
ln -sTf bootstrap-leader.log fullnode.log
;;
fullnode)
net/scripts/rsync-retry.sh -vPrc "$entrypointIp":~/.cargo/bin/ ~/.cargo/bin/
@ -116,9 +116,8 @@ local|tar)
export SOLANA_CUDA=1
fi
./multinode-demo/setup.sh -t validator $setupArgs
./multinode-demo/validator.sh "$entrypointIp":~/solana "$entrypointIp:8001" > validator.log 2>&1 &
ln -sTf validator.log fullnode.log
./multinode-demo/setup.sh -t fullnode $setupArgs
./multinode-demo/fullnode.sh "$entrypointIp":~/solana "$entrypointIp:8001" > fullnode.log 2>&1 &
;;
*)
echo "Error: unknown node type: $nodeType"

16
snap/hooks/configure vendored
View File

@ -2,7 +2,7 @@
echo Stopping daemons
snapctl stop --disable solana.daemon-drone
snapctl stop --disable solana.daemon-bootstrap-fullnode
snapctl stop --disable solana.daemon-bootstrap-leader
snapctl stop --disable solana.daemon-fullnode
snapctl stop --disable solana.daemon-oom-monitor
snapctl stop --disable solana.daemon-net-stats
@ -18,17 +18,17 @@ num_tokens="${num_tokens:+-n $num_tokens}"
setup_args="$(snapctl get setup-args)"
case $mode in
bootstrap-fullnode+drone)
"$SNAP"/multinode-demo/setup.sh -t leader $num_tokens -p $setup_args
bootstrap-leader+drone)
"$SNAP"/multinode-demo/setup.sh -t bootstrap_leader $num_tokens -p $setup_args
snapctl start --enable solana.daemon-drone
snapctl start --enable solana.daemon-bootstrap-fullnode
snapctl start --enable solana.daemon-bootstrap-leader
;;
bootstrap-fullnode)
"$SNAP"/multinode-demo/setup.sh -t leader $num_tokens -p $setup_args
snapctl start --enable solana.daemon-bootstrap-fullnode
bootstrap-leader)
"$SNAP"/multinode-demo/setup.sh -t bootstrap_leader $num_tokens -p $setup_args
snapctl start --enable solana.daemon-bootstrap-leader
;;
fullnode)
"$SNAP"/multinode-demo/setup.sh -t validator -p $setup_args
"$SNAP"/multinode-demo/setup.sh -t fullnode -p $setup_args
snapctl start --enable solana.daemon-fullnode
;;
*)

View File

@ -55,13 +55,13 @@ apps:
- home
daemon-fullnode:
daemon: simple
command: scripts/snap-config-to-env.sh $SNAP/multinode-demo/validator.sh
command: scripts/snap-config-to-env.sh $SNAP/multinode-demo/fullnode.sh
plugs:
- network
- network-bind
daemon-bootstrap-fullnode:
daemon-bootstrap-leader:
daemon: simple
command: scripts/snap-config-to-env.sh $SNAP/multinode-demo/leader.sh
command: scripts/snap-config-to-env.sh $SNAP/multinode-demo/bootstrap-leader.sh
plugs:
- network
- network-bind