project-serum
/
solana
mirror of https://github.com/project-serum/solana.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity

Add Snap fullnode daemon

This commit is contained in:
Michael Vines 2018-06-24 10:10:55 -07:00
parent ed6ba55261
commit ce361c2cdc
12 changed files with 340 additions and 73 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

6
.gitignore vendored
View File

@ -2,3 +2,9 @@ Cargo.lock
/target/ /target/
**/*.rs.bk **/*.rs.bk
.cargo .cargo
# node configuration files
/config/
/config-drone/
/config-validator/
/config-client-demo/

84
README.md
View File

@ -84,13 +84,12 @@ Now start the server:
$ ./multinode-demo/leader.sh $ ./multinode-demo/leader.sh
``` ```
To run a performance-enhanced fullnode on Linux, download `libcuda_verify_ed25519.a`. Enable To run a performance-enhanced fullnode on Linux,
it by adding `--features=cuda` to the line that runs `solana-fullnode` in [CUDA 9.2](https://developer.nvidia.com/cuda-downloads) must be installed on
`leader.sh`. [CUDA 9.2](https://developer.nvidia.com/cuda-downloads) must be installed on your system. your system:
```bash ```bash
$ ./fetch-perf-libs.sh $ ./fetch-perf-libs.sh
$ cargo run --release --features=cuda --bin solana-fullnode -- -l leader.json < genesis.log $ SOLANA_CUDA=1 ./multinode-demo/leader.sh
``` ```
Wait a few seconds for the server to initialize. It will print "Ready." when it's ready to Wait a few seconds for the server to initialize. It will print "Ready." when it's ready to
@ -102,14 +101,15 @@ Multinode Testnet
To run a multinode testnet, after starting a leader node, spin up some validator nodes: To run a multinode testnet, after starting a leader node, spin up some validator nodes:
```bash ```bash
$ ./multinode-demo/validator.sh ubuntu@10.0.1.51:~/solana #The leader machine $ ./multinode-demo/validator.sh ubuntu@10.0.1.51:~/solana 10.0.1.51
``` ```
As with the leader node, you can run a performance-enhanced validator fullnode by adding To run a performance-enhanced fullnode on Linux,
`--features=cuda` to the line that runs `solana-fullnode` in `validator.sh`. [CUDA 9.2](https://developer.nvidia.com/cuda-downloads) must be installed on
your system:
```bash ```bash
$ cargo run --release --features=cuda --bin solana-fullnode -- -l validator.json -v leader.json < genesis.log $ ./fetch-perf-libs.sh
$ SOLANA_CUDA=1 ./multinode-demo/leader.sh ubuntu@10.0.1.51:~/solana 10.0.1.51
``` ```
@ -152,6 +152,70 @@ $ snap info solana
$ sudo snap refresh solana --devmode $ sudo snap refresh solana --devmode
``` ```
### Daemon support
The snap supports running a leader, validator or leader+drone node as a system
daemon.
Run `sudo snap get solana` to view the current daemon configuration, and
`sudo snap logs -f solana` to view the daemon logs.
Disable the daemon at any time by running:
```bash
$ sudo snap set solana mode=
```
Runtime configuration files for the daemon can be found in
`/var/snap/solana/current/config`.
#### Leader daemon
```bash
$ sudo snap set solana mode=leader
```
If CUDA is available:
```bash
$ sudo snap set solana mode=leader enable-cuda=1
```
`rsync` must be configured and running on the leader.
1. Ensure rsync is installed with `sudo apt-get -y install rsync`
2. Edit `/etc/rsyncd.conf` to include the following
```
[config]
path = /var/snap/solana/current/config
hosts allow = *
read only = true
```
3. Run `sudo systemctl enable rsync; sudo systemctl start rsync`
4. Test by running `rsync -Pzravv rsync://<ip-address-of-leader>/config
solana-config` from another machine. If the leader is running on a cloud
provider it may be necessary to configure the Firewall rules to permit ingress
to port tcp:873, tcp:9900 and the port range udp:8000-udp:10000
To run both the Leader and Drone:
```bash
$ sudo snap set solana mode=leader+drone
```
#### Validator daemon
```bash
$ sudo snap set solana mode=validator
```
If CUDA is available:
```bash
$ sudo snap set solana mode=validator enable-cuda=1
```
By default the validator will connect to **testnet.solana.com**, override
the leader IP address by running:
```bash
$ sudo snap set solana mode=validator leader-address=127.0.0.1 #<-- change IP address
```
It's assumed that the leader will be running `rsync` configured as described in
the previous **Leader daemon** section.
Developing Developing
=== ===

3
ci/snap.sh
View File

@ -36,5 +36,8 @@ set -x
echo --- build echo --- build
snapcraft snapcraft
source ci/upload_ci_artifact.sh
upload_ci_artifact solana_*.snap
echo --- publish echo --- publish
$DRYRUN snapcraft push solana_*.snap --release $SNAP_CHANNEL $DRYRUN snapcraft push solana_*.snap --release $SNAP_CHANNEL

18
ci/upload_ci_artifact.sh Normal file
View File

@ -0,0 +1,18 @@
# |source| me
upload_ci_artifact() {
echo "--- artifact: $1"
if [[ -r "$1" ]]; then
ls -l "$1"
if ${BUILDKITE:-false}; then
(
set -x
buildkite-agent artifact upload "$1"
)
fi
else
echo ^^^ +++
echo "$1 not found"
fi
}

29
multinode-demo/client.sh
View File

@ -1,17 +1,22 @@
#!/bin/bash #!/bin/bash
#
# usage: $0 <network path to solana repo on leader machine> <number of nodes in the network>"
#
if [[ -z $1 ]]; then here=$(dirname "$0")
echo "usage: $0 [network path to solana repo on leader machine] <number of nodes in the network>" # shellcheck source=multinode-demo/common.sh
exit 1 source "$here"/common.sh
fi SOLANA_CONFIG_DIR=config-client-demo
LEADER=$1 leader=${1:-${here}/..} # Default to local solana repo
COUNT=${2:-1} count=${2:-1}
rsync -vz "$LEADER"/{leader.json,mint-demo.json} . || exit $? set -ex
mkdir -p $SOLANA_CONFIG_DIR
rsync -vz "$leader"/config/leader.json $SOLANA_CONFIG_DIR/
rsync -vz "$leader"/config/mint-demo.json $SOLANA_CONFIG_DIR/
# if RUST_LOG is unset, default to info # shellcheck disable=SC2086 # $solana_client_demo should not be quoted
export RUST_LOG=${RUST_LOG:-solana=info} exec $solana_client_demo \
-n "$count" -l $SOLANA_CONFIG_DIR/leader.json -d \
cargo run --release --bin solana-client-demo -- \ < $SOLANA_CONFIG_DIR/mint-demo.json 2>&1 | tee client-demo.log
-n "$COUNT" -l leader.json -d < mint-demo.json 2>&1 | tee client.log

41
multinode-demo/common.sh Normal file
View File

@ -0,0 +1,41 @@
# |source| this file
#
# Disable complaints about unused variables in this file:
# shellcheck disable=2034
if [[ -d "$SNAP" ]]; then # Running as a Linux Snap?
solana_program() {
local program="$1"
printf "%s/command-%s.wrapper" "$SNAP" "$program"
}
SOLANA_CUDA="$(snapctl get enable-cuda)"
elif [[ -n "$USE_INSTALL" ]]; then # Assume |cargo install| was run
solana_program() {
local program="$1"
printf "solana-%s" "$program"
}
else
solana_program() {
local program="$1"
local features=""
if [[ "$program" =~ ^(.*)-cuda$ ]]; then
program=${BASH_REMATCH[1]}
features="--features=cuda,erasure"
fi
printf "cargo run --release --bin solana-%s %s -- " "$program" "$features"
}
fi
solana_client_demo=$(solana_program client-demo)
solana_drone=$(solana_program drone)
solana_fullnode=$(solana_program fullnode)
solana_fullnode_config=$(solana_program fullnode-config)
solana_fullnode_cuda=$(solana_program fullnode-cuda)
solana_genesis_demo=$(solana_program genesis-demo)
solana_mint_demo=$(solana_program mint-demo)
export RUST_LOG=${RUST_LOG:-solana=info} # if RUST_LOG is unset, default to info
export RUST_BACKTRACE=1
[[ $(uname) = Linux ]] && (set -x; sudo sysctl -w net.core.rmem_max=26214400 1>/dev/null 2>/dev/null)
SOLANA_CONFIG_DIR=${SNAP_DATA:-$PWD}/config

34
multinode-demo/drone.sh Executable file
View File

@ -0,0 +1,34 @@
#!/bin/bash
#
# usage: $0 <network path to solana repo on leader machine>
#
here=$(dirname "$0")
# shellcheck source=multinode-demo/common.sh
source "$here"/common.sh
SOLANA_CONFIG_DIR=config-drone
if [[ -d "$SNAP" ]]; then
# Exit if mode is not yet configured
# (typically the case after the Snap is first installed)
[[ -n "$(snapctl get mode)" ]] || exit 0
# Select leader from the Snap configuration
leader_address="$(snapctl get leader-address)"
if [[ -z "$leader_address" ]]; then
# Assume drone is running on the same node as the leader by default
leader_address="localhost"
fi
leader=rsync://"$leader_address"
else
leader=${1:-${here}/..} # Default to local solana repo
fi
set -ex
mkdir -p $SOLANA_CONFIG_DIR
rsync -vz "$leader"/config/leader.json $SOLANA_CONFIG_DIR/
rsync -vz "$leader"/config/mint-demo.json $SOLANA_CONFIG_DIR/
# shellcheck disable=SC2086 # $solana_drone should not be quoted
exec $solana_drone \
-l $SOLANA_CONFIG_DIR/leader.json < $SOLANA_CONFIG_DIR/mint-demo.json

40
multinode-demo/leader.sh
View File

@ -1,28 +1,28 @@
#!/bin/bash #!/bin/bash
here=$(dirname "$0") here=$(dirname "$0")
# shellcheck source=multinode-demo/common.sh
source "$here"/common.sh
# shellcheck source=/dev/null if [[ -d "$SNAP" ]]; then
. "${here}"/myip.sh # Exit if mode is not yet configured
# (typically the case after the Snap is first installed)
[[ -n "$(snapctl get mode)" ]] || exit 0
fi
myip=$(myip) || exit $? [[ -f "$SOLANA_CONFIG_DIR"/leader.json ]] || {
echo "$SOLANA_CONFIG_DIR/leader.json not found, run ${here}/setup.sh first"
[[ -f leader-"${myip}".json ]] || {
echo "I can't find a matching leader config file for \"${myip}\"...
Please run ${here}/setup.sh first.
"
exit 1 exit 1
} }
# if RUST_LOG is unset, default to info if [[ -n "$SOLANA_CUDA" ]]; then
export RUST_LOG=${RUST_LOG:-solana=info} program="$solana_fullnode_cuda"
else
program="$solana_fullnode"
fi
[[ $(uname) = Linux ]] && sudo sysctl -w net.core.rmem_max=26214400 1>/dev/null 2>/dev/null # shellcheck disable=SC2086 # $program should not be quoted
exec $program \
# this makes a leader.json file available alongside genesis, etc. for -l "$SOLANA_CONFIG_DIR"/leader.json \
# validators and clients < "$SOLANA_CONFIG_DIR"/genesis.log "$SOLANA_CONFIG_DIR"/tx-*.log \
cp leader-"${myip}".json leader.json > "$SOLANA_CONFIG_DIR"/tx-"$(date -u +%Y%m%d%H%M%S%N)".log
cargo run --release --bin solana-fullnode -- \
-l leader-"${myip}".json \
< genesis.log tx-*.log \
> tx-"$(date -u +%Y%m%d%H%M%S%N)".log

32
multinode-demo/setup.sh
View File

@ -1,15 +1,27 @@
#!/bin/bash #!/bin/bash
here=$(dirname "$0")
# shellcheck source=/dev/null
. "${here}"/myip.sh
myip=$(myip) || exit $?
num_tokens=${1:-1000000000} num_tokens=${1:-1000000000}
cargo run --release --bin solana-mint-demo <<<"${num_tokens}" > mint-demo.json here=$(dirname "$0")
cargo run --release --bin solana-genesis-demo < mint-demo.json > genesis.log # shellcheck source=multinode-demo/common.sh
source "$here"/common.sh
cargo run --release --bin solana-fullnode-config -- -d > leader-"${myip}".json set -e
cargo run --release --bin solana-fullnode-config -- -b 9000 -d > validator-"${myip}".json
echo "Cleaning $SOLANA_CONFIG_DIR"
rm -rvf "$SOLANA_CONFIG_DIR"
mkdir -p "$SOLANA_CONFIG_DIR"
echo "Creating $SOLANA_CONFIG_DIR/mint-demo.json with $num_tokens tokens"
$solana_mint_demo <<<"$num_tokens" > "$SOLANA_CONFIG_DIR"/mint-demo.json
echo "Creating $SOLANA_CONFIG_DIR/genesis.log"
$solana_genesis_demo < "$SOLANA_CONFIG_DIR"/mint-demo.json > "$SOLANA_CONFIG_DIR"/genesis.log
echo "Creating $SOLANA_CONFIG_DIR/leader.json"
$solana_fullnode_config -d > "$SOLANA_CONFIG_DIR"/leader.json
echo "Creating $SOLANA_CONFIG_DIR/validator.json"
$solana_fullnode_config -d -b 9000 > "$SOLANA_CONFIG_DIR"/validator.json
ls -lh "$SOLANA_CONFIG_DIR/"

64
multinode-demo/validator.sh
View File

@ -1,32 +1,54 @@
#!/bin/bash #!/bin/bash
here=$(dirname "$0") here=$(dirname "$0")
# shellcheck source=multinode-demo/common.sh
source "$here"/common.sh
# shellcheck source=/dev/null usage() {
. "${here}"/myip.sh echo "usage: $0 [network path to solana repo on leader machine] [network ip address of leader]"
leader=$1
[[ -z ${leader} ]] && {
echo "usage: $0 [network path to solana repo on leader machine]"
exit 1 exit 1
} }
myip=$(myip) || exit $? if [[ "$1" = "-h" || -n "$3" ]]; then
usage
fi
[[ -f validator-"$myip".json ]] || { if [[ -d "$SNAP" ]]; then
echo "I can't find a matching validator config file for \"${myip}\"... # Exit if mode is not yet configured
Please run ${here}/setup.sh first. # (typically the case after the Snap is first installed)
" [[ -n "$(snapctl get mode)" ]] || exit 0
# Select leader from the Snap configuration
leader_address="$(snapctl get leader-address)"
if [[ -z "$leader_address" ]]; then
# Assume drone is running on the same node as the leader by default
leader_address="localhost"
fi
leader=rsync://"$leader_address"
else
leader=${1:-${here}/..} # Default to local solana repo
leader_address=${2:-127.0.0.1} # Default to local leader
fi
leader_port=8001
if [[ -n "$SOLANA_CUDA" ]]; then
program="$solana_fullnode_cuda"
else
program="$solana_fullnode"
fi
[[ -f "$SOLANA_CONFIG_DIR"/validator.json ]] || {
echo "$SOLANA_CONFIG_DIR/validator.json not found, run ${here}/setup.sh first"
exit 1 exit 1
} }
rsync -vz "${leader}"/{mint-demo.json,leader.json,genesis.log,tx-*.log} . || exit $? set -ex
SOLANA_LEADER_CONFIG_DIR="$SOLANA_CONFIG_DIR"/leader-config
rm -rf "$SOLANA_LEADER_CONFIG_DIR"
rsync -vPrz "${leader}"/config/ "$SOLANA_LEADER_CONFIG_DIR"
ls -lh "$SOLANA_LEADER_CONFIG_DIR"
[[ $(uname) = Linux ]] && sudo sysctl -w net.core.rmem_max=26214400 1>/dev/null 2>/dev/null # shellcheck disable=SC2086 # $program should not be quoted
exec $program \
# if RUST_LOG is unset, default to info -l "$SOLANA_CONFIG_DIR"/validator.json -t "$leader_address:$leader_port" \
export RUST_LOG=${RUST_LOG:-solana=info} < "$SOLANA_LEADER_CONFIG_DIR"/genesis.log "$SOLANA_LEADER_CONFIG_DIR"/tx-*.log
cargo run --release --bin solana-fullnode -- \
-l validator-"${myip}".json -v leader.json \
< genesis.log tx-*.log

30
snap/hooks/configure vendored Executable file
View File

@ -0,0 +1,30 @@
#!/bin/bash -e
echo Stopping daemons
snapctl stop --disable solana.daemon-drone
snapctl stop --disable solana.daemon-leader
snapctl stop --disable solana.daemon-validator
mode="$(snapctl get mode)"
if [[ -z "$mode" ]]; then
exit 0
fi
$SNAP/bin/setup.sh "$(snapctl get num-tokens)"
case $mode in
leader+drone)
snapctl start --enable solana.daemon-leader
snapctl start --enable solana.daemon-drone
;;
leader)
snapctl start --enable solana.daemon-leader
;;
validator)
snapctl start --enable solana.daemon-validator
;;
*)
echo "Error: Unknown mode: $mode"
exit 1
;;
esac

32
snap/snapcraft.yaml
View File

@ -10,6 +10,10 @@ grade: devel
# CUDA dependency, so use 'devmode' confinement for now # CUDA dependency, so use 'devmode' confinement for now
confinement: devmode confinement: devmode
hooks:
configure:
plugs: []
apps: apps:
drone: drone:
command: solana-drone command: solana-drone
@ -44,7 +48,35 @@ apps:
client-demo: client-demo:
command: solana-client-demo command: solana-client-demo
daemon-validator:
daemon: simple
command: validator.sh
daemon-leader:
daemon: simple
command: leader.sh
daemon-drone:
daemon: simple
command: drone.sh
parts: parts:
solana-rsync:
plugin: nil
override-build: |
# TODO: build rsync from source instead of sneaking it in from the host
# system...
set -x
mkdir -p $SNAPCRAFT_PART_INSTALL/bin
cp -av /usr/bin/rsync $SNAPCRAFT_PART_INSTALL/bin/
solana-scripts:
plugin: nil
override-build: |
set -x
mkdir -p $SNAPCRAFT_PART_INSTALL/bin
cp -av multinode-demo/* $SNAPCRAFT_PART_INSTALL/bin/
solana-cuda: solana-cuda:
plugin: rust plugin: rust
rust-channel: stable rust-channel: stable