From 76b83ac0f415db74c46113db54b17587524e99e4 Mon Sep 17 00:00:00 2001 From: Greg Fitzgerald Date: Sun, 25 Nov 2018 23:09:50 -0700 Subject: [PATCH] Move testnet demos into the book Have git readme focus on fullnode development and the book focus on users. --- README.md | 234 +---------------------------------------- src/SUMMARY.md | 2 + src/getting-started.md | 223 +++++++++++++++++++++++++++++++++++++++ 3 files changed, 227 insertions(+), 232 deletions(-) create mode 100644 src/getting-started.md diff --git a/README.md b/README.md index e94e5b7deb..eb35a60308 100644 --- a/README.md +++ b/README.md @@ -23,244 +23,14 @@ It's possible for a centralized database to process 710,000 transactions per sec Furthermore, and much to our surprise, it can be implemented using a mechanism that has existed in Bitcoin since day one. The Bitcoin feature is called nLocktime and it can be used to postdate transactions using block height instead of a timestamp. As a Bitcoin client, you'd use block height instead of a timestamp if you don't trust the network. Block height turns out to be an instance of what's being called a Verifiable Delay Function in cryptography circles. It's a cryptographically secure way to say time has passed. In Solana, we use a far more granular verifiable delay function, a SHA 256 hash chain, to checkpoint the ledger and coordinate consensus. With it, we implement Optimistic Concurrency Control and are now well en route towards that theoretical limit of 710,000 transactions per second. - -Testnet Demos +Architecture === -The Solana repo contains all the scripts you might need to spin up your own -local testnet. Depending on what you're looking to achieve, you may want to -run a different variation, as the full-fledged, performance-enhanced -multinode testnet is considerably more complex to set up than a Rust-only, -singlenode testnode. If you are looking to develop high-level features, such -as experimenting with smart contracts, save yourself some setup headaches and -stick to the Rust-only singlenode demo. If you're doing performance optimization -of the transaction pipeline, consider the enhanced singlenode demo. If you're -doing consensus work, you'll need at least a Rust-only multinode demo. If you want -to reproduce our TPS metrics, run the enhanced multinode demo. - -For all four variations, you'd need the latest Rust toolchain and the Solana -source code: - -First, install Rust's package manager Cargo. - -```bash -$ curl https://sh.rustup.rs -sSf | sh -$ source $HOME/.cargo/env -``` - -Now checkout the code from github: - -```bash -$ git clone https://github.com/solana-labs/solana.git -$ cd solana -``` - -The demo code is sometimes broken between releases as we add new low-level -features, so if this is your first time running the demo, you'll improve -your odds of success if you check out the -[latest release](https://github.com/solana-labs/solana/releases) -before proceeding: - -```bash -$ TAG=$(git describe --tags $(git rev-list --tags --max-count=1)) -$ git checkout $TAG -``` - -Configuration Setup ---- - -The network is initialized with a genesis ledger and leader/validator configuration files. -These files can be generated by running the following script. - -```bash -$ ./multinode-demo/setup.sh -``` - -Drone ---- - -In order for the leader, client and validators to work, we'll need to -spin up a drone to give out some test tokens. The drone delivers Milton -Friedman-style "air drops" (free tokens to requesting clients) to be used in -test transactions. - -Start the drone on the leader node with: - -```bash -$ ./multinode-demo/drone.sh -``` - -Singlenode Testnet ---- - -Before you start a fullnode, make sure you know the IP address of the machine you -want to be the leader for the demo, and make sure that udp ports 8000-10000 are -open on all the machines you want to test with. - -Now start the server in a separate shell: - -```bash -$ ./multinode-demo/leader.sh -``` - -Wait a few seconds for the server to initialize. It will print "leader ready..." when it's ready to -receive transactions. The leader will request some tokens from the drone if it doesn't have any. -The drone does not need to be running for subsequent leader starts. - -Multinode Testnet ---- - -To run a multinode testnet, after starting a leader node, spin up some validator nodes in -separate shells: - -```bash -$ ./multinode-demo/validator.sh -``` - -To run a performance-enhanced leader or validator (on Linux), -[CUDA 10.0](https://developer.nvidia.com/cuda-downloads) must be installed on -your system: - -```bash -$ ./fetch-perf-libs.sh -$ SOLANA_CUDA=1 ./multinode-demo/leader.sh -$ SOLANA_CUDA=1 ./multinode-demo/validator.sh -``` - - -Testnet Client Demo ---- - -Now that your singlenode or multinode testnet is up and running let's send it -some transactions! - -In a separate shell start the client: - -```bash -$ ./multinode-demo/client.sh # runs against localhost by default -``` - -What just happened? The client demo spins up several threads to send 500,000 transactions -to the testnet as quickly as it can. The client then pings the testnet periodically to see -how many transactions it processed in that time. Take note that the demo intentionally -floods the network with UDP packets, such that the network will almost certainly drop a -bunch of them. This ensures the testnet has an opportunity to reach 710k TPS. The client -demo completes after it has convinced itself the testnet won't process any additional -transactions. You should see several TPS measurements printed to the screen. In the -multinode variation, you'll see TPS measurements for each validator node as well. - -Public Testnet --------------- -In this example the client connects to our public testnet. To run validators on the testnet you would need to open udp ports `8000-10000`. - -```bash -$ ./multinode-demo/client.sh --network $(dig +short testnet.solana.com):8001 --identity config-private/client-id.json --duration 60 -``` - -You can observe the effects of your client's transactions on our [dashboard](https://metrics.solana.com:3000/d/testnet/testnet-hud?orgId=2&from=now-30m&to=now&refresh=5s&var-testnet=testnet) - - -Linux Snap ---- -A Linux [Snap](https://snapcraft.io/) is available, which can be used to easily -get Solana running on supported Linux systems without building anything from -source for evaluation. Note that CUDA is not supported by the Snap so -performance will be limited. - -The `edge` Snap channel is updated daily with the latest -development from the `master` branch. To install: - -```bash -$ sudo snap install solana --edge --devmode -``` - -Once installed the usual Solana programs will be available as `solona.*` instead -of `solana-*`. For example, `solana.fullnode` instead of `solana-fullnode`. - -Update to the latest version at any time with: - -```bash -$ snap info solana -$ sudo snap refresh solana --devmode -``` - -### Daemon support -The snap supports running a leader, validator or leader+drone node as a system -daemon. - -Run `sudo snap get solana` to view the current daemon configuration. To view -daemon logs: -1. Run `sudo snap logs -n=all solana` to view the daemon initialization log -2. Runtime logging can be found under `/var/snap/solana/current/leader/`, -`/var/snap/solana/current/validator/`, or `/var/snap/solana/current/drone/` depending -on which `mode=` was selected. Within each log directory the file `current` -contains the latest log, and the files `*.s` (if present) contain older rotated -logs. - -Disable the daemon at any time by running: - -```bash -$ sudo snap set solana mode= -``` - -Runtime configuration files for the daemon can be found in -`/var/snap/solana/current/config`. - -#### Leader daemon - -```bash -$ sudo snap set solana mode=leader -``` - -`rsync` must be configured and running on the leader. - -1. Ensure rsync is installed with `sudo apt-get -y install rsync` -2. Edit `/etc/rsyncd.conf` to include the following -``` -[config] -path = /var/snap/solana/current/config -hosts allow = * -read only = true -``` -3. Run `sudo systemctl enable rsync; sudo systemctl start rsync` -4. Test by running `rsync -Pzravv rsync:///config -solana-config` from another machine. **If the leader is running on a cloud -provider it may be necessary to configure the Firewall rules to permit ingress -to port tcp:873, tcp:9900 and the port range udp:8000-udp:10000** - - -To run both the Leader and Drone: - -```bash -$ sudo snap set solana mode=leader+drone - -``` - -#### Validator daemon - -```bash -$ sudo snap set solana mode=validator - -``` - -By default the validator will connect to **testnet.solana.com**, override -the leader IP address by running: - -```bash -$ sudo snap set solana mode=validator leader-address=127.0.0.1 #<-- change IP address -``` - -It's assumed that the leader will be running `rsync` configured as described in -the previous **Leader daemon** section. +Before you jump into the code, review the online book [Solana: Blockchain Rebuilt for Scale](https://solana-labs.github.io/solana/). Developing === -Architecture ---- - -Before you jump into the code, review the online book [Solana: Blockchain Rebuilt for Scale](https://solana-labs.github.io/solana/). - Building --- diff --git a/src/SUMMARY.md b/src/SUMMARY.md index a9b07b641f..346955820c 100644 --- a/src/SUMMARY.md +++ b/src/SUMMARY.md @@ -4,6 +4,8 @@ - [Terminology](terminology.md) +- [Getting Started](getting-started.md) + - [Programming Model](programs.md) - [A Solana Cluster](cluster.md) diff --git a/src/getting-started.md b/src/getting-started.md new file mode 100644 index 0000000000..8f99176573 --- /dev/null +++ b/src/getting-started.md @@ -0,0 +1,223 @@ +# Getting Started + +The Solana git repository contains all the scripts you might need to spin up your +own local testnet. Depending on what you're looking to achieve, you may want to +run a different variation, as the full-fledged, performance-enhanced +multinode testnet is considerably more complex to set up than a Rust-only, +singlenode testnode. If you are looking to develop high-level features, such +as experimenting with smart contracts, save yourself some setup headaches and +stick to the Rust-only singlenode demo. If you're doing performance optimization +of the transaction pipeline, consider the enhanced singlenode demo. If you're +doing consensus work, you'll need at least a Rust-only multinode demo. If you want +to reproduce our TPS metrics, run the enhanced multinode demo. + +For all four variations, you'd need the latest Rust toolchain and the Solana +source code: + +First, install Rust's package manager Cargo. + +```bash +$ curl https://sh.rustup.rs -sSf | sh +$ source $HOME/.cargo/env +``` + +Now checkout the code from github: + +```bash +$ git clone https://github.com/solana-labs/solana.git +$ cd solana +``` + +The demo code is sometimes broken between releases as we add new low-level +features, so if this is your first time running the demo, you'll improve +your odds of success if you check out the +[latest release](https://github.com/solana-labs/solana/releases) +before proceeding: + +```bash +$ TAG=$(git describe --tags $(git rev-list --tags --max-count=1)) +$ git checkout $TAG +``` + +### Configuration Setup + +The network is initialized with a genesis ledger and leader/validator configuration files. +These files can be generated by running the following script. + +```bash +$ ./multinode-demo/setup.sh +``` + +### Drone + +In order for the leader, client and validators to work, we'll need to +spin up a drone to give out some test tokens. The drone delivers Milton +Friedman-style "air drops" (free tokens to requesting clients) to be used in +test transactions. + +Start the drone on the leader node with: + +```bash +$ ./multinode-demo/drone.sh +``` + +### Singlenode Testnet + +Before you start a fullnode, make sure you know the IP address of the machine you +want to be the leader for the demo, and make sure that udp ports 8000-10000 are +open on all the machines you want to test with. + +Now start the server in a separate shell: + +```bash +$ ./multinode-demo/leader.sh +``` + +Wait a few seconds for the server to initialize. It will print "leader ready..." when it's ready to +receive transactions. The leader will request some tokens from the drone if it doesn't have any. +The drone does not need to be running for subsequent leader starts. + +### Multinode Testnet + +To run a multinode testnet, after starting a leader node, spin up some validator nodes in +separate shells: + +```bash +$ ./multinode-demo/validator.sh +``` + +To run a performance-enhanced leader or validator (on Linux), +[CUDA 10.0](https://developer.nvidia.com/cuda-downloads) must be installed on +your system: + +```bash +$ ./fetch-perf-libs.sh +$ SOLANA_CUDA=1 ./multinode-demo/leader.sh +$ SOLANA_CUDA=1 ./multinode-demo/validator.sh +``` + + +### Testnet Client Demo + +Now that your singlenode or multinode testnet is up and running let's send it +some transactions! + +In a separate shell start the client: + +```bash +$ ./multinode-demo/client.sh # runs against localhost by default +``` + +What just happened? The client demo spins up several threads to send 500,000 transactions +to the testnet as quickly as it can. The client then pings the testnet periodically to see +how many transactions it processed in that time. Take note that the demo intentionally +floods the network with UDP packets, such that the network will almost certainly drop a +bunch of them. This ensures the testnet has an opportunity to reach 710k TPS. The client +demo completes after it has convinced itself the testnet won't process any additional +transactions. You should see several TPS measurements printed to the screen. In the +multinode variation, you'll see TPS measurements for each validator node as well. + +## Public Testnet + +In this example the client connects to our public testnet. To run validators on the testnet you would need to open udp ports `8000-10000`. + +```bash +$ ./multinode-demo/client.sh --network $(dig +short testnet.solana.com):8001 --identity config-private/client-id.json --duration 60 +``` + +You can observe the effects of your client's transactions on our [dashboard](https://metrics.solana.com:3000/d/testnet/testnet-hud?orgId=2&from=now-30m&to=now&refresh=5s&var-testnet=testnet) + + +## Linux Snap + +A Linux [Snap](https://snapcraft.io/) is available, which can be used to easily +get Solana running on supported Linux systems without building anything from +source for evaluation. Note that CUDA is not supported by the Snap so +performance will be limited. + +The `edge` Snap channel is updated daily with the latest +development from the `master` branch. To install: + +```bash +$ sudo snap install solana --edge --devmode +``` + +Once installed the usual Solana programs will be available as `solona.*` instead +of `solana-*`. For example, `solana.fullnode` instead of `solana-fullnode`. + +Update to the latest version at any time with: + +```bash +$ snap info solana +$ sudo snap refresh solana --devmode +``` + +### Daemon support +The snap supports running a leader, validator or leader+drone node as a system +daemon. + +Run `sudo snap get solana` to view the current daemon configuration. To view +daemon logs: +1. Run `sudo snap logs -n=all solana` to view the daemon initialization log +2. Runtime logging can be found under `/var/snap/solana/current/leader/`, +`/var/snap/solana/current/validator/`, or `/var/snap/solana/current/drone/` depending +on which `mode=` was selected. Within each log directory the file `current` +contains the latest log, and the files `*.s` (if present) contain older rotated +logs. + +Disable the daemon at any time by running: + +```bash +$ sudo snap set solana mode= +``` + +Runtime configuration files for the daemon can be found in +`/var/snap/solana/current/config`. + +#### Leader daemon + +```bash +$ sudo snap set solana mode=leader +``` + +`rsync` must be configured and running on the leader. + +1. Ensure rsync is installed with `sudo apt-get -y install rsync` +2. Edit `/etc/rsyncd.conf` to include the following +``` +[config] +path = /var/snap/solana/current/config +hosts allow = * +read only = true +``` +3. Run `sudo systemctl enable rsync; sudo systemctl start rsync` +4. Test by running `rsync -Pzravv rsync:///config +solana-config` from another machine. **If the leader is running on a cloud +provider it may be necessary to configure the Firewall rules to permit ingress +to port tcp:873, tcp:9900 and the port range udp:8000-udp:10000** + + +To run both the Leader and Drone: + +```bash +$ sudo snap set solana mode=leader+drone + +``` + +#### Validator daemon + +```bash +$ sudo snap set solana mode=validator + +``` + +By default the validator will connect to **testnet.solana.com**, override +the leader IP address by running: + +```bash +$ sudo snap set solana mode=validator leader-address=127.0.0.1 #<-- change IP address +``` + +It's assumed that the leader will be running `rsync` configured as described in +the previous **Leader daemon** section. +