blockworks-foundation
/
solana
mirror of https://github.com/blockworks-foundation/solana.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
solana/docs/src/running-validator/validator-start.md

15 KiB
Raw Blame History

title
Starting a Validator

Configure Solana CLI

The solana cli includes get and set configuration commands to automatically set the --url argument for cli commands. For example:

solana config set --url http://devnet.solana.com

While this section demonstrates how to connect to the Devnet cluster, the steps are similar for the other Solana Clusters.

Confirm The Cluster Is Reachable

Before attaching a validator node, sanity check that the cluster is accessible to your machine by fetching the transaction count:

solana transaction-count

View the metrics dashboard for more detail on cluster activity.

Confirm your Installation

Try running following command to join the gossip network and view all the other nodes in the cluster:

solana-gossip spy --entrypoint entrypoint.devnet.solana.com:8001
# Press ^C to exit

Enabling CUDA

If your machine has a GPU with CUDA installed Linux-only currently, include the --cuda argument to solana-validator.

When your validator is started look for the following log message to indicate that CUDA is enabled: "[<timestamp> solana::validator] CUDA is enabled"

System Tuning

Linux

Automatic

The solana repo includes a daemon to adjust system settings to optimize performance (namely by increasing the OS UDP buffer and file mapping limits).

The daemon (solana-sys-tuner) is included in the solana binary release. Restart it, before restarting your validator, after each software upgrade to ensure that the latest recommended settings are applied.

To run it:

sudo solana-sys-tuner --user $(whoami) > sys-tuner.log 2>&1 &

Manual

If you would prefer to manage system settings on your own, you may do so with the following commands.

Increase UDP buffers
sudo bash -c "cat >/etc/sysctl.d/20-solana-udp-buffers.conf <<EOF
# Increase UDP buffer size
net.core.rmem_default = 134217728
net.core.rmem_max = 134217728
net.core.wmem_default = 134217728
net.core.wmem_max = 134217728
EOF"

sudo sysctl -p /etc/sysctl.d/20-solana-udp-buffers.conf
Increased memory mapped files limit
sudo bash -c "cat >/etc/sysctl.d/20-solana-mmaps.conf <<EOF
# Increase memory mapped files limit
vm.max_map_count = 700000
EOF"

sudo sysctl -p /etc/sysctl.d/20-solana-mmaps.conf

Add

LimitNOFILE=700000

to the [Service] section of your systemd service file, if you use one, otherwise add

DefaultLimitNOFILE=700000

to the [Manager] section of /etc/systemd/system.conf.

sudo systemctl daemon-reload

sudo bash -c "cat >/etc/security/limits.d/90-solana-nofiles.conf <<EOF
# Increase process file descriptor count limit
* - nofile 700000
EOF"

### Close all open sessions (log out then, in again) ###

Generate identity

Create an identity keypair for your validator by running:

solana-keygen new -o ~/validator-keypair.json

The identity public key can now be viewed by running:

solana-keygen pubkey ~/validator-keypair.json

Note: The "validator-keypair.json” file is also your ed25519 private key.

Paper Wallet identity

You can create a paper wallet for your identity file instead of writing the keypair file to disk with:

solana-keygen new --no-outfile

The corresponding identity public key can now be viewed by running:

solana-keygen pubkey ASK

and then entering your seed phrase.

See Paper Wallet Usage for more info.

Vanity Keypair

You can generate a custom vanity keypair using solana-keygen. For instance:

solana-keygen grind --starts-with e1v1s:1

You may request that the generated vanity keypair be expressed as a seed phrase which allows recovery of the keypair from the seed phrase and an optionally supplied passphrase (note that this is significantly slower than grinding without a mnemonic):

solana-keygen grind --use-mnemonic --starts-with e1v1s:1

Depending on the string requested, it may take days to find a match...

Your validator identity keypair uniquely identifies your validator within the network. It is crucial to back-up this information.

If you don’t back up this information, you WILL NOT BE ABLE TO RECOVER YOUR VALIDATOR if you lose access to it. If this happens, YOU WILL LOSE YOUR ALLOCATION OF SOL TOO.

To back-up your validator identify keypair, back-up your "validator-keypair.json” file or your seed phrase to a secure location.

More Solana CLI Configuration

Now that you have a keypair, set the solana configuration to use your validator keypair for all following commands:

solana config set --keypair ~/validator-keypair.json

You should see the following output:

Wallet Config Updated: /home/solana/.config/solana/wallet/config.yml
* url: http://devnet.solana.com
* keypair: /home/solana/validator-keypair.json

Airdrop & Check Validator Balance

Airdrop yourself some SOL to get started:

solana airdrop 1

Note that airdrops are only available on Devnet and Testnet. Both are limited to 1 SOL per request.

To view your current balance:

solana balance

Or to see in finer detail:

solana balance --lamports

Read more about the difference between SOL and lamports here.

Create Vote Account

If you haven’t already done so, create a vote-account keypair and create the vote account on the network. If you have completed this step, you should see the “vote-account-keypair.json” in your Solana runtime directory:

solana-keygen new -o ~/vote-account-keypair.json

The following command can be used to create your vote account on the blockchain with all the default options:

solana create-vote-account ~/vote-account-keypair.json ~/validator-keypair.json

Read more about creating and managing a vote account.

Trusted validators

If you know and trust other validator nodes, you can specify this on the command line with the --trusted-validator <PUBKEY> argument to solana-validator. You can specify multiple ones by repeating the argument --trusted-validator <PUBKEY1> --trusted-validator <PUBKEY2>. This has two effects, one is when the validator is booting with --no-untrusted-rpc, it will only ask that set of trusted nodes for downloading genesis and snapshot data. Another is that in combination with the --halt-on-trusted-validator-hash-mismatch option, it will monitor the merkle root hash of the entire accounts state of other trusted nodes on gossip and if the hashes produce any mismatch, the validator will halt the node to prevent the validator from voting or processing potentially incorrect state values. At the moment, the slot that the validator publishes the hash on is tied to the snapshot interval. For the feature to be effective, all validators in the trusted set should be set to the same snapshot interval value or multiples of the same.

It is highly recommended you use these options to prevent malicious snapshot state download or account state divergence.

Connect Your Validator

Connect to the cluster by running:

solana-validator \
  --identity ~/validator-keypair.json \
  --vote-account ~/vote-account-keypair.json \
  --rpc-port 8899 \
  --entrypoint entrypoint.devnet.solana.com:8001 \
  --limit-ledger-size \
  --log ~/solana-validator.log

To force validator logging to the console add a --log - argument, otherwise the validator will automatically log to a file.

The ledger will be placed in the ledger/ directory by default, use the --ledger argument to specify a different location.

Note: You can use a paper wallet seed phrase for your --identity and/or --authorized-voter keypairs. To use these, pass the respective argument as solana-validator --identity ASK ... --authorized-voter ASK ... and you will be prompted to enter your seed phrases and optional passphrase.

Confirm your validator connected to the network by opening a new terminal and running:

solana-gossip spy --entrypoint entrypoint.devnet.solana.com:8001

If your validator is connected, its public key and IP address will appear in the list.

Controlling local network port allocation

By default the validator will dynamically select available network ports in the 8000-10000 range, and may be overridden with --dynamic-port-range. For example, solana-validator --dynamic-port-range 11000-11010 ... will restrict the validator to ports 11000-11010.

Limiting ledger size to conserve disk space

The --limit-ledger-size parameter allows you to specify how many ledger shreds your node retains on disk. If you do not include this parameter, the validator will keep the entire ledger until it runs out of disk space.

The default value attempts to keep the ledger disk usage under 500GB. More or less disk usage may be requested by adding an argument to --limit-ledger-size if desired. Check solana-validator --help for the default limit value used by --limit-ledger-size. More information about selecting a custom limit value is available here.

Systemd Unit

Running the validator as a systemd unit is one easy way to manage running in the background.

Assuming you have a user called sol on your machine, create the file /etc/systemd/system/sol.service with the following:

[Unit]
Description=Solana Validator
After=network.target
Wants=solana-sys-tuner.service
StartLimitIntervalSec=0

[Service]
Type=simple
Restart=always
RestartSec=1
User=sol
LimitNOFILE=700000
LogRateLimitIntervalSec=0
Environment="PATH=/bin:/usr/bin:/home/sol/.local/share/solana/install/active_release/bin"
ExecStart=/home/sol/bin/validator.sh

[Install]
WantedBy=multi-user.target

Now create /home/sol/bin/validator.sh to include the desired solana-validator command-line. Ensure that running /home/sol/bin/validator.sh manually starts the validator as expected. Don't forget to mark it executable with chmod +x /home/sol/bin/validator.sh

Start the service with:

$ sudo systemctl enable --now sol

Logging

Log output tuning

The messages that a validator emits to the log can be controlled by the RUST_LOG environment variable. Details can by found in the documentation for the env_logger Rust crate.

Note that if logging output is reduced, this may make it difficult to debug issues encountered later. Should support be sought from the team, any changes will need to be reverted and the issue reproduced before help can be provided.

Log rotation

The validator log file, as specified by --log ~/solana-validator.log, can get very large over time and it's recommended that log rotation be configured.

The validator will re-open its when it receives the USR1 signal, which is the basic primitive that enables log rotation.

If the validator is being started by a wrapper shell script, it is important to launch the process with exec (exec solana-validator ...) when using logrotate. This will prevent the USR1 signal from being sent to the script's process instead of the validator's, which will kill them both.

Using logrotate

An example setup for the logrotate, which assumes that the validator is running as a systemd service called sol.service and writes a log file at /home/sol/solana-validator.log:

# Setup log rotation

cat > logrotate.sol <<EOF
/home/sol/solana-validator.log {
  rotate 7
  daily
  missingok
  postrotate
    systemctl kill -s USR1 sol.service
  endscript
}
EOF
sudo cp logrotate.sol /etc/logrotate.d/sol
systemctl restart logrotate.service

Disable port checks to speed up restarts

Once your validator is operating normally, you can reduce the time it takes to restart your validator by adding the --no-port-check flag to your solana-validator command-line.

Disable snapshot compression to reduce CPU usage

If you are not serving snapshots to other validators, snapshot compression can be disabled to reduce CPU load at the expense of slightly more disk usage for local snapshot storage.

Add the --snapshot-compression none argument to your solana-validator command-line arguments and restart the validator.

Using a ramdisk with spill-over into swap for the accounts database to reduce SSD wear

If your machine has plenty of RAM, a tmpfs ramdisk (tmpfs) may be used to hold the accounts database

When using tmpfs it's essential to also configure swap on your machine as well to avoid running out of tmpfs space periodically.

A 300GB tmpfs partition is recommended, with an accompanying 250GB swap partition.

Example configuration:

  1. sudo mkdir /mnt/solana-accounts
  2. Add a 300GB tmpfs parition by adding a new line containing tmpfs /mnt/solana-accounts tmpfs rw,size=300G,user=sol 0 0 to /etc/fstab (assuming your validator is running under the user "sol"). CAREFUL: If you incorrectly edit /etc/fstab your machine may no longer boot
  3. Create at least 250GB of swap space
  • Choose a device to use in place of SWAPDEV for the remainder of these instructions. Ideally select a free disk partition of 250GB or greater on a fast disk. If one is not available, create a swap file with sudo dd if=/dev/zero of=/swapfile bs=1MiB count=250KiB, set its permissions with sudo chmod 0600 /swapfile and use /swapfile as SWAPDEV for the remainder of these instructions
  • Format the device for usage as swap with sudo mkswap SWAPDEV
  1. Add the swap file to /etc/fstab with a new line containing SWAPDEV swap swap defaults 0 0
  2. Enable swap with sudo swapon -a and mount the tmpfs with sudo mount /mnt/solana-accounts/
  3. Confirm swap is active with free -g and the tmpfs is mounted with mount

Now add the --accounts /mnt/solana-accounts argument to your solana-validator command-line arguments and restart the validator.

Account indexing

As the number of populated accounts on the cluster grows, account-data RPC requests that scan the entire account set -- like getProgramAccounts and SPL-token-specific requests -- may perform poorly. If your validator needs to support any of these requests, you can use the --account-index parameter to activate one or more in-memory account indexes that significantly improve RPC performance by indexing accounts by the key field. Currently supports the following parameter values: