Commit Graph

265 Commits

Author SHA1 Message Date
Michael Vines 0f5acb86d3
wallet: Refuse to delegate stake to a vote account with a stale root slot (#5282)
* Refuse to delegate stake to a vote account with a stale root slot

* Remove sdk-c from the virtual manifest temporarily

For an unknown reason |cargo clippy| is getting stuck in CI
intermittently when trying to build this crate.
2019-08-01 21:08:24 -07:00
Michael Vines 870503ee36
Introduce delegate-stake.sh for adding stake to a validator.sh (#5380) 2019-08-01 13:48:00 -07:00
Michael Vines f859243191
Remove unused var 2019-07-31 10:51:30 -07:00
Michael Vines 9ddc25283c
Adapt validator sanity args 2019-07-31 10:46:25 -07:00
Michael Vines 388d4a8592
Remove obsolete --generate-snapshots argument 2019-07-31 10:26:22 -07:00
Michael Vines 9b8b7dbfd7
Avoid setting RUST_LOG to the empty string (#5338) 2019-07-31 10:13:30 -07:00
Michael Vines c209e14e40
validator.sh now supports an --entrypoint arg, mimicking the solana-validator CLI API (#5363) 2019-07-31 09:54:39 -07:00
Michael Vines 6df1f6450f
Drop rsync address 2019-07-31 09:24:49 -07:00
Michael Vines bd7e269280 Kill rsync (#5336)
automerge
2019-07-30 22:43:47 -07:00
Michael Vines 294d9288d2
Update remote-node.sh to use bootstrap-leader.sh (#5352) 2019-07-30 15:53:03 -07:00
Michael Vines 2214d2dbb5
Eject bootstrap-leader support from fullnode.sh (#5301) 2019-07-29 21:25:28 -07:00
Michael Vines 252257fe66
Rewrite multinode-demo/replicator.sh to avoid fullnode.sh (#5299) 2019-07-26 19:00:34 -07:00
Michael Vines 5f81a67298
Add --no-deploy option to allow restarting nodes without a software update (#5182) 2019-07-22 21:38:26 -07:00
Michael Vines 3a69459645
Surface validator pubkey in metrics (#5227) 2019-07-22 16:08:21 -07:00
Dan Albert c95cda51c9
Fix internal node lamport funding and staking allocation logic (#5192)
* Plumb node funding from genesis

* Cleanup naming convention

*  Fix balance vs stake yml file logic

* Lamps not Stakes
2019-07-19 12:51:38 -06:00
Michael Vines 06cd7c1020
Disable restart 2019-07-19 08:35:22 -07:00
Michael Vines b58558ea4e
net/: startnode/stopnode now works for blockstreamer/replicator nodes (#5146)
* startnode/stopnode now works for blockstreamer/replicator nodes

* Plumb --skip-ledger-verify through net/
2019-07-17 19:26:23 -07:00
Dan Albert 440d006ec1
Plumb --no-snapshot in from CI (#5077)
* Plumb --no-snapshot in from CI
2019-07-14 13:17:30 -06:00
Michael Vines 8781aebe06
Pass SOLANA_METRICS_CONFIG along to oom-monitor.sh (#5021) 2019-07-10 20:11:55 -07:00
Michael Vines 32b55e6703
Fund solana-install deployments from the mint keypair to avoid airdrops (#4997) 2019-07-09 16:45:28 -07:00
Michael Vines 0a949677f0 net/ plumbing to manage LetsEncrypt TLS certificates (#4985)
automerge
2019-07-09 15:45:46 -07:00
Michael Vines 27221e28f6
Use default pubkey for solana-install sanity check 2019-06-26 21:49:22 -07:00
Michael Vines 465cd45833 Various Snapshot generation improvements
* Only a single snapshot is maintained to avoid unbounded disk growth
* Snapshot is stored as a compressed tar archive for faster rsyncing
* Any validator node may now generate snapshots
* Updated testnet scripts to generate snapshots on the blockstreamer node
2019-06-18 20:11:09 -07:00
Pankaj Garg e43a634944
Calculate bench client lamports based on signature fee (#4713)
* use fee calculator to compute max fee

* review comments

* shellcheck
2019-06-18 14:44:53 -07:00
Pankaj Garg 7fe10ba060
Don't start drone if primordial accounts are created for nodes (#4704)
* disable wallet sanity if no airdrops
2019-06-17 18:15:22 -07:00
Pankaj Garg ee68b9800e
Wait for nodes to boot up before launching other nodes and client (#4682)
* Wait for nodes to bootup in testnet

* increase timeout (as with multiple clients it takes even longer)
2019-06-13 19:37:36 -07:00
Pankaj Garg e3f895d7d4
Create bench exchange accounts in genesis block (#4655)
* fix script

* review comments
2019-06-13 11:51:35 -07:00
Pankaj Garg 288a3bdcd9
Provision bench client accounts in genesis block (#4648)
* fixes to script

* shellcheck

* address review comments
2019-06-11 18:47:35 -07:00
Pankaj Garg 9259d342ac
Facility to provision primordial accounts for fullnodes in genesis block (#4631)
* updated usage

* shellcheck

* support replicators

* disable airdrops if primordial accounts are used

* review comments
2019-06-10 19:42:49 -07:00
Michael Vines 8676b5d40c
Use more -f 2019-06-07 22:18:55 -07:00
Michael Vines a9f73ea321 solana-keygen no longer blindly overwrites a keypair, or assumes "new" (#4599)
automerge
2019-06-07 17:54:54 -07:00
Michael Vines 8e46c44f3e
Deploy mac OS solana-install update package to testnets (#4595) 2019-06-07 12:59:58 -07:00
Michael Vines aa4a7b0c73
Disable |solana-install| check for edge/beta testnets (#4564)
The release tarball URL changes for these testnets, which causes the
normal |solana-install| check to fail and the testnet is unnecessarily
rebooted.
2019-06-05 15:31:29 -07:00
Michael Vines 9c47ce30a7
shift 2019-06-05 12:06:54 -07:00
Michael Vines d26fd27bf9
Avoid sudo in tune-system.sh unless requested by the user (#4556) 2019-06-05 09:10:23 -07:00
Michael Vines 5c98c1d306 Sanity check that runs on the blockstreamer node now checks that node instead of the bootstrap leader (#4551)
automerge
2019-06-04 22:46:48 -07:00
Michael Vines e3365529de
Enable transaction fees for multinode-demo/ and net/ (#4527)
* Collect fees at the end of a slot

* Enable transaction fees for multinode-demo/ and net/
2019-06-04 14:51:52 -07:00
Pankaj Garg 40a4ac15f1 Remove per transaction fee (#4521)
automerge
2019-06-03 13:00:08 -07:00
Michael Vines e15246746d Enable non-zero fees for all testnets (#4513)
automerge
2019-05-31 22:33:55 -07:00
Michael Vines ec5cca41bc
Separate bootstrap leader's stake lamports from its identity lamports (#4510)
* Revert "Prevent run.sh from running beyond the first epoch under normal use (#4498)"

This reverts commit d343c409e6.

* Separate bootstrap leader's stake lamports from its identity lamports
2019-05-31 19:58:52 -07:00
Michael Vines ce04d2bfc2
Add replicator support to net/ (#4494) 2019-05-31 15:27:31 -07:00
Michael Vines 1eceb4831d
Use nohup and sleep a little to improve stability when launching a node 2019-05-27 13:57:40 -07:00
Michael Vines 471465a5f4
net/: Add solana-install test to sanity (#4438)
* Add instance creation date to motd

* Setup localtime

* Add solana-install test
2019-05-26 11:17:07 -07:00
Michael Vines fb2eac20bb
Rename solana-fullnode to solana-validator (#4411) 2019-05-23 15:06:01 -07:00
Michael Vines 392a39dd54
Poh subsystem cleanup, genesis plumbing, enable real PoH on edge testnet (#4292)
* Remove unused PohServiceConfig::Step

* Clarify variable name

* Poh::hash() now takes an iteration counter

* man -> max

* Inline functions with single call site

* Move PohServiceConfig into GenesisBlock

* Add plumbing to enable real PoH on testnets

* Batch hashes to improve PoH hash rate

* Ensure a constant hashes_per_tick

* Remove PohEntry mixin field

* Poh/PohEntry no longer maintains tick_height

* Ensure a constant hashes_per_tick

* ci/localnet-sanity.sh: Use real PoH

* Rework Poh/PohService to keep PohRecorder unlocked as much as possible while hashing
2019-05-18 14:01:36 -07:00
Michael Vines 915956b94b Remove disable leader rotation option (#4299)
automerge
2019-05-15 15:16:45 -07:00
Michael Vines 1393d26f63
Remove obsolete internal multinode-demo/ logging (#4265) 2019-05-13 10:51:18 -07:00
Michael Vines f2b727b534
Update mint keypair filename 2019-05-09 07:27:13 -07:00
Michael Vines 7609a007c6
Add FeeCalculator to the genesis block (#4196) 2019-05-07 20:28:41 -07:00
Michael Vines 9b50583641
wallet: add --url, remove --host/--rpc-host/--rpc-port/-rpc-tls (#4153)
Also by default the wallet now talks to testnet.solana.com instead of
localhost
2019-05-06 07:38:26 -07:00
Michael Vines 0139e5db21
Correct blockstreamer node args 2019-05-04 08:22:36 -07:00
Michael Vines 586fb15c2c
fullnode positional arguments may now be mixed with optional arguments (#4151) 2019-05-03 20:49:24 -07:00
Michael Vines f3f416b7ba
Rename --network argument to --entrypoint (#4149) 2019-05-03 15:00:19 -07:00
Michael Vines 7fe3c75c6b
Add a node-specific ip echo service to remove dependency on ifconfig.co (#4137) 2019-05-03 11:01:35 -07:00
Michael Vines f02ec31c68
Flip if/else 2019-04-30 08:56:53 -07:00
Michael Vines d21fa4a177
v0.14: various net/ fixes for large clusters (#4080)
* net.sh: Add -F to discard validator nodes that didn't bootup successfully

* Relax sanity node count when validator bootup failure is permitted

* Less sanity for testnet-demo

* net.sh: Add -F to discard validator nodes that didn't bootup successfully
2019-04-29 21:38:32 -07:00
Michael Vines 0cc3956693
testnet-demo now runs across more GCE zones (#4053)
* testnet-demo now runs across more GCE zones

* Save zone info to config file

* Add geoip whitelist for common data centers

* Skip more of start

* Include -x for config

* Fetch private key from first validator node if necessary

* Correct -r propagation
2019-04-28 19:50:52 -07:00
Michael Vines b67b0bff05
Do not enable rpc exit on the blockstreamer node (#3977) 2019-04-25 09:30:59 -07:00
Pankaj Garg 3ba93aa8fe Facility to pass extra arguments to bench clients from net.sh command (#3952)
automerge
2019-04-23 15:13:29 -07:00
Michael Vines 05bcb7f292
Add stop node command to solana-gossip (#3928) 2019-04-22 14:51:20 -07:00
sakridge 95a16426f3
Cleanup bench-exchange (#3919)
* bench-exchange changes

Generate new keypair for each bench-exchange
Add metrics
Tweak network sleep parameters for better reliability.

* Bench-exchange contract metrics
2019-04-22 13:16:28 -07:00
Jack May 1a9ac62f60
Add bench-exchange support to the net framework (#3893) 2019-04-19 09:56:01 -07:00
Michael Vines 78d5ace754
Refactor multinode-demo/ scripts to avoid shipping fullnode-x.sh (#3835) 2019-04-17 18:03:58 -07:00
Michael Vines 63d66ece57
net/ testnet nodes now stake more lamports (#3812)
* Add --bootstrap-leader-lamports

* Generalize --no-stake into --stake NUM

* Use a large stake for net/ fullnodes

* Setup vote account before starting fullnode to avoid mixed log output
2019-04-16 13:03:01 -07:00
Michael Vines c67cc694ae
Remove stake from `./net sanity` ephemeral validator (#3797) 2019-04-15 21:42:25 -07:00
Michael Vines 2f1b0bf4f5 Add solana-install deployments to the testnets 2019-04-11 10:03:35 -07:00
Michael Vines 6ac919c71a Set warn log level only for perf testnets 2019-04-09 11:09:16 -07:00
Tyera Eulberg acbc261891 Add gossip to build script, and fix bash strings 2019-04-04 00:18:48 -07:00
Tyera Eulberg 92c66a411b Remove bench-tps converge-only 2019-04-01 23:05:25 -06:00
Michael Vines 0a5b6154e8 Use same gossip port for all testnet nodes 2019-03-22 00:16:58 -07:00
Michael Vines e6b9babf53 Run a drone on blockstreamer nodes 2019-03-21 22:09:44 -07:00
Michael Vines 3b5a9f512c Get client-id.json out of the genesis ledger directory 2019-03-21 16:35:40 -07:00
Michael Vines 242bcf44db
Replace stale --no-signer usage with --no-voting 2019-03-13 13:50:30 -07:00
Michael Vines ebd540972d
Remove duplicate --rpc-drone-address 2019-03-13 13:24:02 -07:00
Michael Vines fedbae6f8c
Enable rpc for all testnet nodes 2019-03-13 10:49:40 -07:00
Michael Vines 5d27f221f7 Drop socat for iptables 2019-03-13 12:03:56 -05:00
Michael Vines 61db74d98e
Run socat in the background 2019-03-13 08:15:58 -07:00
Sagar Dhawan 866d3f467f
Fix flag to disable leader-rotation (#3243) 2019-03-12 16:35:13 -07:00
Rob Walker a799f8f4b1
tell blockexplorer to run on port 8080 (#3237)
* tell blockexplorer to run on port 8080

* forward port 80 to 5000 for a blockexplorer node
2019-03-12 13:39:09 -07:00
Michael Vines 7b49c9f09c Delete fullnode-config/ 2019-03-04 15:49:02 -08:00
Michael Vines cd4bccfd12 Remove snap support 2019-03-02 17:41:09 -08:00
Tyera Eulberg 18d48f09f8 Plumb blockstreamer name through testnet scripts 2019-02-21 17:24:29 -07:00
Tyera Eulberg d8f6865338 Rename EntryStream to Blockstream 2019-02-21 17:24:29 -07:00
Michael Vines 3f14466965
Limit blockexplorer versions to 1.x.y
Per semver semantics when blockexplorer 2.0.0 is released it will be
incompatible in some way with 1.x.y and thus should be opt in.
2019-02-18 16:48:33 -08:00
Michael Vines 2793404116
Ensure blockexplorer comes back up when nodes are updated instead of restarted 2019-02-17 20:07:12 -08:00
Michael Vines 858068cdc0
Drop sudo, it's now handled internally by the block explorer 2019-02-17 12:29:53 -08:00
Michael Vines 65fb307d0f
Avoid '' argument to fullnode.sh 2019-02-17 11:43:41 -08:00
Michael Vines 4ff4fb6c38 Add support for an API node that hosts the block explorer 2019-02-17 10:39:27 -08:00
Michael Vines bae7612f36
Revert "Wait until the node successfully boots"
This reverts commit e84f1f6de7.
2019-01-23 11:27:08 -08:00
Michael Vines e84f1f6de7 Wait until the node successfully boots 2019-01-23 00:02:30 -08:00
Michael Vines a84b6bc7e4 Overhaul wallet rpc/drone command-line arguments 2019-01-17 08:36:05 -08:00
Pankaj Garg 00c4c30d72
Fix testnet bootup issue (#2465)
* Fix testnet bootup issue

* address review comments
2019-01-16 19:18:32 -08:00
Michael Vines be0eefb0af
Add timeout to prevent stuck bench-tps when a cluster goes bad 2019-01-09 19:21:53 -07:00
Michael Vines 28431ff22c Add configurable RUST_LOG for ./net.sh sanity 2019-01-09 12:12:50 -08:00
Michael Vines 639bed2f6d Reorder sanity.
1. Check for presence of nodes
2. Check for functioning RPC API
3. Then try the wallet
2019-01-09 12:05:30 -08:00
Michael Vines eb37aa2bba Kill monitoring scripts by process group to ensure a full shutdown 2019-01-09 11:59:01 -08:00
Michael Vines 048fe371aa set -x for more detailed logs 2019-01-09 11:59:01 -08:00
Michael Vines 1b7598e351 Add retries to RPC API probe 2019-01-08 08:50:51 -08:00
Michael Vines 1531a1777a Add RPC API check 2018-12-24 22:51:36 -08:00
Michael Vines 04d46ea33f Run oom-monitor as root 2018-12-24 22:51:36 -08:00
Sathish 154e20484d
Use hostname in database if env is set (#2101) 2018-12-10 22:59:38 -08:00
Michael Vines 094f0a8be3 Leader rotation flag plumbing 2018-12-10 14:07:59 -08:00
Michael Vines b2ddac610c Add option to skip setup during cluster start 2018-12-10 07:47:15 -08:00
Michael Vines 872a3317b5 Fully switch to bootstrap-leader for command-line args 2018-12-07 16:57:02 -08:00
Michael Vines 1db6a882bb rsync of genesis ledger now works for non-snap deployments 2018-12-07 16:57:02 -08:00
Michael Vines af11562627 Correct ledger path 2018-12-07 11:32:08 -08:00
Michael Vines 286f08f095 Drop old validator name, use fullnode instead 2018-12-07 11:32:08 -08:00
Michael Vines 70c149c7da Rename leader/validator to bootstrap-leader/fullnode
Only rsyncing the genesis ledger snuck in here as well
2018-12-06 19:44:47 -08:00
Michael Vines c4b8f0cd2f bench-tps will now generate an ephemeral identity if not provided with one
Also simplify scripts as a result
2018-12-06 16:30:48 -08:00
Michael Vines f0fe089013
Adapt testnet-deploy metric datapoint names to {,bootnode-}fullnode 2018-12-06 08:04:33 -08:00
Michael Vines a6312ba98f Switch snap to bootstrap-fullnode/fullnode naming 2018-12-05 18:59:43 -08:00
Michael Vines 04a0652614 Generalize net/ from leader/validator to bootstrap-fullnode/fullnode 2018-12-05 17:11:16 -08:00
Michael Vines 5d80edd969 Properly check for failure (can't rely on `set -e` here) 2018-12-05 13:26:06 -08:00
carllin cf95708c18 Set drone address to always be the initial network entry point (#1847)
* Set drone address to always be the initial network entry point, so that even when leaders rotate the client can still find the drone

* Extract drone address as a separate argument to bench-tps

* Add drone port to client.sh instead of setting it in bench-tps

* Add drone entrypoint to scripts

* Fix build error
2018-11-16 19:56:26 -08:00
Sathish c973de1d76
Decouple log and metrics rate (#1839)
Use separate env for log and metrics rate.

Set default log level to WARN if unset.
2018-11-15 22:27:16 -08:00
Michael Vines 83fc3c10cf Setup CUDA env for local builds 2018-11-15 08:00:52 -08:00
Michael Vines 017c281eaf Remove CUDA support from Snap 2018-11-12 20:31:16 -08:00
Michael Vines 9e7b9487b0 perf-libs now drives setting CUDA_HOME 2018-11-12 18:49:15 -08:00
Michael Vines def7d156f6 codemod --extensions sh '#!/usr/bin/env bash -e' '#!/usr/bin/env bash\nset -e' 2018-11-11 16:24:36 -08:00
Michael Vines 33aab094ef codemod --extensions sh '#!/bin/bash' '#!/usr/bin/env bash' 2018-11-11 16:24:36 -08:00
Michael Vines cf6f344ccc Add CUDA_HOME env var to permit overriding the CUDA install location 2018-11-11 16:24:18 -08:00
Michael Vines b02b636b36
Support local tarball deploys 2018-11-07 14:44:40 -08:00
Michael Vines a537154c28
Remove all cuda dependencies from release tarball beyond solana-fullnode-cuda 2018-11-07 14:44:40 -08:00
Michael Vines dd4fb7aa90 Add AWS-based nets 2018-11-07 07:47:39 -08:00
Michael Vines 7b58bd621a
Remove node check from client start-up
If the network loses a validator or two, it's the job of the sanity
check to detect this not the bench clients
2018-11-06 13:57:06 -08:00
Pankaj Garg 85869552e0
Update testnet scripts to use release tar ball (#1660)
* Update testnet scripts to use release tar ball

* use curl instead of s3cmd
2018-10-30 18:05:38 -07:00
Pankaj Garg 3cc78d3a41
Added a new remote node configuration script to set rmem/wmem (#1647)
* Added a new remote node configuration script to set rmem/wmem

* Update common.sh for rmem/wmem configuration
2018-10-30 09:17:35 -07:00
Pankaj Garg fbde9bb731
Run bench-tps for longer duration in testnet (#1638)
- Increased to 2+ hours
2018-10-29 15:03:08 -07:00
Pankaj Garg 7abd456d45
Increase rmem and wmem for remote nodes in testnet (#1635) 2018-10-29 13:04:54 -07:00
Pankaj Garg 32fc0cd7e9
Fix bug introduced during RUST_LOG escaping (#1507)
* Fix bug introduced during RUST_LOG escaping
- remote node configuration should not be quoted

* shellcheck disable SC2090
2018-10-15 16:49:22 -07:00
Pankaj Garg 9fc30f6db4
Escape RUST_LOG configuration in remote-node.sh (#1489)
* Escape RUST_LOG configuration in remote-node.sh

- If it was set to #, it was causing other parameters to be commented out

* escape other variables as well

* disabled shell check

* Fix shellcheck error
2018-10-13 13:35:54 -07:00
Tyera Eulberg f273351789 Add missing port number 2018-09-18 09:36:54 -06:00
Tyera Eulberg 0125163190 Remove wallet.sh, update entrypoint syntax for wallet network argument 2018-09-17 11:53:33 -06:00
Michael Vines f06113500d bench-tps/net sanity: add ability to check for unexpected extra nodes 2018-09-12 15:38:57 -07:00
Pankaj Garg 1c17c6dd2b
Report UDP network statistics (#1176)
* Report UDP network statistics

Fixes #1093

* Address review comments

* Address additional review comments

* Fix shellcheck errors
2018-09-10 15:52:08 -07:00
Michael Vines ebcac3c2d1 Use a common solana user on all testnet instances 2018-09-08 22:34:26 -07:00
Michael Vines 5afcdcbbe6
More log grooming 2018-09-08 14:16:34 -07:00
Michael Vines 3840b4b516
Groom log output 2018-09-08 14:10:18 -07:00
Michael Vines 1d6c4aacae Retry rsync a couple times before failing 2018-09-08 13:59:45 -07:00
Michael Vines 9f5c86e60c Install earlyoom at gce instance startup 2018-09-08 13:59:45 -07:00
Michael Vines 9f413fd656 Establish net/scripts/... for better scoping 2018-09-08 13:59:45 -07:00
Michael Vines c3af0d9d25 Improve client.log 2018-09-07 21:20:00 -07:00
Michael Vines 932c994dc9 Use new bench-tps command-line args 2018-09-07 21:20:00 -07:00
Michael Vines ddd1871840 Install libssl1.1 for solanalabs/rust docker image compat 2018-09-07 19:57:41 -07:00
Michael Vines 73a8441add /var/snap is not writable by most users 2018-09-07 17:41:20 -07:00
Rob Walker 51b27779c9
client changes for TODOs and looping (#1138)
* remove client.sh from snap
* default to ephemeral instead of ~/.config key
* rework CLI for bench-tps
* remote multinode-demo stuff from remote-client.sh
* remove multinode-demo from remote-sanity and localnet-sanity
2018-09-08 07:07:10 +09:00
Michael Vines 0d945e6a92 Groom testnet-sanity logging 2018-09-07 12:45:48 -07:00
Michael Vines ee682d5bc3 Move wallet-sanity.sh out of multinode-demo/ 2018-09-07 12:01:43 -07:00
Michael Vines ab208ddb77 Clean up arg handling 2018-09-07 12:01:43 -07:00
Michael Vines 09a48d773a Run bench-tps in a tmux 2018-09-07 12:01:43 -07:00