Commit Graph

176 Commits

Author SHA1 Message Date
Trent Nelson 6bb22902cc net: net.sh - Enable deploying testnets on debug binaries (#5627)
automerge
2019-08-23 18:31:18 -07:00
Michael Vines 9f6c9c428b
Move genesis/snapshot archive download into Rust (#5515) 2019-08-14 19:25:22 -07:00
Michael Vines cc0e455a51
Skip sanity on blockstreamer node at cluster boot.
It may not have caught up to the bootstrap leader yet...
2019-08-05 17:11:28 -07:00
Michael Vines 5f81a67298
Add --no-deploy option to allow restarting nodes without a software update (#5182) 2019-07-22 21:38:26 -07:00
Dan Albert c95cda51c9
Fix internal node lamport funding and staking allocation logic (#5192)
* Plumb node funding from genesis

* Cleanup naming convention

*  Fix balance vs stake yml file logic

* Lamps not Stakes
2019-07-19 12:51:38 -06:00
sakridge a12e7a2e33
Separate build and deploy steps in net/net.sh (#5184)
So one can keep the network up while a new experiment is building
2019-07-18 18:59:47 -07:00
Michael Vines b58558ea4e
net/: startnode/stopnode now works for blockstreamer/replicator nodes (#5146)
* startnode/stopnode now works for blockstreamer/replicator nodes

* Plumb --skip-ledger-verify through net/
2019-07-17 19:26:23 -07:00
sakridge 5825b967d2
Check for valid pid before kill in node stop script (#5126) 2019-07-16 14:31:15 -07:00
Dan Albert 440d006ec1
Plumb --no-snapshot in from CI (#5077)
* Plumb --no-snapshot in from CI
2019-07-14 13:17:30 -06:00
Michael Vines 0a949677f0 net/ plumbing to manage LetsEncrypt TLS certificates (#4985)
automerge
2019-07-09 15:45:46 -07:00
Dan Albert d111223085
Fix always passing in remote filename, even if no accounts file (#4993)
* Fix always passing in remote filename, even if no accounts file

* typo
2019-07-09 16:07:31 -06:00
Dan Albert 1ca7e9f67b
Add testnet-tds support to testnet manager (#4762)
* Add testnet-tds support to testnet scripts
2019-07-09 14:39:55 -06:00
Michael Vines 8620d0a3b2
Add curl retries 2019-07-02 08:37:18 -07:00
Pankaj Garg e43a634944
Calculate bench client lamports based on signature fee (#4713)
* use fee calculator to compute max fee

* review comments

* shellcheck
2019-06-18 14:44:53 -07:00
Sathish Ambley 44967abd1c update storage len 2019-06-17 22:48:27 -07:00
Pankaj Garg e3f895d7d4
Create bench exchange accounts in genesis block (#4655)
* fix script

* review comments
2019-06-13 11:51:35 -07:00
Pankaj Garg 288a3bdcd9
Provision bench client accounts in genesis block (#4648)
* fixes to script

* shellcheck

* address review comments
2019-06-11 18:47:35 -07:00
Pankaj Garg 9259d342ac
Facility to provision primordial accounts for fullnodes in genesis block (#4631)
* updated usage

* shellcheck

* support replicators

* disable airdrops if primordial accounts are used

* review comments
2019-06-10 19:42:49 -07:00
Michael Vines 8e46c44f3e
Deploy mac OS solana-install update package to testnets (#4595) 2019-06-07 12:59:58 -07:00
Michael Vines aa4a7b0c73
Disable |solana-install| check for edge/beta testnets (#4564)
The release tarball URL changes for these testnets, which causes the
normal |solana-install| check to fail and the testnet is unnecessarily
rebooted.
2019-06-05 15:31:29 -07:00
Michael Vines 5c98c1d306 Sanity check that runs on the blockstreamer node now checks that node instead of the bootstrap leader (#4551)
automerge
2019-06-04 22:46:48 -07:00
Michael Vines ce04d2bfc2
Add replicator support to net/ (#4494) 2019-05-31 15:27:31 -07:00
sakridge 8611b40074
Add argument to net/net to specify number of nodes (#4429)
Allows for testing different node counts without recreating the network.
2019-05-24 16:20:14 -07:00
Michael Vines 94beb4b8c2
More fullnode -> validator renaming (#4414)
* s/fullnode_config/validator_config/g

* s/FullnodeConfig/ValidatorConfig/g

* mv core/lib/fullnode.rs core/lib/validator.rs

* s/Fullnode/Validator/g

* Add replicator-x.sh

* Rename fullnode.md to validator.md

* cargo fmt
2019-05-23 22:05:16 -07:00
sakridge 90e73515ed Remove ls which is innacurate since we never clean up the logs (#4392)
automerge
2019-05-22 13:11:20 -07:00
Michael Vines cbebc7a80c
Expand shortArgs correctly 2019-05-18 19:00:48 -07:00
Michael Vines 392a39dd54
Poh subsystem cleanup, genesis plumbing, enable real PoH on edge testnet (#4292)
* Remove unused PohServiceConfig::Step

* Clarify variable name

* Poh::hash() now takes an iteration counter

* man -> max

* Inline functions with single call site

* Move PohServiceConfig into GenesisBlock

* Add plumbing to enable real PoH on testnets

* Batch hashes to improve PoH hash rate

* Ensure a constant hashes_per_tick

* Remove PohEntry mixin field

* Poh/PohEntry no longer maintains tick_height

* Ensure a constant hashes_per_tick

* ci/localnet-sanity.sh: Use real PoH

* Rework Poh/PohService to keep PohRecorder unlocked as much as possible while hashing
2019-05-18 14:01:36 -07:00
Michael Vines 915956b94b Remove disable leader rotation option (#4299)
automerge
2019-05-15 15:16:45 -07:00
Michael Vines 7fe3c75c6b
Add a node-specific ip echo service to remove dependency on ifconfig.co (#4137) 2019-05-03 11:01:35 -07:00
Pankaj Garg 598f765960 Fix net.sh for running on macos (#4107)
automerge
2019-05-01 16:03:35 -07:00
Michael Vines c5f8b4960c
Stop nodes in parallel 2019-04-30 10:42:59 -07:00
Michael Vines d21fa4a177
v0.14: various net/ fixes for large clusters (#4080)
* net.sh: Add -F to discard validator nodes that didn't bootup successfully

* Relax sanity node count when validator bootup failure is permitted

* Less sanity for testnet-demo

* net.sh: Add -F to discard validator nodes that didn't bootup successfully
2019-04-29 21:38:32 -07:00
Pankaj Garg 13fcfcb964
Blockstreamer annotation fix for non buildkite deployments (#4045) 2019-04-27 20:37:36 -07:00
sakridge b07290df81 Add usage to net.sh when it encounters an invalid argument (#4042)
automerge
2019-04-27 16:12:13 -07:00
Michael Vines 4ce2105548 Switch from a S3 URL to release.solana.com for release assets (#4017)
automerge
2019-04-26 13:06:55 -07:00
Michael Vines 4dc0495a1b
Fix tar version check 2019-04-25 11:16:49 -07:00
Pankaj Garg 9c7ccc0e2b More metrics to track memory usage (#3966)
automerge
2019-04-24 11:11:30 -07:00
Michael Vines 3eed6a6090 Annotate testnet buildkite logs with blockexplorer URL (#3964)
automerge
2019-04-24 08:09:37 -07:00
Dan Albert 4e7e5ace9d
Add support for Azure instances in testnet creation (#3905)
* Add support for Azure instances in testnet creation

* Fixup

* Fix shellcheck errors

* More shellcheck and cleanup node creation and deletion

* More shellcheck and cleanup node creation and deletion

* Fixup instance wait API

* Fix revieew comments and add GPU installation extension
2019-04-23 16:41:45 -06:00
Pankaj Garg 3ba93aa8fe Facility to pass extra arguments to bench clients from net.sh command (#3952)
automerge
2019-04-23 15:13:29 -07:00
Sagar Dhawan 69e67d06a7
Default to Docker based testnet deployment on unsupported Ubuntu versions (#3937)
* Run docker if Ubuntu version is unsupported

* ShellCheck fixes
2019-04-22 18:22:43 -07:00
Pankaj Garg bec094bb3e
Start bench tps clients as default (#3917) 2019-04-21 20:18:52 -07:00
Jack May 1a9ac62f60
Add bench-exchange support to the net framework (#3893) 2019-04-19 09:56:01 -07:00
Jack May aab410380e
Fix net for macos and local (#3872) 2019-04-18 14:49:35 -07:00
Michael Vines 9293a54234
Remove stray + 2019-04-15 20:28:11 -07:00
Michael Vines 666882fbbd
-r does not require an argument 2019-04-15 09:40:34 -07:00
Michael Vines 2f1b0bf4f5 Add solana-install deployments to the testnets 2019-04-11 10:03:35 -07:00
Michael Vines 88e5b14afc
Exit faster on sanity failures 2019-04-09 17:16:15 -07:00
Michael Vines 0b95a5c121 Include blockstreamer node in sanity 2019-04-09 16:52:57 -07:00
Pankaj Garg efbb49d579 Don't use external node ssh key if one is not configured 2019-04-02 14:20:00 -07:00
Pankaj Garg 97bd7a00f1 Support for configuring testnet nodes across multiple cloud services 2019-04-01 17:11:41 -07:00
Michael Vines ed8bada439 Kill all node processes (blockexplorer) 2019-03-21 22:09:44 -07:00
Michael Vines 791ead6053 Include TARGET in release URL to make room for future targets 2019-03-20 13:54:32 -07:00
Michael Vines 8f25548781 Overhaul cargo/rustc version management 2019-03-14 20:22:46 -07:00
Michael Vines 34a8d591fa Switch version file from .txt to .yaml; add target tuple to version.yml 2019-03-13 16:30:07 -07:00
Michael Vines 5d27f221f7 Drop socat for iptables 2019-03-13 12:03:56 -05:00
Michael Vines 61db74d98e
Run socat in the background 2019-03-13 08:15:58 -07:00
Michael Vines cd4bccfd12 Remove snap support 2019-03-02 17:41:09 -08:00
Tyera Eulberg 18d48f09f8 Plumb blockstreamer name through testnet scripts 2019-02-21 17:24:29 -07:00
Pankaj Garg bc52fce810 Fix the custom programs command in net.sh 2019-02-19 13:53:43 -07:00
Tyera Eulberg 760a82cb08
Add optional deploy of custom programs (#2817)
* Add optional deploy of custom programs

* Review comments
2019-02-18 11:43:36 -07:00
Michael Vines 4ff4fb6c38 Add support for an API node that hosts the block explorer 2019-02-17 10:39:27 -08:00
Michael Vines ab6c7f6ca3 /it/ti/ 2019-02-16 10:40:47 -08:00
Michael Vines d5f27f9b1e
shellcheck 2019-01-09 22:06:58 -07:00
Michael Vines 86f19a3ab3
Propagate PS4 to prevent unintentional buildkite log unfolding 2019-01-09 22:02:31 -07:00
Michael Vines 28431ff22c Add configurable RUST_LOG for ./net.sh sanity 2019-01-09 12:12:50 -08:00
Michael Vines eb37aa2bba Kill monitoring scripts by process group to ensure a full shutdown 2019-01-09 11:59:01 -08:00
Michael Vines 87c9af142f Preserve config/ when skipSetup 2019-01-09 11:59:01 -08:00
Michael Vines aedab3f83f Run sanity when previous ledger/setup is preserved 2019-01-08 21:11:43 -08:00
Michael Vines 3c835b692b
Use netLogDir 2018-12-23 10:33:43 -08:00
Michael Vines a6fd1ca3db Add logs subcommand to fetch remote logs from each network node 2018-12-23 10:19:10 -08:00
Michael Vines c3c955b02e Build/install native programs within cargo-install-all.sh 2018-12-19 11:53:08 -08:00
Michael Vines 5c396c222a Clean up install-native-programs.sh usage 2018-12-11 23:29:05 -08:00
Michael Vines 088bab61a4 Remove |cargo install| duplication 2018-12-11 23:29:05 -08:00
Michael Vines b2d7b34082 Add |./net.sh update| command to live update all network nodes 2018-12-11 09:40:22 -08:00
Michael Vines 094f0a8be3 Leader rotation flag plumbing 2018-12-10 14:07:59 -08:00
Michael Vines b2ddac610c Add option to skip setup during cluster start 2018-12-10 07:47:15 -08:00
Michael Vines f5794de636 Clean up bootstrap leader terminology in comments and variable names 2018-12-09 15:25:27 -08:00
Carl b9743957fa Make directory to hold programs 2018-12-09 08:38:41 -08:00
Michael Vines f5569e76db Relocate native programs to deps/ subdirectory of the current executable
This layout is `cargo build` compatible, no post-build file moves
required.
2018-12-08 16:31:01 -08:00
Michael Vines 872a3317b5 Fully switch to bootstrap-leader for command-line args 2018-12-07 16:57:02 -08:00
Michael Vines 6516c2532d Ensure native programs for the correct platform are installed 2018-12-07 11:32:08 -08:00
Michael Vines fa58da2401 Explicitly specific build variant when installing native programs 2018-12-07 11:32:08 -08:00
Michael Vines f0fe089013
Adapt testnet-deploy metric datapoint names to {,bootnode-}fullnode 2018-12-06 08:04:33 -08:00
Michael Vines 04a0652614 Generalize net/ from leader/validator to bootstrap-fullnode/fullnode 2018-12-05 17:11:16 -08:00
Michael Vines 33a5d5fe93 Enable debug builds by default for better backtraces 2018-11-17 10:52:08 -08:00
Michael Vines d96a6b42a5 Move drone into its own crate 2018-11-16 20:42:21 -08:00
Michael Vines 83fc3c10cf Setup CUDA env for local builds 2018-11-15 08:00:52 -08:00
Michael Vines 7f76403d0a Clean ~/solana during network start to avoid tripping over leftover files 2018-11-12 15:09:14 -08:00
Michael Vines def7d156f6 codemod --extensions sh '#!/usr/bin/env bash -e' '#!/usr/bin/env bash\nset -e' 2018-11-11 16:24:36 -08:00
Michael Vines 33aab094ef codemod --extensions sh '#!/bin/bash' '#!/usr/bin/env bash' 2018-11-11 16:24:36 -08:00
Michael Vines b8261d7d83
Determine network version for tar and local deploys 2018-11-08 22:02:42 -08:00
Michael Vines b02b636b36
Support local tarball deploys 2018-11-07 14:44:40 -08:00
Michael Vines c4bc331663 Add support for using a release tar 2018-11-07 07:47:39 -08:00
Michael Vines 7b58bd621a
Remove node check from client start-up
If the network loses a validator or two, it's the job of the sanity
check to detect this not the bench clients
2018-11-06 13:57:06 -08:00
Michael Vines 8041461a07
Bump EC2 validator machine type 2018-11-05 08:47:51 -08:00
Michael Vines f3b04894b9 Try harder to snap download 2018-11-03 00:29:13 +00:00
Pankaj Garg 85869552e0
Update testnet scripts to use release tar ball (#1660)
* Update testnet scripts to use release tar ball

* use curl instead of s3cmd
2018-10-30 18:05:38 -07:00
Michael Vines 5c523716aa Ship native programs 2018-10-10 16:49:48 -07:00
Pankaj Garg 0a39722719
Add support to trigger testnet from a PR (#1434)
* Add support for different node counts

* Update variable names

* Delete network even after failures

* Add array for node counts

* Changed number of nodes to a space separated string of numbers

* Adjust number of nodes

* Snap will not be published if the env variable DO_NOT_PUBLISH_SNAP is set

* Address review comments

* Replaced influx db URL
2018-10-05 16:32:05 -07:00