Commit Graph

204 Commits

Author SHA1 Message Date
sakridge 8611b40074
Add argument to net/net to specify number of nodes (#4429)
Allows for testing different node counts without recreating the network.
2019-05-24 16:20:14 -07:00
Michael Vines 94beb4b8c2
More fullnode -> validator renaming (#4414)
* s/fullnode_config/validator_config/g

* s/FullnodeConfig/ValidatorConfig/g

* mv core/lib/fullnode.rs core/lib/validator.rs

* s/Fullnode/Validator/g

* Add replicator-x.sh

* Rename fullnode.md to validator.md

* cargo fmt
2019-05-23 22:05:16 -07:00
sakridge 90e73515ed Remove ls which is innacurate since we never clean up the logs (#4392)
automerge
2019-05-22 13:11:20 -07:00
Michael Vines cbebc7a80c
Expand shortArgs correctly 2019-05-18 19:00:48 -07:00
Michael Vines 392a39dd54
Poh subsystem cleanup, genesis plumbing, enable real PoH on edge testnet (#4292)
* Remove unused PohServiceConfig::Step

* Clarify variable name

* Poh::hash() now takes an iteration counter

* man -> max

* Inline functions with single call site

* Move PohServiceConfig into GenesisBlock

* Add plumbing to enable real PoH on testnets

* Batch hashes to improve PoH hash rate

* Ensure a constant hashes_per_tick

* Remove PohEntry mixin field

* Poh/PohEntry no longer maintains tick_height

* Ensure a constant hashes_per_tick

* ci/localnet-sanity.sh: Use real PoH

* Rework Poh/PohService to keep PohRecorder unlocked as much as possible while hashing
2019-05-18 14:01:36 -07:00
Michael Vines 915956b94b Remove disable leader rotation option (#4299)
automerge
2019-05-15 15:16:45 -07:00
Michael Vines 7fe3c75c6b
Add a node-specific ip echo service to remove dependency on ifconfig.co (#4137) 2019-05-03 11:01:35 -07:00
Pankaj Garg 598f765960 Fix net.sh for running on macos (#4107)
automerge
2019-05-01 16:03:35 -07:00
Michael Vines c5f8b4960c
Stop nodes in parallel 2019-04-30 10:42:59 -07:00
Michael Vines d21fa4a177
v0.14: various net/ fixes for large clusters (#4080)
* net.sh: Add -F to discard validator nodes that didn't bootup successfully

* Relax sanity node count when validator bootup failure is permitted

* Less sanity for testnet-demo

* net.sh: Add -F to discard validator nodes that didn't bootup successfully
2019-04-29 21:38:32 -07:00
Pankaj Garg 13fcfcb964
Blockstreamer annotation fix for non buildkite deployments (#4045) 2019-04-27 20:37:36 -07:00
sakridge b07290df81 Add usage to net.sh when it encounters an invalid argument (#4042)
automerge
2019-04-27 16:12:13 -07:00
Michael Vines 4ce2105548 Switch from a S3 URL to release.solana.com for release assets (#4017)
automerge
2019-04-26 13:06:55 -07:00
Michael Vines 4dc0495a1b
Fix tar version check 2019-04-25 11:16:49 -07:00
Pankaj Garg 9c7ccc0e2b More metrics to track memory usage (#3966)
automerge
2019-04-24 11:11:30 -07:00
Michael Vines 3eed6a6090 Annotate testnet buildkite logs with blockexplorer URL (#3964)
automerge
2019-04-24 08:09:37 -07:00
Dan Albert 4e7e5ace9d
Add support for Azure instances in testnet creation (#3905)
* Add support for Azure instances in testnet creation

* Fixup

* Fix shellcheck errors

* More shellcheck and cleanup node creation and deletion

* More shellcheck and cleanup node creation and deletion

* Fixup instance wait API

* Fix revieew comments and add GPU installation extension
2019-04-23 16:41:45 -06:00
Pankaj Garg 3ba93aa8fe Facility to pass extra arguments to bench clients from net.sh command (#3952)
automerge
2019-04-23 15:13:29 -07:00
Sagar Dhawan 69e67d06a7
Default to Docker based testnet deployment on unsupported Ubuntu versions (#3937)
* Run docker if Ubuntu version is unsupported

* ShellCheck fixes
2019-04-22 18:22:43 -07:00
Pankaj Garg bec094bb3e
Start bench tps clients as default (#3917) 2019-04-21 20:18:52 -07:00
Jack May 1a9ac62f60
Add bench-exchange support to the net framework (#3893) 2019-04-19 09:56:01 -07:00
Jack May aab410380e
Fix net for macos and local (#3872) 2019-04-18 14:49:35 -07:00
Michael Vines 9293a54234
Remove stray + 2019-04-15 20:28:11 -07:00
Michael Vines 666882fbbd
-r does not require an argument 2019-04-15 09:40:34 -07:00
Michael Vines 2f1b0bf4f5 Add solana-install deployments to the testnets 2019-04-11 10:03:35 -07:00
Michael Vines 88e5b14afc
Exit faster on sanity failures 2019-04-09 17:16:15 -07:00
Michael Vines 0b95a5c121 Include blockstreamer node in sanity 2019-04-09 16:52:57 -07:00
Pankaj Garg efbb49d579 Don't use external node ssh key if one is not configured 2019-04-02 14:20:00 -07:00
Pankaj Garg 97bd7a00f1 Support for configuring testnet nodes across multiple cloud services 2019-04-01 17:11:41 -07:00
Michael Vines ed8bada439 Kill all node processes (blockexplorer) 2019-03-21 22:09:44 -07:00
Michael Vines 791ead6053 Include TARGET in release URL to make room for future targets 2019-03-20 13:54:32 -07:00
Michael Vines 8f25548781 Overhaul cargo/rustc version management 2019-03-14 20:22:46 -07:00
Michael Vines 34a8d591fa Switch version file from .txt to .yaml; add target tuple to version.yml 2019-03-13 16:30:07 -07:00
Michael Vines 5d27f221f7 Drop socat for iptables 2019-03-13 12:03:56 -05:00
Michael Vines 61db74d98e
Run socat in the background 2019-03-13 08:15:58 -07:00
Michael Vines cd4bccfd12 Remove snap support 2019-03-02 17:41:09 -08:00
Tyera Eulberg 18d48f09f8 Plumb blockstreamer name through testnet scripts 2019-02-21 17:24:29 -07:00
Pankaj Garg bc52fce810 Fix the custom programs command in net.sh 2019-02-19 13:53:43 -07:00
Tyera Eulberg 760a82cb08
Add optional deploy of custom programs (#2817)
* Add optional deploy of custom programs

* Review comments
2019-02-18 11:43:36 -07:00
Michael Vines 4ff4fb6c38 Add support for an API node that hosts the block explorer 2019-02-17 10:39:27 -08:00
Michael Vines ab6c7f6ca3 /it/ti/ 2019-02-16 10:40:47 -08:00
Michael Vines d5f27f9b1e
shellcheck 2019-01-09 22:06:58 -07:00
Michael Vines 86f19a3ab3
Propagate PS4 to prevent unintentional buildkite log unfolding 2019-01-09 22:02:31 -07:00
Michael Vines 28431ff22c Add configurable RUST_LOG for ./net.sh sanity 2019-01-09 12:12:50 -08:00
Michael Vines eb37aa2bba Kill monitoring scripts by process group to ensure a full shutdown 2019-01-09 11:59:01 -08:00
Michael Vines 87c9af142f Preserve config/ when skipSetup 2019-01-09 11:59:01 -08:00
Michael Vines aedab3f83f Run sanity when previous ledger/setup is preserved 2019-01-08 21:11:43 -08:00
Michael Vines 3c835b692b
Use netLogDir 2018-12-23 10:33:43 -08:00
Michael Vines a6fd1ca3db Add logs subcommand to fetch remote logs from each network node 2018-12-23 10:19:10 -08:00
Michael Vines c3c955b02e Build/install native programs within cargo-install-all.sh 2018-12-19 11:53:08 -08:00
Michael Vines 5c396c222a Clean up install-native-programs.sh usage 2018-12-11 23:29:05 -08:00
Michael Vines 088bab61a4 Remove |cargo install| duplication 2018-12-11 23:29:05 -08:00
Michael Vines b2d7b34082 Add |./net.sh update| command to live update all network nodes 2018-12-11 09:40:22 -08:00
Michael Vines 094f0a8be3 Leader rotation flag plumbing 2018-12-10 14:07:59 -08:00
Michael Vines b2ddac610c Add option to skip setup during cluster start 2018-12-10 07:47:15 -08:00
Michael Vines f5794de636 Clean up bootstrap leader terminology in comments and variable names 2018-12-09 15:25:27 -08:00
Carl b9743957fa Make directory to hold programs 2018-12-09 08:38:41 -08:00
Michael Vines f5569e76db Relocate native programs to deps/ subdirectory of the current executable
This layout is `cargo build` compatible, no post-build file moves
required.
2018-12-08 16:31:01 -08:00
Michael Vines 872a3317b5 Fully switch to bootstrap-leader for command-line args 2018-12-07 16:57:02 -08:00
Michael Vines 6516c2532d Ensure native programs for the correct platform are installed 2018-12-07 11:32:08 -08:00
Michael Vines fa58da2401 Explicitly specific build variant when installing native programs 2018-12-07 11:32:08 -08:00
Michael Vines f0fe089013
Adapt testnet-deploy metric datapoint names to {,bootnode-}fullnode 2018-12-06 08:04:33 -08:00
Michael Vines 04a0652614 Generalize net/ from leader/validator to bootstrap-fullnode/fullnode 2018-12-05 17:11:16 -08:00
Michael Vines 33a5d5fe93 Enable debug builds by default for better backtraces 2018-11-17 10:52:08 -08:00
Michael Vines d96a6b42a5 Move drone into its own crate 2018-11-16 20:42:21 -08:00
Michael Vines 83fc3c10cf Setup CUDA env for local builds 2018-11-15 08:00:52 -08:00
Michael Vines 7f76403d0a Clean ~/solana during network start to avoid tripping over leftover files 2018-11-12 15:09:14 -08:00
Michael Vines def7d156f6 codemod --extensions sh '#!/usr/bin/env bash -e' '#!/usr/bin/env bash\nset -e' 2018-11-11 16:24:36 -08:00
Michael Vines 33aab094ef codemod --extensions sh '#!/bin/bash' '#!/usr/bin/env bash' 2018-11-11 16:24:36 -08:00
Michael Vines b8261d7d83
Determine network version for tar and local deploys 2018-11-08 22:02:42 -08:00
Michael Vines b02b636b36
Support local tarball deploys 2018-11-07 14:44:40 -08:00
Michael Vines c4bc331663 Add support for using a release tar 2018-11-07 07:47:39 -08:00
Michael Vines 7b58bd621a
Remove node check from client start-up
If the network loses a validator or two, it's the job of the sanity
check to detect this not the bench clients
2018-11-06 13:57:06 -08:00
Michael Vines 8041461a07
Bump EC2 validator machine type 2018-11-05 08:47:51 -08:00
Michael Vines f3b04894b9 Try harder to snap download 2018-11-03 00:29:13 +00:00
Pankaj Garg 85869552e0
Update testnet scripts to use release tar ball (#1660)
* Update testnet scripts to use release tar ball

* use curl instead of s3cmd
2018-10-30 18:05:38 -07:00
Michael Vines 5c523716aa Ship native programs 2018-10-10 16:49:48 -07:00
Pankaj Garg 0a39722719
Add support to trigger testnet from a PR (#1434)
* Add support for different node counts

* Update variable names

* Delete network even after failures

* Add array for node counts

* Changed number of nodes to a space separated string of numbers

* Adjust number of nodes

* Snap will not be published if the env variable DO_NOT_PUBLISH_SNAP is set

* Address review comments

* Replaced influx db URL
2018-10-05 16:32:05 -07:00
Pankaj Garg be7cce1fd2
Tweak GCE scripts for higher node count (#1229)
* Tweak GCE scripts for higher node count

- Some validators were unable to rsync config from leader when
  the node count was high (e.g. 25). Looks like the leader node was
  getting more rsync requests in parallel than it count handle.
- This change staggers the validators bootup, and rsync time

* Address review comments
2018-09-14 17:17:08 -07:00
Michael Vines f06113500d bench-tps/net sanity: add ability to check for unexpected extra nodes 2018-09-12 15:38:57 -07:00
Pankaj Garg 1c17c6dd2b
Report UDP network statistics (#1176)
* Report UDP network statistics

Fixes #1093

* Address review comments

* Address additional review comments

* Fix shellcheck errors
2018-09-10 15:52:08 -07:00
Michael Vines ebcac3c2d1 Use a common solana user on all testnet instances 2018-09-08 22:34:26 -07:00
Michael Vines 5afcdcbbe6
More log grooming 2018-09-08 14:16:34 -07:00
Michael Vines 7aeb6d642b Display log file 2018-09-08 13:59:45 -07:00
Michael Vines db825788fa Document how to get ssh access into CD testnets 2018-09-07 19:41:13 -07:00
Michael Vines 1090254ba5 Add datapoints for leader/validator start 2018-09-07 12:45:48 -07:00
Michael Vines a2631e89f6 Use consistent style 2018-09-07 12:01:43 -07:00
Michael Vines 09a48d773a Run bench-tps in a tmux 2018-09-07 12:01:43 -07:00
Michael Vines 5a57d9b5d9 de-y 2018-09-06 19:54:39 -07:00
Michael Vines 03e87e4169 Add more metrics 2018-09-06 19:54:39 -07:00
Michael Vines 31dee553d5 Split start/version reporting 2018-09-06 19:54:39 -07:00
Michael Vines a3178c3bc7 Remove unused name tag 2018-09-06 19:54:39 -07:00
Michael Vines eaef9be710 Clarify -f 2018-09-06 19:54:39 -07:00
Michael Vines 48c19d3100 Enable cargo features to be specified 2018-09-05 09:02:02 -07:00
Michael Vines e677cda027 Private IP networks now work, and are the default 2018-09-05 09:02:02 -07:00
Michael Vines db9219ccc8 Improve error monitoring 2018-09-05 09:02:02 -07:00
Michael Vines 6ad4a81123 s/_/-/g in filenames 2018-09-05 09:02:02 -07:00
Michael Vines bcaa0fdcb1 net/ can now deploy Snaps 2018-09-05 09:02:02 -07:00
Michael Vines 9365a47d42 Employ a startup script 2018-09-05 09:02:02 -07:00
Michael Vines ec3e62dd58 Add net/ sanity 2018-09-05 09:02:02 -07:00
Michael Vines fa07c49cc9 net/ can now deploy Snaps 2018-09-05 09:02:02 -07:00
Michael Vines 43b4207101 Run oom-monitor in net/ testnets 2018-09-05 09:02:02 -07:00
Michael Vines ff991b87da Add support for deploying from non-Linux machines 2018-09-05 09:02:02 -07:00
Michael Vines 399caf343c Morph gce_multinode-based scripts into net/ 2018-09-05 09:02:02 -07:00