Commit Graph

375 Commits

Author SHA1 Message Date
Michael Vines cbebc7a80c
Expand shortArgs correctly 2019-05-18 19:00:48 -07:00
Michael Vines 392a39dd54
Poh subsystem cleanup, genesis plumbing, enable real PoH on edge testnet (#4292)
* Remove unused PohServiceConfig::Step

* Clarify variable name

* Poh::hash() now takes an iteration counter

* man -> max

* Inline functions with single call site

* Move PohServiceConfig into GenesisBlock

* Add plumbing to enable real PoH on testnets

* Batch hashes to improve PoH hash rate

* Ensure a constant hashes_per_tick

* Remove PohEntry mixin field

* Poh/PohEntry no longer maintains tick_height

* Ensure a constant hashes_per_tick

* ci/localnet-sanity.sh: Use real PoH

* Rework Poh/PohService to keep PohRecorder unlocked as much as possible while hashing
2019-05-18 14:01:36 -07:00
Michael Vines 458ae3fdac Switch to instances with AVX-512 if possible for better interop with dev machines (#4328)
automerge
2019-05-17 20:06:07 -07:00
Michael Vines 915956b94b Remove disable leader rotation option (#4299)
automerge
2019-05-15 15:16:45 -07:00
Michael Vines 1393d26f63
Remove obsolete internal multinode-demo/ logging (#4265) 2019-05-13 10:51:18 -07:00
Michael Vines 50f79e495e net/ improvements (#4257)
automerge
2019-05-11 22:54:50 -07:00
Michael Vines dd12db2f06
Correctly handle more zones than additional nodes 2019-05-11 14:47:27 -07:00
Michael Vines bfc65e829e
Use zone[0] for any left over nodes 2019-05-11 14:07:36 -07:00
Michael Vines f2b727b534
Update mint keypair filename 2019-05-09 07:27:13 -07:00
Pankaj Garg 5719b8f251 Change remote node's ssh config to allow more login retries (#4215)
automerge
2019-05-08 11:20:06 -07:00
Michael Vines 7609a007c6
Add FeeCalculator to the genesis block (#4196) 2019-05-07 20:28:41 -07:00
Michael Vines 9b50583641
wallet: add --url, remove --host/--rpc-host/--rpc-port/-rpc-tls (#4153)
Also by default the wallet now talks to testnet.solana.com instead of
localhost
2019-05-06 07:38:26 -07:00
Michael Vines 0139e5db21
Correct blockstreamer node args 2019-05-04 08:22:36 -07:00
Michael Vines 586fb15c2c
fullnode positional arguments may now be mixed with optional arguments (#4151) 2019-05-03 20:49:24 -07:00
Michael Vines f3f416b7ba
Rename --network argument to --entrypoint (#4149) 2019-05-03 15:00:19 -07:00
Michael Vines 7fe3c75c6b
Add a node-specific ip echo service to remove dependency on ifconfig.co (#4137) 2019-05-03 11:01:35 -07:00
Pankaj Garg 598f765960 Fix net.sh for running on macos (#4107)
automerge
2019-05-01 16:03:35 -07:00
Michael Vines 950d8494ba earlyoom: Stop using unsupported -k option (#4096)
automerge
2019-05-01 11:29:02 -07:00
Michael Vines c5f8b4960c
Stop nodes in parallel 2019-04-30 10:42:59 -07:00
Michael Vines f02ec31c68
Flip if/else 2019-04-30 08:56:53 -07:00
Michael Vines d21fa4a177
v0.14: various net/ fixes for large clusters (#4080)
* net.sh: Add -F to discard validator nodes that didn't bootup successfully

* Relax sanity node count when validator bootup failure is permitted

* Less sanity for testnet-demo

* net.sh: Add -F to discard validator nodes that didn't bootup successfully
2019-04-29 21:38:32 -07:00
Michael Vines 0cc3956693
testnet-demo now runs across more GCE zones (#4053)
* testnet-demo now runs across more GCE zones

* Save zone info to config file

* Add geoip whitelist for common data centers

* Skip more of start

* Include -x for config

* Fetch private key from first validator node if necessary

* Correct -r propagation
2019-04-28 19:50:52 -07:00
Michael Vines 6f56501034
Correctly terminate instances across multiple zones 2019-04-28 09:09:02 -07:00
Pankaj Garg 697cd17b59
Use GPU nodes for blockstreamer as well if rest of testnet has GPUs (#4046)
- The blockstreamer crashes otherwise, as sigverify() looks for CUDA libs
2019-04-27 20:45:38 -07:00
Pankaj Garg 13fcfcb964
Blockstreamer annotation fix for non buildkite deployments (#4045) 2019-04-27 20:37:36 -07:00
sakridge b07290df81 Add usage to net.sh when it encounters an invalid argument (#4042)
automerge
2019-04-27 16:12:13 -07:00
Michael Vines 4ce2105548 Switch from a S3 URL to release.solana.com for release assets (#4017)
automerge
2019-04-26 13:06:55 -07:00
Dan Albert d12705f9b0
Remove wait loops in non-GPU instance creation and add SSD option as default disk type (#3992) 2019-04-25 13:43:42 -06:00
Michael Vines 4dc0495a1b
Fix tar version check 2019-04-25 11:16:49 -07:00
Michael Vines b67b0bff05
Do not enable rpc exit on the blockstreamer node (#3977) 2019-04-25 09:30:59 -07:00
Pankaj Garg e867ce0944
Find unique zones and delete nodes in each zone (#3978) 2019-04-24 17:50:42 -07:00
Pankaj Garg 29a25990d3
Add provision in testnet scripts to ignore validator nodes that failed to bootup (#3972)
* Skip writing to config file if the node didn't bootup
* Detect dead nodes quicker
2019-04-24 16:23:26 -07:00
Pankaj Garg 9c7ccc0e2b More metrics to track memory usage (#3966)
automerge
2019-04-24 11:11:30 -07:00
Michael Vines 3eed6a6090 Annotate testnet buildkite logs with blockexplorer URL (#3964)
automerge
2019-04-24 08:09:37 -07:00
Dan Albert 4e7e5ace9d
Add support for Azure instances in testnet creation (#3905)
* Add support for Azure instances in testnet creation

* Fixup

* Fix shellcheck errors

* More shellcheck and cleanup node creation and deletion

* More shellcheck and cleanup node creation and deletion

* Fixup instance wait API

* Fix revieew comments and add GPU installation extension
2019-04-23 16:41:45 -06:00
Pankaj Garg 3ba93aa8fe Facility to pass extra arguments to bench clients from net.sh command (#3952)
automerge
2019-04-23 15:13:29 -07:00
Sagar Dhawan 69e67d06a7
Default to Docker based testnet deployment on unsupported Ubuntu versions (#3937)
* Run docker if Ubuntu version is unsupported

* ShellCheck fixes
2019-04-22 18:22:43 -07:00
Michael Vines 05bcb7f292
Add stop node command to solana-gossip (#3928) 2019-04-22 14:51:20 -07:00
sakridge 95a16426f3
Cleanup bench-exchange (#3919)
* bench-exchange changes

Generate new keypair for each bench-exchange
Add metrics
Tweak network sleep parameters for better reliability.

* Bench-exchange contract metrics
2019-04-22 13:16:28 -07:00
Pankaj Garg bec094bb3e
Start bench tps clients as default (#3917) 2019-04-21 20:18:52 -07:00
Pankaj Garg d83a71d89f
More AWS regions for testnet deployment (#3911)
- also some minor fixes to gce.sh
2019-04-19 17:46:14 -07:00
Jack May 1a9ac62f60
Add bench-exchange support to the net framework (#3893) 2019-04-19 09:56:01 -07:00
Jack May aab410380e
Fix net for macos and local (#3872) 2019-04-18 14:49:35 -07:00
Pankaj Garg 8999bfef65
Try to delete nodes in all cloud zones (#3874) 2019-04-18 13:16:14 -07:00
sakridge 684e1c73dd
Allow for custom cpu config on gce and use 20gb ram for clients (#3856) 2019-04-18 09:36:11 -07:00
Michael Vines 78d5ace754
Refactor multinode-demo/ scripts to avoid shipping fullnode-x.sh (#3835) 2019-04-17 18:03:58 -07:00
Pankaj Garg d0fb55d9b1
Allow testnet creation with no validator nodes (#3846) 2019-04-17 17:38:18 -07:00
Michael Vines 63d66ece57
net/ testnet nodes now stake more lamports (#3812)
* Add --bootstrap-leader-lamports

* Generalize --no-stake into --stake NUM

* Use a large stake for net/ fullnodes

* Setup vote account before starting fullnode to avoid mixed log output
2019-04-16 13:03:01 -07:00
Michael Vines c67cc694ae
Remove stake from `./net sanity` ephemeral validator (#3797) 2019-04-15 21:42:25 -07:00
Michael Vines 9293a54234
Remove stray + 2019-04-15 20:28:11 -07:00
Michael Vines 666882fbbd
-r does not require an argument 2019-04-15 09:40:34 -07:00
Michael Vines 2f1b0bf4f5 Add solana-install deployments to the testnets 2019-04-11 10:03:35 -07:00
Michael Vines 88e5b14afc
Exit faster on sanity failures 2019-04-09 17:16:15 -07:00
Michael Vines 0b95a5c121 Include blockstreamer node in sanity 2019-04-09 16:52:57 -07:00
Stephen Akridge 76e524ae48 Remove check for 0 additional nodes
Network with 1 leader is valid.
2019-04-09 11:16:55 -07:00
Michael Vines 6ac919c71a Set warn log level only for perf testnets 2019-04-09 11:09:16 -07:00
Michael Vines 1ba4806f8c Document recent -z and -x command-line arg changes 2019-04-09 10:39:55 -07:00
Pankaj Garg 0904df327d
Parallelize cloud node deployment commands in case of multiple zones (#3657) 2019-04-07 08:13:48 -07:00
Pankaj Garg 9cd555cad5 AWS script change for additional zones and regions 2019-04-04 15:59:59 -07:00
Tyera Eulberg acbc261891 Add gossip to build script, and fix bash strings 2019-04-04 00:18:48 -07:00
Pankaj Garg efbb49d579 Don't use external node ssh key if one is not configured 2019-04-02 14:20:00 -07:00
Tyera Eulberg 92c66a411b Remove bench-tps converge-only 2019-04-01 23:05:25 -06:00
Pankaj Garg 97bd7a00f1 Support for configuring testnet nodes across multiple cloud services 2019-04-01 17:11:41 -07:00
Pankaj Garg cb3adea94f Increase node count in beta testnet 2019-04-01 11:06:24 -07:00
Pankaj Garg 15b945a652 Fix EC2 scripts for blockstream startup 2019-03-28 15:37:23 -07:00
Pankaj Garg a94bc80383 fix clippy errors 2019-03-27 18:05:17 -07:00
Pankaj Garg 586b6fc3d7 review comments 2019-03-27 18:05:17 -07:00
Pankaj Garg a14c202d60 fix the ip address that's stored in the config file 2019-03-27 18:05:17 -07:00
Pankaj Garg ed48c495a3 fix shell-check errors 2019-03-27 18:05:17 -07:00
Pankaj Garg f0abd06a46 Added support for multi-region cloud testnet 2019-03-27 18:05:17 -07:00
Pankaj Garg e8814b1297 Add support for influx cloud 2019-03-25 17:10:38 -07:00
Michael Vines 0a5b6154e8 Use same gossip port for all testnet nodes 2019-03-22 00:16:58 -07:00
Michael Vines e6b9babf53 Run a drone on blockstreamer nodes 2019-03-21 22:09:44 -07:00
Michael Vines ed8bada439 Kill all node processes (blockexplorer) 2019-03-21 22:09:44 -07:00
Michael Vines 3b5a9f512c Get client-id.json out of the genesis ledger directory 2019-03-21 16:35:40 -07:00
Michael Vines 791ead6053 Include TARGET in release URL to make room for future targets 2019-03-20 13:54:32 -07:00
Michael Vines 7498488f5f
cloud_DeleteInstances() now waits for the instances to be terminated 2019-03-14 21:15:00 -07:00
Michael Vines ede99d5913
Revert "Block until instances are confirmed to be deleted"
This reverts commit 47ddbbe53b.
2019-03-14 20:53:10 -07:00
Michael Vines 8f25548781 Overhaul cargo/rustc version management 2019-03-14 20:22:46 -07:00
Michael Vines 47ddbbe53b Block until instances are confirmed to be deleted 2019-03-14 16:20:18 -07:00
Michael Vines 34a8d591fa Switch version file from .txt to .yaml; add target tuple to version.yml 2019-03-13 16:30:07 -07:00
Michael Vines 242bcf44db
Replace stale --no-signer usage with --no-voting 2019-03-13 13:50:30 -07:00
Michael Vines ebd540972d
Remove duplicate --rpc-drone-address 2019-03-13 13:24:02 -07:00
Michael Vines fedbae6f8c
Enable rpc for all testnet nodes 2019-03-13 10:49:40 -07:00
Michael Vines 5d27f221f7 Drop socat for iptables 2019-03-13 12:03:56 -05:00
Michael Vines 61db74d98e
Run socat in the background 2019-03-13 08:15:58 -07:00
Pankaj Garg 76feb2098e
Use same VM type for validators as leader, if CUDA is enabled (#3253)
- Since all nodes are created equal
2019-03-12 17:42:47 -07:00
Sagar Dhawan 866d3f467f
Fix flag to disable leader-rotation (#3243) 2019-03-12 16:35:13 -07:00
Rob Walker a799f8f4b1
tell blockexplorer to run on port 8080 (#3237)
* tell blockexplorer to run on port 8080

* forward port 80 to 5000 for a blockexplorer node
2019-03-12 13:39:09 -07:00
Michael Vines 7b49c9f09c Delete fullnode-config/ 2019-03-04 15:49:02 -08:00
Michael Vines cd4bccfd12 Remove snap support 2019-03-02 17:41:09 -08:00
Tyera Eulberg 18d48f09f8 Plumb blockstreamer name through testnet scripts 2019-02-21 17:24:29 -07:00
Tyera Eulberg d8f6865338 Rename EntryStream to Blockstream 2019-02-21 17:24:29 -07:00
Pankaj Garg bc52fce810 Fix the custom programs command in net.sh 2019-02-19 13:53:43 -07:00
Michael Vines a444cac2aa Switch to upstream AMIs for non-CUDA EC2 testnets 2019-02-18 18:59:56 -08:00
Michael Vines 1e714eb6b2 Generate ec2 security group programmatically 2019-02-18 18:59:56 -08:00
Michael Vines 3f14466965
Limit blockexplorer versions to 1.x.y
Per semver semantics when blockexplorer 2.0.0 is released it will be
incompatible in some way with 1.x.y and thus should be opt in.
2019-02-18 16:48:33 -08:00
Tyera Eulberg 760a82cb08
Add optional deploy of custom programs (#2817)
* Add optional deploy of custom programs

* Review comments
2019-02-18 11:43:36 -07:00
Michael Vines 2793404116
Ensure blockexplorer comes back up when nodes are updated instead of restarted 2019-02-17 20:07:12 -08:00
Michael Vines 858068cdc0
Drop sudo, it's now handled internally by the block explorer 2019-02-17 12:29:53 -08:00
Michael Vines 65fb307d0f
Avoid '' argument to fullnode.sh 2019-02-17 11:43:41 -08:00
Michael Vines 4ff4fb6c38 Add support for an API node that hosts the block explorer 2019-02-17 10:39:27 -08:00
Michael Vines ab6c7f6ca3 /it/ti/ 2019-02-16 10:40:47 -08:00
Michael Vines 9eb8b67b5c
Install blockexplorer dependencies 2019-02-15 20:17:46 -08:00
Michael Vines bae7612f36
Revert "Wait until the node successfully boots"
This reverts commit e84f1f6de7.
2019-01-23 11:27:08 -08:00
Michael Vines e84f1f6de7 Wait until the node successfully boots 2019-01-23 00:02:30 -08:00
Michael Vines a84b6bc7e4 Overhaul wallet rpc/drone command-line arguments 2019-01-17 08:36:05 -08:00
Pankaj Garg 00c4c30d72
Fix testnet bootup issue (#2465)
* Fix testnet bootup issue

* address review comments
2019-01-16 19:18:32 -08:00
Michael Vines 6015a0ff15 Add info command 2019-01-16 10:24:00 -08:00
Michael Vines d5f27f9b1e
shellcheck 2019-01-09 22:06:58 -07:00
Michael Vines 86f19a3ab3
Propagate PS4 to prevent unintentional buildkite log unfolding 2019-01-09 22:02:31 -07:00
Michael Vines be0eefb0af
Add timeout to prevent stuck bench-tps when a cluster goes bad 2019-01-09 19:21:53 -07:00
Michael Vines 28431ff22c Add configurable RUST_LOG for ./net.sh sanity 2019-01-09 12:12:50 -08:00
Michael Vines 639bed2f6d Reorder sanity.
1. Check for presence of nodes
2. Check for functioning RPC API
3. Then try the wallet
2019-01-09 12:05:30 -08:00
Michael Vines eb37aa2bba Kill monitoring scripts by process group to ensure a full shutdown 2019-01-09 11:59:01 -08:00
Michael Vines 048fe371aa set -x for more detailed logs 2019-01-09 11:59:01 -08:00
Michael Vines 87c9af142f Preserve config/ when skipSetup 2019-01-09 11:59:01 -08:00
Michael Vines e0c68bf9ad docs: -z is a common option 2019-01-08 21:11:43 -08:00
Michael Vines aedab3f83f Run sanity when previous ledger/setup is preserved 2019-01-08 21:11:43 -08:00
Michael Vines 1b7598e351 Add retries to RPC API probe 2019-01-08 08:50:51 -08:00
Michael Vines 1531a1777a Add RPC API check 2018-12-24 22:51:36 -08:00
Michael Vines 04d46ea33f Run oom-monitor as root 2018-12-24 22:51:36 -08:00
Michael Vines f5bbc5e961
Fix args 2018-12-23 20:56:13 -08:00
Michael Vines 753a783ba9
Add solana user to adm group for /var/log/syslog access 2018-12-23 17:28:35 -08:00
Michael Vines 3c835b692b
Use netLogDir 2018-12-23 10:33:43 -08:00
Michael Vines a6fd1ca3db Add logs subcommand to fetch remote logs from each network node 2018-12-23 10:19:10 -08:00
Pankaj Garg 41f8764232
Ignore error while enabling nvidia persistence mode (#2265) 2018-12-21 12:37:51 -08:00
Pankaj Garg 4bf797c8f1
Load nvidia drivers on node startup (#2263)
* Load nvidia drivers on node startup

* added new script to enable nvidia driver persistent mode

* remove set -ex
2018-12-21 11:43:52 -08:00
Michael Vines c3c955b02e Build/install native programs within cargo-install-all.sh 2018-12-19 11:53:08 -08:00
Michael Vines 5c396c222a Clean up install-native-programs.sh usage 2018-12-11 23:29:05 -08:00
Michael Vines 088bab61a4 Remove |cargo install| duplication 2018-12-11 23:29:05 -08:00
Michael Vines b2d7b34082 Add |./net.sh update| command to live update all network nodes 2018-12-11 09:40:22 -08:00
Sathish 154e20484d
Use hostname in database if env is set (#2101) 2018-12-10 22:59:38 -08:00
Michael Vines 094f0a8be3 Leader rotation flag plumbing 2018-12-10 14:07:59 -08:00
Michael Vines b2ddac610c Add option to skip setup during cluster start 2018-12-10 07:47:15 -08:00
Michael Vines b54b0a1d25 Document that -P is now available for |config| 2018-12-09 15:25:27 -08:00
Michael Vines f5794de636 Clean up bootstrap leader terminology in comments and variable names 2018-12-09 15:25:27 -08:00
Carl b9743957fa Make directory to hold programs 2018-12-09 08:38:41 -08:00
Michael Vines f5569e76db Relocate native programs to deps/ subdirectory of the current executable
This layout is `cargo build` compatible, no post-build file moves
required.
2018-12-08 16:31:01 -08:00
Michael Vines 872a3317b5 Fully switch to bootstrap-leader for command-line args 2018-12-07 16:57:02 -08:00
Michael Vines 1db6a882bb rsync of genesis ledger now works for non-snap deployments 2018-12-07 16:57:02 -08:00
Michael Vines af11562627 Correct ledger path 2018-12-07 11:32:08 -08:00
Michael Vines 286f08f095 Drop old validator name, use fullnode instead 2018-12-07 11:32:08 -08:00
Michael Vines 6516c2532d Ensure native programs for the correct platform are installed 2018-12-07 11:32:08 -08:00
Michael Vines fa58da2401 Explicitly specific build variant when installing native programs 2018-12-07 11:32:08 -08:00
Michael Vines 70c149c7da Rename leader/validator to bootstrap-leader/fullnode
Only rsyncing the genesis ledger snuck in here as well
2018-12-06 19:44:47 -08:00
Michael Vines b34e197424
Add newline at end of file 2018-12-06 17:46:46 -08:00
Michael Vines c4b8f0cd2f bench-tps will now generate an ephemeral identity if not provided with one
Also simplify scripts as a result
2018-12-06 16:30:48 -08:00
carllin aecb06cd2a
Update versions in install-libssl-compatibility.sh (#2044) 2018-12-06 15:57:30 -08:00
Michael Vines f0fe089013
Adapt testnet-deploy metric datapoint names to {,bootnode-}fullnode 2018-12-06 08:04:33 -08:00