Commit Graph

518 Commits

Author SHA1 Message Date
Rob Walker a1fe6265fd
use pubkeys in genesis (#6750) 2019-11-06 11:18:25 -08:00
sakridge ec50c20400
Add time in net/logs path (#6701) 2019-11-06 10:43:12 -08:00
Trent Nelson a91bf296d7
Add some addition packages to DC installer scripts (#6755)
* Add 'cmake' to default DC node installer

* Add 'sysstat' to default DC node installer

For 'iostat'

* Add 'perf' to default DC node installer

* Add 'iftop' to default DC node installer
2019-11-06 09:48:45 -07:00
Pankaj Garg 8993b15248 Integrated use of netem with testnet scripts (#6746)
automerge
2019-11-05 15:04:06 -08:00
Michael Vines fba1af6ea9
ledger-tool can now load a ledger snapshot (#6729) 2019-11-04 22:14:55 -07:00
Sagar Dhawan 3133ee2401
Fix limited iftop output and failure to stop iftop (#6723)
* Fix limited iftop output and failure to stop iftop

* Shellcheck

* Ignore shellcheck
2019-11-04 18:12:07 -08:00
Trent Nelson d085c8626f GCE: Add instances self-destruct (#6363)
automerge
2019-11-04 10:30:26 -08:00
Dan Albert 7b6e3a23be
Add new pubkey to auth keys (#6687) 2019-11-01 14:44:10 -06:00
Dan Albert 1cc8956f74
Get Azure provider working again (#6659)
* Wait for node creation before continuing

* Programatically set networking rules

* Add network security group to nodes upon creation

* shellcheck
2019-11-01 14:43:31 -06:00
TristanDebrunner e6c8bfd008
Add --use-move flag to cargo-install-all.sh and net/net.sh (#6670) 2019-11-01 07:53:30 -07:00
Michael Vines f131255066
Add ~/.cargo/bin to PATH (#6641) 2019-10-30 19:41:24 -07:00
Michael Vines 7bb224f54a Install ag on nodes (#6634)
automerge
2019-10-30 16:43:16 -07:00
Tyera Eulberg 4ec95043d7
Update sol:lamport ratio to base-10 (#6611)
* Update sol:lamport ratio

* Update various SOL quantities in bash scripts
2019-10-29 20:03:48 -06:00
Michael Vines d952b38f93
Ensure nofiles is not capped at 1024 on a node reboot 2019-10-28 23:21:34 -07:00
Michael Vines 1e2ab89b47
Ensure redis-server is started on a reboot 2019-10-28 20:58:46 -07:00
Dan Albert 9ee65009cd
Implement allowing validator boot failure into automation (#6589)
* Pass allow boot failures through create AND start

* Extend sleep timeout to all nodes

* Add 100 node testcase

* Reduce consistent sleep
2019-10-28 16:43:40 -06:00
Trent Nelson 96e209db49
Colo: Don't fail without a message (#6558) 2019-10-28 09:20:49 -06:00
Michael Vines 0c14ca58c7 Invoke on-reboot from cloud startup script to avoid racing with cron (#6579)
automerge
2019-10-27 10:56:16 -07:00
Pankaj Garg e174af7838 Use iftop to collect network bandwidth usage (#6560)
* Use iftop to collect network bandwidth usage

* fix shellcheck

* more shellchecks

* review comments
2019-10-26 00:06:46 -07:00
Michael Vines be74801236
Add NET_NUM_xyz variables 2019-10-25 23:00:14 -07:00
Michael Vines e966c96644
Disable sigverify on blockstreamer node
This node get overloaded at high TPS trying to manage both a validator
and the blockexplorer.  Reduce it's workload by turning off sigverify,
which doesn't really matter since this node doesn't even vote
2019-10-25 21:33:08 -07:00
Dan Albert a2a9d54985
Increase node start stagger (#6566) 2019-10-25 17:35:29 -06:00
Justin Starry ea2b26e5f5 Fix scp client mint keypair (#6565) 2019-10-25 16:23:52 -07:00
Michael Vines e103789994
Ignore exit code when the first mount fails 2019-10-25 10:11:32 -07:00
Michael Vines 90461245f9
Reduce TdS fees to 1 lamport per sig, and slots_per_epoch/2 (#6542) 2019-10-24 20:37:23 -07:00
Michael Vines 1c91c1e880
Remount /mnt/extra-disk on reboot 2019-10-24 20:14:26 -07:00
Dan Albert dadcb632d8
Specify machine type without necessarily enabling GPU (#6529)
* Specifiy machine type without necessarily enabling GPU

* Make long arg, extend --enable-gpu to automation

* Set machine types only in one place

* Fixup

* Fixup flag in automation

* Typo

* shellcheck
2019-10-24 15:12:25 -06:00
Michael Vines 2de2fbd5e3
Remove stray setup_secondary_mounts 2019-10-24 13:48:57 -07:00
Michael Vines 14eca5aea6
Remove setup_secondary_mount knowledge from multinode-demo/ (#6530) 2019-10-24 13:40:16 -07:00
Justin Starry 7a7abe692e
Add mint keypair to solana clients for convenience (#6536) 2019-10-24 14:31:06 -04:00
Justin Starry 88033bccbb
Add mint keypair to validators for convenience (#6531) 2019-10-24 12:50:32 -04:00
Michael Vines 35d6196384
Surface nvidia-smi errors in CI 2019-10-23 10:59:30 -07:00
Michael Vines 26b8747014
Exit cleanly for idle clients 2019-10-23 09:56:05 -07:00
Michael Vines bedb05bdeb
Plumb GEOLOCATION_API_KEY down to the blockexplorer (#6514) 2019-10-23 09:53:06 -07:00
Justin Starry 6829b8a6fb
Ensure solana commands are added to idle clients (#6513) 2019-10-23 11:15:00 -04:00
Michael Vines e462a7d1d5
net: Add ability to only start/stop client nodes (#6503)
* Add info --eval

* net: Add ability to start idle client nodes
2019-10-22 16:08:49 -07:00
Sagar Dhawan 4c515d0ef1
Sagar: Add ssh keys for colo (#6507) 2019-10-22 15:59:39 -07:00
Michael Vines f80a5b8c34
Remove some TODOs (#6488)
* Remove stale TODOs

* Ban TODO markers from markdown

* Scrub all TODOs from ci/ and book/
2019-10-21 22:25:06 -07:00
Greg Fitzgerald 3b9b9b1500 Rename remaining uses of fullnode to validator (#6476)
automerge
2019-10-21 20:21:21 -07:00
Dan Albert 00809a67c0
Push perf test results to slack app (#6371)
* Add script to publish testnet results to slack

* Obscure webhook URL

* fixup

* Replace read with cat redirection

* Turn back on net restart

* Pick nits

* Make symlink before trying to delete its contents

* Display test config in slack and pick Trents nit not to maybe rm -rf /*

* Clean up results print

* Minor nits

* Turn the test settings back up to 11

* typo

* Shellcheck

* Just a few more fields

* fix payload formatting

* Del clear-config.sh

* Mount secondary

* Add commit SHA link and Grafana time range URL

* Add fancy buttons instead of text URLs

* Tighten up test config display

* Fixup display nits

* chellsheck

* Rebase and fix typo
2019-10-21 20:00:17 -04:00
Michael Vines 3fb70b8d47
Ban XXX, TBD, FIXME comments (#6486) 2019-10-21 16:43:11 -07:00
Trent Nelson 564c14a2c6 net.sh: Ensure external disk link is setup before cleaning config dir (#6481)
automerge
2019-10-21 15:38:58 -07:00
sakridge 6996f45d54 Print machine hostname in log (#6480)
automerge
2019-10-21 14:59:03 -07:00
sakridge b1c2c6009e Exclude net/log in rsync script (#6475)
automerge
2019-10-21 14:06:36 -07:00
Trent Nelson 934f69b660 Colo verbosity (#6473)
automerge
2019-10-21 13:49:12 -07:00
Sunny Gleason 951e1f8b48 feat: grant access to sunny@ (#6471) 2019-10-21 11:17:06 -07:00
Greg Fitzgerald 9232057e95
Rename replicator to archiver (#6464)
* Rename replicator to archiver

* cargo fmt

* Fix grammar
2019-10-21 11:29:37 -06:00
Trent Nelson 0fc3c7eee2 Bump Trent's keys... (#6445)
automerge
2019-10-18 15:42:50 -07:00
Michael Vines 6f58bdfcb1 Remove validator sanity check (#6435)
automerge
2019-10-18 08:26:08 -07:00
Pankaj Garg 854c62e208 Reduce kernel networking buffer for rmem and wmem (#6422)
automerge
2019-10-17 14:52:24 -07:00
Trent Nelson 1759968c1e Colo: Put NVMe disks to use (#6357)
automerge
2019-10-17 14:44:45 -07:00
Dan Albert b4ed88e0f7
Fail faster on boot up (#6412) 2019-10-17 12:26:12 -04:00
Michael Vines 2d351d3952
Prevent ping stats header from confusing buildkite log folding 2019-10-16 13:36:16 -07:00
Michael Vines 605b477e06
Permit finding more nodes than expected (./gce.sh config) 2019-10-16 13:21:00 -07:00
Michael Vines b7af5f08d6
Avoid more non-standard ping. macOS 💔 2019-10-16 10:35:41 -07:00
Michael Vines 781dfd9dc4
Drop non-standard ping -o option 2019-10-16 10:05:46 -07:00
Michael Vines 9267931ef6 Add support for preemptible GCP instances 2019-10-16 08:10:31 -07:00
Michael Vines 37a29b979f
--force 2019-10-15 15:12:25 -07:00
Michael Vines d89174ee82
Default to no client nodes to avoid unnecesary cost 2019-10-15 14:37:52 -07:00
Michael Vines 8bc9d8988f
- 2019-10-15 07:58:40 -07:00
Michael Vines f7279804b4
Ensure solana-cli has a keypair 2019-10-15 07:47:45 -07:00
Michael Vines 169b772398 Show validators during net sanity 2019-10-14 20:38:51 -07:00
Trent Nelson b75438ff32 gce.sh: Unwind allocation upon failure (#6343)
automerge
2019-10-14 09:36:20 -07:00
Trent Nelson 82fea9ce73 net.sh: Add support for selecting validator GPU mode (#6326)
automerge
2019-10-14 09:33:32 -07:00
Greg Fitzgerald 322fcea6e5
More fullnode to validator renaming (#6337) 2019-10-11 13:30:52 -06:00
Trent Nelson fa64a0b367 gce.sh: Be strict about fullnode count w/o --allow-boot-failures (#6321)
automerge
2019-10-10 17:13:59 -07:00
Trent Nelson 81fb9e6a59 gce.sh: Rename -f flag to better reflect usage (#6318)
automerge
2019-10-10 12:57:03 -07:00
Trent Nelson 4713cb8675 Colo: Prefer public IPs, part 2 (#6297)
automerge
2019-10-09 15:17:24 -07:00
Trent Nelson fdaee4ab17
Colo: Add running process cleanup to delete logic (#6281) 2019-10-09 15:49:33 -06:00
Justin Starry 95d15dc720
Add jstarry to authorized keys (#6293) 2019-10-09 15:04:44 -04:00
Trent Nelson 667f9e0d79 Colo: Factor out inlined scripts to own files (#6266)
automerge
2019-10-07 22:05:36 -07:00
Trent Nelson 57916f8be6 Colo: Prefer public IPs (#6264)
automerge
2019-10-07 20:44:57 -07:00
Michael Vines 18653b825b
Preserve previous fullnode log file on restart 2019-10-04 07:58:33 -07:00
Pankaj Garg a05d772aa9
Add colo access pubkey (#6232)
* Add colo access pubkey

* Change the key to ed25519
2019-10-03 19:55:39 -07:00
Dan Albert 58139ce5ae
Add buildkite-agent key for colo access (#6205) 2019-10-01 13:24:04 -07:00
Michael Vines 8e888059d8
Use built-in solana-gossip timeout for better error messages (#6189) 2019-10-01 12:30:11 -07:00
Dan Albert db18611c86
Add ability to manually create a db (#6151) 2019-09-27 12:03:20 -07:00
sakridge f97d33e3a7
Add sakridge pubkey (#6142) 2019-09-27 10:55:38 -07:00
sakridge 06b445ac07
Skip if --custom-cpu is used as well. (#6130) 2019-09-26 15:52:03 -07:00
Michael Vines b4da83a3ab
Remove CUDA feature (#6094) 2019-09-26 13:36:51 -07:00
Trent Nelson c4ed80d544 colo-utils: Disable StrictHostKeyChecking for SSH calls (#6117)
automerge
2019-09-26 11:22:07 -07:00
Dan Albert 93ad637c5c
typo 2019-09-25 16:58:53 -04:00
Trent Nelson 02647c25a9 net: Add Trent's work laptop pubkey (#6022)
automerge
2019-09-23 10:25:36 -07:00
Michael Vines 4c49566a89
Enable nvidia persistence mode on instance reboots 2019-09-21 10:45:20 -07:00
Michael Vines 8bbc8343ff
Place verison.yml in the right location 2019-09-19 22:41:27 -07:00
Trent Nelson 2636a9c9f1 Add script for managing colo resourse ala gce.sh (#5854)
automerge
2019-09-19 14:08:22 -07:00
Trent Nelson 4c54245969 net/gce.sh: Sync cloud_CreateInstances docs and usage (#5982)
automerge
2019-09-19 13:28:25 -07:00
Sunny Gleason 51b3451e20 feat: use redis version 5+ via ppa:chris-lea (#5981) 2019-09-19 12:04:06 -07:00
Michael Vines fee5c6c057
testnet-edge/testnet-beta now update while preserving the ledger (#5979)
* Check if an update is current before deploying it again

* Add (new) update command to deploy testnet updates

* Add --deploy-if-newer flag to permit conditional net updates
2019-09-19 12:03:47 -07:00
Michael Vines 9917ece826
Kill the old blockexplorer harder 2019-09-19 10:37:27 -07:00
Michael Vines 355564e486
`net/net.sh start --skip-setup ...` now works again (#5977) 2019-09-19 08:31:22 -07:00
Dan Albert 742562fc2e
Set maintenance policy to terminate and restart for GCE (#5935) 2019-09-18 10:38:38 -07:00
Michael Vines de1636c792 Enable --limit-ledger-size on testnets (#5927)
automerge
2019-09-17 10:05:41 -07:00
Michael Vines e1f4e8a84a Add solana-crate-features workaround to avoid cargo feature thrashing (#5904)
automerge
2019-09-13 23:46:21 -07:00
Michael Vines 92a5979558 net/config/ is now shellcheck compliant (#5888)
automerge
2019-09-12 16:11:13 -07:00
Michael Vines 8b64de0a3c
Add restart-explorer script, to easily restart the network explorer on a testnet (#5886) 2019-09-12 15:12:10 -07:00
Michael Vines bf1ecc2441
Remove ledger verification, it's racy and essentially globaly disabled already (#5867) 2019-09-11 10:53:10 -07:00
Michael Vines fc4aa71193
GCE-based nodes now reboot on maintenance events instead of terminating (#5861) 2019-09-10 12:30:06 -07:00
Parth 468095ede2 Update project to use new account serialization format (#5848) 2019-09-09 16:17:10 -07:00
Trent Nelson 9dc5da7dbd net/net.sh: Add flag to skip build (#5853)
automerge
2019-09-09 15:40:12 -07:00