Commit Graph

1092 Commits

Author SHA1 Message Date
Rob Walker 61abee204f
don't check for snap mode in common.sh, is only relevant to snap daemons (#1113)
snap mode is for daemons, remove it from client (i.e. common.sh)

supply leader info to client via snap
2018-09-04 14:31:54 +09:00
Rob Walker e834c76b40 --count => --num-nodes 2018-09-04 07:07:25 +09:00
Rob Walker d1f1cbe88f leader-address=>leader-ip 2018-09-01 04:38:18 +09:00
Rob Walker a4e7b6e90c more fixups for client.sh changes 2018-09-01 03:33:21 +09:00
Rob Walker fbc7c9c431 fix client_start to deal with new client.sh 2018-09-01 03:23:05 +09:00
Rob Walker 8b248dcf09 specify port 2018-09-01 02:56:24 +09:00
Rob Walker 4938aad939 fixups 2018-09-01 02:21:46 +09:00
Rob Walker 7e882dfe62 inform all snaps where the network is 2018-09-01 02:21:46 +09:00
Rob Walker 9d1eb4f9ea remove 'localhost' leader (redundant, un-dig-friendly) 2018-08-31 23:21:07 +09:00
Rob Walker 176e806d94 rework of netwrk rendezvous
* rename NodeInfo field of Node from "data" to "info"
      (touches a lot of files)

  * update client to use gossip to find leader, a la drone

  * rework multinode scripts
      * move more stuff into rust
      * added usage to all
      * no more rsync unless you're a validator (TODO: whack that, too)
  * fullnode doesn't bail if drone isn't up yet, just keeps trying
  * drone doesn't bail if network isn't up yet, just keeps trying
2018-08-31 23:21:07 +09:00
Rob Walker 63e44dcc35 continue rendezvous refactor for gossip and repair
* remove trailing whitespace in ci/audit.sh

  * code review fixups
     * rename GOSSIP_PORT_RANGE => SOLANA_PORT_RANGE
     * remove out-of-date TODO in localnet-sanity.sh

  * remove features=test and code that was using it (localhost prohibitions in
      crdt) added TODO in crdt.rs, maybe we should boot localhost in production
      networks?

  * boot tvu_window from NodeInfo: instead, send repair requests from the repair
      socket (to gossip on peer) and answer repair requests via the sockaddr
      from the repair request

  * remove various unused pub functions

  * banish SocketAddr parse().unwrap() to a macro that can also accept simpler stuff
2018-08-31 23:21:07 +09:00
Rob Walker 1af4cee63b fix #1079
* move gossip/NCP off assuming anything about its address
  * use a single socket to send and receive gossip
  * remove --addr/-a from CLIs
  * rearrange networking utility code
  * use Arc<UdpSocket> to share the Sync-safe UdpSocket among threads
  * rename TestNode to Node

TODO:

  * re-enable 127.0.0.1 as a valid address in crdt
  * change repair request/response to a similar, single socket
  * pick cloned sockets or Arc<UdpSocket> for all these (rpu uses tryclone())
  * update contact_info with network truthiness instead of what the node
      says?
2018-08-31 23:21:07 +09:00
Rob Walker cb52a335bd re-enable localnet-sanity 2018-08-31 23:21:07 +09:00
Michael Vines 73a0bf8d30 Avoid unbounded /var/tmp growth 2018-08-28 08:11:44 -07:00
Michael Vines 9d53208d68 Use gcloud_DeleteInstances 2018-08-28 08:11:44 -07:00
Michael Vines c8e3ce26a9 Start of scripts/gcloud.sh 2018-08-27 22:35:14 -07:00
Rob Walker 51d911e3f4
Update testnet-sanity.sh 2018-08-27 15:44:10 -07:00
Michael Vines bd5c6158ae Move some common scripts from multinode-demo/ to scripts/ 2018-08-27 13:52:38 -07:00
Rob Walker 0efd64df6f no need for sudo, move ledger copy out of SNAP_DATA 2018-08-28 05:42:05 +09:00
Rob Walker 28bdf346f6 clean up after ledger sanity 2018-08-28 05:42:05 +09:00
anatoly yakovenko 738247ad44
advertise valid gossip address in drone and wallet (#1066)
* advertize valid gossip address in drone and wallet

get rid of asserts

check for valid ip address

check for valid address

ip address

* tests

* cleanup

* cleanup

* print error

* bump

* disable tests

* disable nightly
2018-08-26 11:36:27 -07:00
anatoly yakovenko f7c0d30167
Disallow localhost in deployment (#1064)
* disallow localhost in deployment

* tests

* fmt

* integration tests do not have a flag to check

* fmt
2018-08-25 21:09:18 -07:00
Greg Fitzgerald db3fb3a27c
Boot criterion (#1032)
* Revert benchmarks back to libtest

Criterion has too many dependencies, it's execution as slower, and
we didn't see the kind of precision we had hoped for to use it to
block CI builds.

* Ignore benchmarks that take more than a few milliseconds per iteration

* Revert "Ignore benchmarks that take more than a few milliseconds per iteration"

This reverts commit b87cdf6ef4a2549ec09e282014391de695473699.

* Don't run benchmarks in CI

They are already built in the nightly build. Executing them in CI
doesn't add much value until the results are precise enough to act
on.
2018-08-22 08:57:07 -06:00
Rob Walker 0b48aea937 echo commands, use PID (good form) 2018-08-21 11:41:00 +09:00
Rob Walker cdec0cead2 files have to appear in the snap 2018-08-21 11:41:00 +09:00
Rob Walker 831709ce7e fixups 2018-08-21 10:36:03 +09:00
Rob Walker b7b8a31532 make a copy of the ledger for sanity check
we can't verify a live ledger, unfortunately, fixes #985
2018-08-21 10:36:03 +09:00
Rob Walker 5aced8224f Revert "make a copy of the ledger for sanity check"
This reverts commit af20a43b77.
2018-08-21 10:34:52 +09:00
Rob Walker af20a43b77 make a copy of the ledger for sanity check
we can't verify a live ledger, unfortunately, fixes #985
2018-08-21 09:45:52 +09:00
Greg Fitzgerald 39c3280860 Don't block on large network test 2018-08-20 16:48:37 -06:00
Michael Vines a1ec549630 Pin nightly rust for more controlled updating 2018-08-20 10:55:55 -07:00
Michael Vines ecddff98f5 Add --nopull argument 2018-08-20 10:55:55 -07:00
Michael Vines 10066d67bf
Add llvm deb repository 2018-08-19 09:01:36 -07:00
Michael Vines a07f7435c6
\ 2018-08-19 08:49:29 -07:00
Michael Vines d3523ebbe5 Nightly image now derives from stable image 2018-08-19 08:47:59 -07:00
Tyera Eulberg 1bf15ae907 Temporarily disable cargo audit CI failure 2018-08-18 12:29:49 -06:00
Michael Vines 877920e61b Compute snap channel using ci/channel-info.sh 2018-08-17 23:15:48 -07:00
Michael Vines 3d1e908dad Add script to fetch latest channel info 2018-08-17 23:15:48 -07:00
Michael Vines 6880c2bef0 Exclude ci/semver_bash/; don't want to diverge from upstream 2018-08-17 23:15:48 -07:00
Michael Vines 78872ffb4b Vendor https://github.com/cloudflare/semver_bash/tree/c1133faf0e 2018-08-17 23:15:48 -07:00
Michael Vines 229d825fe0
Fix master-perf basename 2018-08-17 21:59:36 -07:00
Michael Vines edc5fc098e
Make SNAP_CHANNEL more visible in build log 2018-08-17 21:39:54 -07:00
Michael Vines 82e7725a42
Invert logic 2018-08-17 21:16:35 -07:00
Michael Vines dc61cf1c8d
Keep v0.7 snap off the edge channel 2018-08-17 21:12:10 -07:00
Michael Vines aba63e2c6c
Log expansion directive must be on its own line 2018-08-17 20:58:14 -07:00
Michael Vines c2ddd056e2
Add option to skip ledger verification 2018-08-17 20:41:30 -07:00
Rob Walker 7aeef27b99 not quite banishing build.rs, but better 2018-08-16 22:33:31 -07:00
Michael Vines 98d0ef6df5 Add some wget retries 2018-08-16 20:22:49 -07:00
Rob Walker fc0d7f5982 updated nightly versions 2018-08-16 13:17:29 -07:00
Rob Walker f697632edb update clippy install instructions, from here:
https://github.com/rust-lang-nursery/rust-clippy

fixes #947 ?
2018-08-16 13:17:29 -07:00
Rob Walker 73797c789b back to 4 TX threads 2018-08-16 12:02:11 -07:00
Rob Walker 036fcced31 test -t nproc 2018-08-16 12:02:11 -07:00
Rob Walker 1d3157fb80 fixups 2018-08-16 12:02:11 -07:00
Rob Walker 0b11c2e119 restart testnet clients in case airdrop fails 2018-08-16 12:02:11 -07:00
Michael Vines 757dfd36a3 Report errors better in build log 2018-08-14 11:44:26 -07:00
Rob Walker ab10b7676a use stable cache 2018-08-13 16:23:30 -07:00
Rob Walker fa44a71d3e move bench to a seprate, parallel step 2018-08-13 16:23:30 -07:00
Rob Walker 9e22e23ce6
increase stable timeout until tomorrow 2018-08-13 15:45:50 -07:00
Michael Vines df5cde74b0
Back out pre-0.7.1 workaround 2018-08-13 12:13:00 -07:00
Michael Vines c2ba72fe1f Fix up validator sanity 2018-08-13 10:23:35 -07:00
Michael Vines d93786c86a Revert "turn off validator sanity while I work on it"
This reverts commit d4304eea28.
2018-08-13 10:23:35 -07:00
Rob Walker 8eed120c38 add missing backslash 2018-08-10 23:45:08 -07:00
Rob Walker d4304eea28 turn off validator sanity while I work on it 2018-08-10 14:56:46 -07:00
Rob Walker 7f71e1e09f fixups 2018-08-10 11:41:31 -07:00
Rob Walker bb7eccd542 check validator startup in testnet-sanity.sh 2018-08-10 11:41:31 -07:00
Rob Walker b04c71acd9 check issue 910 in testnet-sanity 2018-08-10 11:41:31 -07:00
Rob Walker 8b41c415b7 add equal sign 2018-08-10 08:05:48 -07:00
Michael Vines 8ca514a5ca
Remove unnecessary : 2018-08-08 22:45:39 -07:00
Michael Vines b605552079 Record network version in testnet-deploy start datapoint 2018-08-08 22:41:02 -07:00
Michael Vines 74f5538bd3 Verify the ledger as a part of sanity 2018-08-08 16:10:54 -07:00
Pankaj Garg a6857dbaaa Updated node count to 230. Increased wmem on CI large 2018-08-08 13:13:18 -07:00
Michael Vines 054298d957
Retry snap install 3 times, sometimes the snap server 503s 2018-08-08 08:56:05 -07:00
Michael Vines cca240c279
Add SOLANA_NET_NAME, rename SOLANA_NET_URL to SOLANA_NET_ENTRYPOINT 2018-08-08 08:49:30 -07:00
Michael Vines 89f17ceecf
Route setup-args 2018-08-08 08:32:23 -07:00
pgarg66 75854cc234
Update dynamic network test with more nodes (#904)
- Check for correct OS params in test-large-network.sh
2018-08-08 06:52:57 -07:00
Michael Vines ecea41a0ab Install EarlyOOM on testnet nodes 2018-08-07 16:58:46 -07:00
Michael Vines e3cf1e6598 Bundle metrics_write_datapoint.sh in Snap 2018-08-07 13:35:01 -07:00
pgarg66 16772d3d51
Coalesce multiple blobs if received close together (#869)
- This helps reduce unnecessary growth of window if small blobs
  are received in small space of time
2018-08-07 10:29:57 -07:00
Michael Vines 1c38e40dee Validate ledger once all the tests complete 2018-08-07 10:00:52 -07:00
Michael Vines 91741e20fa Add rustc/cargo version check 2018-08-06 09:32:08 -07:00
Michael Vines a13058b6c4 Look for 3 nodes (1 leader, 2 validators) 2018-08-03 20:30:29 -07:00
Rob Walker c998199954 fixups, add validator-x to sanity 2018-08-03 15:34:11 -07:00
Michael Vines 15a6179b97 Stop installing rustfmt-preview, it's already present 2018-08-03 14:27:11 -07:00
Michael Vines 83b308983f Include rustfmt-preview 2018-08-03 14:11:42 -07:00
Rob Walker f2b1a04bca cargo fmt fixups 2018-08-03 11:59:25 -07:00
Michael Vines 3e36e6dcf8
Upgrade to rust 1.28 2018-08-03 11:30:40 -07:00
Michael Vines 6feb6a27be Run localnet-sanity in test-stable-perf 2018-08-03 10:46:48 -07:00
Michael Vines c230360f4c Wait until recycled machines are reachable before provisioning them 2018-08-02 22:13:17 -07:00
Michael Vines a9240a42bf Delete unreachable validators to cause a fresh one to be spawned 2018-08-02 20:45:29 -07:00
Michael Vines a7204d5353 Use a local user to avoid GCP login quota limits 2018-08-02 19:43:35 -07:00
Greg Fitzgerald 2366c1ebaf Enable cargo audit in CI
Fixes #772
2018-08-01 16:24:47 -07:00
Michael Vines 2c9d288ca9 Add a CI metric data point upload timeout to prevent CI build stalls
5 seconds is somewhat arbitrary, seems like enough
2018-08-01 15:49:48 -07:00
Greg Fitzgerald bb0aabae75 Add cmake, which is needed to build cargo-audit 2018-08-01 16:43:49 -06:00
Michael Vines 2b259aeb41 testnet now deploys successfully on days of the month < 10 2018-08-01 14:10:52 -07:00
Michael Vines de7e9b4b4c
Remove retry
This was introduced to mask the occasional failure of racy tests.  But this is misguided as it helps hid the true problem, the racy test, and it causes tries builds that fail deterministically to retry only to fail once again.
2018-08-01 12:02:39 -07:00
Michael Vines 5d936e5c8a Trap SIGINT for clean ^C shutdown 2018-07-30 17:15:50 -07:00
Michael Vines 5a40ea3fd7 Only map HOME when in CI 2018-07-30 16:36:26 -07:00
Michael Vines 102510ac0e Clear apt cache to reduce image size 2018-07-30 16:36:26 -07:00
Michael Vines 2158329058 Switch to docker-rust image 2018-07-30 16:36:26 -07:00
Michael Vines bc484ffe5f Add docker-rust image 2018-07-30 16:36:26 -07:00
Michael Vines 6fcf4584d5 Propagate more BUILDKITE environment variables into containers 2018-07-30 16:36:26 -07:00
Michael Vines 1adc83d148 Add localnet-sanity.sh 2018-07-30 16:36:26 -07:00
Michael Vines 416c141775 `export SKIP_INSTALL=1` to reset the network without reinstalling the snap 2018-07-28 18:04:13 -07:00
Michael Vines 4cdb9a73f8 Skip testnet-sanity on manual deploy 2018-07-28 12:37:29 -07:00
Michael Vines 4433730610 Add support for deploying a locally built snap 2018-07-28 12:37:29 -07:00
Michael Vines 71eb5bdecc Factor out vm_foreach 2018-07-28 12:37:29 -07:00
Michael Vines 80a02359f7 Add script to audit for security vulnerabilities 2018-07-26 13:42:12 -07:00
Michael Vines ab3968e3bf Dedup 2018-07-26 11:45:58 -07:00
Michael Vines 42ebf9502a Agent cleaning is now performed in a separate pipeline 2018-07-26 11:37:36 -07:00
Michael Vines bd4fcf4ac6 Clean out stale buildkite agent build directories 2018-07-26 11:37:36 -07:00
Michael Vines 4dceb73909 Reinstall client nodes in the background to speed up deploys 2018-07-26 09:49:00 -07:00
Rob Walker 5115cd7798 large network back to erasure 2018-07-25 20:45:16 -07:00
Michael Vines 4c4659be13 Add more stdout 2018-07-25 16:38:21 -07:00
Michael Vines 1b79fe73a1 Emit a metrics datapoint if bench-tps terminates 2018-07-25 15:55:02 -07:00
Michael Vines 5fa072cf16 Avoid quotes around net name 2018-07-25 15:55:02 -07:00
Stephen Akridge 80ecef2832 Add --sustained to ci testnet deploy script 2018-07-25 10:16:46 -07:00
Michael Vines dcce475f0b Progagate logging configuration to client nodes 2018-07-24 21:40:02 -07:00
Rob Walker 8f046cb1f8 disable erasure for large network testing 2018-07-24 16:54:52 -07:00
Michael Vines 735a0ee16d Switch back to running bench-tps in 10 minute iterations 2018-07-24 15:43:25 -07:00
Rob Walker 940caf7876 test large network with erasure 2018-07-24 13:04:34 -07:00
Michael Vines 782aa7b23b Cap at 4 threads 2018-07-24 11:35:03 -07:00
Michael Vines 7a71adaa8c Adjust threads by the number of cpus 2018-07-23 21:17:36 -07:00
Michael Vines ce8796bc2e Correctly calculate the expected number of full nodes 2018-07-23 19:55:09 -07:00
Michael Vines c7e1409f7b Not so much |set -x| 2018-07-23 19:55:09 -07:00
Michael Vines 9de9379925 Add support more more than 1 client node 2018-07-23 19:34:34 -07:00
Michael Vines 96a6e09050 Enable metrics in the TPS client 2018-07-23 16:51:39 -07:00
Michael Vines 7ebbaaeb2d Use bench-tps default duration 2018-07-22 16:26:49 -07:00
Michael Vines 5d065133ef Add data point for testnet startup and shutdown 2018-07-21 23:27:24 -07:00
Michael Vines d403808564
Restart `solana.bench-tps` every 10 minutes to work around memory leak
cc: #728
2018-07-21 19:48:13 -07:00
Michael Vines 3ffdca193d Rename client-demo to bench-tps catchup 2018-07-21 15:46:03 -07:00
Greg Fitzgerald 637f890b91 Rename client-demo to bench-tps 2018-07-21 11:36:20 -04:00
Michael Vines 009d5adcba
Tell the client to transact for 1 hour blocks 2018-07-20 17:52:19 -07:00
Michael Vines 52c55a0335
Log to /tmp/solana.log for easy runtime inspection of client activity 2018-07-20 17:45:13 -07:00
Michael Vines 120add0e82 Add support for a client node running continuous transactions on the net 2018-07-20 17:07:36 -07:00
Michael Vines 9cf0bd9b88 Adjust variable name 2018-07-20 09:50:24 -07:00
Michael Vines 9d25d7611a Protect against unsupported configurations to prevent non-obvious errors later 2018-07-20 09:47:01 -07:00
Michael Vines 1abefb2c7a Pass expected node count to testnet-sanity 2018-07-20 09:47:01 -07:00
Michael Vines 68ca9b2cb8 Generalize testnet deployment scripts 2018-07-20 09:10:28 -07:00
Michael Vines 17d927ac74 Count testnet nodes as a part of sanity 2018-07-19 12:05:21 -07:00
Michael Vines d76d3162e5
Slow down deployment more 2018-07-19 10:11:04 -07:00
Michael Vines d0a2d46923 Don't shellcheck in target/ 2018-07-19 09:41:09 -07:00
anatoly yakovenko fece91c4d1
Test multi node dynamic network ci (#696)
Buildkite automation for multinode test.  This test is ignored by default because it requires a large cpu machine to run.
2018-07-19 07:50:44 -07:00
Michael Vines b2ea2455e2 Disable rolling updates 2018-07-18 21:16:03 -07:00
Michael Vines fc1dfd86d2 Disable coverage again :-/ 2018-07-18 12:54:50 -07:00
Greg Fitzgerald 6ff46540b6 Install llvm-cov on nightly to revive coverage
Towards #433
2018-07-18 12:52:13 -04:00
Michael Vines dbab8792e4 Use real default value 2018-07-18 08:23:59 -07:00
Michael Vines 4eb676afaa Tunnel SOLANA_DEFAULT_METRICS_RATE into Snap nodes 2018-07-18 08:23:59 -07:00
Michael Vines 85d6627ee6 Deploy in one ssh login in a further attempt to avoid hitting GCP login quota 2018-07-17 20:45:52 -07:00
Michael Vines 611a005ec9 Avoid |wait| as it masks failures 2018-07-17 19:52:39 -07:00
Michael Vines 5b0418793e Keep Snap fullnode/drone logs out of syslog, we're too spammy 2018-07-17 15:08:35 -07:00
Michael Vines 2bd48b4207 Display better deploy logs 2018-07-17 09:10:55 -07:00
Michael Vines fdf94a77b4 CUDA is now configurable 2018-07-16 16:23:45 -07:00
Michael Vines 194a84c8dd Add testnet-sanity.sh 2018-07-16 12:17:39 -07:00
Michael Vines 51d932dad1 Connect validators to the right leader 2018-07-16 11:05:01 -07:00
Michael Vines 561d31cc13 Add support for master.testnet.s.c 2018-07-16 10:08:58 -07:00
Michael Vines 4e177877c9 Add more error checking, better logging, avoid hitting GCP login quota 2018-07-15 09:27:25 -07:00
Michael Vines 60848b9d95 Testnet sanity test failures will now turn the build red 2018-07-14 21:27:27 -07:00
Michael Vines cfb3736372
Update buildkite-snap.yml 2018-07-14 17:55:03 -07:00
Michael Vines e8d23c17ca
timeout++ 2018-07-14 15:51:32 -07:00
Michael Vines 751c1eba32 Run wallet-sanity against the new testnet 2018-07-13 22:21:41 -07:00
Michael Vines b0ead086a1 Fix bad copy'n'paste 2018-07-13 13:04:38 -07:00
Michael Vines 28d24497a3 Wait for the leader to initialize before starting the validators 2018-07-13 12:32:24 -07:00
Michael Vines 7105136595 Enable CUDA for the leader node 2018-07-13 11:36:12 -07:00
Michael Vines 54db379bf2 Refresh in parallel 2018-07-13 11:19:31 -07:00
Michael Vines effbf0b978 Add script to refresh testnet nodes 2018-07-13 11:19:31 -07:00
Michael Vines b538b67524
Bump timeout for stable build
When the CI build machine caches are empty stable occasionally needs more than 20m
2018-07-13 09:17:45 -07:00
Michael Vines 2b0f6355af Tagged snap builds now correctly publish to the beta channel 2018-07-12 23:43:59 -07:00
Greg Fitzgerald 0b66a6626a Use docker image's clippy 2018-07-12 09:40:40 -06:00
Greg Fitzgerald e8be4d7eae Add clippy to CI 2018-07-12 09:40:40 -06:00
Michael Vines 84c28a077a Use custom rust nightly image with cargo-cov and clippy pre-installed 2018-07-12 07:25:56 -06:00
Greg Fitzgerald e3882950cf Run benchmarks from Rust stable CI 2018-07-11 11:18:18 -06:00
Michael Vines 0a34cb8023 Include hh:mm in image name 2018-07-09 23:07:07 -06:00
Michael Vines 227aa38c8a Add image --family arg 2018-07-09 23:02:46 -06:00
Anatoly Yakovenko 3a90f138b2 dynamit network test
* cleaned up fullnode api
* added debug_id to ReplicatedData and crdt for debugging
2018-07-03 14:00:39 -06:00
Michael Vines a9e0b27772 Speed up snap build
1. Use pre-installed host rust toolchain
2. Build reference/performance fullnode in same part to avoid rebuilding libraries
3. Merge scripts into same part
2018-07-01 17:47:51 -07:00
Michael Vines fba494343f Save/restore target/ directory between builds 2018-06-30 22:30:57 -07:00
Michael Vines 0b878eccf8 Map HOME to grant docker containers access to the ~/.cargo registry cache 2018-06-30 21:50:15 -07:00
Michael Vines 98772b16d6 Generalize solana-snap build trigger 2018-06-30 21:50:15 -07:00
Michael Vines bb82ff0c80 Don't wanna wait 2018-06-30 20:05:27 -07:00
Michael Vines 71af03dc98 Skip snap build for PRs if nothing under snap/ is modified
Additionally relegate push snap build to a secondary CI pipeline
2018-06-30 20:05:27 -07:00
Michael Vines d63493a852
Grant the snap build more time 2018-06-30 12:20:22 -07:00
Michael Vines 4ffb5d157a Disable coverage until issue #433 is resolved 2018-06-29 17:36:26 -07:00
Stephen Akridge a094507bb8 Lower default benchmarking numbers to make CI timeout 2018-06-29 07:14:47 -06:00
Rob Walker a5ce578c72 ... 2018-06-26 16:23:41 -06:00
Michael Vines ce361c2cdc Add Snap fullnode daemon 2018-06-26 12:32:33 -07:00
Michael Vines be5f2ef9b9 Consolidate CI jobs 2018-06-24 22:28:24 -07:00
Greg Fitzgerald adfcb79387 Force install cargo-cov 2018-06-24 15:34:30 -06:00
Greg Fitzgerald 73c4c0ac5f Revert "cargo-cov installed by default in nightly?"
This reverts commit 6fc601f696.
2018-06-24 15:34:30 -06:00
Greg Fitzgerald 6fc601f696 cargo-cov installed by default in nightly? 2018-06-24 12:17:42 -06:00
Greg Fitzgerald 07111fb7bb Use llvm-cov instead of gcov
@marco-c called this a hack, but since grcov isn't working
out-of-the-box (panics on call to gcov), we'll take a stab at using
llvm-cov.
2018-06-24 12:17:42 -06:00
Greg Fitzgerald a06d2170b0 No need for rustfmt on nightly 2018-06-24 12:17:42 -06:00
Greg Fitzgerald 7f53ea3bf3 Generate coverage with Rust nightly
Fixes #177

Thanks @marco-c!
2018-06-24 12:17:42 -06:00
Michael Vines b2accd1c2a Run snap build sooner to better mask the delay 2018-06-24 10:24:32 -07:00
Michael Vines 33bbfdbc9b Retry flaky coverage/cuda builds on initial failure 2018-06-23 16:17:25 -07:00
Michael Vines 5de54f8853 Make cuda/erasure build logs public 2018-06-23 16:17:25 -07:00
Michael Vines e4782b19a3 Document GCP setup 2018-06-23 02:12:20 -07:00
Michael Vines ec86b1dffa Adapt to GCP-based CI 2018-06-23 02:12:20 -07:00
Michael Vines 30e03feb5f Add initial CI subsystem documentation 2018-06-22 15:30:29 -07:00
Michael Vines 17e8ad110f Temporarily disable failing CI to get back to green 2018-06-22 11:29:31 -07:00
Michael Vines c4a59896f8 Run test-erasure in a container 2018-06-21 13:00:40 -07:00
Michael Vines 3eb1608403 Skip --user if SOLANA_DOCKER_RUN_NOSETUID is set 2018-06-21 12:24:52 -07:00
Michael Vines 8fde70d4dc Erasure tests do not require a CUDA agent 2018-06-21 11:42:37 -07:00
Michael Vines 5a047833ed Run snap publishing directly on CUDA agent
This is necessary until we build a docker image that also contains a CUDA
installation
2018-06-21 11:42:37 -07:00
Michael Vines f6c28e6be1 Update snapcraft docker image contain snapcraft 2.42.1 2018-06-21 11:42:37 -07:00
Michael Vines 89fc0ad7a9 Add convenience script to download performance libraries 2018-06-20 16:48:32 -07:00
Greg Fitzgerald 1c97bf50b6 Fix nightly
No longer ignore failures in the nightly build.
2018-06-19 17:38:04 -07:00
Michael Vines e592243a09 De-double quote 2018-06-19 13:20:47 -07:00
Michael Vines 85b6e7293c Add cleanup script to manage build agent disk space 2018-06-19 12:22:45 -07:00
Michael Vines 75997e6c08 Allow BUILDKITE_BRANCH in containers 2018-06-18 22:51:30 -07:00
Michael Vines 9040d00110 Package solana as a snap 2018-06-18 17:36:03 -07:00
Michael Vines d4807790ff Add snapcraft login credentials
This file was created as follows:
$ snapcraft export-login --snaps solana --channels beta,edge snapcraft.credentials
$ openssl aes-256-cbc -e -in snapcraft.credentials -out snapcraft.credentials.enc
2018-06-18 17:36:03 -07:00
Michael Vines 3f763f99e2 Fail fast in CI when |cargo fmt| says no 2018-06-15 17:10:00 -07:00
Grimes efb23f7cf9 Ensure stuck builds eventually timeout 2018-06-07 19:08:03 -07:00
Grimes 013f4674de Target cuda agents 2018-06-07 19:08:03 -07:00
Stephen Akridge 074c9af020 Shellcheck again 2018-06-05 15:32:25 -06:00
Stephen Akridge 2da2d4e365 More shellcheck 2018-06-05 15:32:25 -06:00
Stephen Akridge 8eb76ab2a5 Fix shellcheck 2018-06-05 15:32:25 -06:00
Stephen Akridge 0339f90b40 Fix gf-complete url and symlinks 2018-06-05 15:32:25 -06:00
Stephen Akridge 6843b71a0d Debug erasure ci script 2018-06-05 15:32:25 -06:00
Stephen Akridge 634408b5e8 Add erasure build to ci 2018-06-05 15:32:25 -06:00
Stephen Akridge b0eab8729f Add erasure ci script 2018-06-05 15:32:25 -06:00
Greg Fitzgerald 3aa2907bd6 Restore shellcheck 2018-06-01 13:43:38 -06:00
Anatoly Yakovenko 9ac7070e08 fix ci 2018-05-30 14:04:48 -06:00
Anatoly Yakovenko 46059a37eb skip shell check 2018-05-30 14:04:48 -06:00
Grimes 86a50ae9e1 Add RUST_BACKTRACE 2018-05-28 22:23:25 -07:00
Grimes 26153d9919 Avoid docker buildkite plugin, which is not supported by bkrun 2018-05-28 22:23:25 -07:00
Grimes 5af922722f Add local buildkite CI runner 2018-05-28 22:23:25 -07:00
Grimes b70d730b32 Support local .a, skip if unable to find .a 2018-05-28 22:23:25 -07:00
Grimes bf4b856e0c Don't fail if CODECOV_TOKEN is undefined 2018-05-28 22:23:25 -07:00
Grimes 0cf0ae6755 s/label:/name:/g 2018-05-28 22:23:25 -07:00
Grimes 29061cff39 Delint existing shell scripts 2018-05-28 05:18:46 -06:00
Grimes b7eec4c89f Lint shell scripts in CI 2018-05-28 05:18:46 -06:00
Grimes 6e35f54738 Simplify environment blocks 2018-05-26 14:38:26 -07:00
Grimes 089294a85e 'ignored' step failures are no longer ignored 2018-05-26 11:00:20 -07:00
Grimes 25c0b44641 Run ignored build step in docker 2018-05-26 11:00:20 -07:00
Greg Fitzgerald 75659ca042 Light up coverage build 2018-05-26 00:36:50 -06:00
Greg Fitzgerald 8d26be8b89 Run benchmarks in nightly
And name functions the same way as test functions
2018-05-26 00:36:50 -06:00
Greg Fitzgerald af4e95ae0f Only check formatting in stable build 2018-05-26 00:36:50 -06:00
Greg Fitzgerald b43ae748c3
Update publish.sh 2018-05-25 16:08:14 -06:00
Greg Fitzgerald 5a45eef1dc
Exit cleanup (#252)
* Ignore record_stage exit reason. We only really care about panic exit versus graceful exit.
* Ignore coverage build in CI
2018-05-24 10:03:17 -06:00
Michael Vines ae29641a18 Run most CI steps in docker 2018-05-22 23:16:25 -07:00
Michael Vines 34514d65bc Add in-tree buildkite pipeline 2018-05-21 23:43:27 -07:00