Commit Graph

553 Commits

Author SHA1 Message Date
Michael Vines cf6f344ccc Add CUDA_HOME env var to permit overriding the CUDA install location 2018-11-11 16:24:18 -08:00
Greg Fitzgerald b670b9bcde Regenerate identity files in CI 2018-11-11 09:22:52 -07:00
Greg Fitzgerald cdf1a96e23 Revert "V1 Window/Ledger based on RocksDb (#1712)"
This reverts commit bfcdec95cb.
2018-11-09 20:25:53 -07:00
carllin bfcdec95cb
V1 Window/Ledger based on RocksDb (#1712)
* Add rocksdb

* Implement new ledger module based on RocksDb
2018-11-09 18:30:26 -08:00
Greg Fitzgerald 0926702269 Fix grcov download on macos and upload gcda/gcdo files for debugging 2018-11-09 11:19:28 -07:00
Greg Fitzgerald 0a85347a0d Upgrade Rust stable to 1.30.1
Fixes `cargo doc`
2018-11-09 07:46:51 -08:00
Michael Vines eaa8b9cb1e Publish book 2018-11-09 02:13:59 -07:00
Michael Vines b0405db5a9
Assign static IPs to {edge,beta}.testnet.solana.com 2018-11-07 20:11:00 -08:00
Michael Vines f34f0af6b1
Install native programs in the correct location 2018-11-07 19:44:57 -08:00
Michael Vines 22b6cbb4da
Switch testnet to AWS 2018-11-07 18:57:08 -08:00
Michael Vines 25dd5145bb
Switch to us-west-1a, us-west-1b is causing trouble 2018-11-07 18:23:28 -08:00
Michael Vines 3b41eec199
Shuffle AWS regions 2018-11-07 15:00:55 -08:00
Michael Vines b02b636b36
Support local tarball deploys 2018-11-07 14:44:40 -08:00
Michael Vines a537154c28
Remove all cuda dependencies from release tarball beyond solana-fullnode-cuda 2018-11-07 14:44:40 -08:00
Michael Vines 812a8bcc6c
Permit release tag tarballs 2018-11-07 10:33:58 -08:00
Michael Vines 63807935cb
Switch testnet/testnet-beta to tarball release 2018-11-07 10:30:02 -08:00
Michael Vines 92a8b646df
Fix tarball publishing for tags 2018-11-07 10:26:19 -08:00
Michael Vines d9f9e347ab
Delete testnet-master, testnet-master-perf 2018-11-07 10:08:29 -08:00
Greg Fitzgerald 038a46b5ef Integrate the markdown book into the codebase
This implies that the book should describe exactly what is implemented,
and will not lead the way and eventually bitrot as the RFCs do.
2018-11-07 10:58:47 -07:00
Michael Vines 863a0c3f8f
s/edge/beta/ 2018-11-07 08:54:32 -08:00
Michael Vines dd4fb7aa90 Add AWS-based nets 2018-11-07 07:47:39 -08:00
Michael Vines 2af5aad032 Switch testnet/testnet-perf to the latest beta or stable tag 2018-11-07 07:47:39 -08:00
Michael Vines 9027141ff8 Publish release tarballs for tags 2018-11-07 07:47:39 -08:00
Greg Fitzgerald 8be7c13d2d
Stub out architecture book (#1674)
* Stub out architecture documentation

* Add book HTML generation and book tests to CI

* Add heading

* Better table of contents

* Reference existing documentation

Move ASCII art from code comments into rendered SVG

* Attempt to fix CI

* Add lamport docs

And truncate lines to 80 characters

* Fix links

And reference shorter, newer description of PoH.

* Replace ASCII art with SVG

* Streamline for Pillbox

* Update path before optional install

* Use $CARGO_HOME instead of $HOME

* Delete code

Attempt to describe all data structures without code.

* Boot RPU from docs, add JsonRpcService

Also, use Rust naming conventions in the block diagrams to
minimize the jump from docs to code.

* Latest code uses tick_height

* Rename bob/ folder to art/

A home for any ASCII art

* Import JSON RPC API

* More mdbook docs

* Add Ncp

* Cleanup links

* Move pipelining description into fullnode description

* Move high-level transaction docs into top-level doc

* Delete unused files
2018-11-06 18:00:58 -07:00
Michael Vines d8d23c9971
Remove unused debug trace 2018-11-06 09:29:39 -08:00
Michael Vines 6b5d12a8bb
Set metrics database correctly 2018-11-06 07:25:18 -08:00
Michael Vines c4b9d5d8b9
Remove stray line 2018-11-05 20:53:34 -08:00
Michael Vines 52491b467a
Update testnet deploy docs 2018-11-05 19:12:55 -08:00
Michael Vines 7789fda016 Add testnet-manager pipeline 2018-11-05 17:35:30 -08:00
Pankaj Garg c4346e6191
Add testnet pipeline for prebuilt images (#1708)
* Add testnet pipeline for prebuilt images

- It'll speed up testnet testing for released images

* removed quotes from variable

* address review comments

* fix testnet automation error
2018-11-05 13:50:33 -08:00
Michael Vines 5a85cc4626
Rename buildkite-snap to buildkite-secondary 2018-11-05 08:47:51 -08:00
Michael Vines 2c74815cc9
ci: correct crates.io publishing order 2018-11-02 15:39:24 -07:00
carllin 298bd6479a
Add first leader to genesis (#1681)
* Add first leader to genesis entries, consume in genesis.sh

* Set bootstrap leader in the bank on startup, remove instantiation of bootstrap leader from bin/fullnode

* Remove need to initialize bootstrap leader in leader_scheduler, now can be read from genesis entries

* Add separate interface new_with_leader() in mint for creating genesis leader entries
2018-11-02 14:32:05 -07:00
Pankaj Garg db98f7e0b4
Use env variables to disable validator sanity and ledger verification (#1675) 2018-10-31 12:30:33 -07:00
jackcmay 0529f36fde
Run workspace member's tests (#1666)
Run workspace member's tests
2018-10-30 22:53:36 -07:00
Pankaj Garg 85869552e0
Update testnet scripts to use release tar ball (#1660)
* Update testnet scripts to use release tar ball

* use curl instead of s3cmd
2018-10-30 18:05:38 -07:00
Pankaj Garg 6f9843c14b
Publish a tarball of Solana release binaries (#1656)
* Publish a tarball of solana release binaries

* included native programs in Solana release tar

* Remove PR check from publish script
2018-10-30 15:31:52 -07:00
Michael Vines 37213209c5 Create programs/bpf/c/sdk/ 2018-10-29 19:10:29 -07:00
Michael Vines f6b1b5ab37
Remove unnecessary checks 2018-10-29 13:27:52 -07:00
Michael Vines f12743de38 Create/publish bpf-sdk tarball 2018-10-29 12:54:57 -07:00
Michael Vines 77e10ed757 Add utility to figure the current crate version 2018-10-29 12:54:57 -07:00
Michael Vines f671b7f63f
Publish root crate too 2018-10-25 17:16:18 -07:00
Michael Vines a340b18b19 Upgrade to rust 1.30 2018-10-25 17:13:41 -07:00
Michael Vines 160cff4a30
Check for TRIGGERED_BUILDKITE_TAG 2018-10-25 16:37:54 -07:00
Michael Vines 1733beabf7 mv common/ sdk/ 2018-10-25 13:26:10 -07:00
Michael Vines 471d8f6ff9 Fix up the version references to all other internal crates 2018-10-25 12:54:32 -07:00
Michael Vines b982595c73 Add version check and rustup 2018-10-24 19:48:58 -07:00
Rob Walker 4ea422bcec run integration tests serially 2018-10-17 11:37:10 -07:00
Greg Fitzgerald 6074e4f962 Attempt to stabilize the test suite
The integration tests are allowed to open sockets, so running them
in parallel may cause "Too many open files" errors. This patch
runs the unit tests in parallel and the integration test serially.
2018-10-17 11:37:10 -07:00
Greg Fitzgerald 5b4d4b97bc Upgrade to latest stable Rust, 1.29.2 2018-10-15 09:54:24 -06:00
Michael Vines c5180c8092 Permit RUST_LOG overrides 2018-10-14 12:40:37 -07:00
Pankaj Garg e5ab9a856c
Upload bench output as build artifacts (#1478)
* Upload bench output as build artifacts

* Fix tags types

* Pull previous stats from metrics

* Change the default branch for comparison

* Fix formatting

* Fix build errors

* Address review comments

* Dedup some common code

* Add eval for channel info to find branch name
2018-10-12 15:13:10 -07:00
Michael Vines 5c523716aa Ship native programs 2018-10-10 16:49:48 -07:00
Pankaj Garg c240bb12ae Change buildkite agent for testnet automation 2018-10-09 15:04:55 -07:00
Pankaj Garg 29a8823db1
Env variables for testnet-automation parameters (#1455)
- This will enable us to create custom pipelines for field events
2018-10-09 11:50:56 -07:00
Pankaj Garg a80955eacb
Change format of data for TPS/Finality metrics in testnet automation (#1446)
* Change format of data for TPS/Finality metrics in testnet automation

* Revert number of nodes for testnet automation

* Split python command to its own script

* Fix python command line arguments
2018-10-09 10:35:01 -07:00
Pankaj Garg 0a39722719
Add support to trigger testnet from a PR (#1434)
* Add support for different node counts

* Update variable names

* Delete network even after failures

* Add array for node counts

* Changed number of nodes to a space separated string of numbers

* Adjust number of nodes

* Snap will not be published if the env variable DO_NOT_PUBLISH_SNAP is set

* Address review comments

* Replaced influx db URL
2018-10-05 16:32:05 -07:00
Greg Fitzgerald 9c0fa4d1d2
Upload coverage HTML reports (#1421)
Uploads two reports to Buildkite, one from cargo-cov and one from lcov via grcov.  The lcov one is busted on linux and is what we need to bring codecov.io back up again. It works great on macos if you wanted to generate them locally and prefer lcov HTML reports.

* Also comment out non-coverage build to speed things up.
2018-10-05 10:17:35 -07:00
Michael Vines beb4536841
Run a fullnode+drone automatically when the container starts up 2018-10-02 18:09:35 -07:00
Michael Vines ad5fcf778f Publish minimal Solana docker images to dockerhub 2018-10-02 16:57:48 -07:00
Michael Vines 83b000ae88 Remove SNAP_ prefix 2018-10-02 16:57:48 -07:00
Greg Fitzgerald 94459deb94 Disable codecov.io reporting 2018-09-28 19:19:16 -07:00
Greg Fitzgerald 660af84b8d Use the same versions of llvm-cov and libprofile 2018-09-28 19:19:16 -07:00
Greg Fitzgerald 7b31020903 Add back llvm-dev for llvm-cov 2018-09-28 19:19:16 -07:00
Greg Fitzgerald 9a4143b4d9 Upgrade llvm-dev and boot kcov
Need clang-dev, not llvm-dev because cargo-cov looks for libprofile
in a clang installation directory.
2018-09-28 19:19:16 -07:00
Greg Fitzgerald aebc47ad55 Attempt coverage reporting 2018-09-28 19:19:16 -07:00
Greg Fitzgerald 5bc01cd51a Revive code coverage 2018-09-28 19:19:16 -07:00
Greg Fitzgerald 19a7ff0c43 Pin down nightly in benchmark build 2018-09-28 19:29:50 -06:00
Greg Fitzgerald 5f18403199 Upgrade nightly 2018-09-28 19:29:50 -06:00
Tyera Eulberg 9f325fca09 Re-enable cargo audit 2018-09-28 17:53:41 -06:00
Tyera Eulberg 10d08acefa Reenable cargo audit 2018-09-28 17:53:41 -06:00
Tyera Eulberg 52d50e6bc4 Update for new solana-jsonrpc 2018-09-28 17:53:41 -06:00
Stephen Akridge aa2a3fe201 Add chacha module to encrypt ledger files 2018-09-28 14:23:37 -07:00
Greg Fitzgerald 485ba093b3 Install kcov to CI environment 2018-09-28 11:20:27 -06:00
Michael Vines 8d92232949
Specify zone 2018-09-28 07:32:49 -07:00
Michael Vines e4d8c094a4
Include -z when deleting network 2018-09-27 21:27:09 -07:00
Greg Fitzgerald 675ff64094 Fail CI on clippy warnings 2018-09-27 16:21:12 -06:00
Greg Fitzgerald f9fe6a0f72 Move clippy to Rust stable 2018-09-27 16:21:12 -06:00
Greg Fitzgerald 8d007bd7f7 Upgrade rustc and add clippy to stable 2018-09-27 16:21:12 -06:00
Stephen Akridge 6cdbdfbbcb Enable bench and fix upload-perf 2018-09-27 14:16:56 -07:00
Pankaj Garg 35e6343d61
Update testnet-deploy script to configure GPUs for leader node (#1379) 2018-09-27 13:42:24 -07:00
Michael Vines afc764752c Permit testnets without a GPU 2018-09-26 10:37:41 -07:00
sakridge 3e76efe97e
Fix bench compilation (#1311) 2018-09-24 10:40:42 -07:00
Michael Vines e5e325154b Add --shell argument 2018-09-24 08:05:47 -07:00
Michael Vines 67f1fbab5f Treat rustc warnings as errors in CI 2018-09-20 19:46:20 -07:00
sakridge 2981076a14
Add solana-upload-perf to parse json from bench and upload to influx (#1166) 2018-09-19 13:16:55 -07:00
Michael Vines 3ca80c676c
Disable large-network until it's fixed 2018-09-14 20:13:17 -07:00
Michael Vines a344eb7dd0 Upgrade rust stable to 1.29 2018-09-14 16:37:49 -07:00
Michael Vines f06113500d bench-tps/net sanity: add ability to check for unexpected extra nodes 2018-09-12 15:38:57 -07:00
Rob Walker 97c3125a78
improve localnet-sanity's robustness (#1160)
* fix poll_gossip_for_leader() loop to actually wait
         for 30 seconds
    * reduce reuseaddr use to only when necessary,
         try to avoid already bound sockets
    * move nat.rs to netutil.rs
    * add gossip tracing to thin_client and bench-tps
2018-09-09 04:50:43 +09:00
Michael Vines a77aca75b2
Add NO_VALIDATOR_SANITY back 2018-09-07 22:37:05 -07:00
Michael Vines b1b03ec13b Refine docker image tagging to avoid breaking stabilization branches on updates 2018-09-07 18:42:25 -07:00
Rob Walker 51b27779c9
client changes for TODOs and looping (#1138)
* remove client.sh from snap
* default to ephemeral instead of ~/.config key
* rework CLI for bench-tps
* remote multinode-demo stuff from remote-client.sh
* remove multinode-demo from remote-sanity and localnet-sanity
2018-09-08 07:07:10 +09:00
Michael Vines 0d945e6a92 Groom testnet-sanity logging 2018-09-07 12:45:48 -07:00
Michael Vines e51445d857
🙃 2018-09-07 12:24:34 -07:00
Michael Vines ee682d5bc3 Move wallet-sanity.sh out of multinode-demo/ 2018-09-07 12:01:43 -07:00
Michael Vines 506a81e8cc Assume -y 2018-09-07 12:01:43 -07:00
Michael Vines 88298bf321 Add -n option 2018-09-07 12:01:43 -07:00
Michael Vines 533ebc17f2
Install multilog automatically on a CI machine 2018-09-07 11:56:23 -07:00
Michael Vines e088833b81
s/create/start/ 2018-09-06 21:07:11 -07:00
Michael Vines b1bacf12a6
Add some log sections 2018-09-06 20:38:11 -07:00
Michael Vines 66ff602659 Rewrite ci/testnet-{deploy,sanity}.sh in terms of net/ primitives 2018-09-06 19:54:39 -07:00
Michael Vines 072d0b67e4
Send deploy metrics to the testnet-specific database 2018-09-06 08:30:03 -07:00
Michael Vines 6560b0e2cc s/whoami/id -un/ 2018-09-05 14:26:21 -07:00
Michael Vines 449d7042f0 Configure metrics correctly 2018-09-05 09:02:02 -07:00
Michael Vines 43b4207101 Run oom-monitor in net/ testnets 2018-09-05 09:02:02 -07:00
Michael Vines c81c19234f Improve incremental speed of docker cargo builds outside of CI 2018-09-05 09:02:02 -07:00
Greg Fitzgerald cf2626a1c5 Update instructions to upgrade nightly docker image 2018-09-04 20:56:40 -06:00
Greg Fitzgerald 68c72d6f34 Fix nightly build 2018-09-04 20:56:40 -06:00
Greg Fitzgerald 65f78905cd Install cargo-cov on latest nightly 2018-09-04 20:56:40 -06:00
Michael Vines b4a7a18334
Update README.md 2018-09-04 13:29:00 -07:00
Rob Walker 61abee204f
don't check for snap mode in common.sh, is only relevant to snap daemons (#1113)
snap mode is for daemons, remove it from client (i.e. common.sh)

supply leader info to client via snap
2018-09-04 14:31:54 +09:00
Rob Walker e834c76b40 --count => --num-nodes 2018-09-04 07:07:25 +09:00
Rob Walker d1f1cbe88f leader-address=>leader-ip 2018-09-01 04:38:18 +09:00
Rob Walker a4e7b6e90c more fixups for client.sh changes 2018-09-01 03:33:21 +09:00
Rob Walker fbc7c9c431 fix client_start to deal with new client.sh 2018-09-01 03:23:05 +09:00
Rob Walker 8b248dcf09 specify port 2018-09-01 02:56:24 +09:00
Rob Walker 4938aad939 fixups 2018-09-01 02:21:46 +09:00
Rob Walker 7e882dfe62 inform all snaps where the network is 2018-09-01 02:21:46 +09:00
Rob Walker 9d1eb4f9ea remove 'localhost' leader (redundant, un-dig-friendly) 2018-08-31 23:21:07 +09:00
Rob Walker 176e806d94 rework of netwrk rendezvous
* rename NodeInfo field of Node from "data" to "info"
      (touches a lot of files)

  * update client to use gossip to find leader, a la drone

  * rework multinode scripts
      * move more stuff into rust
      * added usage to all
      * no more rsync unless you're a validator (TODO: whack that, too)
  * fullnode doesn't bail if drone isn't up yet, just keeps trying
  * drone doesn't bail if network isn't up yet, just keeps trying
2018-08-31 23:21:07 +09:00
Rob Walker 63e44dcc35 continue rendezvous refactor for gossip and repair
* remove trailing whitespace in ci/audit.sh

  * code review fixups
     * rename GOSSIP_PORT_RANGE => SOLANA_PORT_RANGE
     * remove out-of-date TODO in localnet-sanity.sh

  * remove features=test and code that was using it (localhost prohibitions in
      crdt) added TODO in crdt.rs, maybe we should boot localhost in production
      networks?

  * boot tvu_window from NodeInfo: instead, send repair requests from the repair
      socket (to gossip on peer) and answer repair requests via the sockaddr
      from the repair request

  * remove various unused pub functions

  * banish SocketAddr parse().unwrap() to a macro that can also accept simpler stuff
2018-08-31 23:21:07 +09:00
Rob Walker 1af4cee63b fix #1079
* move gossip/NCP off assuming anything about its address
  * use a single socket to send and receive gossip
  * remove --addr/-a from CLIs
  * rearrange networking utility code
  * use Arc<UdpSocket> to share the Sync-safe UdpSocket among threads
  * rename TestNode to Node

TODO:

  * re-enable 127.0.0.1 as a valid address in crdt
  * change repair request/response to a similar, single socket
  * pick cloned sockets or Arc<UdpSocket> for all these (rpu uses tryclone())
  * update contact_info with network truthiness instead of what the node
      says?
2018-08-31 23:21:07 +09:00
Rob Walker cb52a335bd re-enable localnet-sanity 2018-08-31 23:21:07 +09:00
Michael Vines 73a0bf8d30 Avoid unbounded /var/tmp growth 2018-08-28 08:11:44 -07:00
Michael Vines 9d53208d68 Use gcloud_DeleteInstances 2018-08-28 08:11:44 -07:00
Michael Vines c8e3ce26a9 Start of scripts/gcloud.sh 2018-08-27 22:35:14 -07:00
Rob Walker 51d911e3f4
Update testnet-sanity.sh 2018-08-27 15:44:10 -07:00
Michael Vines bd5c6158ae Move some common scripts from multinode-demo/ to scripts/ 2018-08-27 13:52:38 -07:00
Rob Walker 0efd64df6f no need for sudo, move ledger copy out of SNAP_DATA 2018-08-28 05:42:05 +09:00
Rob Walker 28bdf346f6 clean up after ledger sanity 2018-08-28 05:42:05 +09:00
anatoly yakovenko 738247ad44
advertise valid gossip address in drone and wallet (#1066)
* advertize valid gossip address in drone and wallet

get rid of asserts

check for valid ip address

check for valid address

ip address

* tests

* cleanup

* cleanup

* print error

* bump

* disable tests

* disable nightly
2018-08-26 11:36:27 -07:00
anatoly yakovenko f7c0d30167
Disallow localhost in deployment (#1064)
* disallow localhost in deployment

* tests

* fmt

* integration tests do not have a flag to check

* fmt
2018-08-25 21:09:18 -07:00
Greg Fitzgerald db3fb3a27c
Boot criterion (#1032)
* Revert benchmarks back to libtest

Criterion has too many dependencies, it's execution as slower, and
we didn't see the kind of precision we had hoped for to use it to
block CI builds.

* Ignore benchmarks that take more than a few milliseconds per iteration

* Revert "Ignore benchmarks that take more than a few milliseconds per iteration"

This reverts commit b87cdf6ef4a2549ec09e282014391de695473699.

* Don't run benchmarks in CI

They are already built in the nightly build. Executing them in CI
doesn't add much value until the results are precise enough to act
on.
2018-08-22 08:57:07 -06:00
Rob Walker 0b48aea937 echo commands, use PID (good form) 2018-08-21 11:41:00 +09:00
Rob Walker cdec0cead2 files have to appear in the snap 2018-08-21 11:41:00 +09:00
Rob Walker 831709ce7e fixups 2018-08-21 10:36:03 +09:00
Rob Walker b7b8a31532 make a copy of the ledger for sanity check
we can't verify a live ledger, unfortunately, fixes #985
2018-08-21 10:36:03 +09:00
Rob Walker 5aced8224f Revert "make a copy of the ledger for sanity check"
This reverts commit af20a43b77.
2018-08-21 10:34:52 +09:00
Rob Walker af20a43b77 make a copy of the ledger for sanity check
we can't verify a live ledger, unfortunately, fixes #985
2018-08-21 09:45:52 +09:00
Greg Fitzgerald 39c3280860 Don't block on large network test 2018-08-20 16:48:37 -06:00
Michael Vines a1ec549630 Pin nightly rust for more controlled updating 2018-08-20 10:55:55 -07:00
Michael Vines ecddff98f5 Add --nopull argument 2018-08-20 10:55:55 -07:00
Michael Vines 10066d67bf
Add llvm deb repository 2018-08-19 09:01:36 -07:00
Michael Vines a07f7435c6
\ 2018-08-19 08:49:29 -07:00
Michael Vines d3523ebbe5 Nightly image now derives from stable image 2018-08-19 08:47:59 -07:00
Tyera Eulberg 1bf15ae907 Temporarily disable cargo audit CI failure 2018-08-18 12:29:49 -06:00
Michael Vines 877920e61b Compute snap channel using ci/channel-info.sh 2018-08-17 23:15:48 -07:00
Michael Vines 3d1e908dad Add script to fetch latest channel info 2018-08-17 23:15:48 -07:00
Michael Vines 6880c2bef0 Exclude ci/semver_bash/; don't want to diverge from upstream 2018-08-17 23:15:48 -07:00
Michael Vines 78872ffb4b Vendor https://github.com/cloudflare/semver_bash/tree/c1133faf0e 2018-08-17 23:15:48 -07:00
Michael Vines 229d825fe0
Fix master-perf basename 2018-08-17 21:59:36 -07:00
Michael Vines edc5fc098e
Make SNAP_CHANNEL more visible in build log 2018-08-17 21:39:54 -07:00
Michael Vines 82e7725a42
Invert logic 2018-08-17 21:16:35 -07:00
Michael Vines dc61cf1c8d
Keep v0.7 snap off the edge channel 2018-08-17 21:12:10 -07:00
Michael Vines aba63e2c6c
Log expansion directive must be on its own line 2018-08-17 20:58:14 -07:00
Michael Vines c2ddd056e2
Add option to skip ledger verification 2018-08-17 20:41:30 -07:00
Rob Walker 7aeef27b99 not quite banishing build.rs, but better 2018-08-16 22:33:31 -07:00
Michael Vines 98d0ef6df5 Add some wget retries 2018-08-16 20:22:49 -07:00
Rob Walker fc0d7f5982 updated nightly versions 2018-08-16 13:17:29 -07:00
Rob Walker f697632edb update clippy install instructions, from here:
https://github.com/rust-lang-nursery/rust-clippy

fixes #947 ?
2018-08-16 13:17:29 -07:00
Rob Walker 73797c789b back to 4 TX threads 2018-08-16 12:02:11 -07:00
Rob Walker 036fcced31 test -t nproc 2018-08-16 12:02:11 -07:00
Rob Walker 1d3157fb80 fixups 2018-08-16 12:02:11 -07:00
Rob Walker 0b11c2e119 restart testnet clients in case airdrop fails 2018-08-16 12:02:11 -07:00
Michael Vines 757dfd36a3 Report errors better in build log 2018-08-14 11:44:26 -07:00
Rob Walker ab10b7676a use stable cache 2018-08-13 16:23:30 -07:00
Rob Walker fa44a71d3e move bench to a seprate, parallel step 2018-08-13 16:23:30 -07:00
Rob Walker 9e22e23ce6
increase stable timeout until tomorrow 2018-08-13 15:45:50 -07:00
Michael Vines df5cde74b0
Back out pre-0.7.1 workaround 2018-08-13 12:13:00 -07:00
Michael Vines c2ba72fe1f Fix up validator sanity 2018-08-13 10:23:35 -07:00
Michael Vines d93786c86a Revert "turn off validator sanity while I work on it"
This reverts commit d4304eea28.
2018-08-13 10:23:35 -07:00
Rob Walker 8eed120c38 add missing backslash 2018-08-10 23:45:08 -07:00
Rob Walker d4304eea28 turn off validator sanity while I work on it 2018-08-10 14:56:46 -07:00
Rob Walker 7f71e1e09f fixups 2018-08-10 11:41:31 -07:00
Rob Walker bb7eccd542 check validator startup in testnet-sanity.sh 2018-08-10 11:41:31 -07:00
Rob Walker b04c71acd9 check issue 910 in testnet-sanity 2018-08-10 11:41:31 -07:00
Rob Walker 8b41c415b7 add equal sign 2018-08-10 08:05:48 -07:00
Michael Vines 8ca514a5ca
Remove unnecessary : 2018-08-08 22:45:39 -07:00
Michael Vines b605552079 Record network version in testnet-deploy start datapoint 2018-08-08 22:41:02 -07:00
Michael Vines 74f5538bd3 Verify the ledger as a part of sanity 2018-08-08 16:10:54 -07:00
Pankaj Garg a6857dbaaa Updated node count to 230. Increased wmem on CI large 2018-08-08 13:13:18 -07:00
Michael Vines 054298d957
Retry snap install 3 times, sometimes the snap server 503s 2018-08-08 08:56:05 -07:00
Michael Vines cca240c279
Add SOLANA_NET_NAME, rename SOLANA_NET_URL to SOLANA_NET_ENTRYPOINT 2018-08-08 08:49:30 -07:00
Michael Vines 89f17ceecf
Route setup-args 2018-08-08 08:32:23 -07:00
pgarg66 75854cc234
Update dynamic network test with more nodes (#904)
- Check for correct OS params in test-large-network.sh
2018-08-08 06:52:57 -07:00
Michael Vines ecea41a0ab Install EarlyOOM on testnet nodes 2018-08-07 16:58:46 -07:00
Michael Vines e3cf1e6598 Bundle metrics_write_datapoint.sh in Snap 2018-08-07 13:35:01 -07:00
pgarg66 16772d3d51
Coalesce multiple blobs if received close together (#869)
- This helps reduce unnecessary growth of window if small blobs
  are received in small space of time
2018-08-07 10:29:57 -07:00
Michael Vines 1c38e40dee Validate ledger once all the tests complete 2018-08-07 10:00:52 -07:00
Michael Vines 91741e20fa Add rustc/cargo version check 2018-08-06 09:32:08 -07:00
Michael Vines a13058b6c4 Look for 3 nodes (1 leader, 2 validators) 2018-08-03 20:30:29 -07:00
Rob Walker c998199954 fixups, add validator-x to sanity 2018-08-03 15:34:11 -07:00
Michael Vines 15a6179b97 Stop installing rustfmt-preview, it's already present 2018-08-03 14:27:11 -07:00
Michael Vines 83b308983f Include rustfmt-preview 2018-08-03 14:11:42 -07:00
Rob Walker f2b1a04bca cargo fmt fixups 2018-08-03 11:59:25 -07:00
Michael Vines 3e36e6dcf8
Upgrade to rust 1.28 2018-08-03 11:30:40 -07:00
Michael Vines 6feb6a27be Run localnet-sanity in test-stable-perf 2018-08-03 10:46:48 -07:00
Michael Vines c230360f4c Wait until recycled machines are reachable before provisioning them 2018-08-02 22:13:17 -07:00
Michael Vines a9240a42bf Delete unreachable validators to cause a fresh one to be spawned 2018-08-02 20:45:29 -07:00
Michael Vines a7204d5353 Use a local user to avoid GCP login quota limits 2018-08-02 19:43:35 -07:00
Greg Fitzgerald 2366c1ebaf Enable cargo audit in CI
Fixes #772
2018-08-01 16:24:47 -07:00
Michael Vines 2c9d288ca9 Add a CI metric data point upload timeout to prevent CI build stalls
5 seconds is somewhat arbitrary, seems like enough
2018-08-01 15:49:48 -07:00
Greg Fitzgerald bb0aabae75 Add cmake, which is needed to build cargo-audit 2018-08-01 16:43:49 -06:00
Michael Vines 2b259aeb41 testnet now deploys successfully on days of the month < 10 2018-08-01 14:10:52 -07:00
Michael Vines de7e9b4b4c
Remove retry
This was introduced to mask the occasional failure of racy tests.  But this is misguided as it helps hid the true problem, the racy test, and it causes tries builds that fail deterministically to retry only to fail once again.
2018-08-01 12:02:39 -07:00
Michael Vines 5d936e5c8a Trap SIGINT for clean ^C shutdown 2018-07-30 17:15:50 -07:00
Michael Vines 5a40ea3fd7 Only map HOME when in CI 2018-07-30 16:36:26 -07:00
Michael Vines 102510ac0e Clear apt cache to reduce image size 2018-07-30 16:36:26 -07:00
Michael Vines 2158329058 Switch to docker-rust image 2018-07-30 16:36:26 -07:00
Michael Vines bc484ffe5f Add docker-rust image 2018-07-30 16:36:26 -07:00
Michael Vines 6fcf4584d5 Propagate more BUILDKITE environment variables into containers 2018-07-30 16:36:26 -07:00
Michael Vines 1adc83d148 Add localnet-sanity.sh 2018-07-30 16:36:26 -07:00
Michael Vines 416c141775 `export SKIP_INSTALL=1` to reset the network without reinstalling the snap 2018-07-28 18:04:13 -07:00
Michael Vines 4cdb9a73f8 Skip testnet-sanity on manual deploy 2018-07-28 12:37:29 -07:00
Michael Vines 4433730610 Add support for deploying a locally built snap 2018-07-28 12:37:29 -07:00
Michael Vines 71eb5bdecc Factor out vm_foreach 2018-07-28 12:37:29 -07:00
Michael Vines 80a02359f7 Add script to audit for security vulnerabilities 2018-07-26 13:42:12 -07:00
Michael Vines ab3968e3bf Dedup 2018-07-26 11:45:58 -07:00
Michael Vines 42ebf9502a Agent cleaning is now performed in a separate pipeline 2018-07-26 11:37:36 -07:00
Michael Vines bd4fcf4ac6 Clean out stale buildkite agent build directories 2018-07-26 11:37:36 -07:00
Michael Vines 4dceb73909 Reinstall client nodes in the background to speed up deploys 2018-07-26 09:49:00 -07:00
Rob Walker 5115cd7798 large network back to erasure 2018-07-25 20:45:16 -07:00
Michael Vines 4c4659be13 Add more stdout 2018-07-25 16:38:21 -07:00
Michael Vines 1b79fe73a1 Emit a metrics datapoint if bench-tps terminates 2018-07-25 15:55:02 -07:00
Michael Vines 5fa072cf16 Avoid quotes around net name 2018-07-25 15:55:02 -07:00
Stephen Akridge 80ecef2832 Add --sustained to ci testnet deploy script 2018-07-25 10:16:46 -07:00
Michael Vines dcce475f0b Progagate logging configuration to client nodes 2018-07-24 21:40:02 -07:00
Rob Walker 8f046cb1f8 disable erasure for large network testing 2018-07-24 16:54:52 -07:00
Michael Vines 735a0ee16d Switch back to running bench-tps in 10 minute iterations 2018-07-24 15:43:25 -07:00
Rob Walker 940caf7876 test large network with erasure 2018-07-24 13:04:34 -07:00
Michael Vines 782aa7b23b Cap at 4 threads 2018-07-24 11:35:03 -07:00
Michael Vines 7a71adaa8c Adjust threads by the number of cpus 2018-07-23 21:17:36 -07:00
Michael Vines ce8796bc2e Correctly calculate the expected number of full nodes 2018-07-23 19:55:09 -07:00
Michael Vines c7e1409f7b Not so much |set -x| 2018-07-23 19:55:09 -07:00
Michael Vines 9de9379925 Add support more more than 1 client node 2018-07-23 19:34:34 -07:00
Michael Vines 96a6e09050 Enable metrics in the TPS client 2018-07-23 16:51:39 -07:00
Michael Vines 7ebbaaeb2d Use bench-tps default duration 2018-07-22 16:26:49 -07:00
Michael Vines 5d065133ef Add data point for testnet startup and shutdown 2018-07-21 23:27:24 -07:00
Michael Vines d403808564
Restart `solana.bench-tps` every 10 minutes to work around memory leak
cc: #728
2018-07-21 19:48:13 -07:00
Michael Vines 3ffdca193d Rename client-demo to bench-tps catchup 2018-07-21 15:46:03 -07:00
Greg Fitzgerald 637f890b91 Rename client-demo to bench-tps 2018-07-21 11:36:20 -04:00
Michael Vines 009d5adcba
Tell the client to transact for 1 hour blocks 2018-07-20 17:52:19 -07:00
Michael Vines 52c55a0335
Log to /tmp/solana.log for easy runtime inspection of client activity 2018-07-20 17:45:13 -07:00
Michael Vines 120add0e82 Add support for a client node running continuous transactions on the net 2018-07-20 17:07:36 -07:00
Michael Vines 9cf0bd9b88 Adjust variable name 2018-07-20 09:50:24 -07:00
Michael Vines 9d25d7611a Protect against unsupported configurations to prevent non-obvious errors later 2018-07-20 09:47:01 -07:00
Michael Vines 1abefb2c7a Pass expected node count to testnet-sanity 2018-07-20 09:47:01 -07:00
Michael Vines 68ca9b2cb8 Generalize testnet deployment scripts 2018-07-20 09:10:28 -07:00
Michael Vines 17d927ac74 Count testnet nodes as a part of sanity 2018-07-19 12:05:21 -07:00