Pankaj Garg
db98f7e0b4
Use env variables to disable validator sanity and ledger verification ( #1675 )
2018-10-31 12:30:33 -07:00
jackcmay
0529f36fde
Run workspace member's tests ( #1666 )
...
Run workspace member's tests
2018-10-30 22:53:36 -07:00
Pankaj Garg
85869552e0
Update testnet scripts to use release tar ball ( #1660 )
...
* Update testnet scripts to use release tar ball
* use curl instead of s3cmd
2018-10-30 18:05:38 -07:00
Pankaj Garg
6f9843c14b
Publish a tarball of Solana release binaries ( #1656 )
...
* Publish a tarball of solana release binaries
* included native programs in Solana release tar
* Remove PR check from publish script
2018-10-30 15:31:52 -07:00
Michael Vines
37213209c5
Create programs/bpf/c/sdk/
2018-10-29 19:10:29 -07:00
Michael Vines
f6b1b5ab37
Remove unnecessary checks
2018-10-29 13:27:52 -07:00
Michael Vines
f12743de38
Create/publish bpf-sdk tarball
2018-10-29 12:54:57 -07:00
Michael Vines
77e10ed757
Add utility to figure the current crate version
2018-10-29 12:54:57 -07:00
Michael Vines
f671b7f63f
Publish root crate too
2018-10-25 17:16:18 -07:00
Michael Vines
a340b18b19
Upgrade to rust 1.30
2018-10-25 17:13:41 -07:00
Michael Vines
160cff4a30
Check for TRIGGERED_BUILDKITE_TAG
2018-10-25 16:37:54 -07:00
Michael Vines
1733beabf7
mv common/ sdk/
2018-10-25 13:26:10 -07:00
Michael Vines
471d8f6ff9
Fix up the version references to all other internal crates
2018-10-25 12:54:32 -07:00
Michael Vines
b982595c73
Add version check and rustup
2018-10-24 19:48:58 -07:00
Rob Walker
4ea422bcec
run integration tests serially
2018-10-17 11:37:10 -07:00
Greg Fitzgerald
6074e4f962
Attempt to stabilize the test suite
...
The integration tests are allowed to open sockets, so running them
in parallel may cause "Too many open files" errors. This patch
runs the unit tests in parallel and the integration test serially.
2018-10-17 11:37:10 -07:00
Greg Fitzgerald
5b4d4b97bc
Upgrade to latest stable Rust, 1.29.2
2018-10-15 09:54:24 -06:00
Michael Vines
c5180c8092
Permit RUST_LOG overrides
2018-10-14 12:40:37 -07:00
Pankaj Garg
e5ab9a856c
Upload bench output as build artifacts ( #1478 )
...
* Upload bench output as build artifacts
* Fix tags types
* Pull previous stats from metrics
* Change the default branch for comparison
* Fix formatting
* Fix build errors
* Address review comments
* Dedup some common code
* Add eval for channel info to find branch name
2018-10-12 15:13:10 -07:00
Michael Vines
5c523716aa
Ship native programs
2018-10-10 16:49:48 -07:00
Pankaj Garg
c240bb12ae
Change buildkite agent for testnet automation
2018-10-09 15:04:55 -07:00
Pankaj Garg
29a8823db1
Env variables for testnet-automation parameters ( #1455 )
...
- This will enable us to create custom pipelines for field events
2018-10-09 11:50:56 -07:00
Pankaj Garg
a80955eacb
Change format of data for TPS/Finality metrics in testnet automation ( #1446 )
...
* Change format of data for TPS/Finality metrics in testnet automation
* Revert number of nodes for testnet automation
* Split python command to its own script
* Fix python command line arguments
2018-10-09 10:35:01 -07:00
Pankaj Garg
0a39722719
Add support to trigger testnet from a PR ( #1434 )
...
* Add support for different node counts
* Update variable names
* Delete network even after failures
* Add array for node counts
* Changed number of nodes to a space separated string of numbers
* Adjust number of nodes
* Snap will not be published if the env variable DO_NOT_PUBLISH_SNAP is set
* Address review comments
* Replaced influx db URL
2018-10-05 16:32:05 -07:00
Greg Fitzgerald
9c0fa4d1d2
Upload coverage HTML reports ( #1421 )
...
Uploads two reports to Buildkite, one from cargo-cov and one from lcov via grcov. The lcov one is busted on linux and is what we need to bring codecov.io back up again. It works great on macos if you wanted to generate them locally and prefer lcov HTML reports.
* Also comment out non-coverage build to speed things up.
2018-10-05 10:17:35 -07:00
Michael Vines
beb4536841
Run a fullnode+drone automatically when the container starts up
2018-10-02 18:09:35 -07:00
Michael Vines
ad5fcf778f
Publish minimal Solana docker images to dockerhub
2018-10-02 16:57:48 -07:00
Michael Vines
83b000ae88
Remove SNAP_ prefix
2018-10-02 16:57:48 -07:00
Greg Fitzgerald
94459deb94
Disable codecov.io reporting
2018-09-28 19:19:16 -07:00
Greg Fitzgerald
660af84b8d
Use the same versions of llvm-cov and libprofile
2018-09-28 19:19:16 -07:00
Greg Fitzgerald
7b31020903
Add back llvm-dev for llvm-cov
2018-09-28 19:19:16 -07:00
Greg Fitzgerald
9a4143b4d9
Upgrade llvm-dev and boot kcov
...
Need clang-dev, not llvm-dev because cargo-cov looks for libprofile
in a clang installation directory.
2018-09-28 19:19:16 -07:00
Greg Fitzgerald
aebc47ad55
Attempt coverage reporting
2018-09-28 19:19:16 -07:00
Greg Fitzgerald
5bc01cd51a
Revive code coverage
2018-09-28 19:19:16 -07:00
Greg Fitzgerald
19a7ff0c43
Pin down nightly in benchmark build
2018-09-28 19:29:50 -06:00
Greg Fitzgerald
5f18403199
Upgrade nightly
2018-09-28 19:29:50 -06:00
Tyera Eulberg
9f325fca09
Re-enable cargo audit
2018-09-28 17:53:41 -06:00
Tyera Eulberg
10d08acefa
Reenable cargo audit
2018-09-28 17:53:41 -06:00
Tyera Eulberg
52d50e6bc4
Update for new solana-jsonrpc
2018-09-28 17:53:41 -06:00
Stephen Akridge
aa2a3fe201
Add chacha module to encrypt ledger files
2018-09-28 14:23:37 -07:00
Greg Fitzgerald
485ba093b3
Install kcov to CI environment
2018-09-28 11:20:27 -06:00
Michael Vines
8d92232949
Specify zone
2018-09-28 07:32:49 -07:00
Michael Vines
e4d8c094a4
Include -z when deleting network
2018-09-27 21:27:09 -07:00
Greg Fitzgerald
675ff64094
Fail CI on clippy warnings
2018-09-27 16:21:12 -06:00
Greg Fitzgerald
f9fe6a0f72
Move clippy to Rust stable
2018-09-27 16:21:12 -06:00
Greg Fitzgerald
8d007bd7f7
Upgrade rustc and add clippy to stable
2018-09-27 16:21:12 -06:00
Stephen Akridge
6cdbdfbbcb
Enable bench and fix upload-perf
2018-09-27 14:16:56 -07:00
Pankaj Garg
35e6343d61
Update testnet-deploy script to configure GPUs for leader node ( #1379 )
2018-09-27 13:42:24 -07:00
Michael Vines
afc764752c
Permit testnets without a GPU
2018-09-26 10:37:41 -07:00
sakridge
3e76efe97e
Fix bench compilation ( #1311 )
2018-09-24 10:40:42 -07:00
Michael Vines
e5e325154b
Add --shell argument
2018-09-24 08:05:47 -07:00
Michael Vines
67f1fbab5f
Treat rustc warnings as errors in CI
2018-09-20 19:46:20 -07:00
sakridge
2981076a14
Add solana-upload-perf to parse json from bench and upload to influx ( #1166 )
2018-09-19 13:16:55 -07:00
Michael Vines
3ca80c676c
Disable large-network until it's fixed
2018-09-14 20:13:17 -07:00
Michael Vines
a344eb7dd0
Upgrade rust stable to 1.29
2018-09-14 16:37:49 -07:00
Michael Vines
f06113500d
bench-tps/net sanity: add ability to check for unexpected extra nodes
2018-09-12 15:38:57 -07:00
Rob Walker
97c3125a78
improve localnet-sanity's robustness ( #1160 )
...
* fix poll_gossip_for_leader() loop to actually wait
for 30 seconds
* reduce reuseaddr use to only when necessary,
try to avoid already bound sockets
* move nat.rs to netutil.rs
* add gossip tracing to thin_client and bench-tps
2018-09-09 04:50:43 +09:00
Michael Vines
a77aca75b2
Add NO_VALIDATOR_SANITY back
2018-09-07 22:37:05 -07:00
Michael Vines
b1b03ec13b
Refine docker image tagging to avoid breaking stabilization branches on updates
2018-09-07 18:42:25 -07:00
Rob Walker
51b27779c9
client changes for TODOs and looping ( #1138 )
...
* remove client.sh from snap
* default to ephemeral instead of ~/.config key
* rework CLI for bench-tps
* remote multinode-demo stuff from remote-client.sh
* remove multinode-demo from remote-sanity and localnet-sanity
2018-09-08 07:07:10 +09:00
Michael Vines
0d945e6a92
Groom testnet-sanity logging
2018-09-07 12:45:48 -07:00
Michael Vines
e51445d857
🙃
2018-09-07 12:24:34 -07:00
Michael Vines
ee682d5bc3
Move wallet-sanity.sh out of multinode-demo/
2018-09-07 12:01:43 -07:00
Michael Vines
506a81e8cc
Assume -y
2018-09-07 12:01:43 -07:00
Michael Vines
88298bf321
Add -n option
2018-09-07 12:01:43 -07:00
Michael Vines
533ebc17f2
Install multilog automatically on a CI machine
2018-09-07 11:56:23 -07:00
Michael Vines
e088833b81
s/create/start/
2018-09-06 21:07:11 -07:00
Michael Vines
b1bacf12a6
Add some log sections
2018-09-06 20:38:11 -07:00
Michael Vines
66ff602659
Rewrite ci/testnet-{deploy,sanity}.sh in terms of net/ primitives
2018-09-06 19:54:39 -07:00
Michael Vines
072d0b67e4
Send deploy metrics to the testnet-specific database
2018-09-06 08:30:03 -07:00
Michael Vines
6560b0e2cc
s/whoami/id -un/
2018-09-05 14:26:21 -07:00
Michael Vines
449d7042f0
Configure metrics correctly
2018-09-05 09:02:02 -07:00
Michael Vines
43b4207101
Run oom-monitor in net/ testnets
2018-09-05 09:02:02 -07:00
Michael Vines
c81c19234f
Improve incremental speed of docker cargo builds outside of CI
2018-09-05 09:02:02 -07:00
Greg Fitzgerald
cf2626a1c5
Update instructions to upgrade nightly docker image
2018-09-04 20:56:40 -06:00
Greg Fitzgerald
68c72d6f34
Fix nightly build
2018-09-04 20:56:40 -06:00
Greg Fitzgerald
65f78905cd
Install cargo-cov on latest nightly
2018-09-04 20:56:40 -06:00
Michael Vines
b4a7a18334
Update README.md
2018-09-04 13:29:00 -07:00
Rob Walker
61abee204f
don't check for snap mode in common.sh, is only relevant to snap daemons ( #1113 )
...
snap mode is for daemons, remove it from client (i.e. common.sh)
supply leader info to client via snap
2018-09-04 14:31:54 +09:00
Rob Walker
e834c76b40
--count => --num-nodes
2018-09-04 07:07:25 +09:00
Rob Walker
d1f1cbe88f
leader-address=>leader-ip
2018-09-01 04:38:18 +09:00
Rob Walker
a4e7b6e90c
more fixups for client.sh changes
2018-09-01 03:33:21 +09:00
Rob Walker
fbc7c9c431
fix client_start to deal with new client.sh
2018-09-01 03:23:05 +09:00
Rob Walker
8b248dcf09
specify port
2018-09-01 02:56:24 +09:00
Rob Walker
4938aad939
fixups
2018-09-01 02:21:46 +09:00
Rob Walker
7e882dfe62
inform all snaps where the network is
2018-09-01 02:21:46 +09:00
Rob Walker
9d1eb4f9ea
remove 'localhost' leader (redundant, un-dig-friendly)
2018-08-31 23:21:07 +09:00
Rob Walker
176e806d94
rework of netwrk rendezvous
...
* rename NodeInfo field of Node from "data" to "info"
(touches a lot of files)
* update client to use gossip to find leader, a la drone
* rework multinode scripts
* move more stuff into rust
* added usage to all
* no more rsync unless you're a validator (TODO: whack that, too)
* fullnode doesn't bail if drone isn't up yet, just keeps trying
* drone doesn't bail if network isn't up yet, just keeps trying
2018-08-31 23:21:07 +09:00
Rob Walker
63e44dcc35
continue rendezvous refactor for gossip and repair
...
* remove trailing whitespace in ci/audit.sh
* code review fixups
* rename GOSSIP_PORT_RANGE => SOLANA_PORT_RANGE
* remove out-of-date TODO in localnet-sanity.sh
* remove features=test and code that was using it (localhost prohibitions in
crdt) added TODO in crdt.rs, maybe we should boot localhost in production
networks?
* boot tvu_window from NodeInfo: instead, send repair requests from the repair
socket (to gossip on peer) and answer repair requests via the sockaddr
from the repair request
* remove various unused pub functions
* banish SocketAddr parse().unwrap() to a macro that can also accept simpler stuff
2018-08-31 23:21:07 +09:00
Rob Walker
1af4cee63b
fix #1079
...
* move gossip/NCP off assuming anything about its address
* use a single socket to send and receive gossip
* remove --addr/-a from CLIs
* rearrange networking utility code
* use Arc<UdpSocket> to share the Sync-safe UdpSocket among threads
* rename TestNode to Node
TODO:
* re-enable 127.0.0.1 as a valid address in crdt
* change repair request/response to a similar, single socket
* pick cloned sockets or Arc<UdpSocket> for all these (rpu uses tryclone())
* update contact_info with network truthiness instead of what the node
says?
2018-08-31 23:21:07 +09:00
Rob Walker
cb52a335bd
re-enable localnet-sanity
2018-08-31 23:21:07 +09:00
Michael Vines
73a0bf8d30
Avoid unbounded /var/tmp growth
2018-08-28 08:11:44 -07:00
Michael Vines
9d53208d68
Use gcloud_DeleteInstances
2018-08-28 08:11:44 -07:00
Michael Vines
c8e3ce26a9
Start of scripts/gcloud.sh
2018-08-27 22:35:14 -07:00
Rob Walker
51d911e3f4
Update testnet-sanity.sh
2018-08-27 15:44:10 -07:00
Michael Vines
bd5c6158ae
Move some common scripts from multinode-demo/ to scripts/
2018-08-27 13:52:38 -07:00
Rob Walker
0efd64df6f
no need for sudo, move ledger copy out of SNAP_DATA
2018-08-28 05:42:05 +09:00
Rob Walker
28bdf346f6
clean up after ledger sanity
2018-08-28 05:42:05 +09:00
anatoly yakovenko
738247ad44
advertise valid gossip address in drone and wallet ( #1066 )
...
* advertize valid gossip address in drone and wallet
get rid of asserts
check for valid ip address
check for valid address
ip address
* tests
* cleanup
* cleanup
* print error
* bump
* disable tests
* disable nightly
2018-08-26 11:36:27 -07:00
anatoly yakovenko
f7c0d30167
Disallow localhost in deployment ( #1064 )
...
* disallow localhost in deployment
* tests
* fmt
* integration tests do not have a flag to check
* fmt
2018-08-25 21:09:18 -07:00
Greg Fitzgerald
db3fb3a27c
Boot criterion ( #1032 )
...
* Revert benchmarks back to libtest
Criterion has too many dependencies, it's execution as slower, and
we didn't see the kind of precision we had hoped for to use it to
block CI builds.
* Ignore benchmarks that take more than a few milliseconds per iteration
* Revert "Ignore benchmarks that take more than a few milliseconds per iteration"
This reverts commit b87cdf6ef4a2549ec09e282014391de695473699.
* Don't run benchmarks in CI
They are already built in the nightly build. Executing them in CI
doesn't add much value until the results are precise enough to act
on.
2018-08-22 08:57:07 -06:00
Rob Walker
0b48aea937
echo commands, use PID (good form)
2018-08-21 11:41:00 +09:00
Rob Walker
cdec0cead2
files have to appear in the snap
2018-08-21 11:41:00 +09:00
Rob Walker
831709ce7e
fixups
2018-08-21 10:36:03 +09:00
Rob Walker
b7b8a31532
make a copy of the ledger for sanity check
...
we can't verify a live ledger, unfortunately, fixes #985
2018-08-21 10:36:03 +09:00
Rob Walker
5aced8224f
Revert "make a copy of the ledger for sanity check"
...
This reverts commit af20a43b77
.
2018-08-21 10:34:52 +09:00
Rob Walker
af20a43b77
make a copy of the ledger for sanity check
...
we can't verify a live ledger, unfortunately, fixes #985
2018-08-21 09:45:52 +09:00
Greg Fitzgerald
39c3280860
Don't block on large network test
2018-08-20 16:48:37 -06:00
Michael Vines
a1ec549630
Pin nightly rust for more controlled updating
2018-08-20 10:55:55 -07:00
Michael Vines
ecddff98f5
Add --nopull argument
2018-08-20 10:55:55 -07:00
Michael Vines
10066d67bf
Add llvm deb repository
2018-08-19 09:01:36 -07:00
Michael Vines
a07f7435c6
\
2018-08-19 08:49:29 -07:00
Michael Vines
d3523ebbe5
Nightly image now derives from stable image
2018-08-19 08:47:59 -07:00
Tyera Eulberg
1bf15ae907
Temporarily disable cargo audit CI failure
2018-08-18 12:29:49 -06:00
Michael Vines
877920e61b
Compute snap channel using ci/channel-info.sh
2018-08-17 23:15:48 -07:00
Michael Vines
3d1e908dad
Add script to fetch latest channel info
2018-08-17 23:15:48 -07:00
Michael Vines
6880c2bef0
Exclude ci/semver_bash/; don't want to diverge from upstream
2018-08-17 23:15:48 -07:00
Michael Vines
78872ffb4b
Vendor https://github.com/cloudflare/semver_bash/tree/c1133faf0e
2018-08-17 23:15:48 -07:00
Michael Vines
229d825fe0
Fix master-perf basename
2018-08-17 21:59:36 -07:00
Michael Vines
edc5fc098e
Make SNAP_CHANNEL more visible in build log
2018-08-17 21:39:54 -07:00
Michael Vines
82e7725a42
Invert logic
2018-08-17 21:16:35 -07:00
Michael Vines
dc61cf1c8d
Keep v0.7 snap off the edge channel
2018-08-17 21:12:10 -07:00
Michael Vines
aba63e2c6c
Log expansion directive must be on its own line
2018-08-17 20:58:14 -07:00
Michael Vines
c2ddd056e2
Add option to skip ledger verification
2018-08-17 20:41:30 -07:00
Rob Walker
7aeef27b99
not quite banishing build.rs, but better
2018-08-16 22:33:31 -07:00
Michael Vines
98d0ef6df5
Add some wget retries
2018-08-16 20:22:49 -07:00
Rob Walker
fc0d7f5982
updated nightly versions
2018-08-16 13:17:29 -07:00
Rob Walker
f697632edb
update clippy install instructions, from here:
...
https://github.com/rust-lang-nursery/rust-clippy
fixes #947 ?
2018-08-16 13:17:29 -07:00
Rob Walker
73797c789b
back to 4 TX threads
2018-08-16 12:02:11 -07:00
Rob Walker
036fcced31
test -t nproc
2018-08-16 12:02:11 -07:00
Rob Walker
1d3157fb80
fixups
2018-08-16 12:02:11 -07:00
Rob Walker
0b11c2e119
restart testnet clients in case airdrop fails
2018-08-16 12:02:11 -07:00
Michael Vines
757dfd36a3
Report errors better in build log
2018-08-14 11:44:26 -07:00
Rob Walker
ab10b7676a
use stable cache
2018-08-13 16:23:30 -07:00
Rob Walker
fa44a71d3e
move bench to a seprate, parallel step
2018-08-13 16:23:30 -07:00
Rob Walker
9e22e23ce6
increase stable timeout until tomorrow
2018-08-13 15:45:50 -07:00
Michael Vines
df5cde74b0
Back out pre-0.7.1 workaround
2018-08-13 12:13:00 -07:00
Michael Vines
c2ba72fe1f
Fix up validator sanity
2018-08-13 10:23:35 -07:00
Michael Vines
d93786c86a
Revert "turn off validator sanity while I work on it"
...
This reverts commit d4304eea28
.
2018-08-13 10:23:35 -07:00
Rob Walker
8eed120c38
add missing backslash
2018-08-10 23:45:08 -07:00
Rob Walker
d4304eea28
turn off validator sanity while I work on it
2018-08-10 14:56:46 -07:00
Rob Walker
7f71e1e09f
fixups
2018-08-10 11:41:31 -07:00
Rob Walker
bb7eccd542
check validator startup in testnet-sanity.sh
2018-08-10 11:41:31 -07:00
Rob Walker
b04c71acd9
check issue 910 in testnet-sanity
2018-08-10 11:41:31 -07:00
Rob Walker
8b41c415b7
add equal sign
2018-08-10 08:05:48 -07:00
Michael Vines
8ca514a5ca
Remove unnecessary :
2018-08-08 22:45:39 -07:00
Michael Vines
b605552079
Record network version in testnet-deploy start datapoint
2018-08-08 22:41:02 -07:00
Michael Vines
74f5538bd3
Verify the ledger as a part of sanity
2018-08-08 16:10:54 -07:00
Pankaj Garg
a6857dbaaa
Updated node count to 230. Increased wmem on CI large
2018-08-08 13:13:18 -07:00
Michael Vines
054298d957
Retry snap install 3 times, sometimes the snap server 503s
2018-08-08 08:56:05 -07:00