Commit Graph

351 Commits

Author SHA1 Message Date
Deirdre Connolly 1a4f8f6573
feat(ci): create disk image after a successful full sync test (#3986)
* Create disk image after a successful full sync test

* Extract full sync height and name zebrad cached state with it

* Read 500 lines to extract sync height

* Restrict log query to just the container output and fix regex syntax for ubuntu

* Explicitly search logs in descending time

Co-authored-by: Gustavo Valverde <gustavo@iterativo.do>
2022-04-06 16:57:45 -04:00
dependabot[bot] b0304c5882
build(deps): bump docker/metadata-action from 3.6.2 to 3.7.0 (#4049)
Bumps [docker/metadata-action](https://github.com/docker/metadata-action) from 3.6.2 to 3.7.0.
- [Release notes](https://github.com/docker/metadata-action/releases)
- [Commits](https://github.com/docker/metadata-action/compare/v3.6.2...v3.7.0)

---
updated-dependencies:
- dependency-name: docker/metadata-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-04-06 20:51:01 +00:00
dependabot[bot] 47c77f57d1
build(deps): bump google-github-actions/auth from 0.6.0 to 0.7.0 (#4050)
Bumps [google-github-actions/auth](https://github.com/google-github-actions/auth) from 0.6.0 to 0.7.0.
- [Release notes](https://github.com/google-github-actions/auth/releases)
- [Changelog](https://github.com/google-github-actions/auth/blob/main/CHANGELOG.md)
- [Commits](https://github.com/google-github-actions/auth/compare/v0.6.0...v0.7.0)

---
updated-dependencies:
- dependency-name: google-github-actions/auth
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-04-06 19:28:39 +00:00
teor 13c216cd3c
1.0.0-beta.7 Release (#4039)
* Update changelog for v1.0.0-beta.7

* Increment all crate versions

* Remove redundant release test that is now covered by CI

* Remove completed NU5 README check task from the release template

* Add Merge Freeze tool to the release checklist

* Simplify release checklist by removing unused steps
2022-04-05 18:06:46 +00:00
dependabot[bot] fa8d1a7074
build(deps): bump tj-actions/changed-files from 18.4 to 18.6 (#4002)
Bumps [tj-actions/changed-files](https://github.com/tj-actions/changed-files) from 18.4 to 18.6.
- [Release notes](https://github.com/tj-actions/changed-files/releases)
- [Changelog](https://github.com/tj-actions/changed-files/blob/main/HISTORY.md)
- [Commits](https://github.com/tj-actions/changed-files/compare/v18.4...v18.6)

---
updated-dependencies:
- dependency-name: tj-actions/changed-files
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-04-04 20:44:19 +00:00
teor eeff71d0bd
Increase test coverage of state rebuild (#4020)
Only run the state rebuild job if the database format version has (likely) changed.

If we have accidentally changed the format, but not changed the version,
we want to run with the old cached state, so this job fails.

If we change the state path without changing the version,
this job will take a few hours, because it will do a full rebuild.
2022-04-01 16:02:55 +00:00
Gustavo Valverde 75ae05b257
imp(ci): do not invalidate cache between PRs (#3996)
When a PR is created and an image is built in a branch, the cache is also pushed to a `buildcache` tag.

As all PRs are using the same tag, sometimes the cache for a specific branch gets invalidated and makes it take longer on further pushes.

This fix might make the first commit take longer, but further ones will be faster if no changes in the code are applied.
2022-03-31 01:34:12 +00:00
Deirdre Connolly 3552eafbb8
Add a job to ci.yml that does `cargo install --locked --path ./zebrad/ zebrad` (#3998)
* Add a job to ci.yml that does cargo install --locked --path . zebrad

* Update .github/workflows/ci.yml

Co-authored-by: teor <teor@riseup.net>

* Update .github/workflows/ci.patch.yml

Co-authored-by: teor <teor@riseup.net>

* Update .github/workflows/ci.yml

Co-authored-by: Gustavo Valverde <gustavo@iterativo.do>

* Update .github/workflows/ci.patch.yml

Co-authored-by: Gustavo Valverde <gustavo@iterativo.do>

Co-authored-by: teor <teor@riseup.net>
Co-authored-by: Gustavo Valverde <gustavo@iterativo.do>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-03-30 21:53:57 +00:00
Deirdre Connolly dcade546c0
Update test.patch.yml with lightwalletd job (#3970)
* Update test.patch.yml with lightwalletd job

* Remove a workflow condition that will always be false

In general, patch workflows need the
opposite conditions to the original workflow.

But in this case, we know the result of the
condition will always be true, so we can just delete it.

Co-authored-by: teor <teor@riseup.net>
2022-03-27 22:40:53 +00:00
Conrado Gouvea ed5e85f8ae
fix syntax in some yml workflows (#3957) 2022-03-24 22:39:16 +00:00
dependabot[bot] 5d7f986183
build(deps): bump tj-actions/changed-files from 18.3 to 18.4 (#3917)
Bumps [tj-actions/changed-files](https://github.com/tj-actions/changed-files) from 18.3 to 18.4.
- [Release notes](https://github.com/tj-actions/changed-files/releases)
- [Changelog](https://github.com/tj-actions/changed-files/blob/main/HISTORY.md)
- [Commits](https://github.com/tj-actions/changed-files/compare/v18.3...v18.4)

---
updated-dependencies:
- dependency-name: tj-actions/changed-files
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-03-23 11:57:55 -04:00
dependabot[bot] de59fc6aa0
build(deps): bump actions/cache from 2 to 3 (#3918)
Bumps [actions/cache](https://github.com/actions/cache) from 2 to 3.
- [Release notes](https://github.com/actions/cache/releases)
- [Commits](https://github.com/actions/cache/compare/v2...v3)

---
updated-dependencies:
- dependency-name: actions/cache
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-03-21 23:08:25 +00:00
Gustavo Valverde 39765534f1
fix(actions): use a specific shortening length for SHAs (#3929)
* fix(actions): use a specific shortening length for SHAs

The rlespinasse/github-slug-action now works without checking out the code, reduce time and improving security with following actions.

This requires to specify the GITHUB_SHA_SHORT variable length, as git uses 8 by default, but docker uses 7 by default.

* fix(actions): target correct rlespinasse/github-slug-action version

* fix(actions): just use major version

* fix(actions): github-slug-action is not being correctly referenced
2022-03-21 23:07:01 +00:00
Gustavo Valverde e1eb916b6d
refactor(ci): use improved OIDC for gcloud authentication (#3885)
* refactor(ci): use improved OIDC authentication

* fix(ci): standardize OIDC on all required jobs

* fix: wrong indentation

* fix(ci): remove non existing depency in clean job
2022-03-18 21:25:35 +00:00
dependabot[bot] 84ee78dee4
build(deps): bump tj-actions/changed-files from 18.1 to 18.3 (#3895)
Bumps [tj-actions/changed-files](https://github.com/tj-actions/changed-files) from 18.1 to 18.3.
- [Release notes](https://github.com/tj-actions/changed-files/releases)
- [Changelog](https://github.com/tj-actions/changed-files/blob/main/HISTORY.md)
- [Commits](https://github.com/tj-actions/changed-files/compare/v18.1...v18.3)

---
updated-dependencies:
- dependency-name: tj-actions/changed-files
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-03-18 10:09:56 +00:00
Janito Vaqueiro Ferreira Filho 57dba9fd17
Add `--nocapture` to lightwalletd integration test (#3859)
Print lightwalletd's output in CI.
2022-03-18 04:31:29 +00:00
teor 413f7fbb1d
fix(ci): remove coverage from mergify because nightly fails (#3886) 2022-03-16 00:07:04 -04:00
dependabot[bot] bf1f7aa973
build(deps): bump tj-actions/changed-files from 17.3 to 18.1 (#3876)
Bumps [tj-actions/changed-files](https://github.com/tj-actions/changed-files) from 17.3 to 18.1.
- [Release notes](https://github.com/tj-actions/changed-files/releases)
- [Changelog](https://github.com/tj-actions/changed-files/blob/main/HISTORY.md)
- [Commits](https://github.com/tj-actions/changed-files/compare/v17.3...v18.1)

---
updated-dependencies:
- dependency-name: tj-actions/changed-files
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-03-16 03:18:33 +00:00
dependabot[bot] c66d6bfa38
build(deps): bump docker/build-push-action from 2.9.0 to 2.10.0 (#3878)
Bumps [docker/build-push-action](https://github.com/docker/build-push-action) from 2.9.0 to 2.10.0.
- [Release notes](https://github.com/docker/build-push-action/releases)
- [Commits](https://github.com/docker/build-push-action/compare/v2.9.0...v2.10.0)

---
updated-dependencies:
- dependency-name: docker/build-push-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-03-15 22:35:18 +00:00
teor 5950083006
fix(ci): path format for cached state rebuild (#3873) 2022-03-15 09:13:31 +00:00
Deirdre Connolly fcfdc5493e
s/pd-extreme/pd-ssd/ to avoid quotas (#3823) 2022-03-10 22:10:59 +00:00
dependabot[bot] 6f3885d126
build(deps): bump tj-actions/changed-files from 17.2 to 17.3 (#3796)
Bumps [tj-actions/changed-files](https://github.com/tj-actions/changed-files) from 17.2 to 17.3.
- [Release notes](https://github.com/tj-actions/changed-files/releases)
- [Changelog](https://github.com/tj-actions/changed-files/blob/main/HISTORY.md)
- [Commits](https://github.com/tj-actions/changed-files/compare/v17.2...v17.3)

---
updated-dependencies:
- dependency-name: tj-actions/changed-files
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-03-10 04:01:31 +00:00
dependabot[bot] b0a4e7d487
build(deps): bump google-github-actions/setup-gcloud from 0.5.1 to 0.6.0 (#3814)
Bumps [google-github-actions/setup-gcloud](https://github.com/google-github-actions/setup-gcloud) from 0.5.1 to 0.6.0.
- [Release notes](https://github.com/google-github-actions/setup-gcloud/releases)
- [Changelog](https://github.com/google-github-actions/setup-gcloud/blob/master/CHANGELOG.md)
- [Commits](https://github.com/google-github-actions/setup-gcloud/compare/v0.5.1...v0.6.0)

---
updated-dependencies:
- dependency-name: google-github-actions/setup-gcloud
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-03-10 04:01:20 +00:00
Gustavo Valverde d8d3f6407c
build(cache): do not use GHA cache for images (#3794)
GitHub's GHA cache gets invalidated at 10Gb, which is very easy to hit when we're building multiple times a day with several commits.

Instead use the registry, which won't get invalidated until a change is identified in the build process.
2022-03-10 01:12:34 +00:00
Conrado Gouvea bbef390944
change(ci): remove Windows support (#3819)
* ci: remove Windows support

* address comments
2022-03-10 10:50:12 +10:00
Gustavo Valverde 03d123b428
refactor(test): cleanup GCP instances on a single PR (#3766)
* refactor(test): reuse same GCP instance on a single PR

This also ensures the deployments are faster, and we only delete the instance when merging or closing the PR, instead of doing it on each push to the PR

* fix(deploy): add zone to updates

* fix: typo

* fix(ci): improve conditions for updates

* fix(deploy): delete old deployments instead of reusing it

* fix(deploy): keep delete command after run

* fix(deploy): always create an instance

* fix(deploy): delete disks on every delete command.

* imp(ci): use better id name

* Update .github/workflows/test.yml

Co-authored-by: Deirdre Connolly <durumcrustulum@gmail.com>

* imp: handle errors correctly on deletion

* fix: do not hide valid errors

* fix: edge case where the container is not ready yet

Co-authored-by: Deirdre Connolly <durumcrustulum@gmail.com>
2022-03-09 17:10:05 -04:00
teor e4e6275983
fix(ci): test full sync syntax 2022-03-09 18:03:25 +10:00
teor ba5c641c85
fix(ci): re-enable manual dispatch for test full sync (#3812) 2022-03-09 18:00:58 +10:00
Conrado Gouvea 0d2c7154ea
build(deps): bump actions/checkout from 2.4.0 to 3.0.0 (#3806) 2022-03-09 03:38:38 +00:00
Gustavo Valverde ce0727e306
build(docker): revert supporting arm64 (#3791)
Building crates it's taking too much time and timing out on GitHub's Action.
2022-03-08 15:22:03 +00:00
Gustavo Valverde 3923c3de9e
imp(test): turn on full backtraces and disable frame filtering (#3763)
* imp(test): turn on full backtraces and disable frame filtering

* imp(verbosity): add RUST_LIB_BACKTRACE

We also need `RUST_LIB_BACKTRACE` as stated in:
https://docs.rs/color-eyre/latest/color_eyre/#multiple-report-format-verbosity-levels
https://doc.rust-lang.org/std/backtrace/index.html#environment-variables

Co-authored-by: teor <teor@riseup.net>

* fix(verbosity): add missing instances for RUST_BACKTRACE, RUST_LIB_BACKTRACE and COLORBT_SHOW_HIDDEN

Co-authored-by: teor <teor@riseup.net>
2022-03-08 15:09:41 +00:00
dependabot[bot] cee770fe2c
build(deps): bump docker/login-action from 1.13.0 to 1.14.1 (#3761)
Bumps [docker/login-action](https://github.com/docker/login-action) from 1.13.0 to 1.14.1.
- [Release notes](https://github.com/docker/login-action/releases)
- [Commits](https://github.com/docker/login-action/compare/v1.13.0...v1.14.1)

---
updated-dependencies:
- dependency-name: docker/login-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-03-08 09:14:10 +00:00
teor f1123e0386
2. refactor(db): split the raw disk serialzation format into modules (#3717)
* doc(db): fix some comments

* refactor(db): split disk serialization types into their own module

* refactor(db): split the disk format into modules

* doc(db/test): explain the RON serialization format
2022-03-08 07:59:41 +00:00
Gustavo Valverde 4a5f0c25ce
refactor: use gcloud to search for cached disk state (#3775) 2022-03-07 22:19:04 -05:00
teor c724c354d7
fix(ci): only run the full sync test on mergify queue PRs (#3773)
* fix(ci): only run the full sync test on mergify queue PRs

* fix inconsistent indenting and syntax

* Update .github/workflows/test-full-sync.yml

Co-authored-by: Gustavo Valverde <gustavo@iterativo.do>
2022-03-08 02:46:12 +00:00
teor d86896b841
fix(ci): revert a test job name change (#3774) 2022-03-07 22:16:48 -04:00
teor 699c0625f3
fix(test/full-sync): ignore snapshot test file changes (#3756) 2022-03-06 21:22:46 -04:00
dependabot[bot] 26d6ef9aa0
build(deps): bump docker/login-action from 1.12.0 to 1.13.0 (#3570)
Bumps [docker/login-action](https://github.com/docker/login-action) from 1.12.0 to 1.13.0.
- [Release notes](https://github.com/docker/login-action/releases)
- [Commits](https://github.com/docker/login-action/compare/v1.12.0...v1.13.0)

---
updated-dependencies:
- dependency-name: docker/login-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-03-05 17:34:54 +00:00
Conrado Gouvea 2f6a48bed8
fix(tests): use TEST_FAKE_ACTIVATION_HEIGHTS at runtime and fix tests (#3749)
* fix: check TEST_FAKE_ACTIVATION_HEIGHTS at runtime

* fix(tests): add TEST_FAKE_ACTIVATION_HEIGHTS variable

This variable ensures the test is activated in the `test-fake-activation-heights` step

* fix(docker): do not run specific tests by default in entrypoint.sh

* fix(test): remove extra TEST_FULL_SYNC argument

* imp(timeout): wait for an average build time

Co-authored-by: Gustavo Valverde <gustavo@iterativo.do>
2022-03-05 13:32:46 -04:00
Deirdre Connolly 3238ddffa7 Extra $ 2022-03-04 16:40:19 -05:00
Deirdre Connolly f875742766 Fix CD 'Deploy mainnet nodes' job conditional 2022-03-04 16:40:19 -05:00
teor 4cb0f36ae5
doc(ci): explain the different ways .txt files are used (#3743) 2022-03-04 15:55:49 -04:00
Gustavo Valverde d9a0f9d657
fix(build, deployment): missing job key and timeout bump for build (#3744)
* fix: add missing job key

* fix(arm64): bump timeout to build without cache

* fix(deployment): apply changes made in 5004c4d3a1

* fix: remove uneeded condition

* refactor(tests):  make vm names refer the test name

This also adds a build step for full sync, as there won't be a reference image when using workflow_dispatch

* fix(deployment): testing depends on the built image
2022-03-04 15:55:32 -04:00
Gustavo Valverde 15949c8c37
refactor(test): decouple full sync from other tests (#3735)
* refactor(test): decouple full sync from other tests

As the full sync requires to be run just once and isolated, we're running this test in a separate workflow, after a PR has been approved.

* fix: revert to previous conditions in job regenerate-stateful-disks

* fix(condition): get disk sha if regeneration is not executed

* fix: typo

* Update .github/workflows/test-full-sync.yml

Co-authored-by: Deirdre Connolly <durumcrustulum@gmail.com>

* fix(build): bump build time for arm64

Co-authored-by: Deirdre Connolly <durumcrustulum@gmail.com>
2022-03-04 08:12:22 +00:00
Gustavo Valverde c822f0ab0c
fix(build): arm64 add extra time when building from scratch (#3738) 2022-03-04 07:27:36 +00:00
Gustavo Valverde 5004c4d3a1
refactor(build): use OCI Image Format Specification for labels (#3728)
* refactor(build): use OCI Image Format Specification for labels

This should also fix when an image gets built multiple times using the cache, as each image differs in labels

* refactor(tags): use PR context sha and ref

Remove the needed of PR Head SHA and Ref, as those can cause conflict depending on how the branch name has been established
2022-03-04 02:33:29 -04:00
teor e7af51db3a
fix(deps): remove an outdated dependabot ignore rule (#3719)
We can use dependabot commands instead.
2022-03-03 13:04:25 +00:00
teor 9ce3a9a62f
fix(ci): check for adityapk00/lightwalletd behaviour in test harness (#3705)
* fix(ci): remove an unused trigger path

* doc(ci): explain lightwalletd trigger paths

* fix(test): check for adityapk00/lightwalletd behaviour in test harness

* fix(ci): work around buildx command error

* fix(ci): revert the workaround
2022-03-03 13:03:06 +00:00
teor 5cc7f46200
fix(codeowners): avoid multiple reviewer assignments (#3718)
Assign more code to the largest review team, to avoid multiple reviewer assignments.
2022-03-03 12:50:10 +00:00
Gustavo Valverde c176e2a423
feat(build): add arm64 support with cross-compilation (#3678)
* add(actions): lightwalletd continous integrations

* refactor(actions): build lightwalletd and reuse it in zebra

- Download lightwalletd source code
- Create a new Dockerfile for lightwalletd
- Use lightwalletd binary in Zebra's image
- Create a specific step to build/update lightwalletd
- Add lightwalletd integration test to the test suite
- Remove lightwalletd.yml, as it was harder to control

* refactor(docker): organize Dockerfiles and remove unused

Fixes: #3344

* feat(build): add arrm64 support

* fix(build): do not install google-compute-engine in arm64

This package is not available for this platform

* fix(build): do not build arm64 for tests

* fix(condition): indent for better visibility

* fix(condition): wrong use of operators
2022-03-03 00:39:41 +00:00