Commit Graph

181 Commits

Author SHA1 Message Date
Arya 0f5450f565
change(release): Reduce the end of support time from 20 weeks to 16 weeks (#8530)
* Reduce the end of support time from 20 weeks to 16 weeks.

* Update release-process section of the book

* Adds changelog entry

* Update CHANGELOG.md

Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>

---------

Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
2024-05-15 21:12:52 +00:00
Alfredo Garcia 0ca01e6a46
chore: Release v1.7.0 (#8507)
* changelog and readme for 1.7.0 release

* chore: Release

* update zebra-scan and zebra-grpc to the release crates dry run script

* update ESTIMATED_RELEASE_HEIGHT

* update the release period to 6 weeks

* update release block

* Apply suggestions from code review

Co-authored-by: Marek <mail@marek.onl>

* Apply suggestions from code review

Co-authored-by: Marek <mail@marek.onl>

* Apply suggestions from code review

Co-authored-by: Marek <mail@marek.onl>

---------

Co-authored-by: Marek <mail@marek.onl>
2024-05-07 14:25:50 +00:00
Alfredo Garcia d869ec6fe4
change(release): Adjust estimated release interval and end of support (#8429)
* change estimated release interval and end of support

* update readme
2024-04-24 19:22:07 +00:00
teor b2e71f41fe
change(doc): Document how to add a column family (#8149)
* Document how to add a column family

* Add the column family to the list in the DB

* Fix rust formatting
2024-01-12 02:11:06 +00:00
Alfredo Garcia 2d217e29e7
rename worflows to fix readme badges (#8116) 2023-12-18 21:07:40 +00:00
teor 828c2aea57
doc(devops): Explain how to change branch protection rules (#7883)
* Explain how to change branch protection rules

* Fix list formatting

* Add missing .

* Fix incorrect link

Co-authored-by: Pili Guerra <mpguerra@users.noreply.github.com>

---------

Co-authored-by: Pili Guerra <mpguerra@users.noreply.github.com>
2023-11-02 16:59:20 +00:00
teor afbe807060
change(docs): Replace doc.zebra.zfnd.org API documentation with docs.rs (#7886)
* Simple replacements of doc.zebra.zfnd.org with docs.rs

* Manual fixes for specific main/internal/external docs

* Point developer docs to doc-internal.zebra.zfnd.org

* fastmod --glob '\!.git' -- doc.zebra.zfnd.org/zebrad docs.rs/zebrad/latest/zebrad

* Manually remove any remaining doc.zfnd.zebra.org links

* Remove the external docs job

* Add changelog entry and fix links

* Fix links that were broken before this PR
2023-11-01 22:09:40 +00:00
teor a126acb160
change(doc): Document state format upgrade implementation and testing (#7799)
* Explain verification modes

* Explain block write modes

* Expand upgrade goals

* Document testing requirements

* Update current format docs

* Move the detailed checklist to the ticket template

* Move an example back

* Reword confusing paragraphs

Co-authored-by: Arya <aryasolhi@gmail.com>

* Explain the difference between major and minor/patch versions

* Simplify upgrade wording

Co-authored-by: Marek <mail@marek.onl>

---------

Co-authored-by: Arya <aryasolhi@gmail.com>
Co-authored-by: Marek <mail@marek.onl>
2023-10-25 22:18:41 +00:00
Alfredo Garcia c173198d4f
docs(build): Add ECC dependencies documentation (#7794)
* add ecc dependencies documentation

* add `cargo deny` suggestion to deny.toml section

Co-authored-by: Arya <aryasolhi@gmail.com>

* change sententence syntax

Co-authored-by: Arya <aryasolhi@gmail.com>

* Apply suggestions from code review

Co-authored-by: teor <teor@riseup.net>

* move consistency check

---------

Co-authored-by: Arya <aryasolhi@gmail.com>
Co-authored-by: teor <teor@riseup.net>
2023-10-25 03:27:16 +00:00
teor aad883b68a
change(consensus): Build Sprout and Sapling parameters into the zebrad binary, so a download server isn't needed (#7800)
* Build sapling parameters into zebrad

* Load the sapling parameters from wagyu-zcash-parameters

* Reword logs

* wget https://github.com/zcash/zcash/raw/master/src/rust/src/sprout-groth16.vk

* Load sprout from binary and remove download code

* Remove download examples binaries

* Remove unused features and dependencies

* Remove docs and comments about downloading parameters

* Add CHANGELOG entry

* Remove parameter cache steps in workflows

* Update comment about downloading

Co-authored-by: Marek <mail@marek.onl>

* Fix outdated timeout comments and short timeouts

---------

Co-authored-by: Marek <mail@marek.onl>
2023-10-24 23:27:24 +00:00
teor bb7eeb6f76
Document how to fix deny.toml warnings (#7806) 2023-10-24 13:12:57 +00:00
teor b04cca9ec5
change(dev doc): Document how to change branch protection rules (#7773)
* Document branch protection rules

* Document automatic jobs for new crates

* Link to patch workflow docs

* Update book/src/dev/continuous-integration.md

Co-authored-by: Marek <mail@marek.onl>

---------

Co-authored-by: Gustavo Valverde <gustavo@iterativo.do>
Co-authored-by: Marek <mail@marek.onl>
2023-10-22 15:40:19 +00:00
teor 64f777274c
fix(security): fix concurrency issues in tree key formats, and CPU usage in genesis tree roots (#7392)
* Add tree key format and cached root upgrades

* Document the changes in the upgrades

* Remove unnecessary clippy::unwrap_in_result

* Fix database type

* Bump state version

* Skip some checks if the database is empty

* Fix tests for a short state upgrade

* Disable format checks in some tests

* Document state performance issues

* Clarify upgrade behaviour

* Clarify panic messages

* Delete incorrect genesis trees write code

* Fix metrics handling for genesis

* Remove an unused import

* Explain why genesis anchors are ok

* Update snapshots

* Debug a failing test

* Fix some tests

* Fix missing imports

* Move the state check in a test

* Fix comment and doc typos

Co-authored-by: Marek <mail@marek.onl>
Co-authored-by: Arya <aryasolhi@gmail.com>

* Clarify what a long upgrade is

* Rename unused function arguments

Co-authored-by: Marek <mail@marek.onl>

* Add all_unordered log regex matching methods

* Fix timing issues with version upgrades and other logs

* Fix argument name in docs

Co-authored-by: Marek <mail@marek.onl>

* Explain match until first for all regexes behaviour better

---------

Co-authored-by: Marek <mail@marek.onl>
Co-authored-by: Arya <aryasolhi@gmail.com>
2023-10-19 14:50:46 +00:00
Gustavo Valverde fc0133e886
ref(workflows): consolidate workflows based on their purpose (#7616)
* ref(workflows): consolidate workflows based on their purpose

This also renames the workflows to make their naming more consistent and adding a naming convention

Fixes: #6166
Fixes: #6167

* fix(workflows): use correct name for patch

* fix(workflow): docker unit tests

* fix(release): validation error

Error:
```
Validation Failed: {"resource":"Release","code":"invalid","field":"target_commitish"}
```

Fixes: https://github.com/release-drafter/release-drafter/issues/1125

* fix(workflows): reference correct name

* fix: remove extra workflow

* fix(workflows): use larger runners

* fix(workflow): remove code already in docker unit-test

* fix(unit-tests): start zebra the right way

* fix: typo in patch name

* chore: move job to logical order

* imp(workflows): use better name for gcp tests

* add: missing merge changes

* chore: use better name for find-disks

* fix(ci): use the `entrypoint.sh` to change the Network

* fix(ci): add missing `ZEBRA_CONF_PATH` variable

* fix(ci): allow to build the entrypoint file with testnet

* fix(entrypoint): allow to create a dir and file with a single variable

* refactor: test config file in CI and CD with a reusable workflow

* fix(ci): wrong name used

* fix(ci): use checkout

* fix(ci): improve docker config tests

* fix(ci): use better name for protection rules

* Fix changed workflow file name in docs patch file

* Apply suggestions from code review

Co-authored-by: teor <teor@riseup.net>
Co-authored-by: Arya <aryasolhi@gmail.com>

* fix(cd): depend on file tests

* fix(docs): adapt to new workflow name

* fix: revert test coverage on CD

* chore: reduce diff

* fix(ci): allow using variable images for reusable workflows

* fix(dockerfile): use variables or default for config path and file

* fix(entrypoint): if `$ZEBRA_CONF_PATH` is set, do not override it

* Fix patch job names and remove failure job testnet dependencies

---------

Co-authored-by: teor <teor@riseup.net>
Co-authored-by: Arya <aryasolhi@gmail.com>
2023-10-18 06:16:02 +00:00
teor bea90dfd62
change(doc): Add instructions for new placeholder crates (#7751)
* Add instructions for new placeholder crates

* minor changes

---------

Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
2023-10-17 21:54:01 +00:00
teor dbdb4be429
add(doc): Add instructions for changing crate owners (#7572)
* Add instructions for changing crate owners

* Explain new owner invites

* Remove duplicate `cargo login` instructions from the release checklist

* Remove owners as well

* Missing log line
2023-09-20 00:40:48 +00:00
teor 2dd57cb639
Delete documentation about fixing job timeouts (#7540) 2023-09-13 08:30:08 +00:00
Arya 94d9155adb
change(state): Add note subtree index handling to zebra-state, but don't write them to the finalized state yet (#7334)
* zebra-chain changes from the subtree-boundaries branch

```sh
git checkout -b subtree-boundaries-zebra-chain main
git checkout origin/subtree-boundaries zebra-chain
git commit
```

* Temporarily populate new subtree fields with None - for revert

This temporary commit needs to be reverted in the next PR.

* Applies suggestions from code review

* removes from_repr_unchecked methods

* simplifies loop

* adds subtrees to zebra-state

* uses split_at, from_repr, & updates state-db-upgrades.md

* Update book/src/dev/state-db-upgrades.md

Co-authored-by: teor <teor@riseup.net>

* renames partial_subtree to subtree_data

* tests that subtree serialization format

* adds raw data format serialization round-trip test

* decrements minor version and skips inserting subtrees in db

---------

Co-authored-by: teor <teor@riseup.net>
2023-08-28 08:50:31 +00:00
teor 512dd9bc5d
change(doc): Document how to upgrade the database format (#7261)
* Move the state format into a new doc

* Add upgrade instructions

* Link to the format upgrade docs from the upgrade code

* Fix typo

Co-authored-by: Marek <mail@marek.onl>

---------

Co-authored-by: Marek <mail@marek.onl>
2023-07-20 01:50:25 +00:00
teor 56a76385f0
git ls-tree --full-tree -r --name-only HEAD | xargs sed -i -e 's/router_verifier/block_verifier_router/g' (#6998)
cargo fmt --all
2023-06-20 07:11:04 +00:00
teor 98478c3b75
fix(doc): Add `fastmod --hidden` to mass-renames.md (#6913)
* Add `fastmod --hidden` to mass-renames.md

* Do multiple skip paths

* Do multiple sed renames in the same sed

---------

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2023-06-15 18:35:44 +00:00
Deirdre Connolly 8bf2c4be3f
change: Rename tower-batch to tower-batch-control (#6907) 2023-06-12 03:59:10 +00:00
teor 0ffa47cc1e
Add instructions for doing mass renames easily (#6865) 2023-06-08 10:50:18 +00:00
Alfredo Garcia eb07bb31d6
rename(state): Rename state verifiers and related code (#6762)
* rename verifiers

* rename `PreparedBlock` to `SemanticallyVerifiedBlock`

* rename `CommitBlock` to `SemanticallyVerifiedBlock`

* rename `FinalizedBlock` to `CheckpointVerifiedBlock`

* rename `CommitFinalizedBlock` to `CommitCheckpointVerifiedBlock`

* rename `FinalizedWithTrees` to `ContextuallyVerifiedBlockWithTrees`

* rename `ContextuallyValidBlock` to `ContextuallyVerifiedBlock`

* change some `finalized` variables or function arguments to `checkpoint_verified`

* fix docs

* document the difference between `CheckpointVerifiedBlock` and `ContextuallyVerifiedBlock`

* fix doc links

* apply suggestions to request

Co-authored-by: Marek <mail@marek.onl>

* apply suggestions to service

Co-authored-by: Marek <mail@marek.onl>

* apply suggestions to finalized_state.rs and write.rs

Co-authored-by: Marek <mail@marek.onl>

* fmt

* change some more variable names

* change a few missing generics

* fix checkpoint log issue

* rename more `prepared` vars `semantically_verified`

* fix test regex

* fix test regex 2

---------

Co-authored-by: Marek <mail@marek.onl>
2023-06-01 12:29:03 +00:00
teor 943d213490
Temporarily delete unused zebra-cli and zebra-client crates (#6726) 2023-05-19 04:45:54 +00:00
Alfredo Garcia 58bd898f5b
feat(zebrad): Refuse to run zebrad when release is too old (#6351)
* refuse to run Zebra if it is too old

* update the release checklist to consider the constants

* bring newline back

* apply new end of support code

* attempt to add tests (not working yet)

* move eos to progress task

* move tests

* add acceptance test (not working)

* fix tests

* change to block height checks (ugly code)

* change warn days

* refactor estimated blocks per day, etc

* move end of support code to its own task

* change test

* fix some docs

* move constants

* remove uneeded conversions

* downgrade tracing

* reduce end of support time, fix ci changing debugs to info again

* update instructions

* add failure messages

* cargo lock update

* unify releaase name constant

* change info msg

* clippy fixes

* add a block explorer

* ignore testnet in end of support task

* change panic to 16 weeks

* add some documentation about end of support

* Tweak docs wording

---------

Co-authored-by: teor <teor@riseup.net>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2023-04-28 14:13:21 +00:00
teor dde63d27e9
change(docs): Document how to get external PRs to merge (#6252)
* Document how to get external PRs to merge

* Fix typos

Co-authored-by: Marek <mail@marek.onl>

---------

Co-authored-by: Marek <mail@marek.onl>
2023-03-01 11:52:48 +00:00
Jack Gavigan fc955152b7
Update zebra-dependencies-for-audit.md (#6141) 2023-02-10 14:20:53 +00:00
teor 4e12a289a6
Update audit scope based on audit-v1.0.0-rc.0 tag (#6109)
* Update audit scope based on audit-v1.0.0-rc.0 branch

* Minor wording fixes and clarifications

Co-authored-by: Arya <aryasolhi@gmail.com>

* Set scope for `reddsa` and explain code movement

* Just base everything on rc.0

* I'm just about to make it into a tag, not a branch

* Make it clearer that the Crates categories are Zebra crates

* Add reddsa and redjubjub

Co-authored-by: Deirdre Connolly <durumcrustulum@gmail.com>

* Fix links to red(jubjub|dsa) signing_key

* Add missing space

* Remove `redjubjub` from "Out of scope"

* Fix formatting

* Make previous audit clearer

* Add missing space

Co-authored-by: Arya <aryasolhi@gmail.com>

---------

Co-authored-by: Arya <aryasolhi@gmail.com>
Co-authored-by: Deirdre Connolly <durumcrustulum@gmail.com>
2023-02-08 00:48:30 +00:00
Alfredo Garcia d72211fbbe
feat(zebra-checkpoints): make `zebra-checkpoints` work for zebrad backend (#5894)
* make `zebra-checkpoint` util work with zebra as the backend

* update snapshots

* update documentation

* applies suggestions from code review

* irefactor zebra-checkpoints to work with zebra using deserialization of the raw block

* fix imports and derives

* rename mode to backend

* remove old stuff

* fix docs

Co-authored-by: arya2 <aryasolhi@gmail.com>
2023-01-23 04:50:45 +00:00
Arya 99cbffbcc0
change(docs): Update 'Design Overview' page of Zebra book (#5892)
* Updates design overview page of book with:
- a simplified graph of service dependencies in design overview
- links to notable blog posts
- removals of some out-of-date text

also corrects issue/pr refs in an rfc

* Updates service dependencies diagram

* Apply suggestions from code review

Co-authored-by: teor <teor@riseup.net>

* updates diagram and adds doc links

* uses opacity to indicate clickability instead of drop shadow

Co-authored-by: teor <teor@riseup.net>
2023-01-10 10:10:45 +00:00
Arya ea21e642dc
Fix typos (#5728)
Co-authored-by: Dimitris Apostolou <dimitris.apostolou@icloud.com>
2022-11-27 22:37:55 +00:00
Pili Guerra 13cd8b9c2c
cleanup(deps): Remove unused `fpe` dependency and cleanup references (#5541)
* Clean up references to fpe that were not removed when we removed orchard code

* Update Cargo.lock

Co-authored-by: Deirdre Connolly <durumcrustulum@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-11-04 09:04:14 +00:00
teor 26d0455d02
fix(doc): Use a more reliable release template branch name and docker command (#5519)
* Fix the branch name in the release template

* Use a docker command with colour and Ctrl-C support

* Make branch name example more readable

* Fix a link typo

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-11-03 00:28:46 +00:00
Alfredo Garcia f27fa21099
docs(audit): Add a list of Zebra dependencies for the audit (#5395)
* add a list of dependencies for auditors

* change intro

Co-authored-by: teor <teor@riseup.net>

* add zebra-checkpoints to audit

* replace colors in the legend as they don't render in github

* make crypto deps out of the scope

* add a reason to keep rocksdb out of the scope

* add a not of why chrono is out of the scope

* add note for bitvec dep

* move structopt to out of scope

* add zcash_script as partially in scope

* applies suggestion, updates versions, bolds ed25519-zebra

* adds zcash_proofs audit notes

* Splits list of zebra deps into sections for full audits, partial audits, and out of scope.

* Update book/src/dev/zebra-dependencies-for-audit.md

Co-authored-by: teor <teor@riseup.net>

* Update book/src/dev/zebra-dependencies-for-audit.md

* Add missing newline

* Update book/src/dev/zebra-dependencies-for-audit.md

Co-authored-by: teor <teor@riseup.net>

* updates crate versions

* Update zcash_script to 0.1.8

* Only link to the audit version of each crate or file

* Clarify out of scope versions

* Fix source paths

* Fix zcash_script paths

* Improve formatting

* Update zebra-dependencies-for-audit.md

Co-authored-by: teor <teor@riseup.net>
Co-authored-by: arya2 <aryasolhi@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-11-02 02:42:20 +00:00
teor 17c0bacca3
add(doc): Add draft audit process, and update some support docs (#5433)
* Document parameter download workaround, particularly for macOS

* Replace outdated build docs with a link to the README

* Remove incorrectly rendered heading links

* Add an initial draft audit process

* Add dependency criteria
2022-10-21 01:20:28 +00:00
Gustavo Valverde 65e9822c43
docs(release): add information on how we tag and release (#5392)
* draft

* docs(release): add information on how we tag and release

* docs(release): remove outdated document

* Apply suggestions from code review

Co-authored-by: teor <teor@riseup.net>

* Remove rough draft analysis of alternatives

Co-authored-by: teor <teor@riseup.net>
2022-10-19 19:31:28 +00:00
Arya de3640ddcc
adds zebra book's CI/CD docs to the sidebar (#5355) 2022-10-06 20:15:16 +00:00
teor 55e5a13fc8
List temporary CI errors that should be retried (#5281) 2022-09-28 13:14:39 +00:00
teor 9a2814a1b2
doc(ci): Explain how to fix Mergify and Zcash parameter download failures (#5240)
* Explain how to find Mergify failures

* Explain how to fix cache errors

* Fix instructions - clear all caches

* Fix which errors need which actions

* Add a newline to appease GitHub markdown renderer

Co-authored-by: Arya <aryasolhi@gmail.com>

Co-authored-by: Arya <aryasolhi@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-09-27 01:32:55 +00:00
teor 12d084f6cc
fix(ci): Delete Google Cloud test instances after 3 days (#5192)
* Delete test instances after 3 days

* Use correct delete command, improve shell quoting

* Use sed to provide the correct zone or region

* Fix quoting

* Fix IFS

* Fix IFS for multiple disks

* Document why we can't quote some shell variables

* Document that instances can get deleted

* Fix exact names in deletion docs
2022-09-21 23:16:38 +00:00
Pili Guerra 64d984340d
Update release-process.md (#5181)
Fixing some small typos
2022-09-16 16:25:43 +02:00
Gustavo Valverde 085367fc37
docs(release): add versioning and releases information (#4917)
* ci(docs): test `cargo doc` warnings and errors

* ci(docs): remove minimal profile

* ci(docs): revert minimal and fix component name

* ci(docs): try cargo doc alone

* docs(release): add versioning and releases information

* dosc(release): apply suggestions from code review

Co-authored-by: teor <teor@riseup.net>

* docs(release): apply multiple fixes from review

* docs(release): allow more flexibility with release cycles

* Apply suggestions from code review

Co-authored-by: teor <teor@riseup.net>

Co-authored-by: teor <teor@riseup.net>
2022-09-15 02:17:09 +00:00
teor c2514ba64a
fix(doc): Update deny.toml developer docs and file comments (#5151)
* Make deny.toml instructions more specific

* Add documentation and depdendency comments to deny.toml
2022-09-14 19:52:32 +00:00
teor dcbbf444b9
fix(ci): Increase the full sync timeout (#5129)
* Increase the full sync timeout to 28 hours

And put the timeout constant with the rest of them.

* Update developer documentation for the new constant location

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-09-13 03:52:23 +00:00
teor 4810f44564
doc(ci): Document how to resolve disk full errors (#5105)
* Document how to resolve disk full errors

* Update to the `main` branch first
2022-09-07 11:16:18 +00:00
teor e4707a6cfd
Explain how Mergify works, and when it can be used (#5100) 2022-09-06 21:47:09 +00:00
teor fec012a006
fix(ci): delete GCP resources, but keep some recent cached state images (#5082)
* Fix delete GCP resources commands

* Don't create a GCP credentials file

* Keep the latest 2 images

* Explain time

* Show the names of disks that are being deleted

* Actually run the image delete steps

* Only delete commit-based instance templates

* Document automated deletion
2022-09-06 02:51:46 +00:00
teor 9d85360243
Explain how to find errors in CI (#5080) 2022-09-05 21:43:24 +00:00
teor 21b1a622e6
fix(deny.toml): Change an exception to the earlier pin-project version (#5070)
* Use the earlier pin-project version in deny.toml

* Change deny.toml instructions to check for updated versions

* Remove trailing whitespace
2022-09-05 20:12:52 +00:00