* Explain how to resolve duplicate dependency warnings
* Explain why the errors happen and add the CI job name
* Add extra steps
* Add some examples, tools, and extra steps
* feat(build): deploy long running instances on release
Previous behavior:
Each time we merged to main new nodes would be deployed, this is an
expected behavior as we need to ensure nodes get deployed and run
without issues, but this could also replace nodes very hastily.
Expected behavior:
We want instances which would run for a longer time, to allow us to
troubleshoot issues or inspect the behavior of this instances for longer
periods of time (2+ weeks)
Applied solution:
Deploy a versioned manage instance group (MiG) using the major version
of the release semver. We just use the first part of the version to
replace old instances, and change it when a major version is released
to keep a segregation between new and old versions.
* ci(build): allow v0 as a major version tag
* fix(build): use rust conventions for versioning
* fix(deploy): improve documentation and trigger on release
* Update .github/workflows/continous-delivery.yml
Co-authored-by: teor <teor@riseup.net>
* fix(versioning): typo
* fix(deploy): use `zebrad-v1` as the instance name, with no SHA
* fix(deploy): create and update MiG must use the same name
* docs(deployments): add Continuous Delivery process
Co-authored-by: teor <teor@riseup.net>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* Save cached state on full syncs and updates
* Add an -update suffix to CI images created by updating cached state
* Make disk image names unique by adding a time suffix
* Use the latest image from any branch, but prefer the current commit if available
* Document Zebra's continuous integration tests
* Fix typos in environmental variable names
* Expand documentation
* Fix variable name typo
* Fix shell syntax
* update zebra to 1.0.0-beta.13
* add changelog
* update the release date
* Update changelog for cryptographic batch PRs
* Add `bitvec` to the changelog
* Update CHANGELOG.md
* Add zebra-network fix and tokio version bump
* Recommend that users update their rust compiler
* Update disk and network usage for recent large blocks
* Fix spacing
* Add disk and network usage update to README
* apply size suggestions
Co-authored-by: teor <teor@riseup.net>
* add known performance issues
* update changelog
* change release date
Co-authored-by: teor <teor@riseup.net>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* docs: include target tier policies
This documentation is based on Rust supported platforms and tiers, and adapted to Zebra
* docs: add supported platforms by Tier
* docs(release): remove references to use RFCs
* docs(release): adapt teams to ZFND reality
* docs(release): remove text that might generate confusion
* docs(release): refactor general wording to our needs
* docs(release): update wording in supported platforms
* docs: clarify we're not having build artifacts for Tier 2
* docs: remove stub requirements
* docs: tidy up
* docs: last fixes
* docs(platforms): improve overall documentation
* docs: remove legal language
* docs: rewording tier 3 summary
* Update book/src/user/supported-platforms.md
Co-authored-by: teor <teor@riseup.net>
* docs: remove reference to core engineering team
* fix(docs): make rust version a separate column
* tiny fix
* If a supported platform isn't pinned to a specific release, link to the latest release
* Add space for readability
* Clarify some phrasing
* Remove Rust-compiler specific language, replace with consensus rules and features
* Update book/src/user/supported-platforms.md
Co-authored-by: teor <teor@riseup.net>
* Update book/src/user/target-tier-policies.md
Co-authored-by: teor <teor@riseup.net>
* Update book/src/user/target-tier-policies.md
Co-authored-by: teor <teor@riseup.net>
* Update book/src/user/target-tier-policies.md
Co-authored-by: teor <teor@riseup.net>
Co-authored-by: teor <teor@riseup.net>
* Rename `block_by_height` to `block_header_by_height` in fin state
* Rename `tx_by_hash` to `tx_loc_by_hash` in both (non & fin) states
* Rename `utxo_by_outpoint` to `utxo_by_out_loc` in finalized state
* Reorder the column families so that they match the docs
* Update `struct Chain` in the RFCs
* Increment `DATABASE_FORMAT_VERSION` to 25
* Remove obsolete docs from `0004-asynchronous-script-verification.md`
* Remove an obsolete `TODO` from `disk_db.rs`
* Delete unused snapshots
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* Update Zebra to 1.0.0-beta.12
* Simplify tower-* versioning by updating both to 0.2.27
* Simplify version search and replace by removing an old version from the docs
* Update Cargo.lock
* Add draft changelog as of PR #4693
* Update CHANGELOG to PR #4680
* Disable the flamegraph feature by default at compile time
* Disable the journald feature by default at compile time
* Also disable inferno dependency, and rearrange features
* Disable the prometheus feature by default at compile time
* Disable the tracing filter reload feature by default at compile time
* Disable tests when corresponding features are disabled
* Add compile-time tracing features to user docs
* Add compile-time features to the metrics user docs
* Document diagnostics as part of the start command tasks and services
* breaking(diagnostics): rename "enable-sentry" feature to "sentry" (#4623)
* Also skip conflict tests when those ports are disabled
* breaking(diagnostics): rename "enable-sentry" feature to "sentry"
This is mostly:
```sh
fastmod enable-sentry sentry
```
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* fix flag
Co-authored-by: Andrew Arnott <Andrew.arnott@Microsoft.com>
* remove zebra from zcash.conf path
Co-authored-by: Andrew Arnott <Andrew.arnott@Microsoft.com>
* update the implementation of lightwalletd
Co-authored-by: Andrew Arnott <Andrew.arnott@Microsoft.com>
* Bump crate versions
* Increment the protocol user agent
* Increment the version in `README.md`
* Increment the version in `install.md`
* Update `README.md`
* Update the changelog
* Increment the versions of crates that depend on other crates
* Update `Cargo.lock`
* Explain what Zebra does when it starts up
* Put some tasks into sub-sections
* Add non-zero progress log line
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
* Explain DNS seeder
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
* bump versions
* add the changelog for beta 10
* Update CHANGELOG.md
Co-authored-by: Marek <mail@marek.onl>
* Distinguish CI and Rust dependency updates
* Include all the dep bumps in PR #4405
* join 2 dependencies
* Include incrementalmerkletree dep bump in changelog
Co-authored-by: Deirdre Connolly <durumcrustulum@gmail.com>
Co-authored-by: Marek <mail@marek.onl>
* fix(ci): sentry is not longer being activated in test builds
This removes sentry from all the test execution, as some tests might fail as sentry wasn't initially built, or it might take more time to build as it will have to build with sentry.
* fix(build): workaround the failed to fetch oauth token error
* Drop sentry dependencies when enable-sentry feature is disabled
* Make lightwalletd gRPC tests depend on a new lightwalletd-grpc-tests feature
* fix(ci): remove enable-sentry feature from tests
* Add lightwalletd-grpc-tests feature for functionality or efficiency
And document where it is just used to stop re-compilations.
* Remove redundant `cmake` and `protobuf-compiler` dependencies
* Document Zebra's optional production and test feature flags
* Minimise dependencies in zcash-params/Dockerfile
* Minimise dependencies in docker/Dockerfile
* Add a workflow TODO
Co-authored-by: Gustavo Valverde <gustavo@iterativo.do>
* Update changelog for 1.0.0-beta.8
* Bump Zebra crate versions
The tower crates and their dependencies haven't changed,
so their versions don't need to be bumped.
Some Zebra crates haven't changed,
but I bumped all the versions to keep them consistent.
* Explicitly say that we support NU5 testnet 2
* Expand PR #3799 into changelog entries
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* Update changelog for v1.0.0-beta.7
* Increment all crate versions
* Remove redundant release test that is now covered by CI
* Remove completed NU5 README check task from the release template
* Add Merge Freeze tool to the release checklist
* Simplify release checklist by removing unused steps
* Re-order column families in design in dependency order
* Minor RFC design tweaks and fixes
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* Simplify the database design using prefix iterators
* Fix typos and missed changes
Co-authored-by: Marek <mail@marek.onl>
Co-authored-by: Marek <mail@marek.onl>
* refactor(db): simplify block height serialization
* refactor(db): make height serialization length generic
* refactor(db): create a TransactionIndex type
This changes the names of some snapshot types,
but doesn't change any data.
* refactor(db): create transparent OutputIndex and OutputLocation types
This keeps the same serialization, to avoid changing the database version.
* doc(rfc/db): make transparent database type names consistent
* doc(rfc/db): fix a bug in the Utxo.is_coinbase derivation
* fix(db): use the correct serialized size for OutputLocation
* doc(README): remove completed Zebra goals
* doc(README): docker now uses bullseye
* doc(README): clarify and expand disk requirements
* doc(README): add network latency requirement
Also note extra network usage after database format changes.
* doc(run): de-duplicate README info
* doc(run): speed up Zebra's performance
* Updating zebra-test to v1.0.0-beta.4
* Updating tower-fallback to v0.2.16
* Updating tower-batch to v0.2.20
* Updating zebra-chain to v1.0.0-beta.4
* Updating zebra-script to v1.0.0-beta.4
* Updating zebra-network to v1.0.0-beta.4
* Updating zebra-state to v1.0.0-beta.4
* Updating zebra-consensus to v1.0.0-beta.4
* Updating zebra-utils to v1.0.0-beta.4
* Updating zebrad to v1.0.0-beta.4
* Update Cargo.lock with updated crate versions
* Add section for 1.0.0-beta.4 in the CHANGELOG
* Update CHANGELOG.md
Co-authored-by: teor <teor@riseup.net>
* Update README.md
Co-authored-by: teor <teor@riseup.net>
* Update book/src/user/install.md
Co-authored-by: teor <teor@riseup.net>
* Update release description to mention Rust 2021
Update the changelog to mention that all crates now use Rust 2021
Edition.
* Elaborate on Section 3.6 documentation entry
Add the title of the section and mention that it's a section of the
Zcash protocol specification.
* Make changelog entries consistent
Use the same format for entries related to consensus rule documentation.
Co-authored-by: Marek <mail@marek.onl>
* Update README about validated consensus rules
Zebra now validates all documented consensus rules.
* Add changelog entry for newly merged PR
Describe the security fix in the changelog.
Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>
Co-authored-by: teor <teor@riseup.net>
Co-authored-by: Marek <mail@marek.onl>
* Avoid sequential borrows in `LatestChainTip`
Calling `watch::Receiver::borrow` more than once in the same scope can
cause a deadlock. The instrumented methods were calling `borrow` twice
to record instrumented fields.
This refactors things to ensure `borrow` is only called once to record
the fields and perform any actions with the chain tip block.
* Remove `borrow()` calls in `ChainTipChange`
Refactor to use a `LatestChainTip` instance instead, which safely
protects the internal `watch::Receiver` so that it is not borrowed more
than once in the same scope.
* Add a paragraph to the Asynchronous guide
Warn against using two borrow guards in the same scope, and describe why
that can lead to a deadlock.
* Draft design for lightwalletd database support
* Explain how to create blocks from headers and transactions
* Make keys smaller to improve database performance
* Support transparent address balances and UTXOs
Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>
* Justify that the ErrorSlot Mutex is deadlock-safe
* Document cancellation safety in the async RFC
* Document task starvation in the async RFC
Co-authored-by: Marek <mail@marek.onl>
* Update the State RFC to match the current database format
* Formatting and name fixes
* Remove redundant generic parameter
* Remove redundant generics
* Fix history tree types
Co-authored-by: Conrado Gouvea <conrado@zfnd.org>
* Fix spacing
Co-authored-by: Conrado Gouvea <conrado@zfnd.org>
* Add CompactSize64 and CompactSizeMessage types
But don't remove read_compactsize and write_compactsize yet.
* Fix CompactSize capitalisation
```sh
fastmod compactSize CompactSize zebra* book
fastmod compactsize CompactSize zebra* book
```
* Make CompactSize patterns consistent with integer lengths
* Replace unwrap_err with asserting is_err
Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>
* Replace a panic with an assertion
* Make generic serialization use CompactSizeMessage
* Fix type inference and borrow-checker errors
* Doctest fixes
Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>
* Improve documentation
* Incorporate text about Zebra from the last newsletter
* Organize README and user docs
* Add table of contents, organize heading levels
* Fix link
* capitalize list items
* fix table of contents
* format spacing issue
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
* Update the signs in the value pools design to match the spec
Also split the definitions and guide sections into transaction value balances
and chain value pools. And explain the sign differences between them in terms of
inputs and outputs.
* fix typo
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
* Design: Fix Transparent Value Sign and Definition
* make the transparent value pool the sum of unspent output values, like other pools
* swap the sign of the transparent value balance
* make the remaining transaction value balance the sum of unspent input values
* update the consensus rules based on recent spec changes
* explain why the transparent chain value pool balance can't be negative
* clarify the guide and consensus rules sections
* Use consistent notation for sprout value balances
* Fix typos
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
* Update state RFC for double-spends and other recent designs
* Update the value pool column family name
* Mark incremental note commitment trees as tentative
* Change history tree type
* Apply suggestions from code review
* add a draft for value pools design
* add some `ValueBalance` operators
* add `value_balance` methods to modules
* fix some minors
* finalize the last part of the implementation design
* replace wrong AllowNegative with correct NegativeAllowed
* add design PR to header
* update scope
* remove details from transaction `value_balance()`
* update definitions
* change method name
* return Result in operators
* fix the TODOs
* implement `UpdateWith<PreparedBlock>` for `ValueBalance`
* add details to `ValueBalance` serialization
* add a panic to block value balance
* fix `remaining_transaction_value()`
* add the block value balance into `FinalizedState`
* populate the `Chain` with the finalized tip value balance
* remove redundant text from definition
* add docs to `Chain` field
* trigger the `remaining_transaction_value()` check
* fix mistake
* add some tests, remove some not needed sections
* add a summary of the implementation
* do some minor fixes to consensus rules text
* clarify some names
* fix `UpdateWith`
* move the remaining transaction value consensus rule
* fix serialization bug
* fix typo
* Add a missing test
* typo
Co-authored-by: Conrado Gouvea <conradoplg@gmail.com>
Co-authored-by: teor <teor@riseup.net>
Co-authored-by: Conrado Gouvea <conradoplg@gmail.com>
* Modify UTXO and state designs for transparent coinbase output checks
* Add missing word
* Clarify unspent transparent transaction outputs
And add a definition for OutPoint
* Fix coinbase flag derivation
* Rename SpendPools to SpendRestriction
* Replace coinbase_height with Coinbase.height
And clarify a height to spend_height
* Make block height italic to match spec formatting
* Provide a value for MIN_TRANSPARENT_COINBASE_MATURITY
* Add new v1-4 transaction ID consensus rule
* Make the design match the existing Utxo code
This minimises the changes we'll need to make.
* Add a justification for parallel coinbase verification
* Change to Response::SpendableUtxo
To avoid confusion.
* Update the state RFC to match the current design
* Move a change to another PR to avoid conflicts
* Fix spacing
* Clarify the future returned by `PendingUtxos::queue(u)`
* Add missing PendingUtxos::queue arguments
* RFC: add more background to atomics
- Provide explicit tested alternatives to atomics
- Explain the differences between x86 and other processors
* Add nomicon link and further explanation
Updates:
- GitHub Issue templates
- GitHub PR templates
- RFC template
Focusing on:
- consensus rule / network reference sections
- design sections
- review/test checklist
Process changes:
- add new team members to RFC approval
- change RFC approval to "most of the team"
And general cleanup:
- delete docs from the checklist, because we now `warn(missing_docs)`
- shorter explanations
- consistent headings
- consistent order
- consistent formatting
* Standardise lints across Zebra crates, and add missing docs
The only remaining module with missing docs is `zebra_test::command`
* Todo -> TODO
* Clarify what a transcript ErrorChecker does
Also change `Error` -> `BoxError`
* TransError -> ExpectedTranscriptError
* Output Descriptions -> Output descriptions
* Make sure the Canopy activation block is a finalized checkpoint block
This enables ZIP-221 chain history from Canopy activation onwards.
* Clarify that the mandatory checkpoint test includes Canopy activation
The test was correct, but the docs and assertion message did not include activation.
* Document that the mandatory checkpoint includes Canopy activation
Co-authored-by: teor <teor@riseup.net>
* add nullifier methods to orchard
* store orchard nullifiers
* bump database version
* update `IntoDisk`
* support V5 in `UpdateWith`
* add a test for finalized state
* Use the latest network upgrade in state proptests
* Initial async RFC version
* Add a table of contents
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
* Add a toc anchor
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
* Add some words that need definitions
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
* Write guide intro based on feedback
* Add a code example for each reference section
* Link to code examples using commit hashes
* Link to PR and commit for each code example
* Fix typos
Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>
* Remove redundant version in docs.rs link
* Link the guide to the reference
And expand the guide descriptions
* Mention TurboWish as a future diagnostic tool
* Add an example of a compiler error that prevents deadlock
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
Co-authored-by: Deirdre Connolly <deirdre@zfnd.org>
* Redesign Sapling data model for V5 shared anchor and spends
The shared anchor is only present if there are any spends.
As part of this change, delete the manual PartialEq impl and its tests,
because we can derive PartialEq now.
* Stop creating a temporary Vec for the spend and output iterators
* Rename TransferData variants
Interactive rename using the following commands:
```sh
fastmod Spends SpendsAndMaybeOutputs
fastmod NoSpends JustOutputs
```
* Refactor out common sprout nullifier code
* Implement the AtLeastOne constrained vector type
This vector wrapper ensures that it always contains at least one element.
* Simplify Sapling TransferData using AtLeastOne
Also update the RFC to use AtLeastOne for Orchard.
* add sapling shielded data to transaction V5
* implement nullifiers
* test v5 in shielded_data_roundtrip
* Explicitly design serialization for Transaction V5
Implement serialization for V4 and V5 spends and outputs, to make sure
that the design works.
* Test serialization for v5 spends and outputs
Also add a few missing v4 tests.
* Delete a disabled proptest
* Make v5 transactions a top-level heading
And add a missing serialized type.
* Fix a comment typo
* v5 transaction RFC: split array serialization
Based on #2017
* RFC: explicitly describe serialized field order
And link to the spec
* RFC: add the shared anchor serialization rule test
Co-authored-by: teor <teor@riseup.net>
* create block subsidy rfc
* add reference in miner subsidy
* remove `or 0` from function descriptions.
It is clear from the signature that the function will return `Error` on any failure(including input errors, for example a wrong height). Describing this will force us to write it for all functions that returns an error so better remove it.
* add links to subsidy categories
* fix design in Funding streams parameter constants
* fix order in FUNDING_STREAM_RECEIVER_NUMERATORS
* return `PayToScriptHash` in `funding_stream_address`
* add shielded coinbase
* Apply suggestions from code review
Co-authored-by: teor <teor@riseup.net>
Co-authored-by: Daira Hopwood <daira@jacaranda.org>
* improve transparent value pool section
* add a test plan section
* rename `subsidy_is_correct` to `subsidy_is_valid`
* add funding streams address constants
* fix funding stream addresses
* add errors
* remove ending dot for error descriptions
Co-authored-by: Jane Lusby <jlusby42@gmail.com>
* modify founders reward
* change ECC to BP
* add note about state needed for transparent pool
* split constants
* Tweak block subsidy RFC wording
Co-authored-by: Jane Lusby <jlusby42@gmail.com>
* Clarify transaction fees definition
* Use consistent fee terminology
* Clarify transparent value pool
* Revise implementation plan based on latest priorities
* Change tests based on new priorities
* Fix markdown
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
* Fix value pool description
* change block and miner subsidy definitions
* fee pool definitions, fix all links to protocol
* add reference to zip209
* add note for MG stream number of addresses
Co-authored-by: teor <teor@riseup.net>
Co-authored-by: Daira Hopwood <daira@jacaranda.org>
Co-authored-by: Jane Lusby <jlusby42@gmail.com>
* rfc: initial draft of release process proposal
This is a version of the notes I posted in slack for preliminary
discussion, slightly reformatted to fit the Zebra RFC template.
@yaahc suggested I post them as an RFC, to have a concrete proposal for
discussion.
* Add draft note and link to ticket and PR
Co-authored-by: teor <teor@riseup.net>
When we're naming the anchors without the corresponding type or struct,
it's not clear if they are shared or per spend.
Rename the fields as `shared_anchor` or `per_spend_anchor`.
* propose a v5 transaction rfc
* define orchard flags
* Update test sections
* Add security section
* move some code into sapling and orchard crates, do renames
* Move sapling value balance into shielded data
* Add AuthorizedAction type
* Order fields based on last serialized data
* Add the proofsOrchard field
* Delete visibility modifiers for readability
All those `pub`s make the design harder to read.
* Model anchor variants as generic trait types
* Be specific about the network upgrade
* Specify a file for each new type
* Document how the Orchard flags are used
Co-authored-by: teor <teor@riseup.net>
teor
Alfredo Garcia
Marek
Gustavo Valverde
Andrew Arnott
Deirdre Connolly
Conrado Gouvea
ebfull
Janito Vaqueiro Ferreira Filho
Pili Guerra
Dimitris Apostolou
Deirdre Connolly
Kirill Fomichev
Henry de Valence