zebra/zebra-state
teor d651ee3c16
change(db): Upgrade subtrees from the tip backwards, for compatibility with wallet syncing (#7531)
* Avoid manual handling of previous sapling trees by using iterator windows instead

* Avoid manual sapling subtree index handling by comparing prev and current subtree indexes instead

* Simplify adding notes by using the exact number of remaining notes

* Simplify by skipping the first block, because it can't complete a subtree

* Re-use existing tree update code

* Apply the sapling changes to orchard subtree updates

* add a reverse database column family iterator function

* Make skipping the lowest tree independent of iteration order

* Move new subtree checks into the iterator, rename to end_height

* Split subtree calculation into a new method

* Split the calculate and write methods

* Quickly check the first subtree before running the full upgrade

* Do the quick checks every time Zebra runs, and refactor slow check error handling

* Do quick checks for orchard as well

* Make orchard tree upgrade match sapling upgrade code

* Upgrade subtrees in reverse height order

* Bump the database patch version so the upgrade runs again

* Reset previous subtree upgrade data before doing this one

* Add extra checks to subtree calculation to diagnose errors

* Use correct heights for subtrees completed at the end of a block

* Add even more checks to diagnose issues

* Instrument upgrade methods to improve diagnostics

* Prevent modification of re-used trees

* Debug with subtree positions as well

* Fix an off-by-one error with completed subtrees

* Fix typos and confusing comments

Co-authored-by: Marek <mail@marek.onl>

* Fix mistaken previous tree handling and end tree comments

* Remove unnecessary subtraction in remaining leaves calc

* Log heights when assertions fail

* Fix new subtree detection filter

* Move new subtree check into a method, cleanup unused code

* Remove redundant assertions

* Wait for subtree upgrade before testing RPCs

* Fix subtree search in quick check

* Temporarily upgrade subtrees in forward height order

* Clarify some comments

* Fix missing test imports

* Fix subtree logging

* Add a comment about a potential hang with future upgrades

* Fix zebrad var ownership

* Log more info when add_subtrees.rs fails

* cargo fmt --all

* Fix unrelated clippy::unnecessary_unwrap

* cargo clippy --fix --all-features --all-targets; cargo fmt --all

* Stop the quick check depending on tree de-duplication

* Refactor waiting for the upgrade into functions

* Wait for state upgrades whenever the cached state is updated

* Wait for the testnet upgrade in the right place

* Fix unused variable

* Fix a subtree detection bug and comments

* Remove an early reference to reverse direction

* Stop skipping subtrees completed at the end of blocks

* Actually fix new subtree code

* Upgrade subtrees in reverse height order

Reverts "Temporarily upgrade subtrees in forward height order"
This reverts commit a9558be214.

* Bump the database patch version to re-run the upgrade (for testing)

* Revert "Remove an early reference to reverse direction"

This reverts commit c206404377.

---------

Co-authored-by: Marek <mail@marek.onl>
2023-09-20 23:41:28 +00:00
..
proptest-regressions/service feat(state): add transparent address indexes to the non-finalized state (#4022) 2022-04-12 17:21:46 +00:00
src change(db): Upgrade subtrees from the tip backwards, for compatibility with wallet syncing (#7531) 2023-09-20 23:41:28 +00:00
tests rename(state): Rename state verifiers and related code (#6762) 2023-06-01 12:29:03 +00:00
Cargo.toml build(deps): bump the log-time group with 2 updates (#7579) 2023-09-20 23:41:05 +00:00