Commit Graph

12 Commits

Author SHA1 Message Date
Arya f0c497160a
add(state): Track spending transaction ids by spent outpoints and revealed nullifiers (#8895)
* Adds new column family for [spent_out_loc] -> [spending_tx_loc] with a read method and an update to `prepare_spending_transparent_tx_ids_batch()` for maintaining it when committing blocks to the finalized state.

Adds TODOs for remaining production changes needed for issue #8837.

* add spending tx ids for spent outpoints to non-finalized chains

* adds a `spending_transaction_hash()` read fn for the new column family

* Adds a `TransactionIdForSpentOutPoint` ReadRequest and a `TransactionId` ReadResponse

* Updates snapshots, removes outdated TODOs, moves a TODO.

* Clarifies `spent_utxos` field docs, fixes an assertion

* import TypedColumnFamily from `finalized_state` instead of from the crate.

* adds db format upgrade for spent outpoints -> tx hash

* adds revealing tx ids for nullifiers in finalized and non-finalized states

* updates nullifiers column families to include revaling transaction locations in db format upgrade

* Renames new read state request to `SpendingTransactionId` and updates its type to a `Spend` enum

* refactor db format upgrade and prepare_nullifiers_batch() to use ZebraDb instead of DiskDb, checks cancel_receiver before every db operation

* Adds acceptance test for checking that the finalized state has spending transaction ids

* Adds variant docs to zebra_state::request::Spend enum

* Updates Zebra book with the latest changes to the rocks db column families

* Updates acceptance test to check non-finalized state

* adds a few log messages to the acceptance test, reduces frequency of logs for progress updates

* fixes docs lint and skips test when there is no cached state

* Avoids returning genesis coinbase tx hash when indexes are missing

* Adds `indexer` compilation feature in zebra-state and build metadata in db format version file

* stops tracking new indexes in finalized state when feature is unselected

* stops tracking new indexes in non-finalized state when indexer feature is unselected

* condenses imports

* - adds build metadata when writing db version file, if any.
- adds the build metadata to the db version file before adding indexes.
- deletes indexes when running without the `indexer` feature

* Replaces dropping cf with deleting range of all items to avoid a panic when trying to open the db with that column family.

* Fixes lint, avoids reading coinbase transactions from disk

* updates db column families table

* Document need for having an indexed cached state and use a multi-threaded tokio runtime in has_spending_transaction_ids test

* fixes call to renamed `future_blocks` test fn

* improves test logs and fixes a disk format deserialization bug

* Replaces a new expr with a previously existing constant, fixes typo
2025-01-27 14:34:48 +00:00
Alfredo Garcia 1511997829
fix clippy lints (#8855) 2024-09-06 15:24:15 +00:00
idky137 4579722833
change(chain): Remove `Copy` trait impl from `Network` (#8354)
* removed derive copy from network, address and ledgerstate

* changed is_max_block_time_enforced to accept ref

* changed NetworkUpgrade::Current to accept ref

* changed NetworkUpgrade::Next to accept ref

* changed NetworkUpgrade::IsActivationHeight to accept ref

* changed NetworkUpgrade::TargetSpacingForHeight to accept ref

* changed NetworkUpgrade::TargetSpacings to accept ref

* changed NetworkUpgrade::MinimumDifficultySpacing_forHeight to accept ref

* changed NetworkUpgrade::IsTestnetMinDifficultyBlock to accept ref

* changed NetworkUpgrade::AveragingWindowTimespanForHeight to accept ref

* changed NetworkUpgrade::ActivationHeight to accept ref

* changed sapling_activation_height to accept ref

* fixed lifetime for target_spacings

* fixed sapling_activation_height

* changed transaction_to_fake_v5 and fake_v5_transactions_for_network to accept ref to network

* changed Input::vec_strategy to accept ref to network

* changed functions in zebra-chain/src/primitives/zcash_history.rs to accept ref to network

* changed functions in zebra-chain/src/history_tree.rs to accept ref to network

* changed functions in zebra-chain/src/history_tree.rs to accept ref to network

* changed functions in zebra-chain/src/primitives/address.rs to accept ref to network

* changed functions in zebra-chain/src/primitives/viewing_key* to accept ref to network

* changed functions in zebra-chain/src/transparent/address.rs to accept ref to network

* changed functions in zebra-chain/src/primitives/zcash_primitives.rs to accept ref to network

* changed functions in zebra-chain/src/primitives/zcash_note_encryption.rs to accept ref to network

* changed functions in zebra-chain/src/primitives/history_tree* to accept ref to network

* changed functions in zebra-chain/src/block* to accept ref to network

* fixed errors in zebra-chain::parameters::network

* fixed errors in zebra-chain::parameters::network

* fixed errors in zebra-chain

* changed NonEmptyHistoryTree and InnerHistoryTree to hold value instead of ref

* changed NonEmptyHistoryTree and InnerHistoryTree to hold value instead of ref

* fixed errors in zebra-chain/src/block/arbitrary.rs

* finished fixing errors in zebra-chain - all crate tests pass

* changed functions in zebra-state::service::finalized_state to accept &Network

* changed functions in zebra-state::service::non_finalized_state to accept &Network

* zebra-state tests run but fail with overflow error

* zebra-state tests all pass

* converted zebra-network -- all crate tests pass

* applied all requested changes from review

* converted zebra-consensus -- all crate tests pass

* converted zebra-scan -- all crate tests pass

* converted zebra-rpc -- all crate tests pass

* converted zebra-grpc -- all crate tests pass

* converted zebrad -- all crate tests pass

* applied all requested changes from review

* fixed all clippy errors

* fixed build error in zebrad/src/components/mempool/crawler.rs
2024-03-19 20:45:27 +00:00
teor be5cfad07f
change(state): Prepare for in-place database format upgrades, but don't make any format changes yet (#7031)
* Move format upgrades to their own module and enum

* Launch a format change thread if needed, and shut it down during shutdown

* Add some TODOs and remove a redundant timer

* Regularly check for panics in the state upgrade task

* Only run example upgrade once, change version field names

* Increment database format to 25.0.2: add format change task

* Log the running and initial disk database format versions on startup

* Add initial disk and running state versions to cached state images in CI

* Fix missing imports

* Fix typo in logs workflow command

* Add a force_save_to_disk argument to the CI workflow

* Move use_internet_connection into zebrad_config()

* fastmod can_spawn_zebrad_for_rpc can_spawn_zebrad_for_test_type zebra*

* Add a spawn_zebrad_without_rpc() function

* Remove unused copy_state() test code

* Assert that upgrades and downgrades happen with the correct versions

* Add a kill_and_return_output() method for tests

* Add a test for new_state_format() versions (no upgrades or downgrades)

* Add use_internet_connection to can_spawn_zebrad_for_test_type()

* Fix workflow parameter passing

* Check that reopening a new database doesn't upgrade (or downgrade) the format

* Allow ephemeral to be set to false even if we don't have a cached state

* Add a test type that will accept any kind of state

* When re-using a directory, configure the state test config with that path

* Actually mark newly created databases with their format versions

* Wait for the state to be opened before testing the format

* Run state format tests on mainnet and testnet configs (no network access)

* run multiple reopens in tests

* Test upgrades run correctly

* Test that version downgrades work as expected (best effort)

* Add a TODO for testing partial updates

* Fix missing test arguments

* clippy if chain

* Fix typo

* another typo

* Pass a database instance to the format upgrade task

* Fix a timing issue in the tests

* Fix version matching in CI

* Use correct env var reference

* Use correct github env file

* Wait for the database to be written before killing Zebra

* Use correct workflow syntax

* Version changes aren't always upgrades

---------

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2023-07-13 21:36:15 +00:00
teor f9b5eb3a15
fix(log): Limit RPC failure log length, add details to RPC failure logs (#6754)
* Limit the RPC failure log length

* Add more specific logs for some RPC tests

* Make sure the block hash is always available for logging
2023-05-25 18:28:25 +00:00
teor d3ce022ecc
change(utils): Add a direct connection mode to zebra-checkpoints (#6516)
* Rename variables so it's clearer what they do

* Fully document zebra-checkpoints arguments

* Remove some outdated references to zcashd

* Add a json-conversion feature for converting JSON to valid Heights

* Simplify zebra-checkpoints code using conversion methods

* Track the last checkpoint height rather than the height gap

* Move all the CLI-specific code into a single function

* Remove cfg(feature) from the test RPC client API

* Move the RpcRequestClient into zebra-node-services

* Fix the case of RpcRequestClient

* Add transport and addr arguments to zebra-checkpoints

* Make zebra-checkpoints support both CLI and direct JSON-RPC connections

* Fix RpcRequestClient compilation

* Add a suggestion for zcashd authentication failures

* Document required features

* Handle differences in CLI & direct parameter and response formats

* Replace a custom function with an existing dependency function

* Add a checkpoint list test for minimum height gaps
2023-04-26 23:35:53 +00:00
teor afdb3a7013
change(ci): add acceptance test for getblocktemplate RPC in CI, and fix RPC bugs (#5761)
* Re-apply: add acceptance test for getblocktemplate method in CI (#5653)

Revert "Revert "change(tests): add acceptance test for getblocktemplate method in CI (#5653)" (#5672)"

This reverts commit 6446e0ec1b.

* Fix incorrect MAX_CONTEXT_BLOCKS assertion in state

* Actually negate the miner fee for the RPC output

* Try the RPC again after waiting for transactions to verify

* Log before the test waits for the mempool to verify transactions

* Use the new ssh key secrets in CI
2022-12-01 19:39:01 +00:00
teor 6446e0ec1b
Revert "change(tests): add acceptance test for getblocktemplate method in CI (#5653)" (#5672)
This reverts commit bd54a2f40e.
2022-11-21 10:44:11 +10:00
Arya bd54a2f40e
change(tests): add acceptance test for getblocktemplate method in CI (#5653)
* adds test for getblocktemplate rpc method

* adds the new test to CI

* adds a couple logs

* Adds example for running the test in acceptance.rs

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-11-18 11:49:54 +00:00
Arya 61af406d35
change(tests): adds RPC request client for condensing shared code (#5619)
* Adds RPCRequestClient

* uses RPCRequestClient in the rest of the rpc calls

* removes duplicate expect call

* fixed mistaken "get_info" method call with "getinfo"

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-11-16 21:23:29 +00:00
Arya c447b03223
fix(tests): add submitblock test to CI, and avoid copying the cached state directory in other tests (#5589)
* updates mod docs for tests that use future blocks

* updates submitblock test to use TestType methods

* prunes redundant code

* adds check_sync_logs_until

* adds assertion for needs cached state & rpc server

* updates get_raw_future_blocks fn with rpc calls

* updates to get_raw_future_blocks fn and submit_block test

* Rename LightwalletdTestType to TestType

* moves TestType and random_known_rpc_port_config to test_type.rs and config.rs

* moves get_raw_future_blocks to cached_state.rs

* updates ci workflows to include submit block test

* adds get_future_blocks fn and uses it in load_transactions_from_future_blocks

* updates CI docker

* Apply suggestions from code review

Co-authored-by: teor <teor@riseup.net>

* Applies suggestions from code review

* Updates misnamed closure param

* updates mod docs for test_type.rs

Co-authored-by: teor <teor@riseup.net>
2022-11-10 03:40:21 +00:00
Arya 2f3b05f8e1
change(rpc): add submitblock RPC method (#5526)
* adds submitblock rpc method

* re-orders imports

* replaces thread::yield_now with async yield_now

* Fix doc warnings and unused variable warnings, add missing docs

* Mark work_id as optional

* Use the same ChainVerifier for downloaded and submitted blocks

* Revert unused changes & minor cleanups

* Document currently-unreachable code

* updates tests and submit_block response for AlreadyVerified error

* Update zebra-rpc/src/methods/get_block_template_rpcs.rs

Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>

* changes names from BlockVerifier to ChainVerifier and block_verifier to chain_verifier to keep it consistent with naming in zebra-consensus

* move how to run the submit_block test example to acceptance.rs

* updates snapshot tests

* moved acceptance test to a separate file

* removes extra tower::ServiceBuilder::new(), updates docs

* updates vectors and snapshot tests, changes hex decoding error in submit_block method from server error to parse error

* hides errors module in zebra-rpc behind a feature flag and adds docs.

* Updates snapshot test, adds mod docs, moves HexData to its own mod, and removes the unrelated make_server_error refactor for now

* update submit block acceptance test mod doc

Co-authored-by: teor <teor@riseup.net>
Co-authored-by: Alfredo Garcia <oxarbitrage@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-11-04 03:57:08 +00:00