ZcashLightClientKit/MIGRATING.md

162 lines
7.4 KiB
Markdown
Raw Permalink Normal View History

[#1001] Remove PendingDb in favor of `v_transactions` and `v_tx_output` Views (#1001) Removes `PendingTransactionEntity` and all of its related components. Pending items are still tracked and visualized by the existing APIs but they are retrieved from the `TransactionRepository` instead by returning `ZcashTransaction.Overview` instead. `pendingDbURL` is removed from every place it was required. Its deletion is responsibility of wallet developers. `ClearedTransactions` are now just `transactions`. `MigrationManager` is deleted. Now all migrations are in charge of the rust welding layer. `PendingTransactionDao.swift` is removed. Implementation of `AccountEntity` called `Account` is now `DbAccount` `ZcashTransaction.Overview` can be checked for "pending-ness" by calling `.isPending(latestHeight:)` latest height must be provided so that minedHeight can be compared with the lastest and the `defaultStaleTolerance` constant. `TransactionRecipient` is now a public type. protocol `PendingTransactionRepository` is removed. `TransactionManagerError` and `PersistentTransactionManager` are deleted. `OutboundTransactionManager` is deleted and replaced by `TransactionEncoder` which now incorporates `submit(encoded:)` functionality `WalletTransactionEncoder` now uses a `LightWalletService` to submit the encoded transactions. Add changelog changes Delete references to PendingDb from tests and documentation. Fixes some typos. Adds the ability to trace transaction repository SQL queries from test Fix rebase conflicts and generate code [#837] Memo tests regarding transparent address Closes #837 Add model for transaction output Point to FFI branch Fix issue where sync wouldn't resume after wipe. Becasue GRPC channel would be closed Fix Tests Fix testPendingTransactionMinedHeightUpdated Fix testLastStates [#921] Fix broken SynchronizerDarksideTests Add ZcashTransaction.Output API to Synchronizer Changelog + comment fix Add Assertions for transaction outputs and recipients Point to FFI 0.3.1 Fix Demo App Compiler errors Fix Demo App Compiler errors fix cacheDb warnings Fix Tests and compiler errors of rebase build demo app Remove `ZcashTransaction.Sent` and `.Received`. Add `.State` and tests Fix SPM warning PR Suggestions Removes errors that are not used anymore fix warnings
2023-05-05 10:30:47 -07:00
# Migrating from previous versions to _Unreleased_
PendingDb is no longer used. Wallet developers should take care about deleting
the database file since the SDK will no longer require it or any of the
information stored.
Failed transactions will be treated as "Expired-Unmined" instead. The SDK won't
track failures on its own. Wallet developers would have to account for those.
# Migrating from previous versions to 0.20.0-beta
The `SDKSynchronizer` no longer uses `NotificationCenter` to send notifications.
Notifications are replaced with `Combine` publishers.
`stateStream` publisher replaces notifications related to `SyncStatus` changes.
These notifications are replaced by `stateStream`:
- .synchronizerStarted
- .synchronizerProgressUpdated
- .synchronizerStatusWillUpdate
- .synchronizerSynced
- .synchronizerStopped
- .synchronizerDisconnected
- .synchronizerSyncing
- .synchronizerEnhancing
- .synchronizerFetching
- .synchronizerFailed
`eventStream` publisher replaces notifications related to transactions and other stuff.
These notifications are replaced by `eventStream`:
- .synchronizerMinedTransaction
- .synchronizerFoundTransactions
- .synchronizerStoredUTXOs
- .synchronizerConnectionStateChanged
`latestState` is also new property that can be used to get the latest SDK state in a synchronous way.
`SDKSynchronizer.status` is no longer public. To get `SyncStatus` either subscribe to `stateStream`
or use `latestState`.
# Migrating from previous versions to 0.18.x
- [#679] Implementation of the File-system based block cache (#679) Closes https://github.com/zcash/ZcashLightClientKit/issues/697 Closes https://github.com/zcash/ZcashLightClientKit/issues/720 Closes https://github.com/zcash/ZcashLightClientKit/issues/587 Closes https://github.com/zcash/ZcashLightClientKit/issues/667 Closes https://github.com/zcash/ZcashLightClientKit/issues/443 Closes https://github.com/zcash/ZcashLightClientKit/issues/754 - [#790] Fix ShieldFundsTests Closes #790 Removes comments on `ShieldFundsTests` since those issues have been fixed Depends on zcash-light-client-ffi changes that adopt newer versions of librustzcash crates `zcash_primitives 0.10`, `zcash_client_backend 0.7`, `zcash_proofs 0.10`, `zcash_client_sqlite 0.5.0`. Also allows wallets to define a shielding_threshold and will set foundations to customize minimum confirmations for balances, spends and shielding operations. **Test Bootstrapping** - `ZcashCompactBlockDescriptor`: struct that holds functions to describe blocks as filenames and compare those filenames `ZcashCompactBlockDescriptor.live` has the actual implementation but it can be replaced by mocks if needed on Tests main implementations are held under `FSCompactBlockRepository.filenameDescription` and `FSCompactBlockRepository.filenameComparison` on a separate extention `DirectoryListingProviders` provide two default implementations of listing a directory deterministically. `FileManager` does not define a sorting and needs to be done in-memory by calling `.sorted()` on the resulting collection. If this is a big toll on performance it can be changed to a POSIX implementation but this is good for now. `ZcashCompactBlockDescriptor` adds a `height` helper function to turn a filename into the height of the block stored. Implemented `func latestHeight() throws -> BlockHeight ` that returns the blockheight by querying the cache directory in a sorted fashion and getting the last value and turning the filename into a `BlockHeight` Added `Meta` struct to ZcashCompactBlock. Tests implemented: - `filterBlockFiles` - `testClearTheCache` - `testLatestHeightEmptyCacheThrows` - `testLatestHeightEmptyCacheThrowsAsync` - `testRewindEmptyCacheDoesNothing` - `testRewindEmptyCacheDoesNothingAsync` - `testWhenBlockIsStoredItFollowsTheDescribedFormat` - `testWhenBlockIsStoredItFollowsTheFilenameConvention` - `testGetLatestHeight` - `testRewindDeletesTheRightBlocks` test - `testPerformanceExample` test. This isn't a real performance test because the API doesn't work with async/await yet adopts `shield_funds` shielding threshold parameter Implements `initBlockMetadataDb` and fix tests Renames dbCache parameter to `fsBlockDbRoot`. Builds but tests don't pass. Removes cacheDb uses from code. Testing utilities still persist. Added needed information in MIGRATING and CHANGELOG. Added helper to perform deletion of legacy db and creation a the new file system backed cache. Renames parameters and changes code where needed. Network Constants turned into `enum` with static methods. DeletelastDownloadedBlock helper from initializer Removes CompactBlockStorage and CompactBlockEntity. Implements `latestCachedBlockHeight` on rustbackend. *Replaces dependencies on ZcashRustWelding with `FSMetadataStore`* This allows the tests to not depend in a particular implementation of either the MockRustBackend of or ZcashRustBackend. Also provides a way to test errors properly and switch implementations of critical areas like `writeBlocks`.
2023-02-02 08:58:12 -08:00
Compact block cache no longer uses a sqlite database. The existing database
should be deleted. `Initializer` now takes an `fsBlockDbRootURL` which is a
URL pointing to a RW directory in the filesystem that will be used to store
the cached blocks and the companion database managed internally by the SDK.
`Initializer` provides a convenience initializer that takes the an optional
URL to the `cacheDb` location to migrate the internal state of the
`CompactBlockProcessor` and delete that database.
````Swift
[#361] Introduce ZcashError (#1005) This PR introduces `ZcashError` enum. This enum represents any error that can be thrown inside of the SDK and outside of the SDK. Also `ZcashError` is used in `LightWalletGRPCService` and handled in `CompactBlockProcessor` as example. Why enum? First I tried this with some structure which contained code, message and underlyingError. Problem was when some specific place in the SDK would like to attach some additional data to error. I didn't want to add some generic dictionary to store anything with the error. So I used enum to identify error. Each member can have specified amount of associated values. So specific error can bring some context data. And it's type safe. Each error has also a code. Relationship between errors and codes is 1:1. It may looks bit redundant but it's important. The client app now can choose how to process errors. Either identify those by the error itself or by code. Definition or errors and codes is in `ZcashErrorDefinition`. And then `ZcashError` and `ZcashErrorCode` are generated by Sourcery. Thanks to this it is easier to maintain the final code. And it gives us ability to generate any kind of documentation for the errors and codes. I created simple example of this in this PR. And it doesn't make the SDK completely dependent on the Sourcery. Final structures aren't super complicated and can be written manually if needed. [#923] Update error handling in DatabaseStorageManager.swift - cleanup of DatabaseStorageManager, not used anymore - ZcashError for SimpleConnectionProvider Revert "[#923] Update error handling in DatabaseStorageManager.swift" This reverts commit 94e028d31a0f6635800c297eb741db74c6a6ff45. Fix script which generates ZcashError [#922] Update error handling in DatabaseMigrationManager.swift Closes #922 [#923] Update error handling in DatabaseStorageManager.swift - The DatabaseStorageManager is not used so I deleted it - SimpleConnectionProvider's errors updated Fix tests [#955] Update error handling in SaplingParameterDownloader Closes #955 [#936] Update error handling in NotesDao Closes #936 [#935] Update error handling in BlockDao Closes #935 [#931] InternalSyncProgress.computeNextState doesn't need to throw Closes #931 [#950] Update error handling in rust backend Closes #949 Closes #950 [#941] Update error handling in AccountEntity Closes #941 [#928] Update error handling in FSCompactBlockRepository Closes #928 [#925] Update error handling in BlockDownloaderService (#974) - BlockDownloaderService errors updated [#924] Update error handling in BlockDownloader (#973) - BlockDownloaderStream nextBlock updated [#942] Update error handling in TransactionEntity (#977) - ZcashTransaction init errors converted to the ZcashError [#939] Update error handling in TransactionDao (#976) - TransactionRepositoryError removed and replaced with ZcashError - all TransactionDAO errors converted to ZcashError [#943] Update error handling in ZcashCompactBlock Closes #943 [#945] Update error handling in Memo Closes #945 [#934] Update error handling in Checkpoint Closes #944 Closes #934 [#938] Update error handling in PendingTransactionDao Closes #938 [#926] Update error handling in BlockEnhancer - WIP, switching to transaction repository to unblock this ticket [#926] Update error handling in BlockEnhancer - enhancer's errors done [#926] Update error handling in BlockEnhancer (#994) - cleanup [#952] Update error handling in DerivationTool Closes #952 [#932] Update error handling in BlockValidator Closes #932 [#940] Update error handling in UnspentTransactionOutputDao Closes #940 [#946] Update error handling in WalletTypes Closes #946 [#954] Update error handling in WalletTransactionEncoder Closes #954 [#953] Update error handling in PersistentTransactionManager Closes #953 [#956] Update error handling in Initializer Closes #956 [#947] Update error handling in Zatoshi (#996) - Zatoshi's errors converted to ZcashError [#927] Update error handling in UTXOFetcher (#997) - UTXOFetcher resolved Use let instead of var where applicable In the SDK `var` was used on places where `let` would be sufficient. And default strategy in Swift should use use `let` until mutability is required. So all these places are changed. No logic is changed. [#933] Update error handling in CompactBlockProcessor - CompactBlockProcessor's errors refactored to ZcashError [#933] Update error handling in CompactBlockProcessor #999 - comments addressed [#951] Update error handling in SDKSynchronizer - SDKSynchronizer's errors refactored to ZcashError [#951] Update error handling in SDKSynchronizer (#1002) - comments resolved Make DerivationTool constructor public DerivationTool.init should be public. This was removed by accident when adopting async. [#361] Add changelog
2023-04-24 14:15:20 -07:00
convenience public init(
- [#679] Implementation of the File-system based block cache (#679) Closes https://github.com/zcash/ZcashLightClientKit/issues/697 Closes https://github.com/zcash/ZcashLightClientKit/issues/720 Closes https://github.com/zcash/ZcashLightClientKit/issues/587 Closes https://github.com/zcash/ZcashLightClientKit/issues/667 Closes https://github.com/zcash/ZcashLightClientKit/issues/443 Closes https://github.com/zcash/ZcashLightClientKit/issues/754 - [#790] Fix ShieldFundsTests Closes #790 Removes comments on `ShieldFundsTests` since those issues have been fixed Depends on zcash-light-client-ffi changes that adopt newer versions of librustzcash crates `zcash_primitives 0.10`, `zcash_client_backend 0.7`, `zcash_proofs 0.10`, `zcash_client_sqlite 0.5.0`. Also allows wallets to define a shielding_threshold and will set foundations to customize minimum confirmations for balances, spends and shielding operations. **Test Bootstrapping** - `ZcashCompactBlockDescriptor`: struct that holds functions to describe blocks as filenames and compare those filenames `ZcashCompactBlockDescriptor.live` has the actual implementation but it can be replaced by mocks if needed on Tests main implementations are held under `FSCompactBlockRepository.filenameDescription` and `FSCompactBlockRepository.filenameComparison` on a separate extention `DirectoryListingProviders` provide two default implementations of listing a directory deterministically. `FileManager` does not define a sorting and needs to be done in-memory by calling `.sorted()` on the resulting collection. If this is a big toll on performance it can be changed to a POSIX implementation but this is good for now. `ZcashCompactBlockDescriptor` adds a `height` helper function to turn a filename into the height of the block stored. Implemented `func latestHeight() throws -> BlockHeight ` that returns the blockheight by querying the cache directory in a sorted fashion and getting the last value and turning the filename into a `BlockHeight` Added `Meta` struct to ZcashCompactBlock. Tests implemented: - `filterBlockFiles` - `testClearTheCache` - `testLatestHeightEmptyCacheThrows` - `testLatestHeightEmptyCacheThrowsAsync` - `testRewindEmptyCacheDoesNothing` - `testRewindEmptyCacheDoesNothingAsync` - `testWhenBlockIsStoredItFollowsTheDescribedFormat` - `testWhenBlockIsStoredItFollowsTheFilenameConvention` - `testGetLatestHeight` - `testRewindDeletesTheRightBlocks` test - `testPerformanceExample` test. This isn't a real performance test because the API doesn't work with async/await yet adopts `shield_funds` shielding threshold parameter Implements `initBlockMetadataDb` and fix tests Renames dbCache parameter to `fsBlockDbRoot`. Builds but tests don't pass. Removes cacheDb uses from code. Testing utilities still persist. Added needed information in MIGRATING and CHANGELOG. Added helper to perform deletion of legacy db and creation a the new file system backed cache. Renames parameters and changes code where needed. Network Constants turned into `enum` with static methods. DeletelastDownloadedBlock helper from initializer Removes CompactBlockStorage and CompactBlockEntity. Implements `latestCachedBlockHeight` on rustbackend. *Replaces dependencies on ZcashRustWelding with `FSMetadataStore`* This allows the tests to not depend in a particular implementation of either the MockRustBackend of or ZcashRustBackend. Also provides a way to test errors properly and switch implementations of critical areas like `writeBlocks`.
2023-02-02 08:58:12 -08:00
cacheDbURL: URL?,
fsBlockDbRoot: URL,
dataDbURL: URL,
pendingDbURL: URL,
endpoint: LightWalletEndpoint,
network: ZcashNetwork,
spendParamsURL: URL,
outputParamsURL: URL,
viewingKeys: [UnifiedFullViewingKey],
walletBirthday: BlockHeight,
alias: String = "",
loggerProxy: Logger? = nil
)
````
We do not make any efforts to extract the cached blocks in the sqlite
`cacheDb` and storing them on disk. Although this might be the logical
step to do, we think such migration as little to gain since a migration
function will be a "run once" function with many different scenarios to
consider and possibly very error prone. On the other hand, we rather delete
the `cacheDb` altogether and free up that space on the users' devices since
we have surveyed that the `cacheDb` as been growing exponentially taking up
many gigabyte of disk space. We forsee that many possible attempts to copy
information from one cache to another, would possibly fail
Consuming block cache information for other purposes is discouraged. Users
must not make assumptions on its contents or rely on its contents in any way.
Maintainers assume that this state is internal and won't consider further
uses other than the intended for the current development. If you consider
your application needs any other information than the ones available through
public APIs, please file the corresponding feature request.
# Migrating from 0.16.x-beta to 0.17.0-alpha.x
## Changes to Demo APP
The demo application now uses the SDKSynchronizer to create addresses and
shield funds.
`DerivationToolViewController` was removed. See `DerivationTool` unit tests
for sample code.
`GetAddressViewController` now derives transparent and sapling addresses
from Unified Address
`SendViewController` uses Unified Spending Key and type-safe `Memo`
## Changes To SDK
### `CompactBlockProcessor`
`public func getUnifiedAddress(accountIndex: Int) -> UnifiedAddress?`
`public func getSaplingAddress(accountIndex: Int) -> SaplingAddress?` derived from UA
`public func getTransparentAddress(accountIndex: Int) -> TransparentAddress?`
is derived from UA
`public func getTransparentBalance(accountIndex: Int) throws -> WalletBalance` now
fetches from account exclusively
`func refreshUTXOs(tAddress: TransparentAddress, startHeight: BlockHeight) async throws -> RefreshedUTXOs`
uses `TransparentAddress`
### Initializer
Migration of DataDB and CacheDB are delegated to `librustzcash`
removed `public func getAddress(index account: Int = 0) -> String`
### Wallet Types
`UnifiedSpendingKey` to represent Unified Spending Keys. This is a binary
2022-12-31 02:50:16 -08:00
encoded not meant to be stored or backed up. This only serves the purpose
of letting clients use the least privilege keys at all times for every
operation.
### Synchronizer
`sendToAddress` and `shieldFunds` now take a `UnifiedSpendingKey` instead
of the respective spending and transparent private keys.
`refreshUTXOs` uses `TransparentAddress`
### KeyDeriving protocol
Addresses should be obtained from the `Synchronizer` by using the `get_address` functions
Transparent and Sapling receivers should be obtained by extracting the receivers of a UA
````Swift
public extension UnifiedAddress {
/// Extracts the sapling receiver from this UA if available
/// - Returns: an `Optional<SaplingAddress>`
func saplingReceiver() -> SaplingAddress? {
try? DerivationTool.saplingReceiver(from: self)
}
/// Extracts the transparent receiver from this UA if available
/// - Returns: an `Optional<TransparentAddress>`
func transparentReceiver() -> TransparentAddress? {
try? DerivationTool.transparentReceiver(from: self)
}
````
**Removed**
`func deriveUnifiedFullViewingKeys(seed: [UInt8], numberOfAccounts: Int) throws -> [UnifiedFullViewingKey]`
`func deriveViewingKey(spendingKey: SaplingExtendedSpendingKey) throws -> SaplingExtendedFullViewingKey`
`func deriveSpendingKeys(seed: [UInt8], numberOfAccounts: Int) throws -> [SaplingExtendedSpendingKey]`
`func deriveUnifiedAddress(from ufvk: UnifiedFullViewingKey) throws -> UnifiedAddress`
`func deriveTransparentAddress(seed: [UInt8], account: Int, index: Int) throws -> TransparentAddress`
`func deriveTransparentAccountPrivateKey(seed: [UInt8], account: Int) throws -> TransparentAccountPrivKey`
`func deriveTransparentAddressFromAccountPrivateKey(_ xprv: TransparentAccountPrivKey, index: Int) throws -> TransparentAddress`
**Added**
`static func saplingReceiver(from unifiedAddress: UnifiedAddress) throws -> SaplingAddress?`
`static func transparentReceiver(from unifiedAddress: UnifiedAddress) throws -> TransparentAddress?`
`static func receiverTypecodesFromUnifiedAddress(_ address: UnifiedAddress) throws -> [UnifiedAddress.ReceiverTypecodes]`
`func deriveUnifiedSpendingKey(seed: [UInt8], accountIndex: Int) throws -> UnifiedSpendingKey`
`public func deriveUnifiedFullViewingKey(from spendingKey: UnifiedSpendingKey) throws -> UnifiedFullViewingKey`
## Notes on Structured Concurrency
`CompactBlockProcessor` is now an Swift Actor. This makes it more robust and have its own
2022-12-31 02:50:16 -08:00
async environment.
2022-12-31 02:50:16 -08:00
SDK Clients will likely be affected by some `async` methods on `SDKSynchronizer`.
- [#679] Implementation of the File-system based block cache (#679) Closes https://github.com/zcash/ZcashLightClientKit/issues/697 Closes https://github.com/zcash/ZcashLightClientKit/issues/720 Closes https://github.com/zcash/ZcashLightClientKit/issues/587 Closes https://github.com/zcash/ZcashLightClientKit/issues/667 Closes https://github.com/zcash/ZcashLightClientKit/issues/443 Closes https://github.com/zcash/ZcashLightClientKit/issues/754 - [#790] Fix ShieldFundsTests Closes #790 Removes comments on `ShieldFundsTests` since those issues have been fixed Depends on zcash-light-client-ffi changes that adopt newer versions of librustzcash crates `zcash_primitives 0.10`, `zcash_client_backend 0.7`, `zcash_proofs 0.10`, `zcash_client_sqlite 0.5.0`. Also allows wallets to define a shielding_threshold and will set foundations to customize minimum confirmations for balances, spends and shielding operations. **Test Bootstrapping** - `ZcashCompactBlockDescriptor`: struct that holds functions to describe blocks as filenames and compare those filenames `ZcashCompactBlockDescriptor.live` has the actual implementation but it can be replaced by mocks if needed on Tests main implementations are held under `FSCompactBlockRepository.filenameDescription` and `FSCompactBlockRepository.filenameComparison` on a separate extention `DirectoryListingProviders` provide two default implementations of listing a directory deterministically. `FileManager` does not define a sorting and needs to be done in-memory by calling `.sorted()` on the resulting collection. If this is a big toll on performance it can be changed to a POSIX implementation but this is good for now. `ZcashCompactBlockDescriptor` adds a `height` helper function to turn a filename into the height of the block stored. Implemented `func latestHeight() throws -> BlockHeight ` that returns the blockheight by querying the cache directory in a sorted fashion and getting the last value and turning the filename into a `BlockHeight` Added `Meta` struct to ZcashCompactBlock. Tests implemented: - `filterBlockFiles` - `testClearTheCache` - `testLatestHeightEmptyCacheThrows` - `testLatestHeightEmptyCacheThrowsAsync` - `testRewindEmptyCacheDoesNothing` - `testRewindEmptyCacheDoesNothingAsync` - `testWhenBlockIsStoredItFollowsTheDescribedFormat` - `testWhenBlockIsStoredItFollowsTheFilenameConvention` - `testGetLatestHeight` - `testRewindDeletesTheRightBlocks` test - `testPerformanceExample` test. This isn't a real performance test because the API doesn't work with async/await yet adopts `shield_funds` shielding threshold parameter Implements `initBlockMetadataDb` and fix tests Renames dbCache parameter to `fsBlockDbRoot`. Builds but tests don't pass. Removes cacheDb uses from code. Testing utilities still persist. Added needed information in MIGRATING and CHANGELOG. Added helper to perform deletion of legacy db and creation a the new file system backed cache. Renames parameters and changes code where needed. Network Constants turned into `enum` with static methods. DeletelastDownloadedBlock helper from initializer Removes CompactBlockStorage and CompactBlockEntity. Implements `latestCachedBlockHeight` on rustbackend. *Replaces dependencies on ZcashRustWelding with `FSMetadataStore`* This allows the tests to not depend in a particular implementation of either the MockRustBackend of or ZcashRustBackend. Also provides a way to test errors properly and switch implementations of critical areas like `writeBlocks`.
2023-02-02 08:58:12 -08:00
We recommend clients that don't support structured concurrency features, to work around this by surrounding the these function calls either in @MainActor contexts either by marking callers as @MainActor or launching tasks on that actor with `Task { @MainActor in ... }`