ZcashLightClientKit/MIGRATING.md

126 lines
6.0 KiB
Markdown
Raw Normal View History

- [#679] Implementation of the File-system based block cache (#679) Closes https://github.com/zcash/ZcashLightClientKit/issues/697 Closes https://github.com/zcash/ZcashLightClientKit/issues/720 Closes https://github.com/zcash/ZcashLightClientKit/issues/587 Closes https://github.com/zcash/ZcashLightClientKit/issues/667 Closes https://github.com/zcash/ZcashLightClientKit/issues/443 Closes https://github.com/zcash/ZcashLightClientKit/issues/754 - [#790] Fix ShieldFundsTests Closes #790 Removes comments on `ShieldFundsTests` since those issues have been fixed Depends on zcash-light-client-ffi changes that adopt newer versions of librustzcash crates `zcash_primitives 0.10`, `zcash_client_backend 0.7`, `zcash_proofs 0.10`, `zcash_client_sqlite 0.5.0`. Also allows wallets to define a shielding_threshold and will set foundations to customize minimum confirmations for balances, spends and shielding operations. **Test Bootstrapping** - `ZcashCompactBlockDescriptor`: struct that holds functions to describe blocks as filenames and compare those filenames `ZcashCompactBlockDescriptor.live` has the actual implementation but it can be replaced by mocks if needed on Tests main implementations are held under `FSCompactBlockRepository.filenameDescription` and `FSCompactBlockRepository.filenameComparison` on a separate extention `DirectoryListingProviders` provide two default implementations of listing a directory deterministically. `FileManager` does not define a sorting and needs to be done in-memory by calling `.sorted()` on the resulting collection. If this is a big toll on performance it can be changed to a POSIX implementation but this is good for now. `ZcashCompactBlockDescriptor` adds a `height` helper function to turn a filename into the height of the block stored. Implemented `func latestHeight() throws -> BlockHeight ` that returns the blockheight by querying the cache directory in a sorted fashion and getting the last value and turning the filename into a `BlockHeight` Added `Meta` struct to ZcashCompactBlock. Tests implemented: - `filterBlockFiles` - `testClearTheCache` - `testLatestHeightEmptyCacheThrows` - `testLatestHeightEmptyCacheThrowsAsync` - `testRewindEmptyCacheDoesNothing` - `testRewindEmptyCacheDoesNothingAsync` - `testWhenBlockIsStoredItFollowsTheDescribedFormat` - `testWhenBlockIsStoredItFollowsTheFilenameConvention` - `testGetLatestHeight` - `testRewindDeletesTheRightBlocks` test - `testPerformanceExample` test. This isn't a real performance test because the API doesn't work with async/await yet adopts `shield_funds` shielding threshold parameter Implements `initBlockMetadataDb` and fix tests Renames dbCache parameter to `fsBlockDbRoot`. Builds but tests don't pass. Removes cacheDb uses from code. Testing utilities still persist. Added needed information in MIGRATING and CHANGELOG. Added helper to perform deletion of legacy db and creation a the new file system backed cache. Renames parameters and changes code where needed. Network Constants turned into `enum` with static methods. DeletelastDownloadedBlock helper from initializer Removes CompactBlockStorage and CompactBlockEntity. Implements `latestCachedBlockHeight` on rustbackend. *Replaces dependencies on ZcashRustWelding with `FSMetadataStore`* This allows the tests to not depend in a particular implementation of either the MockRustBackend of or ZcashRustBackend. Also provides a way to test errors properly and switch implementations of critical areas like `writeBlocks`.
2023-02-02 08:58:12 -08:00
# Migrating from previous versions to <Unreleased>
Compact block cache no longer uses a sqlite database. The existing database
should be deleted. `Initializer` now takes an `fsBlockDbRootURL` which is a
URL pointing to a RW directory in the filesystem that will be used to store
the cached blocks and the companion database managed internally by the SDK.
`Initializer` provides a convenience initializer that takes the an optional
URL to the `cacheDb` location to migrate the internal state of the
`CompactBlockProcessor` and delete that database.
````Swift
convenience public init (
cacheDbURL: URL?,
fsBlockDbRoot: URL,
dataDbURL: URL,
pendingDbURL: URL,
endpoint: LightWalletEndpoint,
network: ZcashNetwork,
spendParamsURL: URL,
outputParamsURL: URL,
viewingKeys: [UnifiedFullViewingKey],
walletBirthday: BlockHeight,
alias: String = "",
loggerProxy: Logger? = nil
)
````
We do not make any efforts to extract the cached blocks in the sqlite
`cacheDb` and storing them on disk. Although this might be the logical
step to do, we think such migration as little to gain since a migration
function will be a "run once" function with many different scenarios to
consider and possibly very error prone. On the other hand, we rather delete
the `cacheDb` altogether and free up that space on the users' devices since
we have surveyed that the `cacheDb` as been growing exponentially taking up
many gigabyte of disk space. We forsee that many possible attempts to copy
information from one cache to another, would possibly fail
Consuming block cache information for other purposes is discouraged. Users
must not make assumptions on its contents or rely on its contents in any way.
Maintainers assume that this state is internal and won't consider further
uses other than the intended for the current development. If you consider
your application needs any other information than the ones available through
public APIs, please file the corresponding feature request.
# Migrating from 0.16.x-beta to 0.17.0-alpha.x
## Changes to Demo APP
The demo application now uses the SDKSynchronizer to create addresses and
shield funds.
`DerivationToolViewController` was removed. See `DerivationTool` unit tests
for sample code.
`GetAddressViewController` now derives transparent and sapling addresses
from Unified Address
`SendViewController` uses Unified Spending Key and type-safe `Memo`
## Changes To SDK
### `CompactBlockProcessor`
`public func getUnifiedAddress(accountIndex: Int) -> UnifiedAddress?`
`public func getSaplingAddress(accountIndex: Int) -> SaplingAddress?` derived from UA
`public func getTransparentAddress(accountIndex: Int) -> TransparentAddress?`
is derived from UA
`public func getTransparentBalance(accountIndex: Int) throws -> WalletBalance` now
fetches from account exclusively
`func refreshUTXOs(tAddress: TransparentAddress, startHeight: BlockHeight) async throws -> RefreshedUTXOs`
uses `TransparentAddress`
### Initializer
Migration of DataDB and CacheDB are delegated to `librustzcash`
removed `public func getAddress(index account: Int = 0) -> String`
### Wallet Types
`UnifiedSpendingKey` to represent Unified Spending Keys. This is a binary
2022-12-31 02:50:16 -08:00
encoded not meant to be stored or backed up. This only serves the purpose
of letting clients use the least privilege keys at all times for every
operation.
### Synchronizer
`sendToAddress` and `shieldFunds` now take a `UnifiedSpendingKey` instead
of the respective spending and transparent private keys.
`refreshUTXOs` uses `TransparentAddress`
### KeyDeriving protocol
Addresses should be obtained from the `Synchronizer` by using the `get_address` functions
Transparent and Sapling receivers should be obtained by extracting the receivers of a UA
````Swift
public extension UnifiedAddress {
/// Extracts the sapling receiver from this UA if available
/// - Returns: an `Optional<SaplingAddress>`
func saplingReceiver() -> SaplingAddress? {
try? DerivationTool.saplingReceiver(from: self)
}
/// Extracts the transparent receiver from this UA if available
/// - Returns: an `Optional<TransparentAddress>`
func transparentReceiver() -> TransparentAddress? {
try? DerivationTool.transparentReceiver(from: self)
}
````
**Removed**
`func deriveUnifiedFullViewingKeys(seed: [UInt8], numberOfAccounts: Int) throws -> [UnifiedFullViewingKey]`
`func deriveViewingKey(spendingKey: SaplingExtendedSpendingKey) throws -> SaplingExtendedFullViewingKey`
`func deriveSpendingKeys(seed: [UInt8], numberOfAccounts: Int) throws -> [SaplingExtendedSpendingKey]`
`func deriveUnifiedAddress(from ufvk: UnifiedFullViewingKey) throws -> UnifiedAddress`
`func deriveTransparentAddress(seed: [UInt8], account: Int, index: Int) throws -> TransparentAddress`
`func deriveTransparentAccountPrivateKey(seed: [UInt8], account: Int) throws -> TransparentAccountPrivKey`
`func deriveTransparentAddressFromAccountPrivateKey(_ xprv: TransparentAccountPrivKey, index: Int) throws -> TransparentAddress`
**Added**
`static func saplingReceiver(from unifiedAddress: UnifiedAddress) throws -> SaplingAddress?`
`static func transparentReceiver(from unifiedAddress: UnifiedAddress) throws -> TransparentAddress?`
`static func receiverTypecodesFromUnifiedAddress(_ address: UnifiedAddress) throws -> [UnifiedAddress.ReceiverTypecodes]`
`func deriveUnifiedSpendingKey(seed: [UInt8], accountIndex: Int) throws -> UnifiedSpendingKey`
`public func deriveUnifiedFullViewingKey(from spendingKey: UnifiedSpendingKey) throws -> UnifiedFullViewingKey`
## Notes on Structured Concurrency
`CompactBlockProcessor` is now an Swift Actor. This makes it more robust and have its own
2022-12-31 02:50:16 -08:00
async environment.
2022-12-31 02:50:16 -08:00
SDK Clients will likely be affected by some `async` methods on `SDKSynchronizer`.
- [#679] Implementation of the File-system based block cache (#679) Closes https://github.com/zcash/ZcashLightClientKit/issues/697 Closes https://github.com/zcash/ZcashLightClientKit/issues/720 Closes https://github.com/zcash/ZcashLightClientKit/issues/587 Closes https://github.com/zcash/ZcashLightClientKit/issues/667 Closes https://github.com/zcash/ZcashLightClientKit/issues/443 Closes https://github.com/zcash/ZcashLightClientKit/issues/754 - [#790] Fix ShieldFundsTests Closes #790 Removes comments on `ShieldFundsTests` since those issues have been fixed Depends on zcash-light-client-ffi changes that adopt newer versions of librustzcash crates `zcash_primitives 0.10`, `zcash_client_backend 0.7`, `zcash_proofs 0.10`, `zcash_client_sqlite 0.5.0`. Also allows wallets to define a shielding_threshold and will set foundations to customize minimum confirmations for balances, spends and shielding operations. **Test Bootstrapping** - `ZcashCompactBlockDescriptor`: struct that holds functions to describe blocks as filenames and compare those filenames `ZcashCompactBlockDescriptor.live` has the actual implementation but it can be replaced by mocks if needed on Tests main implementations are held under `FSCompactBlockRepository.filenameDescription` and `FSCompactBlockRepository.filenameComparison` on a separate extention `DirectoryListingProviders` provide two default implementations of listing a directory deterministically. `FileManager` does not define a sorting and needs to be done in-memory by calling `.sorted()` on the resulting collection. If this is a big toll on performance it can be changed to a POSIX implementation but this is good for now. `ZcashCompactBlockDescriptor` adds a `height` helper function to turn a filename into the height of the block stored. Implemented `func latestHeight() throws -> BlockHeight ` that returns the blockheight by querying the cache directory in a sorted fashion and getting the last value and turning the filename into a `BlockHeight` Added `Meta` struct to ZcashCompactBlock. Tests implemented: - `filterBlockFiles` - `testClearTheCache` - `testLatestHeightEmptyCacheThrows` - `testLatestHeightEmptyCacheThrowsAsync` - `testRewindEmptyCacheDoesNothing` - `testRewindEmptyCacheDoesNothingAsync` - `testWhenBlockIsStoredItFollowsTheDescribedFormat` - `testWhenBlockIsStoredItFollowsTheFilenameConvention` - `testGetLatestHeight` - `testRewindDeletesTheRightBlocks` test - `testPerformanceExample` test. This isn't a real performance test because the API doesn't work with async/await yet adopts `shield_funds` shielding threshold parameter Implements `initBlockMetadataDb` and fix tests Renames dbCache parameter to `fsBlockDbRoot`. Builds but tests don't pass. Removes cacheDb uses from code. Testing utilities still persist. Added needed information in MIGRATING and CHANGELOG. Added helper to perform deletion of legacy db and creation a the new file system backed cache. Renames parameters and changes code where needed. Network Constants turned into `enum` with static methods. DeletelastDownloadedBlock helper from initializer Removes CompactBlockStorage and CompactBlockEntity. Implements `latestCachedBlockHeight` on rustbackend. *Replaces dependencies on ZcashRustWelding with `FSMetadataStore`* This allows the tests to not depend in a particular implementation of either the MockRustBackend of or ZcashRustBackend. Also provides a way to test errors properly and switch implementations of critical areas like `writeBlocks`.
2023-02-02 08:58:12 -08:00
We recommend clients that don't support structured concurrency features, to work around this by surrounding the these function calls either in @MainActor contexts either by marking callers as @MainActor or launching tasks on that actor with `Task { @MainActor in ... }`