ZcashLightClientKit/Tests/DarksideTests/InternalStateConsistencyTes...

149 lines
5.3 KiB
Swift
Raw Normal View History

- [#679] Implementation of the File-system based block cache (#679) Closes https://github.com/zcash/ZcashLightClientKit/issues/697 Closes https://github.com/zcash/ZcashLightClientKit/issues/720 Closes https://github.com/zcash/ZcashLightClientKit/issues/587 Closes https://github.com/zcash/ZcashLightClientKit/issues/667 Closes https://github.com/zcash/ZcashLightClientKit/issues/443 Closes https://github.com/zcash/ZcashLightClientKit/issues/754 - [#790] Fix ShieldFundsTests Closes #790 Removes comments on `ShieldFundsTests` since those issues have been fixed Depends on zcash-light-client-ffi changes that adopt newer versions of librustzcash crates `zcash_primitives 0.10`, `zcash_client_backend 0.7`, `zcash_proofs 0.10`, `zcash_client_sqlite 0.5.0`. Also allows wallets to define a shielding_threshold and will set foundations to customize minimum confirmations for balances, spends and shielding operations. **Test Bootstrapping** - `ZcashCompactBlockDescriptor`: struct that holds functions to describe blocks as filenames and compare those filenames `ZcashCompactBlockDescriptor.live` has the actual implementation but it can be replaced by mocks if needed on Tests main implementations are held under `FSCompactBlockRepository.filenameDescription` and `FSCompactBlockRepository.filenameComparison` on a separate extention `DirectoryListingProviders` provide two default implementations of listing a directory deterministically. `FileManager` does not define a sorting and needs to be done in-memory by calling `.sorted()` on the resulting collection. If this is a big toll on performance it can be changed to a POSIX implementation but this is good for now. `ZcashCompactBlockDescriptor` adds a `height` helper function to turn a filename into the height of the block stored. Implemented `func latestHeight() throws -> BlockHeight ` that returns the blockheight by querying the cache directory in a sorted fashion and getting the last value and turning the filename into a `BlockHeight` Added `Meta` struct to ZcashCompactBlock. Tests implemented: - `filterBlockFiles` - `testClearTheCache` - `testLatestHeightEmptyCacheThrows` - `testLatestHeightEmptyCacheThrowsAsync` - `testRewindEmptyCacheDoesNothing` - `testRewindEmptyCacheDoesNothingAsync` - `testWhenBlockIsStoredItFollowsTheDescribedFormat` - `testWhenBlockIsStoredItFollowsTheFilenameConvention` - `testGetLatestHeight` - `testRewindDeletesTheRightBlocks` test - `testPerformanceExample` test. This isn't a real performance test because the API doesn't work with async/await yet adopts `shield_funds` shielding threshold parameter Implements `initBlockMetadataDb` and fix tests Renames dbCache parameter to `fsBlockDbRoot`. Builds but tests don't pass. Removes cacheDb uses from code. Testing utilities still persist. Added needed information in MIGRATING and CHANGELOG. Added helper to perform deletion of legacy db and creation a the new file system backed cache. Renames parameters and changes code where needed. Network Constants turned into `enum` with static methods. DeletelastDownloadedBlock helper from initializer Removes CompactBlockStorage and CompactBlockEntity. Implements `latestCachedBlockHeight` on rustbackend. *Replaces dependencies on ZcashRustWelding with `FSMetadataStore`* This allows the tests to not depend in a particular implementation of either the MockRustBackend of or ZcashRustBackend. Also provides a way to test errors properly and switch implementations of critical areas like `writeBlocks`.
2023-02-02 08:58:12 -08:00
//
// InternalStateConsistencyTests.swift
// DarksideTests
//
// Created by Francisco Gindre on 1/26/23.
//
import Combine
- [#679] Implementation of the File-system based block cache (#679) Closes https://github.com/zcash/ZcashLightClientKit/issues/697 Closes https://github.com/zcash/ZcashLightClientKit/issues/720 Closes https://github.com/zcash/ZcashLightClientKit/issues/587 Closes https://github.com/zcash/ZcashLightClientKit/issues/667 Closes https://github.com/zcash/ZcashLightClientKit/issues/443 Closes https://github.com/zcash/ZcashLightClientKit/issues/754 - [#790] Fix ShieldFundsTests Closes #790 Removes comments on `ShieldFundsTests` since those issues have been fixed Depends on zcash-light-client-ffi changes that adopt newer versions of librustzcash crates `zcash_primitives 0.10`, `zcash_client_backend 0.7`, `zcash_proofs 0.10`, `zcash_client_sqlite 0.5.0`. Also allows wallets to define a shielding_threshold and will set foundations to customize minimum confirmations for balances, spends and shielding operations. **Test Bootstrapping** - `ZcashCompactBlockDescriptor`: struct that holds functions to describe blocks as filenames and compare those filenames `ZcashCompactBlockDescriptor.live` has the actual implementation but it can be replaced by mocks if needed on Tests main implementations are held under `FSCompactBlockRepository.filenameDescription` and `FSCompactBlockRepository.filenameComparison` on a separate extention `DirectoryListingProviders` provide two default implementations of listing a directory deterministically. `FileManager` does not define a sorting and needs to be done in-memory by calling `.sorted()` on the resulting collection. If this is a big toll on performance it can be changed to a POSIX implementation but this is good for now. `ZcashCompactBlockDescriptor` adds a `height` helper function to turn a filename into the height of the block stored. Implemented `func latestHeight() throws -> BlockHeight ` that returns the blockheight by querying the cache directory in a sorted fashion and getting the last value and turning the filename into a `BlockHeight` Added `Meta` struct to ZcashCompactBlock. Tests implemented: - `filterBlockFiles` - `testClearTheCache` - `testLatestHeightEmptyCacheThrows` - `testLatestHeightEmptyCacheThrowsAsync` - `testRewindEmptyCacheDoesNothing` - `testRewindEmptyCacheDoesNothingAsync` - `testWhenBlockIsStoredItFollowsTheDescribedFormat` - `testWhenBlockIsStoredItFollowsTheFilenameConvention` - `testGetLatestHeight` - `testRewindDeletesTheRightBlocks` test - `testPerformanceExample` test. This isn't a real performance test because the API doesn't work with async/await yet adopts `shield_funds` shielding threshold parameter Implements `initBlockMetadataDb` and fix tests Renames dbCache parameter to `fsBlockDbRoot`. Builds but tests don't pass. Removes cacheDb uses from code. Testing utilities still persist. Added needed information in MIGRATING and CHANGELOG. Added helper to perform deletion of legacy db and creation a the new file system backed cache. Renames parameters and changes code where needed. Network Constants turned into `enum` with static methods. DeletelastDownloadedBlock helper from initializer Removes CompactBlockStorage and CompactBlockEntity. Implements `latestCachedBlockHeight` on rustbackend. *Replaces dependencies on ZcashRustWelding with `FSMetadataStore`* This allows the tests to not depend in a particular implementation of either the MockRustBackend of or ZcashRustBackend. Also provides a way to test errors properly and switch implementations of critical areas like `writeBlocks`.
2023-02-02 08:58:12 -08:00
import XCTest
@testable import TestUtils
@testable import ZcashLightClientKit
final class InternalStateConsistencyTests: ZcashTestCase {
- [#679] Implementation of the File-system based block cache (#679) Closes https://github.com/zcash/ZcashLightClientKit/issues/697 Closes https://github.com/zcash/ZcashLightClientKit/issues/720 Closes https://github.com/zcash/ZcashLightClientKit/issues/587 Closes https://github.com/zcash/ZcashLightClientKit/issues/667 Closes https://github.com/zcash/ZcashLightClientKit/issues/443 Closes https://github.com/zcash/ZcashLightClientKit/issues/754 - [#790] Fix ShieldFundsTests Closes #790 Removes comments on `ShieldFundsTests` since those issues have been fixed Depends on zcash-light-client-ffi changes that adopt newer versions of librustzcash crates `zcash_primitives 0.10`, `zcash_client_backend 0.7`, `zcash_proofs 0.10`, `zcash_client_sqlite 0.5.0`. Also allows wallets to define a shielding_threshold and will set foundations to customize minimum confirmations for balances, spends and shielding operations. **Test Bootstrapping** - `ZcashCompactBlockDescriptor`: struct that holds functions to describe blocks as filenames and compare those filenames `ZcashCompactBlockDescriptor.live` has the actual implementation but it can be replaced by mocks if needed on Tests main implementations are held under `FSCompactBlockRepository.filenameDescription` and `FSCompactBlockRepository.filenameComparison` on a separate extention `DirectoryListingProviders` provide two default implementations of listing a directory deterministically. `FileManager` does not define a sorting and needs to be done in-memory by calling `.sorted()` on the resulting collection. If this is a big toll on performance it can be changed to a POSIX implementation but this is good for now. `ZcashCompactBlockDescriptor` adds a `height` helper function to turn a filename into the height of the block stored. Implemented `func latestHeight() throws -> BlockHeight ` that returns the blockheight by querying the cache directory in a sorted fashion and getting the last value and turning the filename into a `BlockHeight` Added `Meta` struct to ZcashCompactBlock. Tests implemented: - `filterBlockFiles` - `testClearTheCache` - `testLatestHeightEmptyCacheThrows` - `testLatestHeightEmptyCacheThrowsAsync` - `testRewindEmptyCacheDoesNothing` - `testRewindEmptyCacheDoesNothingAsync` - `testWhenBlockIsStoredItFollowsTheDescribedFormat` - `testWhenBlockIsStoredItFollowsTheFilenameConvention` - `testGetLatestHeight` - `testRewindDeletesTheRightBlocks` test - `testPerformanceExample` test. This isn't a real performance test because the API doesn't work with async/await yet adopts `shield_funds` shielding threshold parameter Implements `initBlockMetadataDb` and fix tests Renames dbCache parameter to `fsBlockDbRoot`. Builds but tests don't pass. Removes cacheDb uses from code. Testing utilities still persist. Added needed information in MIGRATING and CHANGELOG. Added helper to perform deletion of legacy db and creation a the new file system backed cache. Renames parameters and changes code where needed. Network Constants turned into `enum` with static methods. DeletelastDownloadedBlock helper from initializer Removes CompactBlockStorage and CompactBlockEntity. Implements `latestCachedBlockHeight` on rustbackend. *Replaces dependencies on ZcashRustWelding with `FSMetadataStore`* This allows the tests to not depend in a particular implementation of either the MockRustBackend of or ZcashRustBackend. Also provides a way to test errors properly and switch implementations of critical areas like `writeBlocks`.
2023-02-02 08:58:12 -08:00
let sendAmount = Zatoshi(1000)
var birthday: BlockHeight = 663150
let defaultLatestHeight: BlockHeight = 663175
var coordinator: TestCoordinator!
var firstSyncExpectation = XCTestExpectation(description: "first sync expectation")
var expectedReorgHeight: BlockHeight = 665188
var expectedRewindHeight: BlockHeight = 665188
var reorgExpectation = XCTestExpectation(description: "reorg")
let branchID = "2bb40e60"
let chainName = "main"
let network = DarksideWalletDNetwork()
var sdkSynchronizerSyncStatusHandler: SDKSynchronizerSyncStatusHandler! = SDKSynchronizerSyncStatusHandler()
2023-02-16 08:14:31 -08:00
2023-03-30 10:01:47 -07:00
override func setUp() async throws {
try await super.setUp()
[#831] Add SDKSynchronizer wrappers for non-async API This change introduces two new protocols: `ClosureSynchronizer` and `CombineSynchronizer`. These two protocols define API that doesn't use `async`. So the client can choose exactly which API it wants to use. This change also introduces two new objects: `ClosureSDKSynchronizer` and `CombineSDKSynchronizer`. These two implement the respective protocols mentioned above. Both are structures. Neither of these two keeps any state. Thanks to this each is very cheap to create. And usage of these two isn't mutually exclusive. So devs can really choose the best SDK API for each part of the client app. [#831] Use async inside of the SDKSynchronizer - In general lot of methods inside the `SDKSynchronizer` and `CompactBlockProcessoer` which weren't async are now async. And other changes are made because of this change. - `CompactBlockProcessor` no longer uses Combine to communicate with `SDKSynchronizer`. Reason for this is that Combine doesn't play great with async. Closure passed to `sink` isn't async. - Because of this and because of how our tests work (receiving signals from CBP directly) `CompactBlockProcessor` must be able to handle more event closures. Not just one. So it now has `eventClosures` dictionary. It's little bit strange but it works fine. - `SyncStatus` inside the `SDKSynchronizer` was previously protected by lock. Now it's protected by simple actor wrapper. - Changes in tests are minimal. Changes were mady only because `CompactBlockProcessor` changes from Combine to closures. [#831] Add tests for ClosureSDKSynchronizer - Added tests are testing in general if the `ClosuresSDKSynchronizer` is correctly calling `Synchronizer` and if the values are correctly returned. - `ClosuresSDKSynchronizer` doesn't contain any logic but it is public API and we should be sure that it works correctly. [#831] Add tests for CombineSDKSynchronizer [#831] Add changelog
2023-03-16 02:11:18 -07:00
2023-03-30 10:01:47 -07:00
// don't use an exact birthday, users never do.
self.coordinator = try await TestCoordinator(
container: mockContainer,
walletBirthday: birthday + 50,
network: network
)
- [#679] Implementation of the File-system based block cache (#679) Closes https://github.com/zcash/ZcashLightClientKit/issues/697 Closes https://github.com/zcash/ZcashLightClientKit/issues/720 Closes https://github.com/zcash/ZcashLightClientKit/issues/587 Closes https://github.com/zcash/ZcashLightClientKit/issues/667 Closes https://github.com/zcash/ZcashLightClientKit/issues/443 Closes https://github.com/zcash/ZcashLightClientKit/issues/754 - [#790] Fix ShieldFundsTests Closes #790 Removes comments on `ShieldFundsTests` since those issues have been fixed Depends on zcash-light-client-ffi changes that adopt newer versions of librustzcash crates `zcash_primitives 0.10`, `zcash_client_backend 0.7`, `zcash_proofs 0.10`, `zcash_client_sqlite 0.5.0`. Also allows wallets to define a shielding_threshold and will set foundations to customize minimum confirmations for balances, spends and shielding operations. **Test Bootstrapping** - `ZcashCompactBlockDescriptor`: struct that holds functions to describe blocks as filenames and compare those filenames `ZcashCompactBlockDescriptor.live` has the actual implementation but it can be replaced by mocks if needed on Tests main implementations are held under `FSCompactBlockRepository.filenameDescription` and `FSCompactBlockRepository.filenameComparison` on a separate extention `DirectoryListingProviders` provide two default implementations of listing a directory deterministically. `FileManager` does not define a sorting and needs to be done in-memory by calling `.sorted()` on the resulting collection. If this is a big toll on performance it can be changed to a POSIX implementation but this is good for now. `ZcashCompactBlockDescriptor` adds a `height` helper function to turn a filename into the height of the block stored. Implemented `func latestHeight() throws -> BlockHeight ` that returns the blockheight by querying the cache directory in a sorted fashion and getting the last value and turning the filename into a `BlockHeight` Added `Meta` struct to ZcashCompactBlock. Tests implemented: - `filterBlockFiles` - `testClearTheCache` - `testLatestHeightEmptyCacheThrows` - `testLatestHeightEmptyCacheThrowsAsync` - `testRewindEmptyCacheDoesNothing` - `testRewindEmptyCacheDoesNothingAsync` - `testWhenBlockIsStoredItFollowsTheDescribedFormat` - `testWhenBlockIsStoredItFollowsTheFilenameConvention` - `testGetLatestHeight` - `testRewindDeletesTheRightBlocks` test - `testPerformanceExample` test. This isn't a real performance test because the API doesn't work with async/await yet adopts `shield_funds` shielding threshold parameter Implements `initBlockMetadataDb` and fix tests Renames dbCache parameter to `fsBlockDbRoot`. Builds but tests don't pass. Removes cacheDb uses from code. Testing utilities still persist. Added needed information in MIGRATING and CHANGELOG. Added helper to perform deletion of legacy db and creation a the new file system backed cache. Renames parameters and changes code where needed. Network Constants turned into `enum` with static methods. DeletelastDownloadedBlock helper from initializer Removes CompactBlockStorage and CompactBlockEntity. Implements `latestCachedBlockHeight` on rustbackend. *Replaces dependencies on ZcashRustWelding with `FSMetadataStore`* This allows the tests to not depend in a particular implementation of either the MockRustBackend of or ZcashRustBackend. Also provides a way to test errors properly and switch implementations of critical areas like `writeBlocks`.
2023-02-02 08:58:12 -08:00
try coordinator.reset(saplingActivation: 663150, branchID: self.branchID, chainName: self.chainName)
}
2023-03-30 10:01:47 -07:00
override func tearDown() async throws {
try await super.tearDown()
let coordinator = self.coordinator!
self.coordinator = nil
sdkSynchronizerSyncStatusHandler = nil
try await coordinator.stop()
- [#679] Implementation of the File-system based block cache (#679) Closes https://github.com/zcash/ZcashLightClientKit/issues/697 Closes https://github.com/zcash/ZcashLightClientKit/issues/720 Closes https://github.com/zcash/ZcashLightClientKit/issues/587 Closes https://github.com/zcash/ZcashLightClientKit/issues/667 Closes https://github.com/zcash/ZcashLightClientKit/issues/443 Closes https://github.com/zcash/ZcashLightClientKit/issues/754 - [#790] Fix ShieldFundsTests Closes #790 Removes comments on `ShieldFundsTests` since those issues have been fixed Depends on zcash-light-client-ffi changes that adopt newer versions of librustzcash crates `zcash_primitives 0.10`, `zcash_client_backend 0.7`, `zcash_proofs 0.10`, `zcash_client_sqlite 0.5.0`. Also allows wallets to define a shielding_threshold and will set foundations to customize minimum confirmations for balances, spends and shielding operations. **Test Bootstrapping** - `ZcashCompactBlockDescriptor`: struct that holds functions to describe blocks as filenames and compare those filenames `ZcashCompactBlockDescriptor.live` has the actual implementation but it can be replaced by mocks if needed on Tests main implementations are held under `FSCompactBlockRepository.filenameDescription` and `FSCompactBlockRepository.filenameComparison` on a separate extention `DirectoryListingProviders` provide two default implementations of listing a directory deterministically. `FileManager` does not define a sorting and needs to be done in-memory by calling `.sorted()` on the resulting collection. If this is a big toll on performance it can be changed to a POSIX implementation but this is good for now. `ZcashCompactBlockDescriptor` adds a `height` helper function to turn a filename into the height of the block stored. Implemented `func latestHeight() throws -> BlockHeight ` that returns the blockheight by querying the cache directory in a sorted fashion and getting the last value and turning the filename into a `BlockHeight` Added `Meta` struct to ZcashCompactBlock. Tests implemented: - `filterBlockFiles` - `testClearTheCache` - `testLatestHeightEmptyCacheThrows` - `testLatestHeightEmptyCacheThrowsAsync` - `testRewindEmptyCacheDoesNothing` - `testRewindEmptyCacheDoesNothingAsync` - `testWhenBlockIsStoredItFollowsTheDescribedFormat` - `testWhenBlockIsStoredItFollowsTheFilenameConvention` - `testGetLatestHeight` - `testRewindDeletesTheRightBlocks` test - `testPerformanceExample` test. This isn't a real performance test because the API doesn't work with async/await yet adopts `shield_funds` shielding threshold parameter Implements `initBlockMetadataDb` and fix tests Renames dbCache parameter to `fsBlockDbRoot`. Builds but tests don't pass. Removes cacheDb uses from code. Testing utilities still persist. Added needed information in MIGRATING and CHANGELOG. Added helper to perform deletion of legacy db and creation a the new file system backed cache. Renames parameters and changes code where needed. Network Constants turned into `enum` with static methods. DeletelastDownloadedBlock helper from initializer Removes CompactBlockStorage and CompactBlockEntity. Implements `latestCachedBlockHeight` on rustbackend. *Replaces dependencies on ZcashRustWelding with `FSMetadataStore`* This allows the tests to not depend in a particular implementation of either the MockRustBackend of or ZcashRustBackend. Also provides a way to test errors properly and switch implementations of critical areas like `writeBlocks`.
2023-02-02 08:58:12 -08:00
try? FileManager.default.removeItem(at: coordinator.databases.fsCacheDbRoot)
try? FileManager.default.removeItem(at: coordinator.databases.dataDB)
}
func testInternalStateIsConsistentWhenMigrating() async throws {
sdkSynchronizerSyncStatusHandler.subscribe(
to: coordinator.synchronizer.stateStream,
expectations: [.stopped: firstSyncExpectation]
)
- [#679] Implementation of the File-system based block cache (#679) Closes https://github.com/zcash/ZcashLightClientKit/issues/697 Closes https://github.com/zcash/ZcashLightClientKit/issues/720 Closes https://github.com/zcash/ZcashLightClientKit/issues/587 Closes https://github.com/zcash/ZcashLightClientKit/issues/667 Closes https://github.com/zcash/ZcashLightClientKit/issues/443 Closes https://github.com/zcash/ZcashLightClientKit/issues/754 - [#790] Fix ShieldFundsTests Closes #790 Removes comments on `ShieldFundsTests` since those issues have been fixed Depends on zcash-light-client-ffi changes that adopt newer versions of librustzcash crates `zcash_primitives 0.10`, `zcash_client_backend 0.7`, `zcash_proofs 0.10`, `zcash_client_sqlite 0.5.0`. Also allows wallets to define a shielding_threshold and will set foundations to customize minimum confirmations for balances, spends and shielding operations. **Test Bootstrapping** - `ZcashCompactBlockDescriptor`: struct that holds functions to describe blocks as filenames and compare those filenames `ZcashCompactBlockDescriptor.live` has the actual implementation but it can be replaced by mocks if needed on Tests main implementations are held under `FSCompactBlockRepository.filenameDescription` and `FSCompactBlockRepository.filenameComparison` on a separate extention `DirectoryListingProviders` provide two default implementations of listing a directory deterministically. `FileManager` does not define a sorting and needs to be done in-memory by calling `.sorted()` on the resulting collection. If this is a big toll on performance it can be changed to a POSIX implementation but this is good for now. `ZcashCompactBlockDescriptor` adds a `height` helper function to turn a filename into the height of the block stored. Implemented `func latestHeight() throws -> BlockHeight ` that returns the blockheight by querying the cache directory in a sorted fashion and getting the last value and turning the filename into a `BlockHeight` Added `Meta` struct to ZcashCompactBlock. Tests implemented: - `filterBlockFiles` - `testClearTheCache` - `testLatestHeightEmptyCacheThrows` - `testLatestHeightEmptyCacheThrowsAsync` - `testRewindEmptyCacheDoesNothing` - `testRewindEmptyCacheDoesNothingAsync` - `testWhenBlockIsStoredItFollowsTheDescribedFormat` - `testWhenBlockIsStoredItFollowsTheFilenameConvention` - `testGetLatestHeight` - `testRewindDeletesTheRightBlocks` test - `testPerformanceExample` test. This isn't a real performance test because the API doesn't work with async/await yet adopts `shield_funds` shielding threshold parameter Implements `initBlockMetadataDb` and fix tests Renames dbCache parameter to `fsBlockDbRoot`. Builds but tests don't pass. Removes cacheDb uses from code. Testing utilities still persist. Added needed information in MIGRATING and CHANGELOG. Added helper to perform deletion of legacy db and creation a the new file system backed cache. Renames parameters and changes code where needed. Network Constants turned into `enum` with static methods. DeletelastDownloadedBlock helper from initializer Removes CompactBlockStorage and CompactBlockEntity. Implements `latestCachedBlockHeight` on rustbackend. *Replaces dependencies on ZcashRustWelding with `FSMetadataStore`* This allows the tests to not depend in a particular implementation of either the MockRustBackend of or ZcashRustBackend. Also provides a way to test errors properly and switch implementations of critical areas like `writeBlocks`.
2023-02-02 08:58:12 -08:00
let fullSyncLength = 10000
- [#679] Implementation of the File-system based block cache (#679) Closes https://github.com/zcash/ZcashLightClientKit/issues/697 Closes https://github.com/zcash/ZcashLightClientKit/issues/720 Closes https://github.com/zcash/ZcashLightClientKit/issues/587 Closes https://github.com/zcash/ZcashLightClientKit/issues/667 Closes https://github.com/zcash/ZcashLightClientKit/issues/443 Closes https://github.com/zcash/ZcashLightClientKit/issues/754 - [#790] Fix ShieldFundsTests Closes #790 Removes comments on `ShieldFundsTests` since those issues have been fixed Depends on zcash-light-client-ffi changes that adopt newer versions of librustzcash crates `zcash_primitives 0.10`, `zcash_client_backend 0.7`, `zcash_proofs 0.10`, `zcash_client_sqlite 0.5.0`. Also allows wallets to define a shielding_threshold and will set foundations to customize minimum confirmations for balances, spends and shielding operations. **Test Bootstrapping** - `ZcashCompactBlockDescriptor`: struct that holds functions to describe blocks as filenames and compare those filenames `ZcashCompactBlockDescriptor.live` has the actual implementation but it can be replaced by mocks if needed on Tests main implementations are held under `FSCompactBlockRepository.filenameDescription` and `FSCompactBlockRepository.filenameComparison` on a separate extention `DirectoryListingProviders` provide two default implementations of listing a directory deterministically. `FileManager` does not define a sorting and needs to be done in-memory by calling `.sorted()` on the resulting collection. If this is a big toll on performance it can be changed to a POSIX implementation but this is good for now. `ZcashCompactBlockDescriptor` adds a `height` helper function to turn a filename into the height of the block stored. Implemented `func latestHeight() throws -> BlockHeight ` that returns the blockheight by querying the cache directory in a sorted fashion and getting the last value and turning the filename into a `BlockHeight` Added `Meta` struct to ZcashCompactBlock. Tests implemented: - `filterBlockFiles` - `testClearTheCache` - `testLatestHeightEmptyCacheThrows` - `testLatestHeightEmptyCacheThrowsAsync` - `testRewindEmptyCacheDoesNothing` - `testRewindEmptyCacheDoesNothingAsync` - `testWhenBlockIsStoredItFollowsTheDescribedFormat` - `testWhenBlockIsStoredItFollowsTheFilenameConvention` - `testGetLatestHeight` - `testRewindDeletesTheRightBlocks` test - `testPerformanceExample` test. This isn't a real performance test because the API doesn't work with async/await yet adopts `shield_funds` shielding threshold parameter Implements `initBlockMetadataDb` and fix tests Renames dbCache parameter to `fsBlockDbRoot`. Builds but tests don't pass. Removes cacheDb uses from code. Testing utilities still persist. Added needed information in MIGRATING and CHANGELOG. Added helper to perform deletion of legacy db and creation a the new file system backed cache. Renames parameters and changes code where needed. Network Constants turned into `enum` with static methods. DeletelastDownloadedBlock helper from initializer Removes CompactBlockStorage and CompactBlockEntity. Implements `latestCachedBlockHeight` on rustbackend. *Replaces dependencies on ZcashRustWelding with `FSMetadataStore`* This allows the tests to not depend in a particular implementation of either the MockRustBackend of or ZcashRustBackend. Also provides a way to test errors properly and switch implementations of critical areas like `writeBlocks`.
2023-02-02 08:58:12 -08:00
try FakeChainBuilder.buildChain(darksideWallet: coordinator.service, branchID: branchID, chainName: chainName, length: fullSyncLength)
sleep(1)
// apply the height
try coordinator.applyStaged(blockheight: 664150)
sleep(1)
[#831] Add SDKSynchronizer wrappers for non-async API This change introduces two new protocols: `ClosureSynchronizer` and `CombineSynchronizer`. These two protocols define API that doesn't use `async`. So the client can choose exactly which API it wants to use. This change also introduces two new objects: `ClosureSDKSynchronizer` and `CombineSDKSynchronizer`. These two implement the respective protocols mentioned above. Both are structures. Neither of these two keeps any state. Thanks to this each is very cheap to create. And usage of these two isn't mutually exclusive. So devs can really choose the best SDK API for each part of the client app. [#831] Use async inside of the SDKSynchronizer - In general lot of methods inside the `SDKSynchronizer` and `CompactBlockProcessoer` which weren't async are now async. And other changes are made because of this change. - `CompactBlockProcessor` no longer uses Combine to communicate with `SDKSynchronizer`. Reason for this is that Combine doesn't play great with async. Closure passed to `sink` isn't async. - Because of this and because of how our tests work (receiving signals from CBP directly) `CompactBlockProcessor` must be able to handle more event closures. Not just one. So it now has `eventClosures` dictionary. It's little bit strange but it works fine. - `SyncStatus` inside the `SDKSynchronizer` was previously protected by lock. Now it's protected by simple actor wrapper. - Changes in tests are minimal. Changes were mady only because `CompactBlockProcessor` changes from Combine to closures. [#831] Add tests for ClosureSDKSynchronizer - Added tests are testing in general if the `ClosuresSDKSynchronizer` is correctly calling `Synchronizer` and if the values are correctly returned. - `ClosuresSDKSynchronizer` doesn't contain any logic but it is public API and we should be sure that it works correctly. [#831] Add tests for CombineSDKSynchronizer [#831] Add changelog
2023-03-16 02:11:18 -07:00
try await coordinator.sync(
completion: { _ in
XCTFail("shouldn't have completed")
},
error: handleError
)
- [#679] Implementation of the File-system based block cache (#679) Closes https://github.com/zcash/ZcashLightClientKit/issues/697 Closes https://github.com/zcash/ZcashLightClientKit/issues/720 Closes https://github.com/zcash/ZcashLightClientKit/issues/587 Closes https://github.com/zcash/ZcashLightClientKit/issues/667 Closes https://github.com/zcash/ZcashLightClientKit/issues/443 Closes https://github.com/zcash/ZcashLightClientKit/issues/754 - [#790] Fix ShieldFundsTests Closes #790 Removes comments on `ShieldFundsTests` since those issues have been fixed Depends on zcash-light-client-ffi changes that adopt newer versions of librustzcash crates `zcash_primitives 0.10`, `zcash_client_backend 0.7`, `zcash_proofs 0.10`, `zcash_client_sqlite 0.5.0`. Also allows wallets to define a shielding_threshold and will set foundations to customize minimum confirmations for balances, spends and shielding operations. **Test Bootstrapping** - `ZcashCompactBlockDescriptor`: struct that holds functions to describe blocks as filenames and compare those filenames `ZcashCompactBlockDescriptor.live` has the actual implementation but it can be replaced by mocks if needed on Tests main implementations are held under `FSCompactBlockRepository.filenameDescription` and `FSCompactBlockRepository.filenameComparison` on a separate extention `DirectoryListingProviders` provide two default implementations of listing a directory deterministically. `FileManager` does not define a sorting and needs to be done in-memory by calling `.sorted()` on the resulting collection. If this is a big toll on performance it can be changed to a POSIX implementation but this is good for now. `ZcashCompactBlockDescriptor` adds a `height` helper function to turn a filename into the height of the block stored. Implemented `func latestHeight() throws -> BlockHeight ` that returns the blockheight by querying the cache directory in a sorted fashion and getting the last value and turning the filename into a `BlockHeight` Added `Meta` struct to ZcashCompactBlock. Tests implemented: - `filterBlockFiles` - `testClearTheCache` - `testLatestHeightEmptyCacheThrows` - `testLatestHeightEmptyCacheThrowsAsync` - `testRewindEmptyCacheDoesNothing` - `testRewindEmptyCacheDoesNothingAsync` - `testWhenBlockIsStoredItFollowsTheDescribedFormat` - `testWhenBlockIsStoredItFollowsTheFilenameConvention` - `testGetLatestHeight` - `testRewindDeletesTheRightBlocks` test - `testPerformanceExample` test. This isn't a real performance test because the API doesn't work with async/await yet adopts `shield_funds` shielding threshold parameter Implements `initBlockMetadataDb` and fix tests Renames dbCache parameter to `fsBlockDbRoot`. Builds but tests don't pass. Removes cacheDb uses from code. Testing utilities still persist. Added needed information in MIGRATING and CHANGELOG. Added helper to perform deletion of legacy db and creation a the new file system backed cache. Renames parameters and changes code where needed. Network Constants turned into `enum` with static methods. DeletelastDownloadedBlock helper from initializer Removes CompactBlockStorage and CompactBlockEntity. Implements `latestCachedBlockHeight` on rustbackend. *Replaces dependencies on ZcashRustWelding with `FSMetadataStore`* This allows the tests to not depend in a particular implementation of either the MockRustBackend of or ZcashRustBackend. Also provides a way to test errors properly and switch implementations of critical areas like `writeBlocks`.
2023-02-02 08:58:12 -08:00
[#831] Add SDKSynchronizer wrappers for non-async API This change introduces two new protocols: `ClosureSynchronizer` and `CombineSynchronizer`. These two protocols define API that doesn't use `async`. So the client can choose exactly which API it wants to use. This change also introduces two new objects: `ClosureSDKSynchronizer` and `CombineSDKSynchronizer`. These two implement the respective protocols mentioned above. Both are structures. Neither of these two keeps any state. Thanks to this each is very cheap to create. And usage of these two isn't mutually exclusive. So devs can really choose the best SDK API for each part of the client app. [#831] Use async inside of the SDKSynchronizer - In general lot of methods inside the `SDKSynchronizer` and `CompactBlockProcessoer` which weren't async are now async. And other changes are made because of this change. - `CompactBlockProcessor` no longer uses Combine to communicate with `SDKSynchronizer`. Reason for this is that Combine doesn't play great with async. Closure passed to `sink` isn't async. - Because of this and because of how our tests work (receiving signals from CBP directly) `CompactBlockProcessor` must be able to handle more event closures. Not just one. So it now has `eventClosures` dictionary. It's little bit strange but it works fine. - `SyncStatus` inside the `SDKSynchronizer` was previously protected by lock. Now it's protected by simple actor wrapper. - Changes in tests are minimal. Changes were mady only because `CompactBlockProcessor` changes from Combine to closures. [#831] Add tests for ClosureSDKSynchronizer - Added tests are testing in general if the `ClosuresSDKSynchronizer` is correctly calling `Synchronizer` and if the values are correctly returned. - `ClosuresSDKSynchronizer` doesn't contain any logic but it is public API and we should be sure that it works correctly. [#831] Add tests for CombineSDKSynchronizer [#831] Add changelog
2023-03-16 02:11:18 -07:00
let coordinator = self.coordinator!
DispatchQueue.global().asyncAfter(deadline: .now() + 1) {
Task(priority: .userInitiated) {
[#1001] Remove PendingDb in favor of `v_transactions` and `v_tx_output` Views (#1001) Removes `PendingTransactionEntity` and all of its related components. Pending items are still tracked and visualized by the existing APIs but they are retrieved from the `TransactionRepository` instead by returning `ZcashTransaction.Overview` instead. `pendingDbURL` is removed from every place it was required. Its deletion is responsibility of wallet developers. `ClearedTransactions` are now just `transactions`. `MigrationManager` is deleted. Now all migrations are in charge of the rust welding layer. `PendingTransactionDao.swift` is removed. Implementation of `AccountEntity` called `Account` is now `DbAccount` `ZcashTransaction.Overview` can be checked for "pending-ness" by calling `.isPending(latestHeight:)` latest height must be provided so that minedHeight can be compared with the lastest and the `defaultStaleTolerance` constant. `TransactionRecipient` is now a public type. protocol `PendingTransactionRepository` is removed. `TransactionManagerError` and `PersistentTransactionManager` are deleted. `OutboundTransactionManager` is deleted and replaced by `TransactionEncoder` which now incorporates `submit(encoded:)` functionality `WalletTransactionEncoder` now uses a `LightWalletService` to submit the encoded transactions. Add changelog changes Delete references to PendingDb from tests and documentation. Fixes some typos. Adds the ability to trace transaction repository SQL queries from test Fix rebase conflicts and generate code [#837] Memo tests regarding transparent address Closes #837 Add model for transaction output Point to FFI branch Fix issue where sync wouldn't resume after wipe. Becasue GRPC channel would be closed Fix Tests Fix testPendingTransactionMinedHeightUpdated Fix testLastStates [#921] Fix broken SynchronizerDarksideTests Add ZcashTransaction.Output API to Synchronizer Changelog + comment fix Add Assertions for transaction outputs and recipients Point to FFI 0.3.1 Fix Demo App Compiler errors Fix Demo App Compiler errors fix cacheDb warnings Fix Tests and compiler errors of rebase build demo app Remove `ZcashTransaction.Sent` and `.Received`. Add `.State` and tests Fix SPM warning PR Suggestions Removes errors that are not used anymore fix warnings
2023-05-05 10:30:47 -07:00
coordinator.synchronizer.stop()
[#831] Add SDKSynchronizer wrappers for non-async API This change introduces two new protocols: `ClosureSynchronizer` and `CombineSynchronizer`. These two protocols define API that doesn't use `async`. So the client can choose exactly which API it wants to use. This change also introduces two new objects: `ClosureSDKSynchronizer` and `CombineSDKSynchronizer`. These two implement the respective protocols mentioned above. Both are structures. Neither of these two keeps any state. Thanks to this each is very cheap to create. And usage of these two isn't mutually exclusive. So devs can really choose the best SDK API for each part of the client app. [#831] Use async inside of the SDKSynchronizer - In general lot of methods inside the `SDKSynchronizer` and `CompactBlockProcessoer` which weren't async are now async. And other changes are made because of this change. - `CompactBlockProcessor` no longer uses Combine to communicate with `SDKSynchronizer`. Reason for this is that Combine doesn't play great with async. Closure passed to `sink` isn't async. - Because of this and because of how our tests work (receiving signals from CBP directly) `CompactBlockProcessor` must be able to handle more event closures. Not just one. So it now has `eventClosures` dictionary. It's little bit strange but it works fine. - `SyncStatus` inside the `SDKSynchronizer` was previously protected by lock. Now it's protected by simple actor wrapper. - Changes in tests are minimal. Changes were mady only because `CompactBlockProcessor` changes from Combine to closures. [#831] Add tests for ClosureSDKSynchronizer - Added tests are testing in general if the `ClosuresSDKSynchronizer` is correctly calling `Synchronizer` and if the values are correctly returned. - `ClosuresSDKSynchronizer` doesn't contain any logic but it is public API and we should be sure that it works correctly. [#831] Add tests for CombineSDKSynchronizer [#831] Add changelog
2023-03-16 02:11:18 -07:00
}
- [#679] Implementation of the File-system based block cache (#679) Closes https://github.com/zcash/ZcashLightClientKit/issues/697 Closes https://github.com/zcash/ZcashLightClientKit/issues/720 Closes https://github.com/zcash/ZcashLightClientKit/issues/587 Closes https://github.com/zcash/ZcashLightClientKit/issues/667 Closes https://github.com/zcash/ZcashLightClientKit/issues/443 Closes https://github.com/zcash/ZcashLightClientKit/issues/754 - [#790] Fix ShieldFundsTests Closes #790 Removes comments on `ShieldFundsTests` since those issues have been fixed Depends on zcash-light-client-ffi changes that adopt newer versions of librustzcash crates `zcash_primitives 0.10`, `zcash_client_backend 0.7`, `zcash_proofs 0.10`, `zcash_client_sqlite 0.5.0`. Also allows wallets to define a shielding_threshold and will set foundations to customize minimum confirmations for balances, spends and shielding operations. **Test Bootstrapping** - `ZcashCompactBlockDescriptor`: struct that holds functions to describe blocks as filenames and compare those filenames `ZcashCompactBlockDescriptor.live` has the actual implementation but it can be replaced by mocks if needed on Tests main implementations are held under `FSCompactBlockRepository.filenameDescription` and `FSCompactBlockRepository.filenameComparison` on a separate extention `DirectoryListingProviders` provide two default implementations of listing a directory deterministically. `FileManager` does not define a sorting and needs to be done in-memory by calling `.sorted()` on the resulting collection. If this is a big toll on performance it can be changed to a POSIX implementation but this is good for now. `ZcashCompactBlockDescriptor` adds a `height` helper function to turn a filename into the height of the block stored. Implemented `func latestHeight() throws -> BlockHeight ` that returns the blockheight by querying the cache directory in a sorted fashion and getting the last value and turning the filename into a `BlockHeight` Added `Meta` struct to ZcashCompactBlock. Tests implemented: - `filterBlockFiles` - `testClearTheCache` - `testLatestHeightEmptyCacheThrows` - `testLatestHeightEmptyCacheThrowsAsync` - `testRewindEmptyCacheDoesNothing` - `testRewindEmptyCacheDoesNothingAsync` - `testWhenBlockIsStoredItFollowsTheDescribedFormat` - `testWhenBlockIsStoredItFollowsTheFilenameConvention` - `testGetLatestHeight` - `testRewindDeletesTheRightBlocks` test - `testPerformanceExample` test. This isn't a real performance test because the API doesn't work with async/await yet adopts `shield_funds` shielding threshold parameter Implements `initBlockMetadataDb` and fix tests Renames dbCache parameter to `fsBlockDbRoot`. Builds but tests don't pass. Removes cacheDb uses from code. Testing utilities still persist. Added needed information in MIGRATING and CHANGELOG. Added helper to perform deletion of legacy db and creation a the new file system backed cache. Renames parameters and changes code where needed. Network Constants turned into `enum` with static methods. DeletelastDownloadedBlock helper from initializer Removes CompactBlockStorage and CompactBlockEntity. Implements `latestCachedBlockHeight` on rustbackend. *Replaces dependencies on ZcashRustWelding with `FSMetadataStore`* This allows the tests to not depend in a particular implementation of either the MockRustBackend of or ZcashRustBackend. Also provides a way to test errors properly and switch implementations of critical areas like `writeBlocks`.
2023-02-02 08:58:12 -08:00
}
await fulfillment(of: [firstSyncExpectation], timeout: 2)
- [#679] Implementation of the File-system based block cache (#679) Closes https://github.com/zcash/ZcashLightClientKit/issues/697 Closes https://github.com/zcash/ZcashLightClientKit/issues/720 Closes https://github.com/zcash/ZcashLightClientKit/issues/587 Closes https://github.com/zcash/ZcashLightClientKit/issues/667 Closes https://github.com/zcash/ZcashLightClientKit/issues/443 Closes https://github.com/zcash/ZcashLightClientKit/issues/754 - [#790] Fix ShieldFundsTests Closes #790 Removes comments on `ShieldFundsTests` since those issues have been fixed Depends on zcash-light-client-ffi changes that adopt newer versions of librustzcash crates `zcash_primitives 0.10`, `zcash_client_backend 0.7`, `zcash_proofs 0.10`, `zcash_client_sqlite 0.5.0`. Also allows wallets to define a shielding_threshold and will set foundations to customize minimum confirmations for balances, spends and shielding operations. **Test Bootstrapping** - `ZcashCompactBlockDescriptor`: struct that holds functions to describe blocks as filenames and compare those filenames `ZcashCompactBlockDescriptor.live` has the actual implementation but it can be replaced by mocks if needed on Tests main implementations are held under `FSCompactBlockRepository.filenameDescription` and `FSCompactBlockRepository.filenameComparison` on a separate extention `DirectoryListingProviders` provide two default implementations of listing a directory deterministically. `FileManager` does not define a sorting and needs to be done in-memory by calling `.sorted()` on the resulting collection. If this is a big toll on performance it can be changed to a POSIX implementation but this is good for now. `ZcashCompactBlockDescriptor` adds a `height` helper function to turn a filename into the height of the block stored. Implemented `func latestHeight() throws -> BlockHeight ` that returns the blockheight by querying the cache directory in a sorted fashion and getting the last value and turning the filename into a `BlockHeight` Added `Meta` struct to ZcashCompactBlock. Tests implemented: - `filterBlockFiles` - `testClearTheCache` - `testLatestHeightEmptyCacheThrows` - `testLatestHeightEmptyCacheThrowsAsync` - `testRewindEmptyCacheDoesNothing` - `testRewindEmptyCacheDoesNothingAsync` - `testWhenBlockIsStoredItFollowsTheDescribedFormat` - `testWhenBlockIsStoredItFollowsTheFilenameConvention` - `testGetLatestHeight` - `testRewindDeletesTheRightBlocks` test - `testPerformanceExample` test. This isn't a real performance test because the API doesn't work with async/await yet adopts `shield_funds` shielding threshold parameter Implements `initBlockMetadataDb` and fix tests Renames dbCache parameter to `fsBlockDbRoot`. Builds but tests don't pass. Removes cacheDb uses from code. Testing utilities still persist. Added needed information in MIGRATING and CHANGELOG. Added helper to perform deletion of legacy db and creation a the new file system backed cache. Renames parameters and changes code where needed. Network Constants turned into `enum` with static methods. DeletelastDownloadedBlock helper from initializer Removes CompactBlockStorage and CompactBlockEntity. Implements `latestCachedBlockHeight` on rustbackend. *Replaces dependencies on ZcashRustWelding with `FSMetadataStore`* This allows the tests to not depend in a particular implementation of either the MockRustBackend of or ZcashRustBackend. Also provides a way to test errors properly and switch implementations of critical areas like `writeBlocks`.
2023-02-02 08:58:12 -08:00
[#831] Add SDKSynchronizer wrappers for non-async API This change introduces two new protocols: `ClosureSynchronizer` and `CombineSynchronizer`. These two protocols define API that doesn't use `async`. So the client can choose exactly which API it wants to use. This change also introduces two new objects: `ClosureSDKSynchronizer` and `CombineSDKSynchronizer`. These two implement the respective protocols mentioned above. Both are structures. Neither of these two keeps any state. Thanks to this each is very cheap to create. And usage of these two isn't mutually exclusive. So devs can really choose the best SDK API for each part of the client app. [#831] Use async inside of the SDKSynchronizer - In general lot of methods inside the `SDKSynchronizer` and `CompactBlockProcessoer` which weren't async are now async. And other changes are made because of this change. - `CompactBlockProcessor` no longer uses Combine to communicate with `SDKSynchronizer`. Reason for this is that Combine doesn't play great with async. Closure passed to `sink` isn't async. - Because of this and because of how our tests work (receiving signals from CBP directly) `CompactBlockProcessor` must be able to handle more event closures. Not just one. So it now has `eventClosures` dictionary. It's little bit strange but it works fine. - `SyncStatus` inside the `SDKSynchronizer` was previously protected by lock. Now it's protected by simple actor wrapper. - Changes in tests are minimal. Changes were mady only because `CompactBlockProcessor` changes from Combine to closures. [#831] Add tests for ClosureSDKSynchronizer - Added tests are testing in general if the `ClosuresSDKSynchronizer` is correctly calling `Synchronizer` and if the values are correctly returned. - `ClosuresSDKSynchronizer` doesn't contain any logic but it is public API and we should be sure that it works correctly. [#831] Add tests for CombineSDKSynchronizer [#831] Add changelog
2023-03-16 02:11:18 -07:00
let isSyncing = await coordinator.synchronizer.status.isSyncing
let status = await coordinator.synchronizer.status
XCTAssertFalse(isSyncing, "SDKSynchronizer shouldn't be syncing")
XCTAssertEqual(status, .stopped)
- [#679] Implementation of the File-system based block cache (#679) Closes https://github.com/zcash/ZcashLightClientKit/issues/697 Closes https://github.com/zcash/ZcashLightClientKit/issues/720 Closes https://github.com/zcash/ZcashLightClientKit/issues/587 Closes https://github.com/zcash/ZcashLightClientKit/issues/667 Closes https://github.com/zcash/ZcashLightClientKit/issues/443 Closes https://github.com/zcash/ZcashLightClientKit/issues/754 - [#790] Fix ShieldFundsTests Closes #790 Removes comments on `ShieldFundsTests` since those issues have been fixed Depends on zcash-light-client-ffi changes that adopt newer versions of librustzcash crates `zcash_primitives 0.10`, `zcash_client_backend 0.7`, `zcash_proofs 0.10`, `zcash_client_sqlite 0.5.0`. Also allows wallets to define a shielding_threshold and will set foundations to customize minimum confirmations for balances, spends and shielding operations. **Test Bootstrapping** - `ZcashCompactBlockDescriptor`: struct that holds functions to describe blocks as filenames and compare those filenames `ZcashCompactBlockDescriptor.live` has the actual implementation but it can be replaced by mocks if needed on Tests main implementations are held under `FSCompactBlockRepository.filenameDescription` and `FSCompactBlockRepository.filenameComparison` on a separate extention `DirectoryListingProviders` provide two default implementations of listing a directory deterministically. `FileManager` does not define a sorting and needs to be done in-memory by calling `.sorted()` on the resulting collection. If this is a big toll on performance it can be changed to a POSIX implementation but this is good for now. `ZcashCompactBlockDescriptor` adds a `height` helper function to turn a filename into the height of the block stored. Implemented `func latestHeight() throws -> BlockHeight ` that returns the blockheight by querying the cache directory in a sorted fashion and getting the last value and turning the filename into a `BlockHeight` Added `Meta` struct to ZcashCompactBlock. Tests implemented: - `filterBlockFiles` - `testClearTheCache` - `testLatestHeightEmptyCacheThrows` - `testLatestHeightEmptyCacheThrowsAsync` - `testRewindEmptyCacheDoesNothing` - `testRewindEmptyCacheDoesNothingAsync` - `testWhenBlockIsStoredItFollowsTheDescribedFormat` - `testWhenBlockIsStoredItFollowsTheFilenameConvention` - `testGetLatestHeight` - `testRewindDeletesTheRightBlocks` test - `testPerformanceExample` test. This isn't a real performance test because the API doesn't work with async/await yet adopts `shield_funds` shielding threshold parameter Implements `initBlockMetadataDb` and fix tests Renames dbCache parameter to `fsBlockDbRoot`. Builds but tests don't pass. Removes cacheDb uses from code. Testing utilities still persist. Added needed information in MIGRATING and CHANGELOG. Added helper to perform deletion of legacy db and creation a the new file system backed cache. Renames parameters and changes code where needed. Network Constants turned into `enum` with static methods. DeletelastDownloadedBlock helper from initializer Removes CompactBlockStorage and CompactBlockEntity. Implements `latestCachedBlockHeight` on rustbackend. *Replaces dependencies on ZcashRustWelding with `FSMetadataStore`* This allows the tests to not depend in a particular implementation of either the MockRustBackend of or ZcashRustBackend. Also provides a way to test errors properly and switch implementations of critical areas like `writeBlocks`.
2023-02-02 08:58:12 -08:00
[#209] Add support for multiple instances of the SDKSynchronizer Closes #209. [#845] Introduce ZcashSynchronizerAlias enum Closes #845. [#852] SDKSynchronizer using queues label based on the alias Closes #852. [#847] Remove posibility to use DatabaseStorageManager as singleton Closes #847. [#850] Remove synchronizerConnectionStateChanged notification Closes #850. [#855] Add check if the Alias is already used Closes #855 - Added `UsedAliasesChecker` utility which is used to register aliases that are in use. - `prepare()` and `wipe()` methods now check if the current alias can't be used and if not then `InitializerError.aliasAlreadyInUse` is thrown/emitted. - Some public methods that could cause harm if used with the Alias that is already in use now throw `SynchronizerError.notPrepared`. Thanks to this the client app is forced to call `prepare()` first. And `prepare()` does check for the Alias. - Added tests for new conditions. [#849] Make InternalSyncProgress aware of the Alias Closes #849. [#853] Only instance with default Alias migrates legacy cache DB Closes #853. [#851] Apply the Alias to the URLs Closes #851. - `Initializer` now updates paths according to alias before paths are used anywhere in the SDK. - Paths update can fail. It would be incovenient for the client apps to handle errors thrown from `Initiliazer` constructor. So the error is then handled in `SDKSynchronizer.prepare()` or `SDKSynchronizer.wipe()`. [#846] Stop using SDKMetrics as singleton (#862) - metrics are not longer a singleton - tests fixed - metrics outside init of the synchronizer [#848] Make logger aware of the alias - logger is now an instance passed throughout the sdk instead of a static proxy [#848] Make logger aware of the alias (#868) - comments addressed [#848] Make logger aware of the alias (#868) - returning protocol back Fix typos [#856] Add possibility to test multiple synchronizers in the sample app Closes #856. - Added `alias` property to `Synchronizer`. - Added `SyncBlocksListViewController` which provides UI to use multiple synchronizers at once. [#209] Add changelog - Add changelog for #209. - Overall improve readability of the rendered changelog. Tickets references are now prefixed with `###` instead of `- `. Fix compilation
2023-03-22 05:47:32 -07:00
let internalSyncState = InternalSyncProgress(
alias: .default,
storage: UserDefaults.standard,
logger: logger
)
- [#679] Implementation of the File-system based block cache (#679) Closes https://github.com/zcash/ZcashLightClientKit/issues/697 Closes https://github.com/zcash/ZcashLightClientKit/issues/720 Closes https://github.com/zcash/ZcashLightClientKit/issues/587 Closes https://github.com/zcash/ZcashLightClientKit/issues/667 Closes https://github.com/zcash/ZcashLightClientKit/issues/443 Closes https://github.com/zcash/ZcashLightClientKit/issues/754 - [#790] Fix ShieldFundsTests Closes #790 Removes comments on `ShieldFundsTests` since those issues have been fixed Depends on zcash-light-client-ffi changes that adopt newer versions of librustzcash crates `zcash_primitives 0.10`, `zcash_client_backend 0.7`, `zcash_proofs 0.10`, `zcash_client_sqlite 0.5.0`. Also allows wallets to define a shielding_threshold and will set foundations to customize minimum confirmations for balances, spends and shielding operations. **Test Bootstrapping** - `ZcashCompactBlockDescriptor`: struct that holds functions to describe blocks as filenames and compare those filenames `ZcashCompactBlockDescriptor.live` has the actual implementation but it can be replaced by mocks if needed on Tests main implementations are held under `FSCompactBlockRepository.filenameDescription` and `FSCompactBlockRepository.filenameComparison` on a separate extention `DirectoryListingProviders` provide two default implementations of listing a directory deterministically. `FileManager` does not define a sorting and needs to be done in-memory by calling `.sorted()` on the resulting collection. If this is a big toll on performance it can be changed to a POSIX implementation but this is good for now. `ZcashCompactBlockDescriptor` adds a `height` helper function to turn a filename into the height of the block stored. Implemented `func latestHeight() throws -> BlockHeight ` that returns the blockheight by querying the cache directory in a sorted fashion and getting the last value and turning the filename into a `BlockHeight` Added `Meta` struct to ZcashCompactBlock. Tests implemented: - `filterBlockFiles` - `testClearTheCache` - `testLatestHeightEmptyCacheThrows` - `testLatestHeightEmptyCacheThrowsAsync` - `testRewindEmptyCacheDoesNothing` - `testRewindEmptyCacheDoesNothingAsync` - `testWhenBlockIsStoredItFollowsTheDescribedFormat` - `testWhenBlockIsStoredItFollowsTheFilenameConvention` - `testGetLatestHeight` - `testRewindDeletesTheRightBlocks` test - `testPerformanceExample` test. This isn't a real performance test because the API doesn't work with async/await yet adopts `shield_funds` shielding threshold parameter Implements `initBlockMetadataDb` and fix tests Renames dbCache parameter to `fsBlockDbRoot`. Builds but tests don't pass. Removes cacheDb uses from code. Testing utilities still persist. Added needed information in MIGRATING and CHANGELOG. Added helper to perform deletion of legacy db and creation a the new file system backed cache. Renames parameters and changes code where needed. Network Constants turned into `enum` with static methods. DeletelastDownloadedBlock helper from initializer Removes CompactBlockStorage and CompactBlockEntity. Implements `latestCachedBlockHeight` on rustbackend. *Replaces dependencies on ZcashRustWelding with `FSMetadataStore`* This allows the tests to not depend in a particular implementation of either the MockRustBackend of or ZcashRustBackend. Also provides a way to test errors properly and switch implementations of critical areas like `writeBlocks`.
2023-02-02 08:58:12 -08:00
let latestDownloadHeight = await internalSyncState.latestDownloadedBlockHeight
let latestScanHeight = try await coordinator.synchronizer.initializer.transactionRepository.lastScannedHeight()
- [#679] Implementation of the File-system based block cache (#679) Closes https://github.com/zcash/ZcashLightClientKit/issues/697 Closes https://github.com/zcash/ZcashLightClientKit/issues/720 Closes https://github.com/zcash/ZcashLightClientKit/issues/587 Closes https://github.com/zcash/ZcashLightClientKit/issues/667 Closes https://github.com/zcash/ZcashLightClientKit/issues/443 Closes https://github.com/zcash/ZcashLightClientKit/issues/754 - [#790] Fix ShieldFundsTests Closes #790 Removes comments on `ShieldFundsTests` since those issues have been fixed Depends on zcash-light-client-ffi changes that adopt newer versions of librustzcash crates `zcash_primitives 0.10`, `zcash_client_backend 0.7`, `zcash_proofs 0.10`, `zcash_client_sqlite 0.5.0`. Also allows wallets to define a shielding_threshold and will set foundations to customize minimum confirmations for balances, spends and shielding operations. **Test Bootstrapping** - `ZcashCompactBlockDescriptor`: struct that holds functions to describe blocks as filenames and compare those filenames `ZcashCompactBlockDescriptor.live` has the actual implementation but it can be replaced by mocks if needed on Tests main implementations are held under `FSCompactBlockRepository.filenameDescription` and `FSCompactBlockRepository.filenameComparison` on a separate extention `DirectoryListingProviders` provide two default implementations of listing a directory deterministically. `FileManager` does not define a sorting and needs to be done in-memory by calling `.sorted()` on the resulting collection. If this is a big toll on performance it can be changed to a POSIX implementation but this is good for now. `ZcashCompactBlockDescriptor` adds a `height` helper function to turn a filename into the height of the block stored. Implemented `func latestHeight() throws -> BlockHeight ` that returns the blockheight by querying the cache directory in a sorted fashion and getting the last value and turning the filename into a `BlockHeight` Added `Meta` struct to ZcashCompactBlock. Tests implemented: - `filterBlockFiles` - `testClearTheCache` - `testLatestHeightEmptyCacheThrows` - `testLatestHeightEmptyCacheThrowsAsync` - `testRewindEmptyCacheDoesNothing` - `testRewindEmptyCacheDoesNothingAsync` - `testWhenBlockIsStoredItFollowsTheDescribedFormat` - `testWhenBlockIsStoredItFollowsTheFilenameConvention` - `testGetLatestHeight` - `testRewindDeletesTheRightBlocks` test - `testPerformanceExample` test. This isn't a real performance test because the API doesn't work with async/await yet adopts `shield_funds` shielding threshold parameter Implements `initBlockMetadataDb` and fix tests Renames dbCache parameter to `fsBlockDbRoot`. Builds but tests don't pass. Removes cacheDb uses from code. Testing utilities still persist. Added needed information in MIGRATING and CHANGELOG. Added helper to perform deletion of legacy db and creation a the new file system backed cache. Renames parameters and changes code where needed. Network Constants turned into `enum` with static methods. DeletelastDownloadedBlock helper from initializer Removes CompactBlockStorage and CompactBlockEntity. Implements `latestCachedBlockHeight` on rustbackend. *Replaces dependencies on ZcashRustWelding with `FSMetadataStore`* This allows the tests to not depend in a particular implementation of either the MockRustBackend of or ZcashRustBackend. Also provides a way to test errors properly and switch implementations of critical areas like `writeBlocks`.
2023-02-02 08:58:12 -08:00
let dbHandle = TestDbHandle(originalDb: TestDbBuilder.prePopulatedDarksideCacheDb()!)
try dbHandle.setUp()
if latestDownloadHeight > latestScanHeight {
try await coordinator.synchronizer.blockProcessor.migrateCacheDb(dbHandle.readWriteDb)
let afterMigrationDownloadedHeight = await internalSyncState.latestDownloadedBlockHeight
XCTAssertNotEqual(latestDownloadHeight, afterMigrationDownloadedHeight)
XCTAssertEqual(latestScanHeight, afterMigrationDownloadedHeight)
} else {
try await coordinator.synchronizer.blockProcessor.migrateCacheDb(dbHandle.readWriteDb)
let afterMigrationDownloadedHeight = await internalSyncState.latestDownloadedBlockHeight
XCTAssertEqual(latestDownloadHeight, afterMigrationDownloadedHeight)
XCTAssertEqual(latestScanHeight, afterMigrationDownloadedHeight)
}
XCTAssertFalse(FileManager.default.isReadableFile(atPath: dbHandle.readWriteDb.path))
// clear to simulate a clean slate from the FsBlockDb
try await coordinator.synchronizer.blockProcessor.storage.clear()
// Now let's resume scanning and see how it goes.
let secondSyncAttemptExpectation = XCTestExpectation(description: "second sync attempt")
[#831] Add SDKSynchronizer wrappers for non-async API This change introduces two new protocols: `ClosureSynchronizer` and `CombineSynchronizer`. These two protocols define API that doesn't use `async`. So the client can choose exactly which API it wants to use. This change also introduces two new objects: `ClosureSDKSynchronizer` and `CombineSDKSynchronizer`. These two implement the respective protocols mentioned above. Both are structures. Neither of these two keeps any state. Thanks to this each is very cheap to create. And usage of these two isn't mutually exclusive. So devs can really choose the best SDK API for each part of the client app. [#831] Use async inside of the SDKSynchronizer - In general lot of methods inside the `SDKSynchronizer` and `CompactBlockProcessoer` which weren't async are now async. And other changes are made because of this change. - `CompactBlockProcessor` no longer uses Combine to communicate with `SDKSynchronizer`. Reason for this is that Combine doesn't play great with async. Closure passed to `sink` isn't async. - Because of this and because of how our tests work (receiving signals from CBP directly) `CompactBlockProcessor` must be able to handle more event closures. Not just one. So it now has `eventClosures` dictionary. It's little bit strange but it works fine. - `SyncStatus` inside the `SDKSynchronizer` was previously protected by lock. Now it's protected by simple actor wrapper. - Changes in tests are minimal. Changes were mady only because `CompactBlockProcessor` changes from Combine to closures. [#831] Add tests for ClosureSDKSynchronizer - Added tests are testing in general if the `ClosuresSDKSynchronizer` is correctly calling `Synchronizer` and if the values are correctly returned. - `ClosuresSDKSynchronizer` doesn't contain any logic but it is public API and we should be sure that it works correctly. [#831] Add tests for CombineSDKSynchronizer [#831] Add changelog
2023-03-16 02:11:18 -07:00
do {
try await coordinator.sync(
completion: { _ in
XCTAssertTrue(true)
secondSyncAttemptExpectation.fulfill()
},
error: { [weak self] error in
secondSyncAttemptExpectation.fulfill()
self?.handleError(error)
}
)
} catch {
handleError(error)
- [#679] Implementation of the File-system based block cache (#679) Closes https://github.com/zcash/ZcashLightClientKit/issues/697 Closes https://github.com/zcash/ZcashLightClientKit/issues/720 Closes https://github.com/zcash/ZcashLightClientKit/issues/587 Closes https://github.com/zcash/ZcashLightClientKit/issues/667 Closes https://github.com/zcash/ZcashLightClientKit/issues/443 Closes https://github.com/zcash/ZcashLightClientKit/issues/754 - [#790] Fix ShieldFundsTests Closes #790 Removes comments on `ShieldFundsTests` since those issues have been fixed Depends on zcash-light-client-ffi changes that adopt newer versions of librustzcash crates `zcash_primitives 0.10`, `zcash_client_backend 0.7`, `zcash_proofs 0.10`, `zcash_client_sqlite 0.5.0`. Also allows wallets to define a shielding_threshold and will set foundations to customize minimum confirmations for balances, spends and shielding operations. **Test Bootstrapping** - `ZcashCompactBlockDescriptor`: struct that holds functions to describe blocks as filenames and compare those filenames `ZcashCompactBlockDescriptor.live` has the actual implementation but it can be replaced by mocks if needed on Tests main implementations are held under `FSCompactBlockRepository.filenameDescription` and `FSCompactBlockRepository.filenameComparison` on a separate extention `DirectoryListingProviders` provide two default implementations of listing a directory deterministically. `FileManager` does not define a sorting and needs to be done in-memory by calling `.sorted()` on the resulting collection. If this is a big toll on performance it can be changed to a POSIX implementation but this is good for now. `ZcashCompactBlockDescriptor` adds a `height` helper function to turn a filename into the height of the block stored. Implemented `func latestHeight() throws -> BlockHeight ` that returns the blockheight by querying the cache directory in a sorted fashion and getting the last value and turning the filename into a `BlockHeight` Added `Meta` struct to ZcashCompactBlock. Tests implemented: - `filterBlockFiles` - `testClearTheCache` - `testLatestHeightEmptyCacheThrows` - `testLatestHeightEmptyCacheThrowsAsync` - `testRewindEmptyCacheDoesNothing` - `testRewindEmptyCacheDoesNothingAsync` - `testWhenBlockIsStoredItFollowsTheDescribedFormat` - `testWhenBlockIsStoredItFollowsTheFilenameConvention` - `testGetLatestHeight` - `testRewindDeletesTheRightBlocks` test - `testPerformanceExample` test. This isn't a real performance test because the API doesn't work with async/await yet adopts `shield_funds` shielding threshold parameter Implements `initBlockMetadataDb` and fix tests Renames dbCache parameter to `fsBlockDbRoot`. Builds but tests don't pass. Removes cacheDb uses from code. Testing utilities still persist. Added needed information in MIGRATING and CHANGELOG. Added helper to perform deletion of legacy db and creation a the new file system backed cache. Renames parameters and changes code where needed. Network Constants turned into `enum` with static methods. DeletelastDownloadedBlock helper from initializer Removes CompactBlockStorage and CompactBlockEntity. Implements `latestCachedBlockHeight` on rustbackend. *Replaces dependencies on ZcashRustWelding with `FSMetadataStore`* This allows the tests to not depend in a particular implementation of either the MockRustBackend of or ZcashRustBackend. Also provides a way to test errors properly and switch implementations of critical areas like `writeBlocks`.
2023-02-02 08:58:12 -08:00
}
await fulfillment(of: [secondSyncAttemptExpectation], timeout: 10)
- [#679] Implementation of the File-system based block cache (#679) Closes https://github.com/zcash/ZcashLightClientKit/issues/697 Closes https://github.com/zcash/ZcashLightClientKit/issues/720 Closes https://github.com/zcash/ZcashLightClientKit/issues/587 Closes https://github.com/zcash/ZcashLightClientKit/issues/667 Closes https://github.com/zcash/ZcashLightClientKit/issues/443 Closes https://github.com/zcash/ZcashLightClientKit/issues/754 - [#790] Fix ShieldFundsTests Closes #790 Removes comments on `ShieldFundsTests` since those issues have been fixed Depends on zcash-light-client-ffi changes that adopt newer versions of librustzcash crates `zcash_primitives 0.10`, `zcash_client_backend 0.7`, `zcash_proofs 0.10`, `zcash_client_sqlite 0.5.0`. Also allows wallets to define a shielding_threshold and will set foundations to customize minimum confirmations for balances, spends and shielding operations. **Test Bootstrapping** - `ZcashCompactBlockDescriptor`: struct that holds functions to describe blocks as filenames and compare those filenames `ZcashCompactBlockDescriptor.live` has the actual implementation but it can be replaced by mocks if needed on Tests main implementations are held under `FSCompactBlockRepository.filenameDescription` and `FSCompactBlockRepository.filenameComparison` on a separate extention `DirectoryListingProviders` provide two default implementations of listing a directory deterministically. `FileManager` does not define a sorting and needs to be done in-memory by calling `.sorted()` on the resulting collection. If this is a big toll on performance it can be changed to a POSIX implementation but this is good for now. `ZcashCompactBlockDescriptor` adds a `height` helper function to turn a filename into the height of the block stored. Implemented `func latestHeight() throws -> BlockHeight ` that returns the blockheight by querying the cache directory in a sorted fashion and getting the last value and turning the filename into a `BlockHeight` Added `Meta` struct to ZcashCompactBlock. Tests implemented: - `filterBlockFiles` - `testClearTheCache` - `testLatestHeightEmptyCacheThrows` - `testLatestHeightEmptyCacheThrowsAsync` - `testRewindEmptyCacheDoesNothing` - `testRewindEmptyCacheDoesNothingAsync` - `testWhenBlockIsStoredItFollowsTheDescribedFormat` - `testWhenBlockIsStoredItFollowsTheFilenameConvention` - `testGetLatestHeight` - `testRewindDeletesTheRightBlocks` test - `testPerformanceExample` test. This isn't a real performance test because the API doesn't work with async/await yet adopts `shield_funds` shielding threshold parameter Implements `initBlockMetadataDb` and fix tests Renames dbCache parameter to `fsBlockDbRoot`. Builds but tests don't pass. Removes cacheDb uses from code. Testing utilities still persist. Added needed information in MIGRATING and CHANGELOG. Added helper to perform deletion of legacy db and creation a the new file system backed cache. Renames parameters and changes code where needed. Network Constants turned into `enum` with static methods. DeletelastDownloadedBlock helper from initializer Removes CompactBlockStorage and CompactBlockEntity. Implements `latestCachedBlockHeight` on rustbackend. *Replaces dependencies on ZcashRustWelding with `FSMetadataStore`* This allows the tests to not depend in a particular implementation of either the MockRustBackend of or ZcashRustBackend. Also provides a way to test errors properly and switch implementations of critical areas like `writeBlocks`.
2023-02-02 08:58:12 -08:00
}
func handleError(_ error: Error?) {
guard let testError = error else {
XCTFail("failed with nil error")
return
}
XCTFail("Failed with error: \(testError)")
}
- [#679] Implementation of the File-system based block cache (#679) Closes https://github.com/zcash/ZcashLightClientKit/issues/697 Closes https://github.com/zcash/ZcashLightClientKit/issues/720 Closes https://github.com/zcash/ZcashLightClientKit/issues/587 Closes https://github.com/zcash/ZcashLightClientKit/issues/667 Closes https://github.com/zcash/ZcashLightClientKit/issues/443 Closes https://github.com/zcash/ZcashLightClientKit/issues/754 - [#790] Fix ShieldFundsTests Closes #790 Removes comments on `ShieldFundsTests` since those issues have been fixed Depends on zcash-light-client-ffi changes that adopt newer versions of librustzcash crates `zcash_primitives 0.10`, `zcash_client_backend 0.7`, `zcash_proofs 0.10`, `zcash_client_sqlite 0.5.0`. Also allows wallets to define a shielding_threshold and will set foundations to customize minimum confirmations for balances, spends and shielding operations. **Test Bootstrapping** - `ZcashCompactBlockDescriptor`: struct that holds functions to describe blocks as filenames and compare those filenames `ZcashCompactBlockDescriptor.live` has the actual implementation but it can be replaced by mocks if needed on Tests main implementations are held under `FSCompactBlockRepository.filenameDescription` and `FSCompactBlockRepository.filenameComparison` on a separate extention `DirectoryListingProviders` provide two default implementations of listing a directory deterministically. `FileManager` does not define a sorting and needs to be done in-memory by calling `.sorted()` on the resulting collection. If this is a big toll on performance it can be changed to a POSIX implementation but this is good for now. `ZcashCompactBlockDescriptor` adds a `height` helper function to turn a filename into the height of the block stored. Implemented `func latestHeight() throws -> BlockHeight ` that returns the blockheight by querying the cache directory in a sorted fashion and getting the last value and turning the filename into a `BlockHeight` Added `Meta` struct to ZcashCompactBlock. Tests implemented: - `filterBlockFiles` - `testClearTheCache` - `testLatestHeightEmptyCacheThrows` - `testLatestHeightEmptyCacheThrowsAsync` - `testRewindEmptyCacheDoesNothing` - `testRewindEmptyCacheDoesNothingAsync` - `testWhenBlockIsStoredItFollowsTheDescribedFormat` - `testWhenBlockIsStoredItFollowsTheFilenameConvention` - `testGetLatestHeight` - `testRewindDeletesTheRightBlocks` test - `testPerformanceExample` test. This isn't a real performance test because the API doesn't work with async/await yet adopts `shield_funds` shielding threshold parameter Implements `initBlockMetadataDb` and fix tests Renames dbCache parameter to `fsBlockDbRoot`. Builds but tests don't pass. Removes cacheDb uses from code. Testing utilities still persist. Added needed information in MIGRATING and CHANGELOG. Added helper to perform deletion of legacy db and creation a the new file system backed cache. Renames parameters and changes code where needed. Network Constants turned into `enum` with static methods. DeletelastDownloadedBlock helper from initializer Removes CompactBlockStorage and CompactBlockEntity. Implements `latestCachedBlockHeight` on rustbackend. *Replaces dependencies on ZcashRustWelding with `FSMetadataStore`* This allows the tests to not depend in a particular implementation of either the MockRustBackend of or ZcashRustBackend. Also provides a way to test errors properly and switch implementations of critical areas like `writeBlocks`.
2023-02-02 08:58:12 -08:00
}