ZcashLightClientKit/Tests/NetworkTests/DownloadTests.swift

82 lines
2.5 KiB
Swift
Raw Normal View History

//
[476] CompactBlockProcessor to async/await - getting rid of the Operation Queue - the cleanup is needed - the update of tests is needed - tested and it successfully finishes the sync process [476] CompactBlockProcessor to async/await - old processNewBlocks() removed [476] CompactBlockProcessor to async/await - unused operations removed [476] CompactBlockProcessor to async/await - unit tests update [476] CompactBlockProcessor to async/await - unit tests refactored [476] CompactBlockProcessor to async/await - cleanup of deprecated method [476] CompactBlockProcessor to async/await - fail(error) was called even for canceled tasks but that must be excluded [476] CompactBlockProcessor to async/await - removal of all ZcashOperations from the code (unit test will follow) [476] CompactBlockProcessor to async/await - network tests in building and success order again [476] CompactBlockProcessor to async/await - offline tests in building and success order [476] CompactBlockProcessor to async/await (519) - cleanup of suspending the task [476] CompactBlockProcessor to async/await (519) - most comments resolved [476] CompactBlockProcessor to async/await (519) - thread safe state for both sync and async context [476] CompactBlockProcessor to async/await (519) - fixed build for a sample project [476] CompactBlockProcessor to async/await (519) - func testStartNotifiesSuscriptors() reverted [476] CompactBlockProcessor to async/await (519) - TODO added to track why we used NSLock instead of an Actor - Task priority enhanced [476] CompactBlockProcessor to async/await (519) - cleanup in Tasks and priorities
2022-09-01 05:58:41 -07:00
// DownloadTests.swift
// ZcashLightClientKitTests
//
// Created by Francisco Gindre on 10/16/19.
// Copyright © 2019 Electric Coin Company. All rights reserved.
//
import XCTest
import SQLite
2022-02-28 09:03:20 -08:00
@testable import TestUtils
@testable import ZcashLightClientKit
2021-09-23 06:26:41 -07:00
[476] CompactBlockProcessor to async/await - getting rid of the Operation Queue - the cleanup is needed - the update of tests is needed - tested and it successfully finishes the sync process [476] CompactBlockProcessor to async/await - old processNewBlocks() removed [476] CompactBlockProcessor to async/await - unused operations removed [476] CompactBlockProcessor to async/await - unit tests update [476] CompactBlockProcessor to async/await - unit tests refactored [476] CompactBlockProcessor to async/await - cleanup of deprecated method [476] CompactBlockProcessor to async/await - fail(error) was called even for canceled tasks but that must be excluded [476] CompactBlockProcessor to async/await - removal of all ZcashOperations from the code (unit test will follow) [476] CompactBlockProcessor to async/await - network tests in building and success order again [476] CompactBlockProcessor to async/await - offline tests in building and success order [476] CompactBlockProcessor to async/await (519) - cleanup of suspending the task [476] CompactBlockProcessor to async/await (519) - most comments resolved [476] CompactBlockProcessor to async/await (519) - thread safe state for both sync and async context [476] CompactBlockProcessor to async/await (519) - fixed build for a sample project [476] CompactBlockProcessor to async/await (519) - func testStartNotifiesSuscriptors() reverted [476] CompactBlockProcessor to async/await (519) - TODO added to track why we used NSLock instead of an Actor - Task priority enhanced [476] CompactBlockProcessor to async/await (519) - cleanup in Tasks and priorities
2022-09-01 05:58:41 -07:00
class DownloadTests: XCTestCase {
- [#679] Implementation of the File-system based block cache (#679) Closes https://github.com/zcash/ZcashLightClientKit/issues/697 Closes https://github.com/zcash/ZcashLightClientKit/issues/720 Closes https://github.com/zcash/ZcashLightClientKit/issues/587 Closes https://github.com/zcash/ZcashLightClientKit/issues/667 Closes https://github.com/zcash/ZcashLightClientKit/issues/443 Closes https://github.com/zcash/ZcashLightClientKit/issues/754 - [#790] Fix ShieldFundsTests Closes #790 Removes comments on `ShieldFundsTests` since those issues have been fixed Depends on zcash-light-client-ffi changes that adopt newer versions of librustzcash crates `zcash_primitives 0.10`, `zcash_client_backend 0.7`, `zcash_proofs 0.10`, `zcash_client_sqlite 0.5.0`. Also allows wallets to define a shielding_threshold and will set foundations to customize minimum confirmations for balances, spends and shielding operations. **Test Bootstrapping** - `ZcashCompactBlockDescriptor`: struct that holds functions to describe blocks as filenames and compare those filenames `ZcashCompactBlockDescriptor.live` has the actual implementation but it can be replaced by mocks if needed on Tests main implementations are held under `FSCompactBlockRepository.filenameDescription` and `FSCompactBlockRepository.filenameComparison` on a separate extention `DirectoryListingProviders` provide two default implementations of listing a directory deterministically. `FileManager` does not define a sorting and needs to be done in-memory by calling `.sorted()` on the resulting collection. If this is a big toll on performance it can be changed to a POSIX implementation but this is good for now. `ZcashCompactBlockDescriptor` adds a `height` helper function to turn a filename into the height of the block stored. Implemented `func latestHeight() throws -> BlockHeight ` that returns the blockheight by querying the cache directory in a sorted fashion and getting the last value and turning the filename into a `BlockHeight` Added `Meta` struct to ZcashCompactBlock. Tests implemented: - `filterBlockFiles` - `testClearTheCache` - `testLatestHeightEmptyCacheThrows` - `testLatestHeightEmptyCacheThrowsAsync` - `testRewindEmptyCacheDoesNothing` - `testRewindEmptyCacheDoesNothingAsync` - `testWhenBlockIsStoredItFollowsTheDescribedFormat` - `testWhenBlockIsStoredItFollowsTheFilenameConvention` - `testGetLatestHeight` - `testRewindDeletesTheRightBlocks` test - `testPerformanceExample` test. This isn't a real performance test because the API doesn't work with async/await yet adopts `shield_funds` shielding threshold parameter Implements `initBlockMetadataDb` and fix tests Renames dbCache parameter to `fsBlockDbRoot`. Builds but tests don't pass. Removes cacheDb uses from code. Testing utilities still persist. Added needed information in MIGRATING and CHANGELOG. Added helper to perform deletion of legacy db and creation a the new file system backed cache. Renames parameters and changes code where needed. Network Constants turned into `enum` with static methods. DeletelastDownloadedBlock helper from initializer Removes CompactBlockStorage and CompactBlockEntity. Implements `latestCachedBlockHeight` on rustbackend. *Replaces dependencies on ZcashRustWelding with `FSMetadataStore`* This allows the tests to not depend in a particular implementation of either the MockRustBackend of or ZcashRustBackend. Also provides a way to test errors properly and switch implementations of critical areas like `writeBlocks`.
2023-02-02 08:58:12 -08:00
let testTempDirectory = URL(fileURLWithPath: NSString(
string: NSTemporaryDirectory()
)
.appendingPathComponent("tmp-\(Int.random(in: 0 ... .max))"))
let testFileManager = FileManager()
2021-07-28 09:59:10 -07:00
var network = ZcashNetworkBuilder.network(for: .testnet)
2021-09-23 06:26:41 -07:00
- [#679] Implementation of the File-system based block cache (#679) Closes https://github.com/zcash/ZcashLightClientKit/issues/697 Closes https://github.com/zcash/ZcashLightClientKit/issues/720 Closes https://github.com/zcash/ZcashLightClientKit/issues/587 Closes https://github.com/zcash/ZcashLightClientKit/issues/667 Closes https://github.com/zcash/ZcashLightClientKit/issues/443 Closes https://github.com/zcash/ZcashLightClientKit/issues/754 - [#790] Fix ShieldFundsTests Closes #790 Removes comments on `ShieldFundsTests` since those issues have been fixed Depends on zcash-light-client-ffi changes that adopt newer versions of librustzcash crates `zcash_primitives 0.10`, `zcash_client_backend 0.7`, `zcash_proofs 0.10`, `zcash_client_sqlite 0.5.0`. Also allows wallets to define a shielding_threshold and will set foundations to customize minimum confirmations for balances, spends and shielding operations. **Test Bootstrapping** - `ZcashCompactBlockDescriptor`: struct that holds functions to describe blocks as filenames and compare those filenames `ZcashCompactBlockDescriptor.live` has the actual implementation but it can be replaced by mocks if needed on Tests main implementations are held under `FSCompactBlockRepository.filenameDescription` and `FSCompactBlockRepository.filenameComparison` on a separate extention `DirectoryListingProviders` provide two default implementations of listing a directory deterministically. `FileManager` does not define a sorting and needs to be done in-memory by calling `.sorted()` on the resulting collection. If this is a big toll on performance it can be changed to a POSIX implementation but this is good for now. `ZcashCompactBlockDescriptor` adds a `height` helper function to turn a filename into the height of the block stored. Implemented `func latestHeight() throws -> BlockHeight ` that returns the blockheight by querying the cache directory in a sorted fashion and getting the last value and turning the filename into a `BlockHeight` Added `Meta` struct to ZcashCompactBlock. Tests implemented: - `filterBlockFiles` - `testClearTheCache` - `testLatestHeightEmptyCacheThrows` - `testLatestHeightEmptyCacheThrowsAsync` - `testRewindEmptyCacheDoesNothing` - `testRewindEmptyCacheDoesNothingAsync` - `testWhenBlockIsStoredItFollowsTheDescribedFormat` - `testWhenBlockIsStoredItFollowsTheFilenameConvention` - `testGetLatestHeight` - `testRewindDeletesTheRightBlocks` test - `testPerformanceExample` test. This isn't a real performance test because the API doesn't work with async/await yet adopts `shield_funds` shielding threshold parameter Implements `initBlockMetadataDb` and fix tests Renames dbCache parameter to `fsBlockDbRoot`. Builds but tests don't pass. Removes cacheDb uses from code. Testing utilities still persist. Added needed information in MIGRATING and CHANGELOG. Added helper to perform deletion of legacy db and creation a the new file system backed cache. Renames parameters and changes code where needed. Network Constants turned into `enum` with static methods. DeletelastDownloadedBlock helper from initializer Removes CompactBlockStorage and CompactBlockEntity. Implements `latestCachedBlockHeight` on rustbackend. *Replaces dependencies on ZcashRustWelding with `FSMetadataStore`* This allows the tests to not depend in a particular implementation of either the MockRustBackend of or ZcashRustBackend. Also provides a way to test errors properly and switch implementations of critical areas like `writeBlocks`.
2023-02-02 08:58:12 -08:00
override func setUpWithError() throws {
try super.setUpWithError()
try self.testFileManager.createDirectory(at: self.testTempDirectory, withIntermediateDirectories: false)
}
override func tearDownWithError() throws {
try super.tearDownWithError()
try? testFileManager.removeItem(at: testTempDirectory)
}
[476] CompactBlockProcessor to async/await - getting rid of the Operation Queue - the cleanup is needed - the update of tests is needed - tested and it successfully finishes the sync process [476] CompactBlockProcessor to async/await - old processNewBlocks() removed [476] CompactBlockProcessor to async/await - unused operations removed [476] CompactBlockProcessor to async/await - unit tests update [476] CompactBlockProcessor to async/await - unit tests refactored [476] CompactBlockProcessor to async/await - cleanup of deprecated method [476] CompactBlockProcessor to async/await - fail(error) was called even for canceled tasks but that must be excluded [476] CompactBlockProcessor to async/await - removal of all ZcashOperations from the code (unit test will follow) [476] CompactBlockProcessor to async/await - network tests in building and success order again [476] CompactBlockProcessor to async/await - offline tests in building and success order [476] CompactBlockProcessor to async/await (519) - cleanup of suspending the task [476] CompactBlockProcessor to async/await (519) - most comments resolved [476] CompactBlockProcessor to async/await (519) - thread safe state for both sync and async context [476] CompactBlockProcessor to async/await (519) - fixed build for a sample project [476] CompactBlockProcessor to async/await (519) - func testStartNotifiesSuscriptors() reverted [476] CompactBlockProcessor to async/await (519) - TODO added to track why we used NSLock instead of an Actor - Task priority enhanced [476] CompactBlockProcessor to async/await (519) - cleanup in Tasks and priorities
2022-09-01 05:58:41 -07:00
func testSingleDownload() async throws {
[#209] Add support for multiple instances of the SDKSynchronizer Closes #209. [#845] Introduce ZcashSynchronizerAlias enum Closes #845. [#852] SDKSynchronizer using queues label based on the alias Closes #852. [#847] Remove posibility to use DatabaseStorageManager as singleton Closes #847. [#850] Remove synchronizerConnectionStateChanged notification Closes #850. [#855] Add check if the Alias is already used Closes #855 - Added `UsedAliasesChecker` utility which is used to register aliases that are in use. - `prepare()` and `wipe()` methods now check if the current alias can't be used and if not then `InitializerError.aliasAlreadyInUse` is thrown/emitted. - Some public methods that could cause harm if used with the Alias that is already in use now throw `SynchronizerError.notPrepared`. Thanks to this the client app is forced to call `prepare()` first. And `prepare()` does check for the Alias. - Added tests for new conditions. [#849] Make InternalSyncProgress aware of the Alias Closes #849. [#853] Only instance with default Alias migrates legacy cache DB Closes #853. [#851] Apply the Alias to the URLs Closes #851. - `Initializer` now updates paths according to alias before paths are used anywhere in the SDK. - Paths update can fail. It would be incovenient for the client apps to handle errors thrown from `Initiliazer` constructor. So the error is then handled in `SDKSynchronizer.prepare()` or `SDKSynchronizer.wipe()`. [#846] Stop using SDKMetrics as singleton (#862) - metrics are not longer a singleton - tests fixed - metrics outside init of the synchronizer [#848] Make logger aware of the alias - logger is now an instance passed throughout the sdk instead of a static proxy [#848] Make logger aware of the alias (#868) - comments addressed [#848] Make logger aware of the alias (#868) - returning protocol back Fix typos [#856] Add possibility to test multiple synchronizers in the sample app Closes #856. - Added `alias` property to `Synchronizer`. - Added `SyncBlocksListViewController` which provides UI to use multiple synchronizers at once. [#209] Add changelog - Add changelog for #209. - Overall improve readability of the rendered changelog. Tickets references are now prefixed with `###` instead of `- `. Fix compilation
2023-03-22 05:47:32 -07:00
let service = LightWalletServiceFactory(endpoint: LightWalletEndpointBuilder.eccTestnet).make()
- [#679] Implementation of the File-system based block cache (#679) Closes https://github.com/zcash/ZcashLightClientKit/issues/697 Closes https://github.com/zcash/ZcashLightClientKit/issues/720 Closes https://github.com/zcash/ZcashLightClientKit/issues/587 Closes https://github.com/zcash/ZcashLightClientKit/issues/667 Closes https://github.com/zcash/ZcashLightClientKit/issues/443 Closes https://github.com/zcash/ZcashLightClientKit/issues/754 - [#790] Fix ShieldFundsTests Closes #790 Removes comments on `ShieldFundsTests` since those issues have been fixed Depends on zcash-light-client-ffi changes that adopt newer versions of librustzcash crates `zcash_primitives 0.10`, `zcash_client_backend 0.7`, `zcash_proofs 0.10`, `zcash_client_sqlite 0.5.0`. Also allows wallets to define a shielding_threshold and will set foundations to customize minimum confirmations for balances, spends and shielding operations. **Test Bootstrapping** - `ZcashCompactBlockDescriptor`: struct that holds functions to describe blocks as filenames and compare those filenames `ZcashCompactBlockDescriptor.live` has the actual implementation but it can be replaced by mocks if needed on Tests main implementations are held under `FSCompactBlockRepository.filenameDescription` and `FSCompactBlockRepository.filenameComparison` on a separate extention `DirectoryListingProviders` provide two default implementations of listing a directory deterministically. `FileManager` does not define a sorting and needs to be done in-memory by calling `.sorted()` on the resulting collection. If this is a big toll on performance it can be changed to a POSIX implementation but this is good for now. `ZcashCompactBlockDescriptor` adds a `height` helper function to turn a filename into the height of the block stored. Implemented `func latestHeight() throws -> BlockHeight ` that returns the blockheight by querying the cache directory in a sorted fashion and getting the last value and turning the filename into a `BlockHeight` Added `Meta` struct to ZcashCompactBlock. Tests implemented: - `filterBlockFiles` - `testClearTheCache` - `testLatestHeightEmptyCacheThrows` - `testLatestHeightEmptyCacheThrowsAsync` - `testRewindEmptyCacheDoesNothing` - `testRewindEmptyCacheDoesNothingAsync` - `testWhenBlockIsStoredItFollowsTheDescribedFormat` - `testWhenBlockIsStoredItFollowsTheFilenameConvention` - `testGetLatestHeight` - `testRewindDeletesTheRightBlocks` test - `testPerformanceExample` test. This isn't a real performance test because the API doesn't work with async/await yet adopts `shield_funds` shielding threshold parameter Implements `initBlockMetadataDb` and fix tests Renames dbCache parameter to `fsBlockDbRoot`. Builds but tests don't pass. Removes cacheDb uses from code. Testing utilities still persist. Added needed information in MIGRATING and CHANGELOG. Added helper to perform deletion of legacy db and creation a the new file system backed cache. Renames parameters and changes code where needed. Network Constants turned into `enum` with static methods. DeletelastDownloadedBlock helper from initializer Removes CompactBlockStorage and CompactBlockEntity. Implements `latestCachedBlockHeight` on rustbackend. *Replaces dependencies on ZcashRustWelding with `FSMetadataStore`* This allows the tests to not depend in a particular implementation of either the MockRustBackend of or ZcashRustBackend. Also provides a way to test errors properly and switch implementations of critical areas like `writeBlocks`.
2023-02-02 08:58:12 -08:00
let realRustBackend = ZcashRustBackend.self
let storage = FSCompactBlockRepository(
fsBlockDbRoot: testTempDirectory,
- [#679] Implementation of the File-system based block cache (#679) Closes https://github.com/zcash/ZcashLightClientKit/issues/697 Closes https://github.com/zcash/ZcashLightClientKit/issues/720 Closes https://github.com/zcash/ZcashLightClientKit/issues/587 Closes https://github.com/zcash/ZcashLightClientKit/issues/667 Closes https://github.com/zcash/ZcashLightClientKit/issues/443 Closes https://github.com/zcash/ZcashLightClientKit/issues/754 - [#790] Fix ShieldFundsTests Closes #790 Removes comments on `ShieldFundsTests` since those issues have been fixed Depends on zcash-light-client-ffi changes that adopt newer versions of librustzcash crates `zcash_primitives 0.10`, `zcash_client_backend 0.7`, `zcash_proofs 0.10`, `zcash_client_sqlite 0.5.0`. Also allows wallets to define a shielding_threshold and will set foundations to customize minimum confirmations for balances, spends and shielding operations. **Test Bootstrapping** - `ZcashCompactBlockDescriptor`: struct that holds functions to describe blocks as filenames and compare those filenames `ZcashCompactBlockDescriptor.live` has the actual implementation but it can be replaced by mocks if needed on Tests main implementations are held under `FSCompactBlockRepository.filenameDescription` and `FSCompactBlockRepository.filenameComparison` on a separate extention `DirectoryListingProviders` provide two default implementations of listing a directory deterministically. `FileManager` does not define a sorting and needs to be done in-memory by calling `.sorted()` on the resulting collection. If this is a big toll on performance it can be changed to a POSIX implementation but this is good for now. `ZcashCompactBlockDescriptor` adds a `height` helper function to turn a filename into the height of the block stored. Implemented `func latestHeight() throws -> BlockHeight ` that returns the blockheight by querying the cache directory in a sorted fashion and getting the last value and turning the filename into a `BlockHeight` Added `Meta` struct to ZcashCompactBlock. Tests implemented: - `filterBlockFiles` - `testClearTheCache` - `testLatestHeightEmptyCacheThrows` - `testLatestHeightEmptyCacheThrowsAsync` - `testRewindEmptyCacheDoesNothing` - `testRewindEmptyCacheDoesNothingAsync` - `testWhenBlockIsStoredItFollowsTheDescribedFormat` - `testWhenBlockIsStoredItFollowsTheFilenameConvention` - `testGetLatestHeight` - `testRewindDeletesTheRightBlocks` test - `testPerformanceExample` test. This isn't a real performance test because the API doesn't work with async/await yet adopts `shield_funds` shielding threshold parameter Implements `initBlockMetadataDb` and fix tests Renames dbCache parameter to `fsBlockDbRoot`. Builds but tests don't pass. Removes cacheDb uses from code. Testing utilities still persist. Added needed information in MIGRATING and CHANGELOG. Added helper to perform deletion of legacy db and creation a the new file system backed cache. Renames parameters and changes code where needed. Network Constants turned into `enum` with static methods. DeletelastDownloadedBlock helper from initializer Removes CompactBlockStorage and CompactBlockEntity. Implements `latestCachedBlockHeight` on rustbackend. *Replaces dependencies on ZcashRustWelding with `FSMetadataStore`* This allows the tests to not depend in a particular implementation of either the MockRustBackend of or ZcashRustBackend. Also provides a way to test errors properly and switch implementations of critical areas like `writeBlocks`.
2023-02-02 08:58:12 -08:00
metadataStore: FSMetadataStore.live(
fsBlockDbRoot: testTempDirectory,
[#209] Add support for multiple instances of the SDKSynchronizer Closes #209. [#845] Introduce ZcashSynchronizerAlias enum Closes #845. [#852] SDKSynchronizer using queues label based on the alias Closes #852. [#847] Remove posibility to use DatabaseStorageManager as singleton Closes #847. [#850] Remove synchronizerConnectionStateChanged notification Closes #850. [#855] Add check if the Alias is already used Closes #855 - Added `UsedAliasesChecker` utility which is used to register aliases that are in use. - `prepare()` and `wipe()` methods now check if the current alias can't be used and if not then `InitializerError.aliasAlreadyInUse` is thrown/emitted. - Some public methods that could cause harm if used with the Alias that is already in use now throw `SynchronizerError.notPrepared`. Thanks to this the client app is forced to call `prepare()` first. And `prepare()` does check for the Alias. - Added tests for new conditions. [#849] Make InternalSyncProgress aware of the Alias Closes #849. [#853] Only instance with default Alias migrates legacy cache DB Closes #853. [#851] Apply the Alias to the URLs Closes #851. - `Initializer` now updates paths according to alias before paths are used anywhere in the SDK. - Paths update can fail. It would be incovenient for the client apps to handle errors thrown from `Initiliazer` constructor. So the error is then handled in `SDKSynchronizer.prepare()` or `SDKSynchronizer.wipe()`. [#846] Stop using SDKMetrics as singleton (#862) - metrics are not longer a singleton - tests fixed - metrics outside init of the synchronizer [#848] Make logger aware of the alias - logger is now an instance passed throughout the sdk instead of a static proxy [#848] Make logger aware of the alias (#868) - comments addressed [#848] Make logger aware of the alias (#868) - returning protocol back Fix typos [#856] Add possibility to test multiple synchronizers in the sample app Closes #856. - Added `alias` property to `Synchronizer`. - Added `SyncBlocksListViewController` which provides UI to use multiple synchronizers at once. [#209] Add changelog - Add changelog for #209. - Overall improve readability of the rendered changelog. Tickets references are now prefixed with `###` instead of `- `. Fix compilation
2023-03-22 05:47:32 -07:00
rustBackend: realRustBackend,
logger: logger
- [#679] Implementation of the File-system based block cache (#679) Closes https://github.com/zcash/ZcashLightClientKit/issues/697 Closes https://github.com/zcash/ZcashLightClientKit/issues/720 Closes https://github.com/zcash/ZcashLightClientKit/issues/587 Closes https://github.com/zcash/ZcashLightClientKit/issues/667 Closes https://github.com/zcash/ZcashLightClientKit/issues/443 Closes https://github.com/zcash/ZcashLightClientKit/issues/754 - [#790] Fix ShieldFundsTests Closes #790 Removes comments on `ShieldFundsTests` since those issues have been fixed Depends on zcash-light-client-ffi changes that adopt newer versions of librustzcash crates `zcash_primitives 0.10`, `zcash_client_backend 0.7`, `zcash_proofs 0.10`, `zcash_client_sqlite 0.5.0`. Also allows wallets to define a shielding_threshold and will set foundations to customize minimum confirmations for balances, spends and shielding operations. **Test Bootstrapping** - `ZcashCompactBlockDescriptor`: struct that holds functions to describe blocks as filenames and compare those filenames `ZcashCompactBlockDescriptor.live` has the actual implementation but it can be replaced by mocks if needed on Tests main implementations are held under `FSCompactBlockRepository.filenameDescription` and `FSCompactBlockRepository.filenameComparison` on a separate extention `DirectoryListingProviders` provide two default implementations of listing a directory deterministically. `FileManager` does not define a sorting and needs to be done in-memory by calling `.sorted()` on the resulting collection. If this is a big toll on performance it can be changed to a POSIX implementation but this is good for now. `ZcashCompactBlockDescriptor` adds a `height` helper function to turn a filename into the height of the block stored. Implemented `func latestHeight() throws -> BlockHeight ` that returns the blockheight by querying the cache directory in a sorted fashion and getting the last value and turning the filename into a `BlockHeight` Added `Meta` struct to ZcashCompactBlock. Tests implemented: - `filterBlockFiles` - `testClearTheCache` - `testLatestHeightEmptyCacheThrows` - `testLatestHeightEmptyCacheThrowsAsync` - `testRewindEmptyCacheDoesNothing` - `testRewindEmptyCacheDoesNothingAsync` - `testWhenBlockIsStoredItFollowsTheDescribedFormat` - `testWhenBlockIsStoredItFollowsTheFilenameConvention` - `testGetLatestHeight` - `testRewindDeletesTheRightBlocks` test - `testPerformanceExample` test. This isn't a real performance test because the API doesn't work with async/await yet adopts `shield_funds` shielding threshold parameter Implements `initBlockMetadataDb` and fix tests Renames dbCache parameter to `fsBlockDbRoot`. Builds but tests don't pass. Removes cacheDb uses from code. Testing utilities still persist. Added needed information in MIGRATING and CHANGELOG. Added helper to perform deletion of legacy db and creation a the new file system backed cache. Renames parameters and changes code where needed. Network Constants turned into `enum` with static methods. DeletelastDownloadedBlock helper from initializer Removes CompactBlockStorage and CompactBlockEntity. Implements `latestCachedBlockHeight` on rustbackend. *Replaces dependencies on ZcashRustWelding with `FSMetadataStore`* This allows the tests to not depend in a particular implementation of either the MockRustBackend of or ZcashRustBackend. Also provides a way to test errors properly and switch implementations of critical areas like `writeBlocks`.
2023-02-02 08:58:12 -08:00
),
blockDescriptor: .live,
[#209] Add support for multiple instances of the SDKSynchronizer Closes #209. [#845] Introduce ZcashSynchronizerAlias enum Closes #845. [#852] SDKSynchronizer using queues label based on the alias Closes #852. [#847] Remove posibility to use DatabaseStorageManager as singleton Closes #847. [#850] Remove synchronizerConnectionStateChanged notification Closes #850. [#855] Add check if the Alias is already used Closes #855 - Added `UsedAliasesChecker` utility which is used to register aliases that are in use. - `prepare()` and `wipe()` methods now check if the current alias can't be used and if not then `InitializerError.aliasAlreadyInUse` is thrown/emitted. - Some public methods that could cause harm if used with the Alias that is already in use now throw `SynchronizerError.notPrepared`. Thanks to this the client app is forced to call `prepare()` first. And `prepare()` does check for the Alias. - Added tests for new conditions. [#849] Make InternalSyncProgress aware of the Alias Closes #849. [#853] Only instance with default Alias migrates legacy cache DB Closes #853. [#851] Apply the Alias to the URLs Closes #851. - `Initializer` now updates paths according to alias before paths are used anywhere in the SDK. - Paths update can fail. It would be incovenient for the client apps to handle errors thrown from `Initiliazer` constructor. So the error is then handled in `SDKSynchronizer.prepare()` or `SDKSynchronizer.wipe()`. [#846] Stop using SDKMetrics as singleton (#862) - metrics are not longer a singleton - tests fixed - metrics outside init of the synchronizer [#848] Make logger aware of the alias - logger is now an instance passed throughout the sdk instead of a static proxy [#848] Make logger aware of the alias (#868) - comments addressed [#848] Make logger aware of the alias (#868) - returning protocol back Fix typos [#856] Add possibility to test multiple synchronizers in the sample app Closes #856. - Added `alias` property to `Synchronizer`. - Added `SyncBlocksListViewController` which provides UI to use multiple synchronizers at once. [#209] Add changelog - Add changelog for #209. - Overall improve readability of the rendered changelog. Tickets references are now prefixed with `###` instead of `- `. Fix compilation
2023-03-22 05:47:32 -07:00
contentProvider: DirectoryListingProviders.defaultSorted,
logger: logger
- [#679] Implementation of the File-system based block cache (#679) Closes https://github.com/zcash/ZcashLightClientKit/issues/697 Closes https://github.com/zcash/ZcashLightClientKit/issues/720 Closes https://github.com/zcash/ZcashLightClientKit/issues/587 Closes https://github.com/zcash/ZcashLightClientKit/issues/667 Closes https://github.com/zcash/ZcashLightClientKit/issues/443 Closes https://github.com/zcash/ZcashLightClientKit/issues/754 - [#790] Fix ShieldFundsTests Closes #790 Removes comments on `ShieldFundsTests` since those issues have been fixed Depends on zcash-light-client-ffi changes that adopt newer versions of librustzcash crates `zcash_primitives 0.10`, `zcash_client_backend 0.7`, `zcash_proofs 0.10`, `zcash_client_sqlite 0.5.0`. Also allows wallets to define a shielding_threshold and will set foundations to customize minimum confirmations for balances, spends and shielding operations. **Test Bootstrapping** - `ZcashCompactBlockDescriptor`: struct that holds functions to describe blocks as filenames and compare those filenames `ZcashCompactBlockDescriptor.live` has the actual implementation but it can be replaced by mocks if needed on Tests main implementations are held under `FSCompactBlockRepository.filenameDescription` and `FSCompactBlockRepository.filenameComparison` on a separate extention `DirectoryListingProviders` provide two default implementations of listing a directory deterministically. `FileManager` does not define a sorting and needs to be done in-memory by calling `.sorted()` on the resulting collection. If this is a big toll on performance it can be changed to a POSIX implementation but this is good for now. `ZcashCompactBlockDescriptor` adds a `height` helper function to turn a filename into the height of the block stored. Implemented `func latestHeight() throws -> BlockHeight ` that returns the blockheight by querying the cache directory in a sorted fashion and getting the last value and turning the filename into a `BlockHeight` Added `Meta` struct to ZcashCompactBlock. Tests implemented: - `filterBlockFiles` - `testClearTheCache` - `testLatestHeightEmptyCacheThrows` - `testLatestHeightEmptyCacheThrowsAsync` - `testRewindEmptyCacheDoesNothing` - `testRewindEmptyCacheDoesNothingAsync` - `testWhenBlockIsStoredItFollowsTheDescribedFormat` - `testWhenBlockIsStoredItFollowsTheFilenameConvention` - `testGetLatestHeight` - `testRewindDeletesTheRightBlocks` test - `testPerformanceExample` test. This isn't a real performance test because the API doesn't work with async/await yet adopts `shield_funds` shielding threshold parameter Implements `initBlockMetadataDb` and fix tests Renames dbCache parameter to `fsBlockDbRoot`. Builds but tests don't pass. Removes cacheDb uses from code. Testing utilities still persist. Added needed information in MIGRATING and CHANGELOG. Added helper to perform deletion of legacy db and creation a the new file system backed cache. Renames parameters and changes code where needed. Network Constants turned into `enum` with static methods. DeletelastDownloadedBlock helper from initializer Removes CompactBlockStorage and CompactBlockEntity. Implements `latestCachedBlockHeight` on rustbackend. *Replaces dependencies on ZcashRustWelding with `FSMetadataStore`* This allows the tests to not depend in a particular implementation of either the MockRustBackend of or ZcashRustBackend. Also provides a way to test errors properly and switch implementations of critical areas like `writeBlocks`.
2023-02-02 08:58:12 -08:00
)
try storage.create()
let blockCount = 100
2021-09-15 05:21:29 -07:00
let activationHeight = network.constants.saplingActivationHeight
2021-07-28 09:59:10 -07:00
let range = activationHeight ... activationHeight + blockCount
[476] CompactBlockProcessor to async/await - getting rid of the Operation Queue - the cleanup is needed - the update of tests is needed - tested and it successfully finishes the sync process [476] CompactBlockProcessor to async/await - old processNewBlocks() removed [476] CompactBlockProcessor to async/await - unused operations removed [476] CompactBlockProcessor to async/await - unit tests update [476] CompactBlockProcessor to async/await - unit tests refactored [476] CompactBlockProcessor to async/await - cleanup of deprecated method [476] CompactBlockProcessor to async/await - fail(error) was called even for canceled tasks but that must be excluded [476] CompactBlockProcessor to async/await - removal of all ZcashOperations from the code (unit test will follow) [476] CompactBlockProcessor to async/await - network tests in building and success order again [476] CompactBlockProcessor to async/await - offline tests in building and success order [476] CompactBlockProcessor to async/await (519) - cleanup of suspending the task [476] CompactBlockProcessor to async/await (519) - most comments resolved [476] CompactBlockProcessor to async/await (519) - thread safe state for both sync and async context [476] CompactBlockProcessor to async/await (519) - fixed build for a sample project [476] CompactBlockProcessor to async/await (519) - func testStartNotifiesSuscriptors() reverted [476] CompactBlockProcessor to async/await (519) - TODO added to track why we used NSLock instead of an Actor - Task priority enhanced [476] CompactBlockProcessor to async/await (519) - cleanup in Tasks and priorities
2022-09-01 05:58:41 -07:00
let processorConfig = CompactBlockProcessor.Configuration.standard(
for: network,
walletBirthday: network.constants.saplingActivationHeight
)
- [#679] Implementation of the File-system based block cache (#679) Closes https://github.com/zcash/ZcashLightClientKit/issues/697 Closes https://github.com/zcash/ZcashLightClientKit/issues/720 Closes https://github.com/zcash/ZcashLightClientKit/issues/587 Closes https://github.com/zcash/ZcashLightClientKit/issues/667 Closes https://github.com/zcash/ZcashLightClientKit/issues/443 Closes https://github.com/zcash/ZcashLightClientKit/issues/754 - [#790] Fix ShieldFundsTests Closes #790 Removes comments on `ShieldFundsTests` since those issues have been fixed Depends on zcash-light-client-ffi changes that adopt newer versions of librustzcash crates `zcash_primitives 0.10`, `zcash_client_backend 0.7`, `zcash_proofs 0.10`, `zcash_client_sqlite 0.5.0`. Also allows wallets to define a shielding_threshold and will set foundations to customize minimum confirmations for balances, spends and shielding operations. **Test Bootstrapping** - `ZcashCompactBlockDescriptor`: struct that holds functions to describe blocks as filenames and compare those filenames `ZcashCompactBlockDescriptor.live` has the actual implementation but it can be replaced by mocks if needed on Tests main implementations are held under `FSCompactBlockRepository.filenameDescription` and `FSCompactBlockRepository.filenameComparison` on a separate extention `DirectoryListingProviders` provide two default implementations of listing a directory deterministically. `FileManager` does not define a sorting and needs to be done in-memory by calling `.sorted()` on the resulting collection. If this is a big toll on performance it can be changed to a POSIX implementation but this is good for now. `ZcashCompactBlockDescriptor` adds a `height` helper function to turn a filename into the height of the block stored. Implemented `func latestHeight() throws -> BlockHeight ` that returns the blockheight by querying the cache directory in a sorted fashion and getting the last value and turning the filename into a `BlockHeight` Added `Meta` struct to ZcashCompactBlock. Tests implemented: - `filterBlockFiles` - `testClearTheCache` - `testLatestHeightEmptyCacheThrows` - `testLatestHeightEmptyCacheThrowsAsync` - `testRewindEmptyCacheDoesNothing` - `testRewindEmptyCacheDoesNothingAsync` - `testWhenBlockIsStoredItFollowsTheDescribedFormat` - `testWhenBlockIsStoredItFollowsTheFilenameConvention` - `testGetLatestHeight` - `testRewindDeletesTheRightBlocks` test - `testPerformanceExample` test. This isn't a real performance test because the API doesn't work with async/await yet adopts `shield_funds` shielding threshold parameter Implements `initBlockMetadataDb` and fix tests Renames dbCache parameter to `fsBlockDbRoot`. Builds but tests don't pass. Removes cacheDb uses from code. Testing utilities still persist. Added needed information in MIGRATING and CHANGELOG. Added helper to perform deletion of legacy db and creation a the new file system backed cache. Renames parameters and changes code where needed. Network Constants turned into `enum` with static methods. DeletelastDownloadedBlock helper from initializer Removes CompactBlockStorage and CompactBlockEntity. Implements `latestCachedBlockHeight` on rustbackend. *Replaces dependencies on ZcashRustWelding with `FSMetadataStore`* This allows the tests to not depend in a particular implementation of either the MockRustBackend of or ZcashRustBackend. Also provides a way to test errors properly and switch implementations of critical areas like `writeBlocks`.
2023-02-02 08:58:12 -08:00
[476] CompactBlockProcessor to async/await - getting rid of the Operation Queue - the cleanup is needed - the update of tests is needed - tested and it successfully finishes the sync process [476] CompactBlockProcessor to async/await - old processNewBlocks() removed [476] CompactBlockProcessor to async/await - unused operations removed [476] CompactBlockProcessor to async/await - unit tests update [476] CompactBlockProcessor to async/await - unit tests refactored [476] CompactBlockProcessor to async/await - cleanup of deprecated method [476] CompactBlockProcessor to async/await - fail(error) was called even for canceled tasks but that must be excluded [476] CompactBlockProcessor to async/await - removal of all ZcashOperations from the code (unit test will follow) [476] CompactBlockProcessor to async/await - network tests in building and success order again [476] CompactBlockProcessor to async/await - offline tests in building and success order [476] CompactBlockProcessor to async/await (519) - cleanup of suspending the task [476] CompactBlockProcessor to async/await (519) - most comments resolved [476] CompactBlockProcessor to async/await (519) - thread safe state for both sync and async context [476] CompactBlockProcessor to async/await (519) - fixed build for a sample project [476] CompactBlockProcessor to async/await (519) - func testStartNotifiesSuscriptors() reverted [476] CompactBlockProcessor to async/await (519) - TODO added to track why we used NSLock instead of an Actor - Task priority enhanced [476] CompactBlockProcessor to async/await (519) - cleanup in Tasks and priorities
2022-09-01 05:58:41 -07:00
let compactBlockProcessor = CompactBlockProcessor(
service: service,
storage: storage,
- [#679] Implementation of the File-system based block cache (#679) Closes https://github.com/zcash/ZcashLightClientKit/issues/697 Closes https://github.com/zcash/ZcashLightClientKit/issues/720 Closes https://github.com/zcash/ZcashLightClientKit/issues/587 Closes https://github.com/zcash/ZcashLightClientKit/issues/667 Closes https://github.com/zcash/ZcashLightClientKit/issues/443 Closes https://github.com/zcash/ZcashLightClientKit/issues/754 - [#790] Fix ShieldFundsTests Closes #790 Removes comments on `ShieldFundsTests` since those issues have been fixed Depends on zcash-light-client-ffi changes that adopt newer versions of librustzcash crates `zcash_primitives 0.10`, `zcash_client_backend 0.7`, `zcash_proofs 0.10`, `zcash_client_sqlite 0.5.0`. Also allows wallets to define a shielding_threshold and will set foundations to customize minimum confirmations for balances, spends and shielding operations. **Test Bootstrapping** - `ZcashCompactBlockDescriptor`: struct that holds functions to describe blocks as filenames and compare those filenames `ZcashCompactBlockDescriptor.live` has the actual implementation but it can be replaced by mocks if needed on Tests main implementations are held under `FSCompactBlockRepository.filenameDescription` and `FSCompactBlockRepository.filenameComparison` on a separate extention `DirectoryListingProviders` provide two default implementations of listing a directory deterministically. `FileManager` does not define a sorting and needs to be done in-memory by calling `.sorted()` on the resulting collection. If this is a big toll on performance it can be changed to a POSIX implementation but this is good for now. `ZcashCompactBlockDescriptor` adds a `height` helper function to turn a filename into the height of the block stored. Implemented `func latestHeight() throws -> BlockHeight ` that returns the blockheight by querying the cache directory in a sorted fashion and getting the last value and turning the filename into a `BlockHeight` Added `Meta` struct to ZcashCompactBlock. Tests implemented: - `filterBlockFiles` - `testClearTheCache` - `testLatestHeightEmptyCacheThrows` - `testLatestHeightEmptyCacheThrowsAsync` - `testRewindEmptyCacheDoesNothing` - `testRewindEmptyCacheDoesNothingAsync` - `testWhenBlockIsStoredItFollowsTheDescribedFormat` - `testWhenBlockIsStoredItFollowsTheFilenameConvention` - `testGetLatestHeight` - `testRewindDeletesTheRightBlocks` test - `testPerformanceExample` test. This isn't a real performance test because the API doesn't work with async/await yet adopts `shield_funds` shielding threshold parameter Implements `initBlockMetadataDb` and fix tests Renames dbCache parameter to `fsBlockDbRoot`. Builds but tests don't pass. Removes cacheDb uses from code. Testing utilities still persist. Added needed information in MIGRATING and CHANGELOG. Added helper to perform deletion of legacy db and creation a the new file system backed cache. Renames parameters and changes code where needed. Network Constants turned into `enum` with static methods. DeletelastDownloadedBlock helper from initializer Removes CompactBlockStorage and CompactBlockEntity. Implements `latestCachedBlockHeight` on rustbackend. *Replaces dependencies on ZcashRustWelding with `FSMetadataStore`* This allows the tests to not depend in a particular implementation of either the MockRustBackend of or ZcashRustBackend. Also provides a way to test errors properly and switch implementations of critical areas like `writeBlocks`.
2023-02-02 08:58:12 -08:00
backend: realRustBackend,
[#209] Add support for multiple instances of the SDKSynchronizer Closes #209. [#845] Introduce ZcashSynchronizerAlias enum Closes #845. [#852] SDKSynchronizer using queues label based on the alias Closes #852. [#847] Remove posibility to use DatabaseStorageManager as singleton Closes #847. [#850] Remove synchronizerConnectionStateChanged notification Closes #850. [#855] Add check if the Alias is already used Closes #855 - Added `UsedAliasesChecker` utility which is used to register aliases that are in use. - `prepare()` and `wipe()` methods now check if the current alias can't be used and if not then `InitializerError.aliasAlreadyInUse` is thrown/emitted. - Some public methods that could cause harm if used with the Alias that is already in use now throw `SynchronizerError.notPrepared`. Thanks to this the client app is forced to call `prepare()` first. And `prepare()` does check for the Alias. - Added tests for new conditions. [#849] Make InternalSyncProgress aware of the Alias Closes #849. [#853] Only instance with default Alias migrates legacy cache DB Closes #853. [#851] Apply the Alias to the URLs Closes #851. - `Initializer` now updates paths according to alias before paths are used anywhere in the SDK. - Paths update can fail. It would be incovenient for the client apps to handle errors thrown from `Initiliazer` constructor. So the error is then handled in `SDKSynchronizer.prepare()` or `SDKSynchronizer.wipe()`. [#846] Stop using SDKMetrics as singleton (#862) - metrics are not longer a singleton - tests fixed - metrics outside init of the synchronizer [#848] Make logger aware of the alias - logger is now an instance passed throughout the sdk instead of a static proxy [#848] Make logger aware of the alias (#868) - comments addressed [#848] Make logger aware of the alias (#868) - returning protocol back Fix typos [#856] Add possibility to test multiple synchronizers in the sample app Closes #856. - Added `alias` property to `Synchronizer`. - Added `SyncBlocksListViewController` which provides UI to use multiple synchronizers at once. [#209] Add changelog - Add changelog for #209. - Overall improve readability of the rendered changelog. Tickets references are now prefixed with `###` instead of `- `. Fix compilation
2023-03-22 05:47:32 -07:00
config: processorConfig,
metrics: SDKMetrics(),
logger: logger
[476] CompactBlockProcessor to async/await - getting rid of the Operation Queue - the cleanup is needed - the update of tests is needed - tested and it successfully finishes the sync process [476] CompactBlockProcessor to async/await - old processNewBlocks() removed [476] CompactBlockProcessor to async/await - unused operations removed [476] CompactBlockProcessor to async/await - unit tests update [476] CompactBlockProcessor to async/await - unit tests refactored [476] CompactBlockProcessor to async/await - cleanup of deprecated method [476] CompactBlockProcessor to async/await - fail(error) was called even for canceled tasks but that must be excluded [476] CompactBlockProcessor to async/await - removal of all ZcashOperations from the code (unit test will follow) [476] CompactBlockProcessor to async/await - network tests in building and success order again [476] CompactBlockProcessor to async/await - offline tests in building and success order [476] CompactBlockProcessor to async/await (519) - cleanup of suspending the task [476] CompactBlockProcessor to async/await (519) - most comments resolved [476] CompactBlockProcessor to async/await (519) - thread safe state for both sync and async context [476] CompactBlockProcessor to async/await (519) - fixed build for a sample project [476] CompactBlockProcessor to async/await (519) - func testStartNotifiesSuscriptors() reverted [476] CompactBlockProcessor to async/await (519) - TODO added to track why we used NSLock instead of an Actor - Task priority enhanced [476] CompactBlockProcessor to async/await (519) - cleanup in Tasks and priorities
2022-09-01 05:58:41 -07:00
)
[476] CompactBlockProcessor to async/await - getting rid of the Operation Queue - the cleanup is needed - the update of tests is needed - tested and it successfully finishes the sync process [476] CompactBlockProcessor to async/await - old processNewBlocks() removed [476] CompactBlockProcessor to async/await - unused operations removed [476] CompactBlockProcessor to async/await - unit tests update [476] CompactBlockProcessor to async/await - unit tests refactored [476] CompactBlockProcessor to async/await - cleanup of deprecated method [476] CompactBlockProcessor to async/await - fail(error) was called even for canceled tasks but that must be excluded [476] CompactBlockProcessor to async/await - removal of all ZcashOperations from the code (unit test will follow) [476] CompactBlockProcessor to async/await - network tests in building and success order again [476] CompactBlockProcessor to async/await - offline tests in building and success order [476] CompactBlockProcessor to async/await (519) - cleanup of suspending the task [476] CompactBlockProcessor to async/await (519) - most comments resolved [476] CompactBlockProcessor to async/await (519) - thread safe state for both sync and async context [476] CompactBlockProcessor to async/await (519) - fixed build for a sample project [476] CompactBlockProcessor to async/await (519) - func testStartNotifiesSuscriptors() reverted [476] CompactBlockProcessor to async/await (519) - TODO added to track why we used NSLock instead of an Actor - Task priority enhanced [476] CompactBlockProcessor to async/await (519) - cleanup in Tasks and priorities
2022-09-01 05:58:41 -07:00
do {
try await compactBlockProcessor.blockDownloaderService.downloadBlockRange(range)
[476] CompactBlockProcessor to async/await - getting rid of the Operation Queue - the cleanup is needed - the update of tests is needed - tested and it successfully finishes the sync process [476] CompactBlockProcessor to async/await - old processNewBlocks() removed [476] CompactBlockProcessor to async/await - unused operations removed [476] CompactBlockProcessor to async/await - unit tests update [476] CompactBlockProcessor to async/await - unit tests refactored [476] CompactBlockProcessor to async/await - cleanup of deprecated method [476] CompactBlockProcessor to async/await - fail(error) was called even for canceled tasks but that must be excluded [476] CompactBlockProcessor to async/await - removal of all ZcashOperations from the code (unit test will follow) [476] CompactBlockProcessor to async/await - network tests in building and success order again [476] CompactBlockProcessor to async/await - offline tests in building and success order [476] CompactBlockProcessor to async/await (519) - cleanup of suspending the task [476] CompactBlockProcessor to async/await (519) - most comments resolved [476] CompactBlockProcessor to async/await (519) - thread safe state for both sync and async context [476] CompactBlockProcessor to async/await (519) - fixed build for a sample project [476] CompactBlockProcessor to async/await (519) - func testStartNotifiesSuscriptors() reverted [476] CompactBlockProcessor to async/await (519) - TODO added to track why we used NSLock instead of an Actor - Task priority enhanced [476] CompactBlockProcessor to async/await (519) - cleanup in Tasks and priorities
2022-09-01 05:58:41 -07:00
} catch {
XCTFail("Download failed with error: \(error)")
}
let latestHeight = await storage.latestHeight()
XCTAssertEqual(latestHeight, range.upperBound)
}
}