Commit Graph

72 Commits

Author SHA1 Message Date
Michal Fousek 3aad1470f1 [#793] Send synchronizerStopped notification only when sync process stops
Closes #793
2023-02-17 08:49:55 +01:00
Michal Fousek 44732b179b Update code to to comply better with linter rules
- Removed some disabled linter rules from code and update code to comply
  with those.
- No logic was changed.
2023-02-13 12:18:08 +01:00
Michal Fousek 1ec12269ae [#764] Stop using Notifications inside the SDK
Closes #764

- All notification that were previously sent from CompactBlockProcessor are now gone.
- `CompactBlockProcessor` now provides `eventStream` to communicate with `SDKSynchronizer`.
- Added `synchronizerStoredUTXOs` notification.
- Tests are updated to not use notifications anymore. Some tests there
  weren't working before are now fixed.
- Added `CompactBlockProcessorEventHandler` utility class. Previously
  expectations were subscribing to notifications. This class replaces
  this functionality. It subscribes to events from
  `CompactBlockProcessor` and fullfill expectations.
2023-02-09 20:09:51 +01:00
Francisco Gindre 67dd061523 Merge branch 'main' into release/0.18.1-beta 2023-02-08 12:48:20 -03:00
Francisco Gindre 2c211732e1 [#767] implement getRecipients() for Synchronizer.
This implements `getRecipients()` function which retrieves the possible
recipients from a sent transaction. These can either be addresses or
internal accounts depending on the transaction being a shielding tx
or a regular outgoing transaction.

Closes #767

Other changes:
Fix version of zcash-light-client-ffi to 0.1.1

Enhance error reporting on a test make Mock comply with protocol

Update CHANGELOG.md
2023-02-08 08:44:29 -03:00
Michal Fousek 385c0a7195 [#746] Modularize GRPC related code
Closes #746

- `LightWalletGRPCService` is no longer public. `LightWalletService` is no longer public.
- `LightWalletGRPCService` shouldn't be used dicrectly. Use `LightWalletServiceFactory` to create instance of the service.
- Moved sending of `blockProcessorConnectivityStateChanged` notification to `Initilizer`.
- All the errors from GRPC are mapped to `LightWalletServiceError`. Handling of `GRPCStatus` in the SDK is no longer required.
- `Service` directory (that contains GRPC code) is moved to `Modules/Service`. We can put more code to `Modules/` if we decide
to modularize more code.
2023-02-03 22:25:54 +01:00
Francisco Gindre fad3aae174
- [#679] Implementation of the File-system based block cache (#679)
Closes https://github.com/zcash/ZcashLightClientKit/issues/697
Closes https://github.com/zcash/ZcashLightClientKit/issues/720
Closes https://github.com/zcash/ZcashLightClientKit/issues/587
Closes https://github.com/zcash/ZcashLightClientKit/issues/667
Closes https://github.com/zcash/ZcashLightClientKit/issues/443
Closes https://github.com/zcash/ZcashLightClientKit/issues/754

- [#790] Fix ShieldFundsTests

Closes #790
Removes comments on `ShieldFundsTests` since those issues have been fixed

Depends on zcash-light-client-ffi changes that adopt newer versions of
 librustzcash crates `zcash_primitives 0.10`, `zcash_client_backend 0.7`,
`zcash_proofs 0.10`, `zcash_client_sqlite 0.5.0`.

Also allows wallets to define a shielding_threshold and will set
foundations to customize minimum confirmations for balances, spends
and shielding operations.

**Test Bootstrapping**

- `ZcashCompactBlockDescriptor`: struct that holds functions to
describe blocks as filenames and compare those filenames

`ZcashCompactBlockDescriptor.live` has the actual implementation
but it can be replaced by mocks if needed on Tests

main implementations are held under `FSCompactBlockRepository.filenameDescription` and `FSCompactBlockRepository.filenameComparison` on a separate extention

`DirectoryListingProviders` provide two default implementations of listing a
directory deterministically. `FileManager` does not define a sorting and needs to be done in-memory by calling `.sorted()` on the resulting collection. If this
is a big toll on performance it can be changed to a POSIX implementation but
this is good for now.

`ZcashCompactBlockDescriptor` adds a `height` helper function to
turn a filename into the height of the block stored.

Implemented `func latestHeight() throws -> BlockHeight ` that
returns the blockheight by querying the cache directory in a sorted
fashion and getting the last value and turning the filename into a
`BlockHeight`

Added `Meta` struct to ZcashCompactBlock.

Tests implemented:
- `filterBlockFiles`
- `testClearTheCache`
- `testLatestHeightEmptyCacheThrows`
- `testLatestHeightEmptyCacheThrowsAsync`
- `testRewindEmptyCacheDoesNothing`
- `testRewindEmptyCacheDoesNothingAsync`
- `testWhenBlockIsStoredItFollowsTheDescribedFormat`
- `testWhenBlockIsStoredItFollowsTheFilenameConvention`
- `testGetLatestHeight`
- `testRewindDeletesTheRightBlocks` test
- `testPerformanceExample` test. This isn't a real performance test because the API doesn't work with async/await yet

adopts `shield_funds` shielding threshold parameter

Implements `initBlockMetadataDb` and fix tests
Renames dbCache parameter to `fsBlockDbRoot`. Builds but tests don't pass.
Removes cacheDb uses from code. Testing utilities still persist.
Added needed information in  MIGRATING and CHANGELOG.
Added helper to perform deletion of legacy db and creation a the
new file system backed cache.

Renames parameters and changes code where needed.
Network Constants turned into `enum` with static methods.
DeletelastDownloadedBlock helper from initializer
Removes CompactBlockStorage and CompactBlockEntity.
Implements `latestCachedBlockHeight` on rustbackend.

*Replaces dependencies on ZcashRustWelding with `FSMetadataStore`*

This allows the tests to not depend in a particular implementation
of either the MockRustBackend of or ZcashRustBackend. Also provides
a way to test errors properly and switch implementations of critical
areas like `writeBlocks`.
2023-02-02 13:58:12 -03:00
Michal Fousek 6bbaad4cb1 [#740] Refator blocks downloading in CompactBlockProcessor
Closes #740

`CompactBlockDownloading` protocol is renamed to `BlockDownloaderService`. `CompactBlockDownloader` is renamed to `BlockDownloaderServiceImpl`.

What is the motivation for this rename?
Goal of this change is to extract code from `CompactBlockProcessor` which is related to downloading. So naturaly it should be called "downloader".
But `CompactBlockDownloader` was already taken. After analysis of `CompactBlockDownloading` and `CompactBlockDownloader` I found out that
`CompactBlockDownloader` doesn't download blocks exactly. It's more like abstraction over `LightWalletService`. And it does very similar job to
`LightWalletService`. That is why I decided to rename it to "service".

What is the motivation for protoco/impl naming scheme?
First `BlockDownloaderService` protocol doesn't describe capability in my eyes. It describe what something is. Also `BlockDownloaderServicing` sounds
strange. I agree that add suffix "Impl" to class name isn't super nice. But it is acceptable. And it makes sense to use nicer type name
(e.g. `BlockDownloaderService`) for protocol because it's used on much more places in the code.

- Added `BlockDownloader` protocol and `BlockDownloaderImpl` class. The code related to downloading was exctracted to this class.
2023-01-31 15:24:41 +01:00
Michal Fousek ce4d7bab45 [#556] Rename Transaction to ZCashTransaction
- Xcode's refactor feature used. Only renamed happened no logic was
  changed.
2023-01-23 10:01:07 +01:00
Michal Fousek 28de8d4ac3 Merge branch 'main' into 556_using_transactions_db_views 2023-01-19 12:44:28 +01:00
Lukas Korba 372ea5af70
[#711] Add Swiftlint plug in to SDK (#722)
- plugin integrated
- errors resolved
- warnings resolved
- package.swift macos platform version 10_15 -> 12
2023-01-18 17:09:04 +01:00
Lukas Korba dcc6ef1757
[#671] Make CompactBlockProcessor Internal (#710)
- sdk and demo app modified
- changelog updated
2023-01-12 13:05:11 +01:00
Michal Fousek cb62101e2d [#683] Add Synchronizer API to fetch memos for transactions
Closes #683

Fix tests
2023-01-10 11:03:57 +01:00
Michal Fousek 169fd3da46 Merge branch 'main' into 556_using_transactions_db_views 2023-01-10 11:02:12 +01:00
Michal Fousek 382905b06a [#657] Fix typos 2023-01-09 13:29:43 +01:00
Michal Fousek c4df36db36 [#556] Rename Transaction to TransactionNG
This change is really only replace one string with other. No logic was
changed.
2023-01-06 10:12:00 +01:00
Michal Fousek ef88023171 [#556] Remove ConfirmedTransaction and ConfirmedTransactionEntity 2023-01-05 09:42:50 +01:00
Michal Fousek f12ace4bf3 [#556] Replace TransactionRepository.findAll methods with new methods
Fix compilation
2023-01-03 13:12:07 +01:00
Michal Fousek a40b311ccd [#556] Replace TransactionRepository.findAllSentTransactions with new method
Make minedHeight optional
2023-01-02 12:29:01 +01:00
Michal Fousek c3b13d8367 [#657] Fix start of SDKSynchronizer when sync is already in process
Previously `SDKSynchronizer` simply refused to start while the sync
process was in progress. That could lead to some troubles.
`CompactBlockProcessor` has already mechanism which can handle this
exact situation. So now `SDKSynchronizer` starts `CompactBlockProcessor`
even when there is sync in progress.
2023-01-02 10:24:02 +01:00
Michal Fousek 526e8e87b8
Merge pull request #688 from Chlup/556_2_replace_findTransactions
[#556] Replace TransactionRepository.findTransactions methods with new methods
2022-12-23 22:06:22 +01:00
Michal Fousek 4b4416b1e1 Merge branch 'master' into 657_merge_master
- Changed how metrics are pushed when downloading and scanning blocks.
2022-12-22 15:35:11 +01:00
Michal Fousek 27188e7e80 [#556] Replace TransactionRepository.findTransactions methods with new methods 2022-12-22 10:35:11 +01:00
Michal Fousek 92256c108f [#556] Replace TransactionRepository.findAllReceivedTransactions with new method 2022-12-22 09:46:54 +01:00
Michal Fousek c7b0e0b5b6 Merge branch 'master' into 556_using_transactions_db_views 2022-12-22 09:41:21 +01:00
Lukas Korba 3c3b97ca42
[#663] Create a benchmarking section on the Demo App that can be automated (#681)
- SDKMetrics singleton for the measurement
- download, validate and scan blocks supported
- Sample app refactored to use SDKMetrics, connected to appropriate notifications and labels
- enhancement metric added
- total sync time metric added
- sample app extended to show summary time and operations summaries
- unit tests for the SDKMetrics
- doc for the SDKMetrics and our approach for the performance testing in general
- changelog update
2022-12-21 13:30:05 +01:00
Michal Fousek 6c5bf8de27 [#677] Add support for wipe into SDK
Closes #677

- Previously when keychain wanted to do wipe wallet had to do it on it's
  own. Which isn't good. Now wallet can simply call
  `synchronizer.wipe()` and it's done.
- This change brings support for wipe. `Synchronizer` has new method
  `wipe()` which takes care of everything. All the database connections
  are closed. Databases are removed. `InternalSyncProgress` is reset.

Change condition when wipe can run
2022-12-20 13:41:20 +01:00
Michal Fousek 349148cd38 [#657] Change how blocks are downloaded and processed
Closes #657

- Now SDK downloads N downloads and scan those. And repeat until sync is
  done. This safes lot of space of disk used to store downloaded blocks.
- Most changes are done in `CompactBlockProcessor.processNewBlocks`.
- `SyncRanges` structure is changed a bit.
- `SyncStatus` is changed. SDK now report that it's syncing with
  progress. It doesn't report each phase of the sync process separately.
  Also appropriate notifications were updated.

Address review comments

- Small changes
- Add tests to check computation of processing ranges for single loop
2022-12-19 09:35:09 +01:00
Michal Fousek cb07484c45 [#556] Replace TransactionRepository.findBy methods with new transaction method 2022-12-19 09:28:28 +01:00
Michal Fousek 7c07b91c5b [#651] Fix rewind when it's called during sync
- When rewind is called during sync then exception is thrown.
2022-11-29 08:33:18 +01:00
Lukas Korba 6029a75038
[#637] Make sapling parameter download part of processing blocks (#650)
- sync process enhanced to try to download sapling params if needed
- tests fixed
- failing import Crypto -> import CryptoKit
- download params only if there are sapling funds
2022-11-29 07:38:28 +01:00
Michal Fousek 1f2ea5e716 Change lastState property to Publisher from Subject in SDKSynchronizer
It's not good practice to make internal combine subjects accessible from
outside of SDK. So `lastState` is changed from `CurrentValueSubject` to
`Publisher`. This doesn't have any effect on public API. It just makes
SDK safer.
2022-11-22 14:10:27 +01:00
Michal Fousek 44a6970b73 [#616] Dont send notification from SDK on main thread
Closes #616

- Added small helper `NotificationSender` which is used to send
  notifications.
- No notificaiton is sent on main thread.
- Change how `status` is stored and update in `SDKSynchronizer`. It is
  now protected by lock. Before this wasn't needed because status was
  updated from main thread everywhere.
- `SyncBlocksViewController` in the sample app now uses
  `SDKSynchronizer` instead of `CompactBlockProcessor` directly.
2022-11-21 16:33:42 +01:00
Michal Fousek 4e5cb505fa [#488] Delete cache db when sync ends
Closes #488

- When sync process is finished cache DB is removed from disk.
- Latest downloaded block is preserved becuase it's required to computes
  in which phase to resume sync process.
- `SDKSynchronizer.latestDownloadedHeight` public API is removed
  it makes no longer sense. This API was reading from cache DB.
2022-11-15 10:31:59 +01:00
Francisco Gindre 6208beec5c [#621] Adds a synchronizer state subject with the last available
state for this synchronizer.

also reports the SychronizerState on synchonizerStarted notification.

Add tests that verify the state being reported properly

Fix `BlockBatchValidationTests`
2022-11-14 14:44:33 -03:00
Francisco Gindre b9e00055e9 [#614] make prepare(with:) and get{pool_type}Address() synchronous
Closes #614

fixes travis ci

Removed `setStartHeight` function

I’m going back in time and this setHeight is not something we are using anywhere else than in the prepare function. This comes from the times we had `initialize(seed:birthday) and we don’t have that anymore

See diff 246d10edaa (diff-414771774e10bc81260094ffcdc7b310a3be48758b2abd2bfae831c83912c02c)

The `func setStartHeight(_ startHeight: BlockHeight)` function assumes that
there might not be a wallet birthday set up or that it might be changed
in-flight which are things that are no longer happening.

remove test warning

Fix test compiler error
2022-11-11 14:07:45 -03:00
Francisco Gindre 01d85f5748 Fix compiler errors 2022-10-31 09:57:10 -03:00
Francisco Gindre dbd4b8934c Merge branch 'master' into merge-master 2022-10-31 09:34:38 -03:00
Francisco Gindre b5d659e8a8 Fix various tests and deleted some that are not useful anymore
Closes #588
Closes #584
Closes #589
Closes #591
2022-10-27 20:09:08 -03:00
Lukas Korba cfe71d5da2
[#523] Make a CompactBlockProcessor an Actor (#565)
- Sample app refactored for the processor being an actor
- tests refactored as well
- dark side tests fixed
- utilities separated to new file
- synchronizer's start and stop are no longer in async context
- updating the UI for the scan fixed
2022-10-27 12:51:38 +02:00
Francisco Gindre 5de1b50b29 make Memo and MemoBytes parameters nullable so they can be omitted when sending to transparent receivers.
update libzcashlc
2022-10-24 22:49:04 -03:00
Kris Nuttycombe 36932a21dd Use PendingTransactionEntity.internalAccount for shielding. 2022-10-18 10:17:07 -06:00
Kris Nuttycombe f5d7aa0f17 Modify PendingTransactionEntity to be able to represent internal shielding tx. 2022-10-18 10:07:23 -06:00
Francisco Gindre 7f480e417d Merge branch 'master' into merge-master 2022-10-03 20:15:40 -03:00
Lukas Korba 9b6ff51b29
[#492] Get rid of blocking API (#551)
- blocking API removed, only latestBlockHeight() stayed
- non-blocking closure based API removed
- unit tests updated to use async API

[#492] Get rid of blocking API (#551)

- forgotten commented code cleaned up
- some comments were still mentioning result (completion closure), removed
2022-10-03 20:05:11 -03:00
Francisco Gindre 7806b5114f
Use UnifiedSpendingKey for shielding and Spending (#535)
[#534] Use UnifiedSpendingKey for shielding and Spending

This commit implements the use of Unified Spending Keys for shielding
and spending as well as rolling Unified Addresses.

Users should obtain addresses by rolling them from the SDK.
USKs replace Sapling Extended Spending keys and TransparentAccountPrivKeys
when shielding or spending

Closes #534

Co-authored-by: Kris Nuttycombe <kris@nutty.land>

* Fix rebase issues

* PR Suggestion. Make `lastError` an Optional

* Fix test `testReOrgRemovesOutboundTxAndIsNeverMined`

Co-authored-by: Kris Nuttycombe <kris@nutty.land>
2022-10-02 23:11:17 -03:00
Francisco Gindre ca5221ac75 Merge branch 'master' into merge-master-to-zip-316 2022-09-30 09:53:34 -03:00
Francisco Gindre 28985a4e38
Merge pull request #536 from LukasKorba/486_sendToAddress_async_await
[#486] sendToAddress async/await
2022-09-23 12:08:31 -07:00
Lukas Korba 2c93802828 [#487] shieldFunds to async/await
- API refactored to async

[#487] shieldFunds to async/await

- unit tests refactored
2022-09-21 16:36:41 +02:00
Lukas Korba 61683a85f3 [486] sendToAddress async/await
- sendToAddress and SaplingParameterDownloader refactored to async
- unit tests WIP

[486] sendToAddress async/await

- testing code cleanup

[#486] sendToAddress async/await

- reducing number of Tasks in Sample app

[#486] sendToAddress async/await

- I finally figure out how to make the dark side tests async, using new sendToAddress APIs and pass, POC done, 18 more tests to refactor

[#486] sendToAddress async/await

- async sendToAddress unit test refactored except NetworkUpgradeTests

[#486] sendToAddress async/await

- NetworkUpgradeTests refactored to async API

[#486] sendToAddress async/await

- cleanup

[#486] sendToAddress async/await (#536)

- PR comments resolved
2022-09-21 13:47:17 +02:00