Commit Graph

66 Commits

Author SHA1 Message Date
Michal Fousek 09bc1332eb Possibly fix testStoringTenSandblastedBlocksFailsAndThrows test 2023-02-06 13:28:19 +01:00
Michal Fousek 385c0a7195 [#746] Modularize GRPC related code
Closes #746

- `LightWalletGRPCService` is no longer public. `LightWalletService` is no longer public.
- `LightWalletGRPCService` shouldn't be used dicrectly. Use `LightWalletServiceFactory` to create instance of the service.
- Moved sending of `blockProcessorConnectivityStateChanged` notification to `Initilizer`.
- All the errors from GRPC are mapped to `LightWalletServiceError`. Handling of `GRPCStatus` in the SDK is no longer required.
- `Service` directory (that contains GRPC code) is moved to `Modules/Service`. We can put more code to `Modules/` if we decide
to modularize more code.
2023-02-03 22:25:54 +01:00
Francisco Gindre fad3aae174
- [#679] Implementation of the File-system based block cache (#679)
Closes https://github.com/zcash/ZcashLightClientKit/issues/697
Closes https://github.com/zcash/ZcashLightClientKit/issues/720
Closes https://github.com/zcash/ZcashLightClientKit/issues/587
Closes https://github.com/zcash/ZcashLightClientKit/issues/667
Closes https://github.com/zcash/ZcashLightClientKit/issues/443
Closes https://github.com/zcash/ZcashLightClientKit/issues/754

- [#790] Fix ShieldFundsTests

Closes #790
Removes comments on `ShieldFundsTests` since those issues have been fixed

Depends on zcash-light-client-ffi changes that adopt newer versions of
 librustzcash crates `zcash_primitives 0.10`, `zcash_client_backend 0.7`,
`zcash_proofs 0.10`, `zcash_client_sqlite 0.5.0`.

Also allows wallets to define a shielding_threshold and will set
foundations to customize minimum confirmations for balances, spends
and shielding operations.

**Test Bootstrapping**

- `ZcashCompactBlockDescriptor`: struct that holds functions to
describe blocks as filenames and compare those filenames

`ZcashCompactBlockDescriptor.live` has the actual implementation
but it can be replaced by mocks if needed on Tests

main implementations are held under `FSCompactBlockRepository.filenameDescription` and `FSCompactBlockRepository.filenameComparison` on a separate extention

`DirectoryListingProviders` provide two default implementations of listing a
directory deterministically. `FileManager` does not define a sorting and needs to be done in-memory by calling `.sorted()` on the resulting collection. If this
is a big toll on performance it can be changed to a POSIX implementation but
this is good for now.

`ZcashCompactBlockDescriptor` adds a `height` helper function to
turn a filename into the height of the block stored.

Implemented `func latestHeight() throws -> BlockHeight ` that
returns the blockheight by querying the cache directory in a sorted
fashion and getting the last value and turning the filename into a
`BlockHeight`

Added `Meta` struct to ZcashCompactBlock.

Tests implemented:
- `filterBlockFiles`
- `testClearTheCache`
- `testLatestHeightEmptyCacheThrows`
- `testLatestHeightEmptyCacheThrowsAsync`
- `testRewindEmptyCacheDoesNothing`
- `testRewindEmptyCacheDoesNothingAsync`
- `testWhenBlockIsStoredItFollowsTheDescribedFormat`
- `testWhenBlockIsStoredItFollowsTheFilenameConvention`
- `testGetLatestHeight`
- `testRewindDeletesTheRightBlocks` test
- `testPerformanceExample` test. This isn't a real performance test because the API doesn't work with async/await yet

adopts `shield_funds` shielding threshold parameter

Implements `initBlockMetadataDb` and fix tests
Renames dbCache parameter to `fsBlockDbRoot`. Builds but tests don't pass.
Removes cacheDb uses from code. Testing utilities still persist.
Added needed information in  MIGRATING and CHANGELOG.
Added helper to perform deletion of legacy db and creation a the
new file system backed cache.

Renames parameters and changes code where needed.
Network Constants turned into `enum` with static methods.
DeletelastDownloadedBlock helper from initializer
Removes CompactBlockStorage and CompactBlockEntity.
Implements `latestCachedBlockHeight` on rustbackend.

*Replaces dependencies on ZcashRustWelding with `FSMetadataStore`*

This allows the tests to not depend in a particular implementation
of either the MockRustBackend of or ZcashRustBackend. Also provides
a way to test errors properly and switch implementations of critical
areas like `writeBlocks`.
2023-02-02 13:58:12 -03:00
Michal Fousek 6bbaad4cb1 [#740] Refator blocks downloading in CompactBlockProcessor
Closes #740

`CompactBlockDownloading` protocol is renamed to `BlockDownloaderService`. `CompactBlockDownloader` is renamed to `BlockDownloaderServiceImpl`.

What is the motivation for this rename?
Goal of this change is to extract code from `CompactBlockProcessor` which is related to downloading. So naturaly it should be called "downloader".
But `CompactBlockDownloader` was already taken. After analysis of `CompactBlockDownloading` and `CompactBlockDownloader` I found out that
`CompactBlockDownloader` doesn't download blocks exactly. It's more like abstraction over `LightWalletService`. And it does very similar job to
`LightWalletService`. That is why I decided to rename it to "service".

What is the motivation for protoco/impl naming scheme?
First `BlockDownloaderService` protocol doesn't describe capability in my eyes. It describe what something is. Also `BlockDownloaderServicing` sounds
strange. I agree that add suffix "Impl" to class name isn't super nice. But it is acceptable. And it makes sense to use nicer type name
(e.g. `BlockDownloaderService`) for protocol because it's used on much more places in the code.

- Added `BlockDownloader` protocol and `BlockDownloaderImpl` class. The code related to downloading was exctracted to this class.
2023-01-31 15:24:41 +01:00
Michal Fousek ce4d7bab45 [#556] Rename Transaction to ZCashTransaction
- Xcode's refactor feature used. Only renamed happened no logic was
  changed.
2023-01-23 10:01:07 +01:00
Michal Fousek 28de8d4ac3 Merge branch 'main' into 556_using_transactions_db_views 2023-01-19 12:44:28 +01:00
Lukas Korba 372ea5af70
[#711] Add Swiftlint plug in to SDK (#722)
- plugin integrated
- errors resolved
- warnings resolved
- package.swift macos platform version 10_15 -> 12
2023-01-18 17:09:04 +01:00
Michal Fousek b4e320b9c3 [#556] Fix sorting when selecting transactions without minedHeight
- `id` property of transcations is no longer Optional. `id` column in DB
  is primary key and it can't be nil. This was oversight in previous
  changes.
- Sorting order of transactions is switched from asc to desc. asc is
  oversight in previous changes. Android is using desc.
- Using `IFNULL` when sorting transactions by minedHeight.
- Update tests accordingly. And revert few changes on darkside tests.
  These changes are no longer needed because using `desc` sorting order
  on transactions.
2023-01-16 13:18:08 +01:00
Michal Fousek 14f1436987 [#556] Fix tests and fix few bugs
- Fix all the tests. In few tests code had to be changes because order
  of transactions is newly defined. It wasn't before.
- Change properties of Transaction.* objects. Some had to be made
  Optional to make all tests work.
- Add new tests for TransactionDAO.
- Removed methods from TransactionDAO that weren't used.
2023-01-10 23:20:46 +01:00
Michal Fousek cb62101e2d [#683] Add Synchronizer API to fetch memos for transactions
Closes #683

Fix tests
2023-01-10 11:03:57 +01:00
Michal Fousek 169fd3da46 Merge branch 'main' into 556_using_transactions_db_views 2023-01-10 11:02:12 +01:00
Michal Fousek c4df36db36 [#556] Rename Transaction to TransactionNG
This change is really only replace one string with other. No logic was
changed.
2023-01-06 10:12:00 +01:00
Michal Fousek becadf21c8 Merge branch 'main' into 657_download_change 2023-01-05 09:50:00 +01:00
Dimitris Apostolou 6c3423181d
Fix typos 2022-12-31 12:50:16 +02:00
Michal Fousek ef2d904260 [#556] Update API of TransactionRepository and make tests work
This change is prerequisity for future changes. API of
`TransactionRepository` is changed a bit. Now it's possible to filter
transaction by kind (all/sent/received) event when
`TransactionNG.Overview` structure is returned.

Tests are now fixed for new methods and new objects. For this mocked
database `darkside_data.db` had to be updated to the latest scheme used
by the Rust code.
2022-12-21 12:33:20 +01:00
Michal Fousek 349148cd38 [#657] Change how blocks are downloaded and processed
Closes #657

- Now SDK downloads N downloads and scan those. And repeat until sync is
  done. This safes lot of space of disk used to store downloaded blocks.
- Most changes are done in `CompactBlockProcessor.processNewBlocks`.
- `SyncRanges` structure is changed a bit.
- `SyncStatus` is changed. SDK now report that it's syncing with
  progress. It doesn't report each phase of the sync process separately.
  Also appropriate notifications were updated.

Address review comments

- Small changes
- Add tests to check computation of processing ranges for single loop
2022-12-19 09:35:09 +01:00
Michal Fousek cb07484c45 [#556] Replace TransactionRepository.findBy methods with new transaction method 2022-12-19 09:28:28 +01:00
Michal Fousek e01c83690f [#646] SDK sync process resumes correctly
- Previously we had one range for each sync which was used for each
  phase of sync process. Newly there is separate range for each phase of
  the sync process.
- I added `InternalSyncProgress` utility. This utility tracks progress
  of some phases. And it is able to compute right ranges which should be
  used for syncing.
- Some unused download code from `CompactBlockProcessor` is removed.

Fix tests

Address review comments

- Rebase to master
- Update how range for `processBatchFinished()` is computed.
- Refactor `InternalSyncProgress`
- Add tests for `InternalSyncProgress`

Address review comments

Change how latest downloaded block is tracked

- Cache DB is no longer used to track which block was downloaded as
  latest. `InternalSyncProgress` is used to do that.
- Thanks to that #660 is fixed. And cache DB can be completely removed
  after sync process.
- Added sleep(1) to some darkside tests when latest block height is set.
  Otherwise lightwalletd in darkside mode doesn't have time to setup and
  tests are flaky.

Fix TransactionEnhancementTests.testBasicEnhancement test
2022-12-12 14:27:30 +01:00
Michal Fousek a7fe75c11e
Cleanup warnings (#655)
- TestCoordinator doesn't need to be used in async manner.
2022-11-29 07:40:45 +01:00
Lukas Korba 6029a75038
[#637] Make sapling parameter download part of processing blocks (#650)
- sync process enhanced to try to download sapling params if needed
- tests fixed
- failing import Crypto -> import CryptoKit
- download params only if there are sapling funds
2022-11-29 07:38:28 +01:00
Francisco Gindre b350f6b220
[#639] Provide an API to estimate TextMemo length limit correctly (#640)
This provides a method for clients to handle UI for text memo
limit correctly handling memos that are visibly fit within the
capacity defined in ZIP-302 but when encoded into UTF-8 bytes
their size exceeds such limit.

Closes #639
2022-11-22 14:02:39 -03:00
Francisco Gindre 6208beec5c [#621] Adds a synchronizer state subject with the last available
state for this synchronizer.

also reports the SychronizerState on synchonizerStarted notification.

Add tests that verify the state being reported properly

Fix `BlockBatchValidationTests`
2022-11-14 14:44:33 -03:00
Francisco Gindre b9e00055e9 [#614] make prepare(with:) and get{pool_type}Address() synchronous
Closes #614

fixes travis ci

Removed `setStartHeight` function

I’m going back in time and this setHeight is not something we are using anywhere else than in the prepare function. This comes from the times we had `initialize(seed:birthday) and we don’t have that anymore

See diff 246d10edaa (diff-414771774e10bc81260094ffcdc7b310a3be48758b2abd2bfae831c83912c02c)

The `func setStartHeight(_ startHeight: BlockHeight)` function assumes that
there might not be a wallet birthday set up or that it might be changed
in-flight which are things that are no longer happening.

remove test warning

Fix test compiler error
2022-11-11 14:07:45 -03:00
Francisco Gindre 59dc56c9b4 Merge branch 'feature/zip-316-and-latest-upstream' into release/0.17.0-alpha.3 2022-11-01 15:09:39 -03:00
Francisco Gindre a12666e932 [#602] Improve error logging for InitializerError and RustWeldingError 2022-11-01 13:57:42 -03:00
Francisco Gindre 01d85f5748 Fix compiler errors 2022-10-31 09:57:10 -03:00
Francisco Gindre dbd4b8934c Merge branch 'master' into merge-master 2022-10-31 09:34:38 -03:00
Francisco Gindre b5d659e8a8 Fix various tests and deleted some that are not useful anymore
Closes #588
Closes #584
Closes #589
Closes #591
2022-10-27 20:09:08 -03:00
Lukas Korba cfe71d5da2
[#523] Make a CompactBlockProcessor an Actor (#565)
- Sample app refactored for the processor being an actor
- tests refactored as well
- dark side tests fixed
- utilities separated to new file
- synchronizer's start and stop are no longer in async context
- updating the UI for the scan fixed
2022-10-27 12:51:38 +02:00
Francisco Gindre 37b060852c
Merge pull request #567 from nuttycom/pending_transaction_entity_recipient
Add internal recipient information to `PendingTransactionEntity`
2022-10-21 18:27:52 -03:00
Francisco Gindre 2fb25dcd35 Fix: Transaction repository tests pass 2022-10-21 18:19:08 -03:00
Francisco Gindre 1979e410b5 Fix pre populated Db to have transactions from darksidewalletd seed 2022-10-21 13:54:53 -03:00
Kris Nuttycombe a483537cb1 Ensure that the persisted test database has had migrations applied. 2022-10-20 14:22:24 -06:00
Francisco Gindre 789cf01188 Add Fee field to Transaction, ConfirmedTransaction, ReceivedTransactions and Pen
dingTransactions.
Update Notes DAOs with new fields.

Make Zatoshi Codable

add tests for Recipient.forEncodedAddress
2022-10-19 17:37:06 -03:00
Kris Nuttycombe 849083ffd9 Fix UInt32 conversions to SQL in PendingTransactionDao 2022-10-19 14:13:18 -06:00
Francisco Gindre 30a9c066ad Replace `db.run` with `db.execute` to fix migration issues 2022-10-19 11:29:15 -03:00
Kris Nuttycombe 0fbf90dc82 Add migration to re-create pending_transactions table with nullable columns. 2022-10-18 19:44:34 -06:00
Kris Nuttycombe f5d7aa0f17 Modify PendingTransactionEntity to be able to represent internal shielding tx. 2022-10-18 10:07:23 -06:00
Francisco Gindre 923eca19ce [#561] Fix unified typecodes tests
Fix tests.

Fix tests.

Use Account 6 vectors on testListTransparentReceivers

only use first tree addresses from the vector for testListTransparentReceivers

Fix unified address generation tests.

Fix build and PR suggestions
2022-10-18 11:40:36 -03:00
Kris Nuttycombe 91ea6f5476 Update to latest librustzcash 2022-10-17 15:00:08 -06:00
Francisco Gindre 8a26e2e469 UFVK validation was validating sapling keys 2022-10-03 21:12:10 -03:00
Francisco Gindre 7f480e417d Merge branch 'master' into merge-master 2022-10-03 20:15:40 -03:00
Lukas Korba 9b6ff51b29
[#492] Get rid of blocking API (#551)
- blocking API removed, only latestBlockHeight() stayed
- non-blocking closure based API removed
- unit tests updated to use async API

[#492] Get rid of blocking API (#551)

- forgotten commented code cleaned up
- some comments were still mentioning result (completion closure), removed
2022-10-03 20:05:11 -03:00
Francisco Gindre 7806b5114f
Use UnifiedSpendingKey for shielding and Spending (#535)
[#534] Use UnifiedSpendingKey for shielding and Spending

This commit implements the use of Unified Spending Keys for shielding
and spending as well as rolling Unified Addresses.

Users should obtain addresses by rolling them from the SDK.
USKs replace Sapling Extended Spending keys and TransparentAccountPrivKeys
when shielding or spending

Closes #534

Co-authored-by: Kris Nuttycombe <kris@nutty.land>

* Fix rebase issues

* PR Suggestion. Make `lastError` an Optional

* Fix test `testReOrgRemovesOutboundTxAndIsNeverMined`

Co-authored-by: Kris Nuttycombe <kris@nutty.land>
2022-10-02 23:11:17 -03:00
Francisco Gindre ca5221ac75 Merge branch 'master' into merge-master-to-zip-316 2022-09-30 09:53:34 -03:00
Francisco Gindre 38b8fd09d8
Merge pull request #531 from zcash/receiver_type_codes
Receiver type codes
2022-09-20 10:27:54 -07:00
Francisco Gindre a9805cfe7e [#530] Implement ability to extract available typecodes from UA
this introduces several types under UnifiedAddress `ReceiverTypecodes`
and `UnifiedAddress.Errors` to host possible errors on third-party
generated UAs and their composition

Implement extracting the typecodes from a UA
2022-09-19 14:52:30 -03:00
Francisco Gindre bee1c9db80 Merge branch 'master' into merge_master_to_zip_316 2022-09-15 11:04:49 -03:00
Lukas Korba 2b65bd46e4 [476] CompactBlockProcessor to async/await
- getting rid of the Operation Queue
- the cleanup is needed
- the update of tests is needed
- tested and it successfully finishes the sync process

[476] CompactBlockProcessor to async/await

- old processNewBlocks() removed

[476] CompactBlockProcessor to async/await

- unused operations removed

[476] CompactBlockProcessor to async/await

- unit tests update

[476] CompactBlockProcessor to async/await

- unit tests refactored

[476] CompactBlockProcessor to async/await

- cleanup of deprecated method

[476] CompactBlockProcessor to async/await

- fail(error) was called even for canceled tasks but that must be excluded

[476] CompactBlockProcessor to async/await

- removal of all ZcashOperations from the code (unit test will follow)

[476] CompactBlockProcessor to async/await

- network tests in building and success order again

[476] CompactBlockProcessor to async/await

- offline tests in building and success order

[476] CompactBlockProcessor to async/await (519)

- cleanup of suspending the task

[476] CompactBlockProcessor to async/await (519)

- most comments resolved

[476] CompactBlockProcessor to async/await (519)

- thread safe state for both sync and async context

[476] CompactBlockProcessor to async/await (519)

- fixed build for a sample project

[476] CompactBlockProcessor to async/await (519)

- func testStartNotifiesSuscriptors() reverted

[476] CompactBlockProcessor to async/await (519)

- TODO added to track why we used NSLock instead of an Actor
- Task priority enhanced

[476] CompactBlockProcessor to async/await (519)

- cleanup in Tasks and priorities
2022-09-15 08:10:12 +02:00
Francisco Gindre 4428f9ffb2 Merge branch 'feature/zip-316-and-latest-upstream' into merge_master_to_zip_316 2022-09-12 13:40:05 -03:00