Commit Graph

153 Commits

Author SHA1 Message Date
Lukas Korba e2956a190e Sync progress merged
- the progress is merged now but boolean value whether funds are spendable or not has been added extra
2025-04-11 14:36:34 +02:00
Lukas Korba f56fde9cb9 Adoption of sync & recovery progresses
Sync and recovery progresses

- FFI bumped to preview 0.15.0
- SDK has been refactored to incorporate recoveryProgress alongside scanProgress

Non-optional syncProgress

0 Denominator fix

OfflineTests fixed
2025-04-10 14:31:38 +02:00
Lukas Korba da00a69b49 Pczt typealias and clean up
- The comments were provided
- Changelog updated
- Code cleaned up
- Tests fixed
- Error codes for the Pczt
2024-12-13 10:13:01 +01:00
Lukas Korba 5179f4e1b0 getAccount for UUID
- rust backend `getAccount` method implemented, it returns Account with associated data
- `listAccount()` has been modified to return an array of Accounts instead of AccountUUIDs
2024-12-06 11:43:20 +01:00
Lukas Korba bb03333458 Performance, Network and Dark side tests
- updated to build and tests that are obsolete are now taken out of the bundle
2024-12-05 12:42:24 +01:00
Lukas Korba 27af6f134c PR Comments addressed and code cleaned up
- Comments in the code updated and cleaned up
- OfflineTests passes again, those failing has been removed from the bundle and marked to be fixed with a TODO

Fixes of build

- The SDK builds again
2024-12-05 10:31:13 +01:00
Lukas Korba 35ddf87a1b [#1512] Ensure that the SDK does not assume a default account anywhere
- The SDK no longer assumes a default account. All business logic with instances of Zip32AccountIndex(<index>) has been refactored.
2024-12-03 13:01:44 +01:00
Lukas Korba 97137510b2 Zip32AccountIndex UInt32 refactor
- Zip32Account(Int32) has been refactored to Zip32AccountIndex(UIn32)
2024-11-25 11:32:55 +01:00
Lukas Korba a0535a6a82 [#1511] Refactor account representation from Int to a dedicated Account structure
- First iteration of all Int account references to an instance of an Account
2024-11-19 11:11:36 +01:00
Kris Nuttycombe 187753fc13 Fix offline test build errors. 2024-10-10 22:47:05 -06:00
Jack Grigg bce8085690 Add APIs for fetching the ZEC-USD exchange rate over Tor. 2024-07-30 11:02:10 +01:00
Lukas Korba beb48dd0bd [#1462] Syncing is broken
- The CompactBlockProcessor's state machine got stuck in some cases at the updateChainTip action.

[#1462] Syncing is broken

- unit test fixed
2024-07-04 08:28:48 +02:00
Francisco Gindre b13b5bc179
Add TreeState fo 663150 2024-06-26 17:03:17 -03:00
Lukas Korba d258fc094b [#1431] Fix swiftlint reported issues
- all reported swiftlint issues have been addressed

[#1431] Fix swiftlint reported issues

- changelog updated

[#1431] Fix swiftlint reported issues (#1432)

- fixme fix
2024-05-30 13:16:46 +02:00
Lukas Korba e1c600b8ff Public API isSeedRelevantToWallet
- New public API `isSeedRelevantToWallet` has been implemented
- 2 swiftlint error have been resolved
- Mocks generated
2024-03-18 14:45:30 +01:00
Francisco Gindre c873e208c4
[#1379] Fulfill Payment from a valid ZIP-321 request
Closes #1379

Adds test for parsing error
Adds test for successful tx creation
Public API changes in CHANGELOG.md
2024-03-08 18:38:50 -03:00
Lukas Korba e51f425c54 [#1369] SynchronizerState refactor
- plural of account balances refactor to singular

[#1369] SynchronizerState refactor

- transaprent balance removed from SynchronizerState
- all balances are now handled with AccountBalance

[#1369] SynchronizerState refactor and balances cleanup

- WalletBalance has been removed from the SDK, replaced with AccountBalance
- public APIs for individual balances (getTransparent, getShielded, getShieldedVerfied) have been removed from the SDK, replaced with getAccountBalance
- tests refactored

[#1369] SynchronizerState refactor and balances cleanup

- dark side tests refactored

[#1369] SynchronizerState refactor and balances cleanup (#1372)

- addressed comments

[#1369] SynchronizerState refactor

- ZcashError for spending key does not belong to the wallet
2024-02-12 17:41:26 +01:00
Lukas Korba 9fab46a2d1 [#1363] Expose PoolBalance Zatoshi values
- new public API with returning AccountBalance

[#1363] Expose PoolBalance Zatoshi values

- accountBalances exposed into the SynchronizationState

[#1363] Expose PoolBalance Zatoshi values

- public vs. internal

[1363] Account balances in the SynchronizerState

- changelog update
- scanAction reports the initial after-scan value immediately instead of after X scans
- SynchronizerState has been refactored, we no longer call 2 rust methods to get shielded verified + total balances
 - tests refactored to reflect this change
- mocks regenerated
- Sourcery version bumped

[#1363] Account balances in the SynchronizerState

- changelog typo fixed

[#1363] Account balances in the SynchronizerState

- internal vs. public
2024-01-31 17:05:13 -08:00
Francisco Gindre 6625ffd711
Create a Dependency that can load checkpoints from bundle
Closes #1315

This PR introduces small changes on each commit.
Things done:

rename Checkpoint+Constants to Checkpoint+helpers

Move `Checkpoint` from Model folder to Checkpoint folder

Remove unused function `ofLatestCheckpoint` from BlockHeight

Create a protocol called `CheckpointSource` that contains the
relevant functionality to get checkpoints from Bundle

Create a set of tests that check that functionality is maintained
when a `CheckpointSource` is used instead of Checkpoint helpers

Implement `BundleCheckpointSource` and add Tests

Code clean up: move `BundleCheckpointURLProvider` to its own file

Code clean up: `Checkpoint+helpers` match file header

Replace use of `Checkpoint.birthday(with:network)` with CheckpointSource

Revert "Remove unused function `ofLatestCheckpoint` from BlockHeight"

addresses PR comment from @daira

This reverts commit d0e154ded7, since it
modifies a public API and it was not the goal of this PR.

Update Sources/ZcashLightClientKit/Checkpoint/BundleCheckpointSource.swift

Use a decent Date Format

Co-authored-by: Daira Emma Hopwood <daira@jacaranda.org>

Improve documentation on BundleCheckpointURLProvider

Co-authored-by: Daira Emma Hopwood <daira@jacaranda.org>

Improve documentation on BundleCheckpointURLProvider

Co-authored-by: Daira Emma Hopwood <daira@jacaranda.org>

use YYYY-mm-dd on file header

author: @daira

Co-authored-by: Daira Emma Hopwood <daira@jacaranda.org>

Add test that verifies that the exact height is returned if available
2023-11-08 15:45:14 -03:00
Lukas Korba 69e38e0667 [#1294] Remove all uses of the incorrect 1000-ZAT fee
- Removed deprecated zip-313 fee of 1000 Zatoshi
- default is now 10k zatoshi, the minimum defined by zip-317

[#1294] Remove all uses of the incorrect 1000-ZAT fee

- changelog update
2023-09-29 20:01:38 +02:00
Kris Nuttycombe 2c526dbcd6 Update `zcash-light-client-ffi` to adopt fix for missing transparent history.
Fixes #1271

Co-Authored-By: Jack Grigg <jack@electriccoin.co>
Co-Authored-By: Lukas Korba <lukas.korba@seznam.cz>
2023-09-19 08:26:03 -06:00
Lukas Korba 4b45c15a96 Comments addressed
- all comments from the review (round 1) resolved
2023-09-13 09:24:26 +02:00
Lukas Korba ec8aca2890 [#1249] Fix linter issues
- all issued fixed, one wasn't clear so new ticket is created to follow up and TODO added
2023-09-11 08:03:16 +02:00
Lukas Korba 2f521d7857 [#1242] Fix unit tests after SBS changes
- DarkSide, Network and Performance tests were modified to be buildable again
2023-09-10 08:43:33 +02:00
Jack Grigg e7621204e5 Update Rust dependencies with account birthdays and scan progress
The previous FFI repo revisions no longer exist; commits between
87faf91096 and here will not build.

Update Rust dependencies with account birthdays and scan progress

- fixes for SampleApp
2023-09-07 09:26:48 +02:00
Jack Grigg 4c74e4d204 Migrate to latest `darkside.proto` to fix Darkside tests 2023-08-18 09:37:41 -04:00
Lukas Korba 34af1f1611 [#1140] ClearCache action before anything starts
- draft

[#1140] ClearCache action before anything starts

- ClearCache action right after the idle action, clearing out metadata so the sync process can be fully restored from the DB and live blockchain values only.
- InternalSyncProgress removed
- InternalSyncProgressStorage removed
- Sync process control logic updated, controlled by latestScannedHeight and firstUnenhancedHeight only
- cleaned up unused code

[#1140] ClearCache action before anything starts

- ChecksBeforeSyncAction removed
- Offline tests fixed

[#1140] ClearCache action before anything starts

- fixed injection of a wallet birthday, the sync range must start with wallet BD instead of lower bound

[#1140] ClearCache action before anything starts

- Network tests fixed
- DarkSideTests partially fixed

[#1140] ClearCache action before anything starts

- rewind actions extension in compact block processor added

[#1140] ClearCache action before anything starts

- draft

[#1140] ClearCache action before anything starts

- DarkSideTests fixed

[#1140] ClearCache action before anything starts

- SyncRanges modified to be even less dependent on ranges, now it holds just 3 values (latest block height, latest scanned height if any, first unenhanced height if any), the rest is computed on the fly

[#1140] ClearCache action before anything starts

- SyncRanges struct not anymore, refactored to SyncControlData, holding just 3 mentioned values

[#1140] ClearCache action before anything starts

- cleanup

[#1140] ClearCache action before anything starts (#1148)

- TODO solved, the UTXOs fetcher doesn't work with range anymore, therefore reporting 100%
2023-06-28 12:34:41 +02:00
Michal Fousek 21a3c06b00 [#700] CompactBlockProcessor as state machine proof of concept
- For now I created `CompactBlockProcessorNG` where I started with
  implementation of the state machine. I did it to not break the rest of
  the SDK. This change will be merged to the feature branch. And before
  it is merged to `main` branch code from `CompactBlockProcessorNG` will
  be moved to `CompactBlockProcessor`.
- The new code is not used. It just shows and explains how it is done.
  It is proof of concept.
- I did put either commented current code or comment to some places to
  explain what should be done there.
- New important data types:
  - `ActionContext` is context that can hold any data that needs to be
    shared between actions. For example sync ranges or current state.
  - `CBPState` is state of the `CompactBlockProcessor`. Each state is
    handled by one action. This doesn't apply to terminal states like
    `finished` or `failed`.
  - `ActionProgress` is very similar to `CompactBlockProgress`.
    Different actions reports progress differently and `ActionProgress`
    represents this.
  - `Action` is protocol that defines API of an action. It has one run
    method that executes the code of the action
- CBP first creates actions for (almost) each state in `makeActions()`
  method. Then the "magic" is done in `CompactBlockProcessorNG.run()` method.
  Here is main loop which takes action for current state and execute it.
  It's expected that action does it's work and then updates the context
  with new state. And this happens until some terminal state
  (`finished`, `failed`, `stopped`) is reached.
- After the transition to state machine API of the
  `CompactBlockProcessor` should stay the same. No changes should be
  required in `SDKSynchronizer`.

[#700] Add documentation for CompactBlockProcessor state machine

- plantuml tool is used to generate diagram.

[#1054] Add config to state machine CBP

Closes #1054

[#1043] Implement DownloadAction

Closes #1043

[#1049] Implement ValidateAction

Closes #1049

[#1050] Implement ValidateServerAction

Closes #1050

[#1056] Add constructors to state machine CBP

Closes #1056

[#1061] Add failure methods for state machine CBP

Closes #1061

[#1055] Implement retry timer to state machine CBP

Closes #1055

[#1057] Implement start for state machine CBP

Closes #1057

[#1058] Implement stop for state machine CBP

Closes #1058

[#1052] Implement AfterSyncHooksManager when using state machine CBP

Closes #1052

[#1060] Implement wipe for state machine CBP

Closes #1060

[#1059] Implement rewind for state machine CBP

Closes #1059

[#700] Add idle state to CBP state machine

This is required so the CBP can detect start of the sync process.

[#700] Implement sending of some events from CompactBlockProcessorNG

[#700] Implement progress reporting in state machine CBP

[#1045] Implement FetchUTXOsAction

- draft of the fetching

[#1045] Implement FetchUTXOsAction

- updated the way Actions communicate data back to the CBP
- used this mechanism to pass result of utxos fetch so it's passed to the SDKSynchronizer as an Event

[#1042] Implement ComputeSyncRangesAction

Closes #1042

[#700] Implement cache clearing when some actions fail

[#1043] Fix batch range computation in DownloadAction

[#1046] Implement SaplingParamsAction

- action for sapling param files finished

[#1048] Implement ScanDownloadedButUnscannedAction

- scan downloaded but unscanned blocks

[#1047] Implement ScanAction

- scan action with the proper ranges computed

[#1047] Implement ScanAction (#1085)

- fixed logger message

[#1044] Implement EnhanceAction

Closes #1044

[#1041] Implement ClearCacheAction

Closes #1041

[#1040] Implement ClearAlreadyScannedBlocksAction

Closes #1040

[#1039] Implement ChecksBeforeSyncAction

Closes #1039

[#700] Make CBP state machine work

[#1050] Implement ValidateServerAction

- broken tests commented out and tracked in the tickets
- new test for ValidateServerAction

[#1051] Update how progress is computed after switch to state machine

Closes #1051

- new proposal for the progress computation
- OverallProgress value is passed to the Synchronizer as a Float
- OverallProgress is a result of fetch, scan and enhance operations
- Order of actions no longer influences the computation
- Actions report partial updates, CompactBlockProgress actor holds current state and computes the OverallProgress

[#1049] Implement ValidateAction

- synchronizer offline tests updated so it compiles, review is requested in a different ticket
- ValidateAction tests added
- BlockValidator mock generated

[#1047] Implement ScanAction

- ScanAction tests
- refactor of validateAction -> validateServerAction
- generated few more mocks for the DI

[#1045] Implement FetchUTXOsAction

- FetchUTXOsAction tests
- UTXOFetcher mocks

[#1045] Implement FetchUTXOsAction

- enhanced with mocked values and more checks

[#1046] Implement SaplingParamsAction

- SaplingParamsAction tests
- added TODO for TestCoordinator reset()
- SaplingParametersHandler mock added

[#1046] Implement SaplingParamsAction

- SaplingParamsAction tests
- added TODO for TestCoordinator reset()
- SaplingParametersHandler mock added

[#1046] Implement SaplingParamsAction

- rebased so I get functionality of improved mock checks
- enhanced SaplingParamsAction tests
- enhanced ValidateAction tests
- enhanced ScanAction tests

[#1046] Implement SaplingParamsAction

- scanAction tests more checks added

[#1044] Implement EnhanceAction

- EnhanceAction tests focused on 2 different methods:
- decideWhatToDoNext covered separately, decisions where the state machine goes next
- run tests for different cases
- new mocks generated for enhacer
- some typos fixed

[#1044] Implement EnhanceAction (#1107)

- empty assert messages fixed

[#700] Get rid of ScanDownloadedButUnscannedAction

Before the state machine download and scan was called in one loop. And
processing range for one batch was same for both of them. Therefore
there was code which scanned downloaded but not scanned blocks.

But now download and scan are independent. So it is possible to remove
`ScanDownloadedButUnscannedAction`.

[#700] Make NetworkTests compilable

Some tests are disabled for now (list is in #1115). And `NetworkTests`
can be compiled and all the enabled tests work.

[#1043] Implement DownloadAction

- DownloadAction tests
- BlockDownloader mock

[#1043] Implement DownloadAction (#1110)

- support functions set to private

[#1039] Implement ChecksBeforeSyncAction

- ChecksBeforeSyncAction tests
- all support functions in Action tests are set to private
- let _ = -> _ = refactor
- CompactBlockRepository mock added

[#1040] Implement ClearAlreadyScannedBlocksAction

- Implement ClearAlreadyScannedBlocksAction tests
- CompactBlockRepository mock added

[#1041] Implement ClearCacheAction

- ClearCacheAction tests
- CompactBlockRepository mock added

[#1042] Implement ComputeSyncRangesAction

- ComputeSyncRangesAction tests
- fixed all tests after merge of latest SDK changes related InternalSyncProgress
- all actions marked as final class

[#1042] Implement ComputeSyncRangesAction (#1120)

- Custom LatestBlocksDataProviderMock removed from the project

[#1122] Implement FileManager protocol and dependency

- ZcashFileManager implemented
- MigrateLegacyCacheDBAction refactored to be dependent on ZcashFileManager
- ZcashFileManager mock added

[#1122] Implement FileManager protocol and dependency (#1124)

- code cleanup

[#1121] Implement MigrateLegacyCacheDBAction

- MigrateLegacyCacheDBAction tests WIP
- tests naming cleanup

[#1121] Implement MigrateLegacyCacheDBAction

- MigrateLegacyCacheDBAction tests finished

[#700] Fix DarksideTests

Closes #1102

Some tests that can't be compiled are disabled for now. List is in #1126.

This PR contains multiple fixes. Most of the fixes is done in the code.
Not in the tests. That is good news.

Fixes:
- `config` inside `CompactBlockProcessor` can be updated during the
  tests. And it must be also updated inside the actions. So
  `ConfigProvider` is added and it is wrapper for config that is passed
  to any instance of `Action` and provides updated config.
- Fixed `EnhanceAction`. Now it should update all the blocks in the
  enhance range even when the remaining count of blocks is lower than
  1000.
- Fixed `fail()` and `validationFailed()`. These two were canceling
  `syncTask`. But that stopped run loop in a bad way.

[#1129] Final check of all State Machine Action tests

- XTCAsset messages checked
- test naming checked and fixed

[#1126] Fix DarksideTests in state machine branch

Closes #1126

Fix offline tests

Closes #1098
Closes #1095
Closes #1094

Most of the tests is removed. Either the code that was tested doesn't
exists. Or now tests for state machine actions do this work.

[#1115] Fix NetworkTests in state machine branch

Closes #1115

[#700] Fix progress reporting

Some actions in the sync process may not run. For example there are no
transactions to enhance and therefore there is no enhance progress. And in
cases like this computation of final progress won't work properly.
So let's fake 100% progress at the end of the sync process.
2023-05-30 15:01:42 +02:00
Michal Fousek 9144f26b35 [#1128] Fix switch to error state after Synchronizer.prepare() call
Closes #1128
2023-05-24 15:25:11 +02:00
Michal Fousek c0c0c426d5 [#1111] Change storage for InternalSyncProgress
Closes #1111

- `InternalSyncProgress` has now new disk storage implementation:
  `InternalSyncProgressDiskStorage`.
- When then `UserDefaults` was used as disk storage there were no IO errors
  to handle. But now the code in `InternalSyncProgressDiskStorage`
  writes to files and reads from files. So there are errors to handle.
  Because of this protocol `InternalSyncProgressStorage` changed a bit
  and API of `InternalSyncProgress` changed a bit. Now all the functions
  throws.
- It is possible to make progress storage IO errors silent but I think
  that storing of the progress is very essential to the sync process. So
  I think that when progress storage fails then the sync process should
  fail.
- `Initializer` has new parameter `generalStorageURL`. It is directory
  where `InternalSyncProgressDiskStorage` stores progress files. In
  future this can be used for storing any data the SDK wants.
2023-05-22 20:42:11 +02:00
Michal Fousek d091660b79 [#1016] Rebuild download stream periodically while downloading
- Rebuild download stream in `BlockDownloaderImpl` each 10 batches. So
  currently stream downloads 1000 blocks and then new stream is created.
- Memory stays between 60MB and 70MB during sync in the Sample app.
2023-05-17 10:36:02 +02:00
Francisco Gindre bb3d7d06c5
[#780] part of Adopt green policy (#1064)
fixes docs and deletes fixme for [#783]

closes #783

[#785]: Fix test testSingleNoteNoChangeTransaction

[#786] fix testVerifyChangeTransaction

Closes #786

[#790] mark as fixed testRewindAfterSendingTransaction

Closes #790

[#865] Fix test testStoringTenSandblastedBlocks

Closes #865

[#791] Fix test testRewindAfterSendingTransaction

Closes #791

[#783]: Fix test testMaxAmountSendBroadcast

Closes #783
2023-05-11 08:13:20 -03:00
Francisco Gindre f98692c0ea
[#781] This fixes test `testMaxAmountMinusOneSend` by creating two se… (#1034)
* [#781] This fixes test `testMaxAmountMinusOneSend` by creating two separate
tests:
  - testMaxAmountMinusOneSendFails
  - testMaxAmountSend

Also includes new functionality that tracks sent transactions so
that users can be notified specifically when they are mined and uses "idea B" of
issue #1033.

closes #1033
closes #781

* Fix tests
2023-05-10 17:13:29 -03:00
Lukas Korba 44e17ca042 [#843] Simplify Synchronizer Status
- SyncStatus refactored to InternalSyncStatus, so the SDK works internally with the SyncStatus as previously
- new SyncStatus is a computed property that maps InternalSyncStatus to simplified SyncStatus

[#843] Simplify Synchronizer Status (#1010)

- simplification of SyncStatus done
- % computation moved to the mapping function so the rest of the SDK works just like before
- fixed unit tests
- new unit tests checking the boundaries of the new progress

[#843] Simplify Synchronizer Status (#1010)

- removed commented code

[#843] Simplify Synchronizer Status (#1010)

- package.resolved

[#843] Simplify Synchronizer Status (#1010)

- fixed dark side tests

[#843] Simplify Synchronizer Status (#1010)

- package.resolved
2023-05-09 12:35:49 +02:00
Michal Fousek 5d2b2298b5 [#1012] Introduce dependency injection in the SDK
Closes #1012

- This is the first change for the dependency injection. This change
  adds `DIContainer` which is used to register and resolve dependencies.
- Then this change shows how `DIContainer` can be used to resolve
  dependencies. And it also shows how to mock dependencies in tests.
- Constructors of `Initializer` are changed to support mocking of
  depedendencies. But it doesn't affect public API in any way.
- Depedencies are registered in `Dependencies.setup()`. Then in the code
  `container.resolve(SomeType.self)` is called to get instance of the
  dependency.
- `ZcashTestCase` class is added. It inherits from `XCTestCase` and
  should be used for base class in tests. `ZcashTestCase` takes care of
  creation of `DIContainer` for dependencies mocking. In future we can
  maybe move there more things that are used in each test.
- Lot is missing here. Not all the code is using dependency injection.
  Tests for `DIContainer` are missing. Only `OfflineTests` work now
  (other tests can't be even compiled). It will be added in future
  changes.
2023-05-08 14:58:49 +02:00
Michal Fousek 84d8002ddb Fix tests related to blocks downloading
- Call `setSyncRange()` in tests.
- Little update to `BlockDownloader` API so it makes more sense.
2023-05-08 11:08:54 +02:00
Francisco Gindre f5e7c027af
[#1001] Remove PendingDb in favor of `v_transactions` and `v_tx_output` Views (#1001)
Removes `PendingTransactionEntity` and all of its related components.
Pending items are still tracked and visualized by the existing APIs
but they are retrieved from the `TransactionRepository` instead by
returning `ZcashTransaction.Overview` instead.

`pendingDbURL` is removed from every place it was required. Its
deletion is responsibility of wallet developers.

`ClearedTransactions` are now just `transactions`.

`MigrationManager` is deleted. Now all migrations are in charge of
the rust welding layer.

`PendingTransactionDao.swift` is removed.

Implementation of `AccountEntity` called `Account` is now `DbAccount`

`ZcashTransaction.Overview` can be checked for "pending-ness" by calling
`.isPending(latestHeight:)` latest height must be provided so that minedHeight
can be compared with the lastest and the `defaultStaleTolerance` constant.

`TransactionRecipient` is now a public type.

protocol `PendingTransactionRepository` is removed.

`TransactionManagerError` and `PersistentTransactionManager` are deleted.

`OutboundTransactionManager` is deleted and replaced by `TransactionEncoder`
which now incorporates `submit(encoded:)` functionality

`WalletTransactionEncoder` now uses a `LightWalletService` to submit the
encoded transactions.

Add changelog changes

Delete references to PendingDb from tests and documentation.

Fixes some typos. Adds the ability to trace transaction repository
SQL queries from test

Fix rebase conflicts and generate code

[#837] Memo tests regarding transparent address

Closes #837

Add model for transaction output

Point to FFI branch

Fix issue where sync wouldn't resume after wipe. Becasue GRPC
channel would be closed

Fix Tests

Fix testPendingTransactionMinedHeightUpdated

Fix testLastStates

[#921] Fix  broken SynchronizerDarksideTests

Add ZcashTransaction.Output API to Synchronizer

Changelog + comment fix

Add Assertions for transaction outputs and recipients

Point to FFI 0.3.1

Fix Demo App Compiler errors

Fix Demo App Compiler errors

fix cacheDb warnings

Fix Tests and compiler errors of rebase

build demo app

Remove `ZcashTransaction.Sent` and `.Received`. Add `.State` and tests

Fix SPM warning

PR Suggestions

Removes errors that are not used anymore

fix warnings
2023-05-05 14:30:47 -03:00
Michal Fousek 0032fedde7 [#442] Introduce parallel downloading and scanning
- `BlockDownloaderImpl` is changed. It now spawns detached Task to
  download blocks. So blocks can be downloaded in parallel with scanning
  process.
- `Synchronizer.stop()` is no longer async.
- Blocks cache cleaning is changed. It's not possible to drop the whole
  cache now. There are some blocks pre-downloaded which weren't scanned
  yet.
2023-05-05 09:35:43 +02:00
Michal Fousek 3ffea42454 [#901] Use fulfillment instead of wait in async context in tests
Closes #901
2023-05-01 09:07:43 +02:00
Michal Fousek 88e120509b
Merge pull request #1007 from Chlup/ZcashKeyDerivationBackend_non_async
Make all methods in ZcashKeyDerivationBackendWelding non async
2023-04-25 16:23:48 +02:00
Michal Fousek 5f162d01f8 Make all methods in ZcashKeyDerivationBackendWelding non async
It looks like that some methods in `ZcashKeyDerivationBackendWelding`
were left async by mistake. There is no reason for any method there to
be async.

After this change no method in `DerivationTool` has to async. It makes
things much easier.
2023-04-25 16:16:04 +02:00
Francisco Gindre 86458ea76a
[#644] AdvancedReOrg Tests fails when running Lightwalletd on `master` (#1006)
Fixes nonce paramenter in darkside api. also corrects tests block staging
for the failing tests

closes #644
2023-04-25 10:41:03 -03:00
Michal Fousek 724d410fad
[#361] Introduce ZcashError (#1005)
This PR introduces `ZcashError` enum. This enum represents any error
that can be thrown inside of the SDK and outside of the SDK. Also
`ZcashError` is used in `LightWalletGRPCService` and handled in
`CompactBlockProcessor` as example.

Why enum? First I tried this with some structure which contained code,
message and underlyingError. Problem was when some specific place in the
SDK would like to attach some additional data to error. I didn't want to
add some generic dictionary to store anything with the error.

So I used enum to identify error. Each member can have specified amount
of associated values. So specific error can bring some context data. And
it's type safe.

Each error has also a code. Relationship between errors and codes is
1:1. It may looks bit redundant but it's important. The client app now
can choose how to process errors. Either identify those by the error
itself or by code.

Definition or errors and codes is in `ZcashErrorDefinition`. And then
`ZcashError` and `ZcashErrorCode` are generated by Sourcery. Thanks to
this it is easier to maintain the final code. And it gives us ability to
generate any kind of documentation for the errors and codes. I created
simple example of this in this PR. And it doesn't make the SDK
completely dependent on the Sourcery. Final structures aren't super
complicated and can be written manually if needed.

[#923] Update error handling in DatabaseStorageManager.swift

- cleanup of DatabaseStorageManager, not used anymore
- ZcashError for SimpleConnectionProvider

Revert "[#923] Update error handling in DatabaseStorageManager.swift"

This reverts commit 94e028d31a.

Fix script which generates ZcashError

[#922] Update error handling in DatabaseMigrationManager.swift

Closes #922

[#923] Update error handling in DatabaseStorageManager.swift

- The DatabaseStorageManager is not used so I deleted it
- SimpleConnectionProvider's errors updated

Fix tests

[#955] Update error handling in SaplingParameterDownloader

Closes #955

[#936] Update error handling in NotesDao

Closes #936

[#935] Update error handling in BlockDao

Closes #935

[#931] InternalSyncProgress.computeNextState doesn't need to throw

Closes #931

[#950] Update error handling in rust backend

Closes #949
Closes #950

[#941] Update error handling in AccountEntity

Closes #941

[#928] Update error handling in FSCompactBlockRepository

Closes #928

[#925] Update error handling in BlockDownloaderService (#974)

- BlockDownloaderService errors updated

[#924] Update error handling in BlockDownloader (#973)

- BlockDownloaderStream nextBlock updated

[#942] Update error handling in TransactionEntity (#977)

- ZcashTransaction init errors converted to the ZcashError

[#939] Update error handling in TransactionDao (#976)

- TransactionRepositoryError removed and replaced with ZcashError
- all TransactionDAO errors converted to ZcashError

[#943] Update error handling in ZcashCompactBlock

Closes #943

[#945] Update error handling in Memo

Closes #945

[#934] Update error handling in Checkpoint

Closes #944
Closes #934

[#938] Update error handling in PendingTransactionDao

Closes #938

[#926] Update error handling in BlockEnhancer

- WIP, switching to transaction repository to unblock this ticket

[#926] Update error handling in BlockEnhancer

- enhancer's errors done

[#926] Update error handling in BlockEnhancer (#994)

- cleanup

[#952] Update error handling in DerivationTool

Closes #952

[#932] Update error handling in BlockValidator

Closes #932

[#940] Update error handling in UnspentTransactionOutputDao

Closes #940

[#946] Update error handling in WalletTypes

Closes #946

[#954] Update error handling in WalletTransactionEncoder

Closes #954

[#953] Update error handling in PersistentTransactionManager

Closes #953

[#956] Update error handling in Initializer

Closes #956

[#947] Update error handling in Zatoshi (#996)

- Zatoshi's errors converted to ZcashError

[#927] Update error handling in UTXOFetcher (#997)

- UTXOFetcher resolved

Use let instead of var where applicable

In the SDK `var` was used on places where `let` would be sufficient. And
default strategy in Swift should use use `let` until mutability is
required. So all these places are changed. No logic is changed.

[#933] Update error handling in CompactBlockProcessor

- CompactBlockProcessor's errors refactored to ZcashError

[#933] Update error handling in CompactBlockProcessor #999

- comments addressed

[#951] Update error handling in SDKSynchronizer

- SDKSynchronizer's errors refactored to ZcashError

[#951] Update error handling in SDKSynchronizer (#1002)

- comments resolved

Make DerivationTool constructor public

DerivationTool.init should be public. This was removed by accident when
adopting async.

[#361] Add changelog
2023-04-24 18:15:20 -03:00
Michal Fousek 34db6b594a Make DerivationTool constructor public
DerivationTool.init should be public. This was removed by accident when
adopting async.
2023-04-24 16:24:36 +02:00
Francisco Gindre 802aaa437d
[#959] Fix `v_transactions` view issues with value (#963)
This change switches to a new (future) version of the rust crates
that will get rid of the sent and received transactions Views in
favor of a `v_transaction` view that will do better accounting of
outgoing and incoming funds. Additionally it will support an
outputs view for seeing the inner details of transactions enabling
the SDKs tell the users the precise movement of value that a tx
causes in its multiple possible ways according to the protocol.

the `v_tx_outputs` view is not yet implemented.

Sent and Received transaction sub-types are kept for compatibility
purposes but they are generated from Overviews instead of queried
from a specific view.

In the transaction Overview the value represents the whole value
transfer for the transaction from the point of view of a given
account including fees. This means that the value for a single
transaction Overview struct represents the addition or subtraction
of ZEC value to the account's balance.

Future updates will give clients the possibility to drill into the
inner workings of those value changes in a per-output basis for
each transaction.

Also, the field `pending_unmined` field was added to `v_transactions`
so that wallets can query `DataDb` for pending but yet unmined txs

This will prepare the field for removing the notion of a "PendingDb"
and its nuances.

Also updated test database `darkside_data.db`

Closes #959

Closes #971 ZcashLightClientKitSample main target broken swiftlint script

Demo App improvements: Show Short date and value on transaction list
2023-04-18 09:10:56 -03:00
Lukas Korba 99b24c2081
[#908] Add last and latest blocks data to Synchronizer State (#911)
- attached to the processor's timer
- added latest scanned time and latest block height
- sample app showing the new values alongside with the %
- unit tests fixed
- latestBlock() API added
- the transaction repository is no longer used to update the exposed values during the syncing
2023-04-14 09:08:37 +02:00
Michal Fousek cd4450e43f Fix SynchronizerDarksideTests 2023-04-12 16:04:28 +02:00
Michal Fousek 2bbc5800bd [#888] Make actor from ZcashRustBackendWelding
Closes #888.

- `ZcashRustBackend` is actor now. So majority of methods in this actor
  are now async.
- Some methods stayed `static` in `ZcashRustBackend`. It would be hard
  to pass instance of the `ZcashRustBackend` to the places where these
  methods are used in static manner. And it would change lot of APIs.
  But it isn't problem from technical perspective because these methods
  would be `nonisolated` otherwise.
- Methods `lastError()` and `getLastError()` in `ZcashRustBackend` are
  now private. This makes sure that ther won't be aby race condition
  between other methods and these two error methods.
- All the methods for which was `lastError()` used in code now throw
  error. So `lastError()` is no longer needed outside of the
  `ZcashRustBackend`.
- There are in the public API related to `DerivationTool`.
- `DerivationTool` now requires instance of the `ZcashRustBackend`. And
  `ZcashRustBackend` isn't public type. So `DerivationTool` doesn't have
  any public constructor now. It can be created only via
  `Initializer.makeDerivationTool()` instance method.
- `deriveUnifiedSpendingKey()` and `deriveUnifiedFullViewingKey()` in
  `DerivationTool` are now async. It is because these are using
  `ZcashRustBackend` inside. `DerivationTool` offers alternative
  (closure and combine) APIs. But downside is that there is no sync API
  to dervie spending key or viewing key.
- Some methods of the `DerivationTool` are now static. These methods
  don't use anything that requires instance of the `DerivationTool`
  inside.

[#888] Use Sourcery to generate mocks

- I wrote mock for `Synchronizer` manually. And it's tedious and long
  and boring work.
- Now `ZcashRustBackendWelding` is changed a lot so it means
  `MockRustBackend` must be changed a lot. So I decided to introduce
  `sourcery` to generate mocks from protocols so we don't have to do it
  manually ever.
- To generate mocks go to `ZcashLightClientKit/Tests/TestUtils/Sourcery`
  directory and run `generateMocks.sh` script.
- Your protocol must be mentioned in `AutoMockable.swift` file.
  Generated mocks are in `AutoMockable.generated.swift` file.

[#888] Fix Offline tests

- Offline tests target now runs and tests are green.
- There is log of changes in tests. But logic is not changed.
- Updated `AutoMockable.stencil` so sourcery is able to generate mock as
  actor when protocol is marked with: `// sourcery: mockActor`.
- Last few updates in `ZcashRustBackendWelding`. In previous PR `rewindCacheToHeight`
  methods was overlooked and it didn't throw error.
- Removed `MockRustBackend` and using generated
  `ZCashRustBackendWeldingMock` instead.
- Using generated `SynchronizerMock`.

[#888] Fix NetworkTests

- Changed a bit how rust backend mock is used in the tests. Introduced
  `RustBackendMockHelper`. There are some state variables that must be
  preserved within one instance of the mock. This helper does exactly
  this. It keeps this state variables in the memory and helping mock to
  work as expected.

[#888] Fix Darkside tests

Create ZcashKeyDeriving internal protocol

Use New DerivationTool that does not require RustBackend

Remove duplicated methods that had been copied over

[#888] Fix potentially broken tests

I broke the tests because I moved `testTempDirectory` from each
`TestCase` to the `Environment`. By this I caused that each tests uses
exactly same URL. Which is directly against purpose of
`testTempDirectory`.

So now each test calls this one and store it to local variable. So each
test has unique URL.

[#888] Add ability to mock nonisolated methods to AutoMockable.stencil

[#888] Add changelog and fix the documentation in ZcashRustBackendWelding

[#888] Rename derivation rust backend protocol and remove static methods

- Renamed `ZcashKeyDeriving` to `ZcashKeyDerivationBackendWelding`. So
  the naming scheme is same as for `ZcashRustBackendWelding`.
- `ZcashKeyDerivationBackend` is now struct instead of enum.
- Methods in `ZcashKeyDerivationBackendWelding` (except one) are no
  longer static. Because of this the respective methods in
  `DerivationTool` aren't also static anymore.
2023-04-11 17:51:28 +02:00
Francisco Gindre 6b7fbdd908
[#895] Add Sync Session ID Synchronizer State (#906)
Closes #895

Add Sync Session ID to `SynchronizerState`.

A SyncSession is an attempt to sync the blockchain within the lifetime of a Synchronizer.
A Synchronizer can sync many times, when synced it will refresh every ~20 seconds +- random padding.
each sync attempt will have a different UUID even if it's from the same instance of SDKSynchronizer.

**How are SyncSessions are delimited? With `SyncStatus` changes.**

changes from [`.unprepared`|`.error`|`.disconnected`|`.stopped`] to `.syncing` assign a new sessionID to the synchronizer.

Any other transitions won't.

`areTwoStatusesDifferent ` was refactored to a helper function of `SyncStatus`

**How are IDs generated?**

ID generation is not mandated but delegated to a protocol `SyncSessionIDGenerator`.
Tests inject their own deterministic generator to avoid test flakiness.

Default implementation of SyncSessionIDGenerator is

````Swift
struct UniqueSyncSessionIDGenerator {}

extension UniqueSyncSessionIDGenerator: SyncSessionIDGenerator {
    func nextID() -> UUID {
        UUID()
    }
}
````

**SyncSession Pseudo-Atomicity and thread safety**

SyncSession is a type alias of a GenericActor holding a UUID

````
typealias SyncSession = GenericActor<UUID>

extension SyncSession {
    /// updates the current sync session to a new value with the given generator
    /// - Parameters generator: a `SyncSessionIDGenerator`
    /// - returns: the `UUID` of the newly updated value.
    @discardableResult
    func newSession(with generator: SyncSessionIDGenerator) async -> UUID {
        return await self.update(generator.nextID())
    }
}
````

Closes #895

SessionTicker struct to control session transitions.

switching to `.unprepared` now makes syncID to be `.nullID`
2023-04-07 09:02:05 -03:00
Michal Fousek 34ac9ff7c5
Merge pull request #891 from Chlup/877_OutboundTransactionManager_async
[#877] OutboundTransactionManager to async
2023-03-31 16:40:05 +02:00