Commit Graph

990 Commits

Author SHA1 Message Date
Michal Fousek 3f513e9c6d [#1059] Implement rewind for state machine CBP
Closes #1059
2023-05-19 10:54:15 +02:00
Michal Fousek 70026af582 [#1060] Implement wipe for state machine CBP
Closes #1060
2023-05-19 10:54:15 +02:00
Michal Fousek 9aba16b69d [#1052] Implement AfterSyncHooksManager when using state machine CBP
Closes #1052
2023-05-19 10:54:15 +02:00
Michal Fousek e1ee62403f [#1058] Implement stop for state machine CBP
Closes #1058
2023-05-19 10:54:15 +02:00
Michal Fousek 268cc8ce6f [#1057] Implement start for state machine CBP
Closes #1057
2023-05-19 10:54:15 +02:00
Michal Fousek f8ea5bb859 [#1055] Implement retry timer to state machine CBP
Closes #1055
2023-05-19 10:54:15 +02:00
Michal Fousek 15680e7b64 [#1061] Add failure methods for state machine CBP
Closes #1061
2023-05-19 10:54:15 +02:00
Michal Fousek eda8977d5d [#1056] Add constructors to state machine CBP
Closes #1056
2023-05-19 10:54:15 +02:00
Michal Fousek 7ded055ef7 [#1050] Implement ValidateServerAction
Closes #1050
2023-05-19 10:54:15 +02:00
Michal Fousek 4b366b68c0 [#1049] Implement ValidateAction
Closes #1049
2023-05-19 10:54:15 +02:00
Michal Fousek 3737b841ef [#1043] Implement DownloadAction
Closes #1043
2023-05-19 10:54:14 +02:00
Michal Fousek 66cdde4a28 Fix docs 2023-05-19 10:53:43 +02:00
Michal Fousek 553b938836 [#1054] Add config to state machine CBP
Closes #1054
2023-05-19 10:53:43 +02:00
Michal Fousek 47386086d7 Action.stop is now async 2023-05-19 10:53:43 +02:00
Michal Fousek 304fcc341f [#700] Add documentation for CompactBlockProcessor state machine
- plantuml tool is used to generate diagram.
2023-05-19 10:53:43 +02:00
Michal Fousek 2501cea4b4 Small updates 2023-05-19 10:53:43 +02:00
Michal Fousek 92994b067e [#700] CompactBlockProcessor as state machine proof of concept
- For now I created `CompactBlockProcessorNG` where I started with
  implementation of the state machine. I did it to not break the rest of
  the SDK. This change will be merged to the feature branch. And before
  it is merged to `main` branch code from `CompactBlockProcessorNG` will
  be moved to `CompactBlockProcessor`.
- The new code is not used. It just shows and explains how it is done.
  It is proof of concept.
- I did put either commented current code or comment to some places to
  explain what should be done there.
- New important data types:
  - `ActionContext` is context that can hold any data that needs to be
    shared between actions. For example sync ranges or current state.
  - `CBPState` is state of the `CompactBlockProcessor`. Each state is
    handled by one action. This doesn't apply to terminal states like
    `finished` or `failed`.
  - `ActionProgress` is very similar to `CompactBlockProgress`.
    Different actions reports progress differently and `ActionProgress`
    represents this.
  - `Action` is protocol that defines API of an action. It has one run
    method that executes the code of the action
- CBP first creates actions for (almost) each state in `makeActions()`
  method. Then the "magic" is done in `CompactBlockProcessorNG.run()` method.
  Here is main loop which takes action for current state and execute it.
  It's expected that action does it's work and then updates the context
  with new state. And this happens until some terminal state
  (`finished`, `failed`, `stopped`) is reached.
- After the transition to state machine API of the
  `CompactBlockProcessor` should stay the same. No changes should be
  required in `SDKSynchronizer`.
2023-05-19 10:53:43 +02:00
Michal Fousek ea4da6e0e6
Merge pull request #1093 from Chlup/1016_stream_per_batch_parallel
[#1016] Rebuild download stream periodically while downloading
2023-05-17 17:00:55 +02:00
Francisco Gindre 630737e511
fix typo on log 2023-05-17 11:22:25 -03:00
Michal Fousek d091660b79 [#1016] Rebuild download stream periodically while downloading
- Rebuild download stream in `BlockDownloaderImpl` each 10 batches. So
  currently stream downloads 1000 blocks and then new stream is created.
- Memory stays between 60MB and 70MB during sync in the Sample app.
2023-05-17 10:36:02 +02:00
Francisco Gindre bb3d7d06c5
[#780] part of Adopt green policy (#1064)
fixes docs and deletes fixme for [#783]

closes #783

[#785]: Fix test testSingleNoteNoChangeTransaction

[#786] fix testVerifyChangeTransaction

Closes #786

[#790] mark as fixed testRewindAfterSendingTransaction

Closes #790

[#865] Fix test testStoringTenSandblastedBlocks

Closes #865

[#791] Fix test testRewindAfterSendingTransaction

Closes #791

[#783]: Fix test testMaxAmountSendBroadcast

Closes #783
2023-05-11 08:13:20 -03:00
Francisco Gindre f98692c0ea
[#781] This fixes test `testMaxAmountMinusOneSend` by creating two se… (#1034)
* [#781] This fixes test `testMaxAmountMinusOneSend` by creating two separate
tests:
  - testMaxAmountMinusOneSendFails
  - testMaxAmountSend

Also includes new functionality that tracks sent transactions so
that users can be notified specifically when they are mined and uses "idea B" of
issue #1033.

closes #1033
closes #781

* Fix tests
2023-05-10 17:13:29 -03:00
Francisco Gindre cf53edb3b8
[#1037] Empty string memo throws `ZcashError.memoTextInputEndsWithNullBytes` (#1038)
Fixes the case where the SDK would read an empty string from the Db and would
throw this error incorrectly
closes #1037
2023-05-10 12:39:34 -03:00
Lukas Korba 355a3fc4f5
Merge pull request #1036 from LukasKorba/1035-Create-ZcashError.unknown
- new ZcashError.unknown that wraps and handles any Error addedCreate ZcashError.unknown
2023-05-10 14:49:03 +02:00
Lukas Korba 9c611b31e3 [#1035] Create ZcashError.unknown
- new ZcashError.unknown that wraps and handles any Error added

[#1035] Create ZcashError.unknown (#1036)

- improved the conversion of error -> ZcashError

[#1035] Create ZcashError.unknown (#1036)

- detailed comments of the unknown case
2023-05-10 14:48:16 +02:00
Lukas Korba c477e483ff
Merge pull request #1010 from LukasKorba/843-Simplify-Synchronizer-Status-internal
[#843] Simplify Synchronizer Status
2023-05-09 16:45:36 +02:00
Lukas Korba 44e17ca042 [#843] Simplify Synchronizer Status
- SyncStatus refactored to InternalSyncStatus, so the SDK works internally with the SyncStatus as previously
- new SyncStatus is a computed property that maps InternalSyncStatus to simplified SyncStatus

[#843] Simplify Synchronizer Status (#1010)

- simplification of SyncStatus done
- % computation moved to the mapping function so the rest of the SDK works just like before
- fixed unit tests
- new unit tests checking the boundaries of the new progress

[#843] Simplify Synchronizer Status (#1010)

- removed commented code

[#843] Simplify Synchronizer Status (#1010)

- package.resolved

[#843] Simplify Synchronizer Status (#1010)

- fixed dark side tests

[#843] Simplify Synchronizer Status (#1010)

- package.resolved
2023-05-09 12:35:49 +02:00
Michal Fousek 336c5a7c38
Merge pull request #1032 from zcash/features/1012_depedency_injection
[#1012] Introduce dependency injection in the SDK
2023-05-08 19:00:00 +02:00
Michal Fousek 5d2b2298b5 [#1012] Introduce dependency injection in the SDK
Closes #1012

- This is the first change for the dependency injection. This change
  adds `DIContainer` which is used to register and resolve dependencies.
- Then this change shows how `DIContainer` can be used to resolve
  dependencies. And it also shows how to mock dependencies in tests.
- Constructors of `Initializer` are changed to support mocking of
  depedendencies. But it doesn't affect public API in any way.
- Depedencies are registered in `Dependencies.setup()`. Then in the code
  `container.resolve(SomeType.self)` is called to get instance of the
  dependency.
- `ZcashTestCase` class is added. It inherits from `XCTestCase` and
  should be used for base class in tests. `ZcashTestCase` takes care of
  creation of `DIContainer` for dependencies mocking. In future we can
  maybe move there more things that are used in each test.
- Lot is missing here. Not all the code is using dependency injection.
  Tests for `DIContainer` are missing. Only `OfflineTests` work now
  (other tests can't be even compiled). It will be added in future
  changes.
2023-05-08 14:58:49 +02:00
Michal Fousek b582e1e4e1
Merge pull request #1030 from LukasKorba/1029-AsyncThrowingStream-update-so-it-does-not-use-Task
[#1029] AsyncThrowingStream update so it doesn't use Task
2023-05-08 14:46:08 +02:00
Michal Fousek ac782bbfa6
Merge pull request #1031 from Chlup/fix_network_test
Fix error handling in BlockDownloaderImpl
2023-05-08 14:45:55 +02:00
Lukas Korba 222c6cf064 [#1029] AsyncThrowingStream update so it doesn't use Task
- refactored

[#1029] AsyncThrowingStream update so it doesn't use Task (#1030)

- resolved file reverted
2023-05-08 13:57:31 +02:00
Michal Fousek 9f333170c7 Fix error handling in BlockDownloaderImpl
`lastError` inside the `BlockDownloaderImpl` was set even when
downloading task was canceled. And because it was canceled the
`lastError` has never been read by CBP before the next sync. And when
next sync was started it was immediately failed because of cancelation
error stored in `lastError`.
2023-05-08 13:49:29 +02:00
Michal Fousek c3335db54d
Merge pull request #1028 from Chlup/fix_tests
Fix tests related to blocks downloading
2023-05-08 11:09:21 +02:00
Michal Fousek 84d8002ddb Fix tests related to blocks downloading
- Call `setSyncRange()` in tests.
- Little update to `BlockDownloader` API so it makes more sense.
2023-05-08 11:08:54 +02:00
Francisco Gindre c736dd37fb
[#1019] Memo has trailing garbled text (#1023)
Changes the way unpadded bytes are turned into a UTF-8 Swift String
without using cString assuming APIs that would overflow memory and
add garbled trailing bytes.

Closes #1019

add changelog entry
2023-05-05 15:51:50 -03:00
Francisco Gindre f5e7c027af
[#1001] Remove PendingDb in favor of `v_transactions` and `v_tx_output` Views (#1001)
Removes `PendingTransactionEntity` and all of its related components.
Pending items are still tracked and visualized by the existing APIs
but they are retrieved from the `TransactionRepository` instead by
returning `ZcashTransaction.Overview` instead.

`pendingDbURL` is removed from every place it was required. Its
deletion is responsibility of wallet developers.

`ClearedTransactions` are now just `transactions`.

`MigrationManager` is deleted. Now all migrations are in charge of
the rust welding layer.

`PendingTransactionDao.swift` is removed.

Implementation of `AccountEntity` called `Account` is now `DbAccount`

`ZcashTransaction.Overview` can be checked for "pending-ness" by calling
`.isPending(latestHeight:)` latest height must be provided so that minedHeight
can be compared with the lastest and the `defaultStaleTolerance` constant.

`TransactionRecipient` is now a public type.

protocol `PendingTransactionRepository` is removed.

`TransactionManagerError` and `PersistentTransactionManager` are deleted.

`OutboundTransactionManager` is deleted and replaced by `TransactionEncoder`
which now incorporates `submit(encoded:)` functionality

`WalletTransactionEncoder` now uses a `LightWalletService` to submit the
encoded transactions.

Add changelog changes

Delete references to PendingDb from tests and documentation.

Fixes some typos. Adds the ability to trace transaction repository
SQL queries from test

Fix rebase conflicts and generate code

[#837] Memo tests regarding transparent address

Closes #837

Add model for transaction output

Point to FFI branch

Fix issue where sync wouldn't resume after wipe. Becasue GRPC
channel would be closed

Fix Tests

Fix testPendingTransactionMinedHeightUpdated

Fix testLastStates

[#921] Fix  broken SynchronizerDarksideTests

Add ZcashTransaction.Output API to Synchronizer

Changelog + comment fix

Add Assertions for transaction outputs and recipients

Point to FFI 0.3.1

Fix Demo App Compiler errors

Fix Demo App Compiler errors

fix cacheDb warnings

Fix Tests and compiler errors of rebase

build demo app

Remove `ZcashTransaction.Sent` and `.Received`. Add `.State` and tests

Fix SPM warning

PR Suggestions

Removes errors that are not used anymore

fix warnings
2023-05-05 14:30:47 -03:00
Michal Fousek 0324d9ace5
Merge pull request #1009 from Chlup/442_downloading_in_parallel_prototype
[#442] Implement parallel downloading and scanning
2023-05-05 17:06:09 +02:00
Michal Fousek 0032fedde7 [#442] Introduce parallel downloading and scanning
- `BlockDownloaderImpl` is changed. It now spawns detached Task to
  download blocks. So blocks can be downloaded in parallel with scanning
  process.
- `Synchronizer.stop()` is no longer async.
- Blocks cache cleaning is changed. It's not possible to drop the whole
  cache now. There are some blocks pre-downloaded which weren't scanned
  yet.
2023-05-05 09:35:43 +02:00
Lukas Korba c1f7707c95
Merge pull request #1020 from LukasKorba/1011-Improve-end-of-the-sync-process
[#1011] Improve end of the sync process
2023-05-04 18:24:05 +02:00
Lukas Korba 4197436668 [#1011] Improve end of the sync process
- added check for mined blocks, if yes, the next batch is called
2023-05-04 10:11:57 +02:00
Francisco Gindre 505ac6a26c
[#1017] Tweak batch size and scan height (#1018)
this bumps up the amount of blocks to scan on sections of the chain
that have full blocks
Closes #1017
2023-05-02 17:15:39 -03:00
Matthew Watt 58b99c8fb1
[#1013] - Enable more granular control over logging behavior (#1014)
* Add new model for logging configuration

* Fix the example app and update the changelog
2023-05-02 16:59:49 -03:00
Michal Fousek 991f199138
Merge pull request #902 from Chlup/901_fix_swift58_warnings
[#901] Use fulfillment instead of wait in async context in tests
2023-05-01 09:29:00 +02:00
Michal Fousek 3ffea42454 [#901] Use fulfillment instead of wait in async context in tests
Closes #901
2023-05-01 09:07:43 +02:00
Michal Fousek 88e120509b
Merge pull request #1007 from Chlup/ZcashKeyDerivationBackend_non_async
Make all methods in ZcashKeyDerivationBackendWelding non async
2023-04-25 16:23:48 +02:00
Michal Fousek 5f162d01f8 Make all methods in ZcashKeyDerivationBackendWelding non async
It looks like that some methods in `ZcashKeyDerivationBackendWelding`
were left async by mistake. There is no reason for any method there to
be async.

After this change no method in `DerivationTool` has to async. It makes
things much easier.
2023-04-25 16:16:04 +02:00
Francisco Gindre 86458ea76a
[#644] AdvancedReOrg Tests fails when running Lightwalletd on `master` (#1006)
Fixes nonce paramenter in darkside api. also corrects tests block staging
for the failing tests

closes #644
2023-04-25 10:41:03 -03:00
Michal Fousek 724d410fad
[#361] Introduce ZcashError (#1005)
This PR introduces `ZcashError` enum. This enum represents any error
that can be thrown inside of the SDK and outside of the SDK. Also
`ZcashError` is used in `LightWalletGRPCService` and handled in
`CompactBlockProcessor` as example.

Why enum? First I tried this with some structure which contained code,
message and underlyingError. Problem was when some specific place in the
SDK would like to attach some additional data to error. I didn't want to
add some generic dictionary to store anything with the error.

So I used enum to identify error. Each member can have specified amount
of associated values. So specific error can bring some context data. And
it's type safe.

Each error has also a code. Relationship between errors and codes is
1:1. It may looks bit redundant but it's important. The client app now
can choose how to process errors. Either identify those by the error
itself or by code.

Definition or errors and codes is in `ZcashErrorDefinition`. And then
`ZcashError` and `ZcashErrorCode` are generated by Sourcery. Thanks to
this it is easier to maintain the final code. And it gives us ability to
generate any kind of documentation for the errors and codes. I created
simple example of this in this PR. And it doesn't make the SDK
completely dependent on the Sourcery. Final structures aren't super
complicated and can be written manually if needed.

[#923] Update error handling in DatabaseStorageManager.swift

- cleanup of DatabaseStorageManager, not used anymore
- ZcashError for SimpleConnectionProvider

Revert "[#923] Update error handling in DatabaseStorageManager.swift"

This reverts commit 94e028d31a.

Fix script which generates ZcashError

[#922] Update error handling in DatabaseMigrationManager.swift

Closes #922

[#923] Update error handling in DatabaseStorageManager.swift

- The DatabaseStorageManager is not used so I deleted it
- SimpleConnectionProvider's errors updated

Fix tests

[#955] Update error handling in SaplingParameterDownloader

Closes #955

[#936] Update error handling in NotesDao

Closes #936

[#935] Update error handling in BlockDao

Closes #935

[#931] InternalSyncProgress.computeNextState doesn't need to throw

Closes #931

[#950] Update error handling in rust backend

Closes #949
Closes #950

[#941] Update error handling in AccountEntity

Closes #941

[#928] Update error handling in FSCompactBlockRepository

Closes #928

[#925] Update error handling in BlockDownloaderService (#974)

- BlockDownloaderService errors updated

[#924] Update error handling in BlockDownloader (#973)

- BlockDownloaderStream nextBlock updated

[#942] Update error handling in TransactionEntity (#977)

- ZcashTransaction init errors converted to the ZcashError

[#939] Update error handling in TransactionDao (#976)

- TransactionRepositoryError removed and replaced with ZcashError
- all TransactionDAO errors converted to ZcashError

[#943] Update error handling in ZcashCompactBlock

Closes #943

[#945] Update error handling in Memo

Closes #945

[#934] Update error handling in Checkpoint

Closes #944
Closes #934

[#938] Update error handling in PendingTransactionDao

Closes #938

[#926] Update error handling in BlockEnhancer

- WIP, switching to transaction repository to unblock this ticket

[#926] Update error handling in BlockEnhancer

- enhancer's errors done

[#926] Update error handling in BlockEnhancer (#994)

- cleanup

[#952] Update error handling in DerivationTool

Closes #952

[#932] Update error handling in BlockValidator

Closes #932

[#940] Update error handling in UnspentTransactionOutputDao

Closes #940

[#946] Update error handling in WalletTypes

Closes #946

[#954] Update error handling in WalletTransactionEncoder

Closes #954

[#953] Update error handling in PersistentTransactionManager

Closes #953

[#956] Update error handling in Initializer

Closes #956

[#947] Update error handling in Zatoshi (#996)

- Zatoshi's errors converted to ZcashError

[#927] Update error handling in UTXOFetcher (#997)

- UTXOFetcher resolved

Use let instead of var where applicable

In the SDK `var` was used on places where `let` would be sufficient. And
default strategy in Swift should use use `let` until mutability is
required. So all these places are changed. No logic is changed.

[#933] Update error handling in CompactBlockProcessor

- CompactBlockProcessor's errors refactored to ZcashError

[#933] Update error handling in CompactBlockProcessor #999

- comments addressed

[#951] Update error handling in SDKSynchronizer

- SDKSynchronizer's errors refactored to ZcashError

[#951] Update error handling in SDKSynchronizer (#1002)

- comments resolved

Make DerivationTool constructor public

DerivationTool.init should be public. This was removed by accident when
adopting async.

[#361] Add changelog
2023-04-24 18:15:20 -03:00
Michal Fousek ed87069c03
Merge pull request #1004 from zcash/fix_derivation_tool_constructor
Make DerivationTool constructor public
2023-04-24 16:29:00 +02:00