Commit Graph

34 Commits

Author SHA1 Message Date
Francisco Gindre f5e7c027af
[#1001] Remove PendingDb in favor of `v_transactions` and `v_tx_output` Views (#1001)
Removes `PendingTransactionEntity` and all of its related components.
Pending items are still tracked and visualized by the existing APIs
but they are retrieved from the `TransactionRepository` instead by
returning `ZcashTransaction.Overview` instead.

`pendingDbURL` is removed from every place it was required. Its
deletion is responsibility of wallet developers.

`ClearedTransactions` are now just `transactions`.

`MigrationManager` is deleted. Now all migrations are in charge of
the rust welding layer.

`PendingTransactionDao.swift` is removed.

Implementation of `AccountEntity` called `Account` is now `DbAccount`

`ZcashTransaction.Overview` can be checked for "pending-ness" by calling
`.isPending(latestHeight:)` latest height must be provided so that minedHeight
can be compared with the lastest and the `defaultStaleTolerance` constant.

`TransactionRecipient` is now a public type.

protocol `PendingTransactionRepository` is removed.

`TransactionManagerError` and `PersistentTransactionManager` are deleted.

`OutboundTransactionManager` is deleted and replaced by `TransactionEncoder`
which now incorporates `submit(encoded:)` functionality

`WalletTransactionEncoder` now uses a `LightWalletService` to submit the
encoded transactions.

Add changelog changes

Delete references to PendingDb from tests and documentation.

Fixes some typos. Adds the ability to trace transaction repository
SQL queries from test

Fix rebase conflicts and generate code

[#837] Memo tests regarding transparent address

Closes #837

Add model for transaction output

Point to FFI branch

Fix issue where sync wouldn't resume after wipe. Becasue GRPC
channel would be closed

Fix Tests

Fix testPendingTransactionMinedHeightUpdated

Fix testLastStates

[#921] Fix  broken SynchronizerDarksideTests

Add ZcashTransaction.Output API to Synchronizer

Changelog + comment fix

Add Assertions for transaction outputs and recipients

Point to FFI 0.3.1

Fix Demo App Compiler errors

Fix Demo App Compiler errors

fix cacheDb warnings

Fix Tests and compiler errors of rebase

build demo app

Remove `ZcashTransaction.Sent` and `.Received`. Add `.State` and tests

Fix SPM warning

PR Suggestions

Removes errors that are not used anymore

fix warnings
2023-05-05 14:30:47 -03:00
Michal Fousek 0032fedde7 [#442] Introduce parallel downloading and scanning
- `BlockDownloaderImpl` is changed. It now spawns detached Task to
  download blocks. So blocks can be downloaded in parallel with scanning
  process.
- `Synchronizer.stop()` is no longer async.
- Blocks cache cleaning is changed. It's not possible to drop the whole
  cache now. There are some blocks pre-downloaded which weren't scanned
  yet.
2023-05-05 09:35:43 +02:00
Michal Fousek 724d410fad
[#361] Introduce ZcashError (#1005)
This PR introduces `ZcashError` enum. This enum represents any error
that can be thrown inside of the SDK and outside of the SDK. Also
`ZcashError` is used in `LightWalletGRPCService` and handled in
`CompactBlockProcessor` as example.

Why enum? First I tried this with some structure which contained code,
message and underlyingError. Problem was when some specific place in the
SDK would like to attach some additional data to error. I didn't want to
add some generic dictionary to store anything with the error.

So I used enum to identify error. Each member can have specified amount
of associated values. So specific error can bring some context data. And
it's type safe.

Each error has also a code. Relationship between errors and codes is
1:1. It may looks bit redundant but it's important. The client app now
can choose how to process errors. Either identify those by the error
itself or by code.

Definition or errors and codes is in `ZcashErrorDefinition`. And then
`ZcashError` and `ZcashErrorCode` are generated by Sourcery. Thanks to
this it is easier to maintain the final code. And it gives us ability to
generate any kind of documentation for the errors and codes. I created
simple example of this in this PR. And it doesn't make the SDK
completely dependent on the Sourcery. Final structures aren't super
complicated and can be written manually if needed.

[#923] Update error handling in DatabaseStorageManager.swift

- cleanup of DatabaseStorageManager, not used anymore
- ZcashError for SimpleConnectionProvider

Revert "[#923] Update error handling in DatabaseStorageManager.swift"

This reverts commit 94e028d31a.

Fix script which generates ZcashError

[#922] Update error handling in DatabaseMigrationManager.swift

Closes #922

[#923] Update error handling in DatabaseStorageManager.swift

- The DatabaseStorageManager is not used so I deleted it
- SimpleConnectionProvider's errors updated

Fix tests

[#955] Update error handling in SaplingParameterDownloader

Closes #955

[#936] Update error handling in NotesDao

Closes #936

[#935] Update error handling in BlockDao

Closes #935

[#931] InternalSyncProgress.computeNextState doesn't need to throw

Closes #931

[#950] Update error handling in rust backend

Closes #949
Closes #950

[#941] Update error handling in AccountEntity

Closes #941

[#928] Update error handling in FSCompactBlockRepository

Closes #928

[#925] Update error handling in BlockDownloaderService (#974)

- BlockDownloaderService errors updated

[#924] Update error handling in BlockDownloader (#973)

- BlockDownloaderStream nextBlock updated

[#942] Update error handling in TransactionEntity (#977)

- ZcashTransaction init errors converted to the ZcashError

[#939] Update error handling in TransactionDao (#976)

- TransactionRepositoryError removed and replaced with ZcashError
- all TransactionDAO errors converted to ZcashError

[#943] Update error handling in ZcashCompactBlock

Closes #943

[#945] Update error handling in Memo

Closes #945

[#934] Update error handling in Checkpoint

Closes #944
Closes #934

[#938] Update error handling in PendingTransactionDao

Closes #938

[#926] Update error handling in BlockEnhancer

- WIP, switching to transaction repository to unblock this ticket

[#926] Update error handling in BlockEnhancer

- enhancer's errors done

[#926] Update error handling in BlockEnhancer (#994)

- cleanup

[#952] Update error handling in DerivationTool

Closes #952

[#932] Update error handling in BlockValidator

Closes #932

[#940] Update error handling in UnspentTransactionOutputDao

Closes #940

[#946] Update error handling in WalletTypes

Closes #946

[#954] Update error handling in WalletTransactionEncoder

Closes #954

[#953] Update error handling in PersistentTransactionManager

Closes #953

[#956] Update error handling in Initializer

Closes #956

[#947] Update error handling in Zatoshi (#996)

- Zatoshi's errors converted to ZcashError

[#927] Update error handling in UTXOFetcher (#997)

- UTXOFetcher resolved

Use let instead of var where applicable

In the SDK `var` was used on places where `let` would be sufficient. And
default strategy in Swift should use use `let` until mutability is
required. So all these places are changed. No logic is changed.

[#933] Update error handling in CompactBlockProcessor

- CompactBlockProcessor's errors refactored to ZcashError

[#933] Update error handling in CompactBlockProcessor #999

- comments addressed

[#951] Update error handling in SDKSynchronizer

- SDKSynchronizer's errors refactored to ZcashError

[#951] Update error handling in SDKSynchronizer (#1002)

- comments resolved

Make DerivationTool constructor public

DerivationTool.init should be public. This was removed by accident when
adopting async.

[#361] Add changelog
2023-04-24 18:15:20 -03:00
Lukas Korba 99b24c2081
[#908] Add last and latest blocks data to Synchronizer State (#911)
- attached to the processor's timer
- added latest scanned time and latest block height
- sample app showing the new values alongside with the %
- unit tests fixed
- latestBlock() API added
- the transaction repository is no longer used to update the exposed values during the syncing
2023-04-14 09:08:37 +02:00
Michal Fousek 7826258e24 [#878] UnspentTransactionOutputRepository to async
Closes #878.
2023-03-31 10:59:46 +02:00
Michal Fousek c4f7f6ff1e [#469] Make ZcashRustBackendWelding async
Closes #469.

- Methods defined in `ZcashRustBackendWelding` protocol which are doing
  some IO operations are now async.
- `Initializer` no longer have methods to get balance. Only
  `Synchronizer` now have these methods. These methods are newly async.
  And `Initilializer` doesn't have alternative APIs.
- There is lot of changes in tests but those are mostly adding `await`.
  No logic is changed.
2023-03-30 19:08:56 +02:00
Michal Fousek 3971f80eef [#484] Use async for TransactionsRepository
Closes #484.

- `TransactionRepository` has async API. `Synchronizer` and alternative
  APIs are updated accordingly.

These methods and properties in the `Synchronizer` are async now:
- `cancelSpend(transaction:)`
- All the variants of the `getMemos(for:)` method.
- All the variants fo the `getRecipients(for:)` method.
- `allConfirmedTransactions(from:limit:)`
- `pendingTransactions`
- `clearedTransactions`
- `sentTransactions`
- `receivedTransactions`
2023-03-28 10:17:14 +02:00
Michal Fousek c2d9c6f1e8 [#871] Cleanup internals of the SDK and use async where applicable
Closes #871.
2023-03-27 16:16:42 +02:00
Michal Fousek cb31acdd81 [#826] Change how the SDK is initialised
Closes #826

- `viewingKeys` and `walletBirthday` are removed from `Initializer`
  constuctor. These parameters are moved to `SDKSynchronizer.prepare`
  function.
- Constructor of the `SDKSynchronizer` no longer throws exception.
- Any value emitted from `lastState` stream before `SDKSynchronizer.prepare`
  is called has `latestScannedHeight` set to 0.
- `Initializer.initialize` function isn't public anymore. To initialize
  SDK call `SDKSynchronizer.prepare` instead.
- Real birthday is now known after the `SDKSynchronizer.prepare` is
  called. But this function isn't async and therefore it's hard to
  pass birthday from here to `CompactBlockProcessor` in synchronous
  manner. We must be sure that it's safe to call start() after prepare
  function finishes.
    For this reason `CompactBlockProcessor` has `walletBirthdayProvider`
  instead of `walletBirthday`. And this new parameter is closure which
  reads birthday from `Initiliazer` on each call. Thanks to this
  `CompactBlockProcessor` has always the right birthday.
- Some cleanup is done in `TestCoordinator`.
2023-03-13 15:32:55 +01:00
Francisco Gindre 67dd061523 Merge branch 'main' into release/0.18.1-beta 2023-02-08 12:48:20 -03:00
Francisco Gindre 2c211732e1 [#767] implement getRecipients() for Synchronizer.
This implements `getRecipients()` function which retrieves the possible
recipients from a sent transaction. These can either be addresses or
internal accounts depending on the transaction being a shielding tx
or a regular outgoing transaction.

Closes #767

Other changes:
Fix version of zcash-light-client-ffi to 0.1.1

Enhance error reporting on a test make Mock comply with protocol

Update CHANGELOG.md
2023-02-08 08:44:29 -03:00
Michal Fousek 385c0a7195 [#746] Modularize GRPC related code
Closes #746

- `LightWalletGRPCService` is no longer public. `LightWalletService` is no longer public.
- `LightWalletGRPCService` shouldn't be used dicrectly. Use `LightWalletServiceFactory` to create instance of the service.
- Moved sending of `blockProcessorConnectivityStateChanged` notification to `Initilizer`.
- All the errors from GRPC are mapped to `LightWalletServiceError`. Handling of `GRPCStatus` in the SDK is no longer required.
- `Service` directory (that contains GRPC code) is moved to `Modules/Service`. We can put more code to `Modules/` if we decide
to modularize more code.
2023-02-03 22:25:54 +01:00
Francisco Gindre fad3aae174
- [#679] Implementation of the File-system based block cache (#679)
Closes https://github.com/zcash/ZcashLightClientKit/issues/697
Closes https://github.com/zcash/ZcashLightClientKit/issues/720
Closes https://github.com/zcash/ZcashLightClientKit/issues/587
Closes https://github.com/zcash/ZcashLightClientKit/issues/667
Closes https://github.com/zcash/ZcashLightClientKit/issues/443
Closes https://github.com/zcash/ZcashLightClientKit/issues/754

- [#790] Fix ShieldFundsTests

Closes #790
Removes comments on `ShieldFundsTests` since those issues have been fixed

Depends on zcash-light-client-ffi changes that adopt newer versions of
 librustzcash crates `zcash_primitives 0.10`, `zcash_client_backend 0.7`,
`zcash_proofs 0.10`, `zcash_client_sqlite 0.5.0`.

Also allows wallets to define a shielding_threshold and will set
foundations to customize minimum confirmations for balances, spends
and shielding operations.

**Test Bootstrapping**

- `ZcashCompactBlockDescriptor`: struct that holds functions to
describe blocks as filenames and compare those filenames

`ZcashCompactBlockDescriptor.live` has the actual implementation
but it can be replaced by mocks if needed on Tests

main implementations are held under `FSCompactBlockRepository.filenameDescription` and `FSCompactBlockRepository.filenameComparison` on a separate extention

`DirectoryListingProviders` provide two default implementations of listing a
directory deterministically. `FileManager` does not define a sorting and needs to be done in-memory by calling `.sorted()` on the resulting collection. If this
is a big toll on performance it can be changed to a POSIX implementation but
this is good for now.

`ZcashCompactBlockDescriptor` adds a `height` helper function to
turn a filename into the height of the block stored.

Implemented `func latestHeight() throws -> BlockHeight ` that
returns the blockheight by querying the cache directory in a sorted
fashion and getting the last value and turning the filename into a
`BlockHeight`

Added `Meta` struct to ZcashCompactBlock.

Tests implemented:
- `filterBlockFiles`
- `testClearTheCache`
- `testLatestHeightEmptyCacheThrows`
- `testLatestHeightEmptyCacheThrowsAsync`
- `testRewindEmptyCacheDoesNothing`
- `testRewindEmptyCacheDoesNothingAsync`
- `testWhenBlockIsStoredItFollowsTheDescribedFormat`
- `testWhenBlockIsStoredItFollowsTheFilenameConvention`
- `testGetLatestHeight`
- `testRewindDeletesTheRightBlocks` test
- `testPerformanceExample` test. This isn't a real performance test because the API doesn't work with async/await yet

adopts `shield_funds` shielding threshold parameter

Implements `initBlockMetadataDb` and fix tests
Renames dbCache parameter to `fsBlockDbRoot`. Builds but tests don't pass.
Removes cacheDb uses from code. Testing utilities still persist.
Added needed information in  MIGRATING and CHANGELOG.
Added helper to perform deletion of legacy db and creation a the
new file system backed cache.

Renames parameters and changes code where needed.
Network Constants turned into `enum` with static methods.
DeletelastDownloadedBlock helper from initializer
Removes CompactBlockStorage and CompactBlockEntity.
Implements `latestCachedBlockHeight` on rustbackend.

*Replaces dependencies on ZcashRustWelding with `FSMetadataStore`*

This allows the tests to not depend in a particular implementation
of either the MockRustBackend of or ZcashRustBackend. Also provides
a way to test errors properly and switch implementations of critical
areas like `writeBlocks`.
2023-02-02 13:58:12 -03:00
Michal Fousek ce4d7bab45 [#556] Rename Transaction to ZCashTransaction
- Xcode's refactor feature used. Only renamed happened no logic was
  changed.
2023-01-23 10:01:07 +01:00
Michal Fousek 14f1436987 [#556] Fix tests and fix few bugs
- Fix all the tests. In few tests code had to be changes because order
  of transactions is newly defined. It wasn't before.
- Change properties of Transaction.* objects. Some had to be made
  Optional to make all tests work.
- Add new tests for TransactionDAO.
- Removed methods from TransactionDAO that weren't used.
2023-01-10 23:20:46 +01:00
Michal Fousek cb62101e2d [#683] Add Synchronizer API to fetch memos for transactions
Closes #683

Fix tests
2023-01-10 11:03:57 +01:00
Michal Fousek 169fd3da46 Merge branch 'main' into 556_using_transactions_db_views 2023-01-10 11:02:12 +01:00
Michal Fousek c4df36db36 [#556] Rename Transaction to TransactionNG
This change is really only replace one string with other. No logic was
changed.
2023-01-06 10:12:00 +01:00
Michal Fousek e8aa451b17 [#657] Update changelog 2023-01-05 15:13:27 +01:00
Michal Fousek f12ace4bf3 [#556] Replace TransactionRepository.findAll methods with new methods
Fix compilation
2023-01-03 13:12:07 +01:00
Michal Fousek a40b311ccd [#556] Replace TransactionRepository.findAllSentTransactions with new method
Make minedHeight optional
2023-01-02 12:29:01 +01:00
Michal Fousek 45d6260688 [#556] Remove unused TransactionRepository.findEncodedTransactionBy method 2022-12-23 22:08:45 +01:00
Michal Fousek 526e8e87b8
Merge pull request #688 from Chlup/556_2_replace_findTransactions
[#556] Replace TransactionRepository.findTransactions methods with new methods
2022-12-23 22:06:22 +01:00
Michal Fousek 27188e7e80 [#556] Replace TransactionRepository.findTransactions methods with new methods 2022-12-22 10:35:11 +01:00
Michal Fousek 92256c108f [#556] Replace TransactionRepository.findAllReceivedTransactions with new method 2022-12-22 09:46:54 +01:00
Michal Fousek ef2d904260 [#556] Update API of TransactionRepository and make tests work
This change is prerequisity for future changes. API of
`TransactionRepository` is changed a bit. Now it's possible to filter
transaction by kind (all/sent/received) event when
`TransactionNG.Overview` structure is returned.

Tests are now fixed for new methods and new objects. For this mocked
database `darkside_data.db` had to be updated to the latest scheme used
by the Rust code.
2022-12-21 12:33:20 +01:00
Michal Fousek 6c5bf8de27 [#677] Add support for wipe into SDK
Closes #677

- Previously when keychain wanted to do wipe wallet had to do it on it's
  own. Which isn't good. Now wallet can simply call
  `synchronizer.wipe()` and it's done.
- This change brings support for wipe. `Synchronizer` has new method
  `wipe()` which takes care of everything. All the database connections
  are closed. Databases are removed. `InternalSyncProgress` is reset.

Change condition when wipe can run
2022-12-20 13:41:20 +01:00
Michal Fousek cb07484c45 [#556] Replace TransactionRepository.findBy methods with new transaction method 2022-12-19 09:28:28 +01:00
Michal Fousek 999ee1e204 [#556] Define structures for db view and add new repository methods
- Structures that represent transactions DB views are newly define.
- New structures are:
  - TransactionNG.Overview
  - TransactionNG.Received
  - TransactionNG.Sent

- `TransactionNG` is used as temporary name. `Transaction` is existing
  structure in the code. `TransactionNG` will be eventually renamed to
  `Transaction` once current `Transaction` is removed from code.

- New methods that are using new structures are added to
  `TransactionRepository`. Those are slightly changed compared to
  predecessors. Those are now little bit simpler to use.

Address review comments
2022-12-15 22:19:14 +01:00
Michal Fousek e01c83690f [#646] SDK sync process resumes correctly
- Previously we had one range for each sync which was used for each
  phase of sync process. Newly there is separate range for each phase of
  the sync process.
- I added `InternalSyncProgress` utility. This utility tracks progress
  of some phases. And it is able to compute right ranges which should be
  used for syncing.
- Some unused download code from `CompactBlockProcessor` is removed.

Fix tests

Address review comments

- Rebase to master
- Update how range for `processBatchFinished()` is computed.
- Refactor `InternalSyncProgress`
- Add tests for `InternalSyncProgress`

Address review comments

Change how latest downloaded block is tracked

- Cache DB is no longer used to track which block was downloaded as
  latest. `InternalSyncProgress` is used to do that.
- Thanks to that #660 is fixed. And cache DB can be completely removed
  after sync process.
- Added sleep(1) to some darkside tests when latest block height is set.
  Otherwise lightwalletd in darkside mode doesn't have time to setup and
  tests are flaky.

Fix TransactionEnhancementTests.testBasicEnhancement test
2022-12-12 14:27:30 +01:00
Michal Fousek 4e5cb505fa [#488] Delete cache db when sync ends
Closes #488

- When sync process is finished cache DB is removed from disk.
- Latest downloaded block is preserved becuase it's required to computes
  in which phase to resume sync process.
- `SDKSynchronizer.latestDownloadedHeight` public API is removed
  it makes no longer sense. This API was reading from cache DB.
2022-11-15 10:31:59 +01:00
Lukas Korba 9b6ff51b29
[#492] Get rid of blocking API (#551)
- blocking API removed, only latestBlockHeight() stayed
- non-blocking closure based API removed
- unit tests updated to use async API

[#492] Get rid of blocking API (#551)

- forgotten commented code cleaned up
- some comments were still mentioning result (completion closure), removed
2022-10-03 20:05:11 -03:00
Lukas Korba 742e6bd8ec
[#464] CompactBlockStorage To async/await (#494)
- await/async APIs provided
- async throws unit tests using new API implemented

[464] CompactBlockStorage To async/await (494)

- removed deprecated closure APIs
- upgraded use of the async APIs
- tests updated
2022-08-26 14:52:12 -03:00
Daniel Haight 86d1257ab5 Remove dependecy on rust sources 2022-02-28 17:03:20 +00:00