Commit Graph

18 Commits

Author SHA1 Message Date
Francisco Gindre fad3aae174
- [#679] Implementation of the File-system based block cache (#679)
Closes https://github.com/zcash/ZcashLightClientKit/issues/697
Closes https://github.com/zcash/ZcashLightClientKit/issues/720
Closes https://github.com/zcash/ZcashLightClientKit/issues/587
Closes https://github.com/zcash/ZcashLightClientKit/issues/667
Closes https://github.com/zcash/ZcashLightClientKit/issues/443
Closes https://github.com/zcash/ZcashLightClientKit/issues/754

- [#790] Fix ShieldFundsTests

Closes #790
Removes comments on `ShieldFundsTests` since those issues have been fixed

Depends on zcash-light-client-ffi changes that adopt newer versions of
 librustzcash crates `zcash_primitives 0.10`, `zcash_client_backend 0.7`,
`zcash_proofs 0.10`, `zcash_client_sqlite 0.5.0`.

Also allows wallets to define a shielding_threshold and will set
foundations to customize minimum confirmations for balances, spends
and shielding operations.

**Test Bootstrapping**

- `ZcashCompactBlockDescriptor`: struct that holds functions to
describe blocks as filenames and compare those filenames

`ZcashCompactBlockDescriptor.live` has the actual implementation
but it can be replaced by mocks if needed on Tests

main implementations are held under `FSCompactBlockRepository.filenameDescription` and `FSCompactBlockRepository.filenameComparison` on a separate extention

`DirectoryListingProviders` provide two default implementations of listing a
directory deterministically. `FileManager` does not define a sorting and needs to be done in-memory by calling `.sorted()` on the resulting collection. If this
is a big toll on performance it can be changed to a POSIX implementation but
this is good for now.

`ZcashCompactBlockDescriptor` adds a `height` helper function to
turn a filename into the height of the block stored.

Implemented `func latestHeight() throws -> BlockHeight ` that
returns the blockheight by querying the cache directory in a sorted
fashion and getting the last value and turning the filename into a
`BlockHeight`

Added `Meta` struct to ZcashCompactBlock.

Tests implemented:
- `filterBlockFiles`
- `testClearTheCache`
- `testLatestHeightEmptyCacheThrows`
- `testLatestHeightEmptyCacheThrowsAsync`
- `testRewindEmptyCacheDoesNothing`
- `testRewindEmptyCacheDoesNothingAsync`
- `testWhenBlockIsStoredItFollowsTheDescribedFormat`
- `testWhenBlockIsStoredItFollowsTheFilenameConvention`
- `testGetLatestHeight`
- `testRewindDeletesTheRightBlocks` test
- `testPerformanceExample` test. This isn't a real performance test because the API doesn't work with async/await yet

adopts `shield_funds` shielding threshold parameter

Implements `initBlockMetadataDb` and fix tests
Renames dbCache parameter to `fsBlockDbRoot`. Builds but tests don't pass.
Removes cacheDb uses from code. Testing utilities still persist.
Added needed information in  MIGRATING and CHANGELOG.
Added helper to perform deletion of legacy db and creation a the
new file system backed cache.

Renames parameters and changes code where needed.
Network Constants turned into `enum` with static methods.
DeletelastDownloadedBlock helper from initializer
Removes CompactBlockStorage and CompactBlockEntity.
Implements `latestCachedBlockHeight` on rustbackend.

*Replaces dependencies on ZcashRustWelding with `FSMetadataStore`*

This allows the tests to not depend in a particular implementation
of either the MockRustBackend of or ZcashRustBackend. Also provides
a way to test errors properly and switch implementations of critical
areas like `writeBlocks`.
2023-02-02 13:58:12 -03:00
Michal Fousek 6bbaad4cb1 [#740] Refator blocks downloading in CompactBlockProcessor
Closes #740

`CompactBlockDownloading` protocol is renamed to `BlockDownloaderService`. `CompactBlockDownloader` is renamed to `BlockDownloaderServiceImpl`.

What is the motivation for this rename?
Goal of this change is to extract code from `CompactBlockProcessor` which is related to downloading. So naturaly it should be called "downloader".
But `CompactBlockDownloader` was already taken. After analysis of `CompactBlockDownloading` and `CompactBlockDownloader` I found out that
`CompactBlockDownloader` doesn't download blocks exactly. It's more like abstraction over `LightWalletService`. And it does very similar job to
`LightWalletService`. That is why I decided to rename it to "service".

What is the motivation for protoco/impl naming scheme?
First `BlockDownloaderService` protocol doesn't describe capability in my eyes. It describe what something is. Also `BlockDownloaderServicing` sounds
strange. I agree that add suffix "Impl" to class name isn't super nice. But it is acceptable. And it makes sense to use nicer type name
(e.g. `BlockDownloaderService`) for protocol because it's used on much more places in the code.

- Added `BlockDownloader` protocol and `BlockDownloaderImpl` class. The code related to downloading was exctracted to this class.
2023-01-31 15:24:41 +01:00
Lukas Korba f69da50ba9
[#747] Remove SampleLogger and replace it with OSLogger (#748)
- SampleLogger removed
- OSLogger used
- category default
2023-01-26 18:14:07 +01:00
Lukas Korba 372ea5af70
[#711] Add Swiftlint plug in to SDK (#722)
- plugin integrated
- errors resolved
- warnings resolved
- package.swift macos platform version 10_15 -> 12
2023-01-18 17:09:04 +01:00
Michal Fousek 349148cd38 [#657] Change how blocks are downloaded and processed
Closes #657

- Now SDK downloads N downloads and scan those. And repeat until sync is
  done. This safes lot of space of disk used to store downloaded blocks.
- Most changes are done in `CompactBlockProcessor.processNewBlocks`.
- `SyncRanges` structure is changed a bit.
- `SyncStatus` is changed. SDK now report that it's syncing with
  progress. It doesn't report each phase of the sync process separately.
  Also appropriate notifications were updated.

Address review comments

- Small changes
- Add tests to check computation of processing ranges for single loop
2022-12-19 09:35:09 +01:00
Michal Fousek e01c83690f [#646] SDK sync process resumes correctly
- Previously we had one range for each sync which was used for each
  phase of sync process. Newly there is separate range for each phase of
  the sync process.
- I added `InternalSyncProgress` utility. This utility tracks progress
  of some phases. And it is able to compute right ranges which should be
  used for syncing.
- Some unused download code from `CompactBlockProcessor` is removed.

Fix tests

Address review comments

- Rebase to master
- Update how range for `processBatchFinished()` is computed.
- Refactor `InternalSyncProgress`
- Add tests for `InternalSyncProgress`

Address review comments

Change how latest downloaded block is tracked

- Cache DB is no longer used to track which block was downloaded as
  latest. `InternalSyncProgress` is used to do that.
- Thanks to that #660 is fixed. And cache DB can be completely removed
  after sync process.
- Added sleep(1) to some darkside tests when latest block height is set.
  Otherwise lightwalletd in darkside mode doesn't have time to setup and
  tests are flaky.

Fix TransactionEnhancementTests.testBasicEnhancement test
2022-12-12 14:27:30 +01:00
Francisco Gindre 59dc56c9b4 Merge branch 'feature/zip-316-and-latest-upstream' into release/0.17.0-alpha.3 2022-11-01 15:09:39 -03:00
Francisco Gindre a12666e932 [#602] Improve error logging for InitializerError and RustWeldingError 2022-11-01 13:57:42 -03:00
Francisco Gindre a0d02c4d57 PR Suggestions
`XCAsyncTestCase` does not work as intended but `wait {}` does what
we need.

restored `needsToStartScanningWhenStopped` as a Bool var on actor

fixed Enhancement test
2022-10-31 20:17:21 -03:00
Francisco Gindre 01d85f5748 Fix compiler errors 2022-10-31 09:57:10 -03:00
Francisco Gindre dbd4b8934c Merge branch 'master' into merge-master 2022-10-31 09:34:38 -03:00
Francisco Gindre b5d659e8a8 Fix various tests and deleted some that are not useful anymore
Closes #588
Closes #584
Closes #589
Closes #591
2022-10-27 20:09:08 -03:00
Lukas Korba cfe71d5da2
[#523] Make a CompactBlockProcessor an Actor (#565)
- Sample app refactored for the processor being an actor
- tests refactored as well
- dark side tests fixed
- utilities separated to new file
- synchronizer's start and stop are no longer in async context
- updating the UI for the scan fixed
2022-10-27 12:51:38 +02:00
Francisco Gindre 7806b5114f
Use UnifiedSpendingKey for shielding and Spending (#535)
[#534] Use UnifiedSpendingKey for shielding and Spending

This commit implements the use of Unified Spending Keys for shielding
and spending as well as rolling Unified Addresses.

Users should obtain addresses by rolling them from the SDK.
USKs replace Sapling Extended Spending keys and TransparentAccountPrivKeys
when shielding or spending

Closes #534

Co-authored-by: Kris Nuttycombe <kris@nutty.land>

* Fix rebase issues

* PR Suggestion. Make `lastError` an Optional

* Fix test `testReOrgRemovesOutboundTxAndIsNeverMined`

Co-authored-by: Kris Nuttycombe <kris@nutty.land>
2022-10-02 23:11:17 -03:00
Kris Nuttycombe 905ee401d1 Use zcash_client_sqlite to manage migrations for the wallet db.
This change removes responsibility for maintaining the state of
the wallet database from `ZcashLightClientKit` in favor of using
the migration system now provided by librustzcash. This will help
to ensure that the structure of the database is kept consistent with
the functions that query and update the database state.

Co-authored-by: Francisco Gindre <francisco.gindre@gmail.com>
2022-08-24 09:38:42 -06:00
Francisco Gindre 2fcfa6fdbe
[#398] Make WalletBirthday an internal type (#414)
This commit makes Renames `WalletBirthday` to `Checkpoint` and
makes Checkpoint an internal type. Public ocurrences of this
type is replaced by `BlockHeight` (Int) and then retrieval of the
Checkpoint is deferred to the appropiate place in the code

Add an extension method to `BlockHeight` to get latest checkpoint
height present in the SDK's bundle

PR Suggestions:

Add test for integer overflow on JSON
File renamed from WalletBirthday to Checkpoint
2022-07-12 16:36:12 -03:00
Francisco Gindre 912abe0128
[#397] Checkpoint format that supports NU5 TreeStates (#399)
* [#397] Checkpoint format that supports NU5 TreeStates

Closes #397

This commit adds `orchardTree` value to WalletBirthday and renames
the value `tree` to `saplingTree`. It also modifies the checkpoint
JSON files accordingly. It adds `.testnet` and `.mainnet` checkpoints
for orchard activation height.

* rename `sapling-checkpoints` to `checkpoints`
2022-06-28 16:17:10 -03:00
Daniel Haight 86d1257ab5 Remove dependecy on rust sources 2022-02-28 17:03:20 +00:00