Commit Graph

147 Commits

Author SHA1 Message Date
Michal Fousek c6af1cace3
Merge pull request #702 from zcash/657_download_change
657 download change
2023-01-09 15:17:07 +01:00
Michal Fousek 382905b06a [#657] Fix typos 2023-01-09 13:29:43 +01:00
Michal Fousek abe647e28d Update how progress is reported while scanning blocks 2023-01-06 16:45:49 +01:00
Francisco Gindre 851023842f
[#612] Remove Support for Cocoapods (#706)
Our main dependency, Swift-GRPC is dropping support for Cocoapods from version 1.8.2.

Even though they make a great work at maintaining backwards compatibility, we will eventually be forced to drop Cocoapods support.

We've reached out partners like NighthawkApps, Horizontal Systems (Unstoppable) and Edge, who have reported not depending on Cocoapods.

We intend to end Cocoapods support on 0.18.0.

Closes #612
2023-01-06 12:26:00 -03:00
Michal Fousek 09e9148b88 [#657] Scan batch size is computed for each scan loop 2023-01-06 13:16:12 +01:00
Michal Fousek e8aa451b17 [#657] Update changelog 2023-01-05 15:13:27 +01:00
Michal Fousek becadf21c8 Merge branch 'main' into 657_download_change 2023-01-05 09:50:00 +01:00
Francisco Gindre f13322225d
Merge branch 'main' into release/0.17.5-beta 2023-01-04 10:52:44 -03:00
Francisco Gindre c0f8d46501 [#698] Updated Checkpoints release 0.17.5-beta 2023-01-03 17:55:02 -03:00
Michal Fousek c3b13d8367 [#657] Fix start of SDKSynchronizer when sync is already in process
Previously `SDKSynchronizer` simply refused to start while the sync
process was in progress. That could lead to some troubles.
`CompactBlockProcessor` has already mechanism which can handle this
exact situation. So now `SDKSynchronizer` starts `CompactBlockProcessor`
even when there is sync in progress.
2023-01-02 10:24:02 +01:00
Dimitris Apostolou 6c3423181d
Fix typos 2022-12-31 12:50:16 +02:00
Michal Fousek 4b4416b1e1 Merge branch 'master' into 657_merge_master
- Changed how metrics are pushed when downloading and scanning blocks.
2022-12-22 15:35:11 +01:00
Lukas Korba 3c3b97ca42
[#663] Create a benchmarking section on the Demo App that can be automated (#681)
- SDKMetrics singleton for the measurement
- download, validate and scan blocks supported
- Sample app refactored to use SDKMetrics, connected to appropriate notifications and labels
- enhancement metric added
- total sync time metric added
- sample app extended to show summary time and operations summaries
- unit tests for the SDKMetrics
- doc for the SDKMetrics and our approach for the performance testing in general
- changelog update
2022-12-21 13:30:05 +01:00
Michal Fousek 6c5bf8de27 [#677] Add support for wipe into SDK
Closes #677

- Previously when keychain wanted to do wipe wallet had to do it on it's
  own. Which isn't good. Now wallet can simply call
  `synchronizer.wipe()` and it's done.
- This change brings support for wipe. `Synchronizer` has new method
  `wipe()` which takes care of everything. All the database connections
  are closed. Databases are removed. `InternalSyncProgress` is reset.

Change condition when wipe can run
2022-12-20 13:41:20 +01:00
Michal Fousek 349148cd38 [#657] Change how blocks are downloaded and processed
Closes #657

- Now SDK downloads N downloads and scan those. And repeat until sync is
  done. This safes lot of space of disk used to store downloaded blocks.
- Most changes are done in `CompactBlockProcessor.processNewBlocks`.
- `SyncRanges` structure is changed a bit.
- `SyncStatus` is changed. SDK now report that it's syncing with
  progress. It doesn't report each phase of the sync process separately.
  Also appropriate notifications were updated.

Address review comments

- Small changes
- Add tests to check computation of processing ranges for single loop
2022-12-19 09:35:09 +01:00
Francisco Gindre 50e63232e3 New Checkpoints + CHANGELOG.md 2022-12-12 17:39:05 -03:00
Michal Fousek e01c83690f [#646] SDK sync process resumes correctly
- Previously we had one range for each sync which was used for each
  phase of sync process. Newly there is separate range for each phase of
  the sync process.
- I added `InternalSyncProgress` utility. This utility tracks progress
  of some phases. And it is able to compute right ranges which should be
  used for syncing.
- Some unused download code from `CompactBlockProcessor` is removed.

Fix tests

Address review comments

- Rebase to master
- Update how range for `processBatchFinished()` is computed.
- Refactor `InternalSyncProgress`
- Add tests for `InternalSyncProgress`

Address review comments

Change how latest downloaded block is tracked

- Cache DB is no longer used to track which block was downloaded as
  latest. `InternalSyncProgress` is used to do that.
- Thanks to that #660 is fixed. And cache DB can be completely removed
  after sync process.
- Added sleep(1) to some darkside tests when latest block height is set.
  Otherwise lightwalletd in darkside mode doesn't have time to setup and
  tests are flaky.

Fix TransactionEnhancementTests.testBasicEnhancement test
2022-12-12 14:27:30 +01:00
Francisco Gindre 3b7202c922
Fix `testShieldFunds()` dataset loading issue. (#659)
There's a problem withstanding:

````
▿ RustWeldingError
  ▿ genericError : 1 element
    - message : "Error while fetching transparent balances for AccountId(0): Invalid column type Null at index: 0, name: address"
````

Which is fixed by commit `d0297bff0cdaaff42ad26f3be5bc261f6c828e8f` of "https://github.com/zcash/librustzcash".

nil-coalesce thrown errors on test so it reaches the end.

Make balance calls throwing to show underlying errors but nil-coalesce in non-throwing APIs

Add missing fulfill() calls

Add changelog entry

document and handle errors on getting shielded balance before
attempting to download sapling parameters.
2022-12-12 10:00:31 -03:00
Michal Fousek cc2c0ffa65 [#660] Fix the situation when any rewind causes full rescan
This is just hotfix that disables removing of cache DB.

Bump Cocoapods version to 0.17.2-beta
2022-12-06 15:12:34 -03:00
Francisco Gindre 1c672cad59 [#625] Release 0.17.1-beta
Closes #625

See CHANGELOG.md for details
2022-11-30 11:08:48 -03:00
Michal Fousek 7c07b91c5b [#651] Fix rewind when it's called during sync
- When rewind is called during sync then exception is thrown.
2022-11-29 08:33:18 +01:00
Lukas Korba 6029a75038
[#637] Make sapling parameter download part of processing blocks (#650)
- sync process enhanced to try to download sapling params if needed
- tests fixed
- failing import Crypto -> import CryptoKit
- download params only if there are sapling funds
2022-11-29 07:38:28 +01:00
Francisco Gindre 7c9fddedf4
Add benchmarking info to SyncBlocksViewController (#649)
Fix Github action
2022-11-28 13:42:07 -03:00
Lukas Korba cc99fbad30
[#590] Fix or delete CompactBlockProcessorTests disabled tests it not needed (#653)
- The tests were fine in the end but Stream needed to be finished in any way, caused "for await" syntax to never finish
2022-11-28 15:08:55 +01:00
Lukas Korba d36c1fbc14
[#631] Verify SHA1 correctness of Sapling files after downloading (#643)
- Verification of the SHA1 after downloading the file(s)
- Private API is encapsulated in private extension
2022-11-23 09:47:56 +01:00
Michal Fousek 96a205644a
Merge pull request #642 from Chlup/sdksynchronizer_status_publisher
Change lastState property to Publisher from Subject in SDKSynchronizer
2022-11-23 09:11:36 +01:00
Francisco Gindre b350f6b220
[#639] Provide an API to estimate TextMemo length limit correctly (#640)
This provides a method for clients to handle UI for text memo
limit correctly handling memos that are visibly fit within the
capacity defined in ZIP-302 but when encoded into UTF-8 bytes
their size exceeds such limit.

Closes #639
2022-11-22 14:02:39 -03:00
Michal Fousek 1f2ea5e716 Change lastState property to Publisher from Subject in SDKSynchronizer
It's not good practice to make internal combine subjects accessible from
outside of SDK. So `lastState` is changed from `CurrentValueSubject` to
`Publisher`. This doesn't have any effect on public API. It just makes
SDK safer.
2022-11-22 14:10:27 +01:00
Michal Fousek 44a6970b73 [#616] Dont send notification from SDK on main thread
Closes #616

- Added small helper `NotificationSender` which is used to send
  notifications.
- No notificaiton is sent on main thread.
- Change how `status` is stored and update in `SDKSynchronizer`. It is
  now protected by lock. Before this wasn't needed because status was
  updated from main thread everywhere.
- `SyncBlocksViewController` in the sample app now uses
  `SDKSynchronizer` instead of `CompactBlockProcessor` directly.
2022-11-21 16:33:42 +01:00
Francisco Gindre 667bda24a7
Merge pull request #618 from Chlup/488_delete_cache
[#488] Delete cache db when sync ends
2022-11-17 08:24:53 -03:00
Francisco Gindre d15cd5e33c [#420] release 0.17.0-beta
Closes #420
    Closes #634
2022-11-16 20:44:10 -03:00
Francisco Gindre 725e5ff291 [#634] Set ZIP-317 flag on sending and shielding as false for 0.17.0
Closes #634

    This turns off ZIP-317 fee strategy in favor of ZIP-313 fixed fees
    until this can be tested thoroughly
2022-11-16 20:27:12 -03:00
Francisco Gindre a85787ff59 Add Checkpoints for Testnet and Mainnet 2022-11-15 09:31:34 -03:00
Michal Fousek 4e5cb505fa [#488] Delete cache db when sync ends
Closes #488

- When sync process is finished cache DB is removed from disk.
- Latest downloaded block is preserved becuase it's required to computes
  in which phase to resume sync process.
- `SDKSynchronizer.latestDownloadedHeight` public API is removed
  it makes no longer sense. This API was reading from cache DB.
2022-11-15 10:31:59 +01:00
Francisco Gindre 6208beec5c [#621] Adds a synchronizer state subject with the last available
state for this synchronizer.

also reports the SychronizerState on synchonizerStarted notification.

Add tests that verify the state being reported properly

Fix `BlockBatchValidationTests`
2022-11-14 14:44:33 -03:00
Francisco Gindre b9e00055e9 [#614] make prepare(with:) and get{pool_type}Address() synchronous
Closes #614

fixes travis ci

Removed `setStartHeight` function

I’m going back in time and this setHeight is not something we are using anywhere else than in the prepare function. This comes from the times we had `initialize(seed:birthday) and we don’t have that anymore

See diff 246d10edaa (diff-414771774e10bc81260094ffcdc7b310a3be48758b2abd2bfae831c83912c02c)

The `func setStartHeight(_ startHeight: BlockHeight)` function assumes that
there might not be a wallet birthday set up or that it might be changed
in-flight which are things that are no longer happening.

remove test warning

Fix test compiler error
2022-11-11 14:07:45 -03:00
Michal Fousek 5e723566ba [#619] Correctly resume syncing process in scanning phase
Closes #619

- When computing next action in `NextStateHelper.nextStateAsync` now
  scanning state is also taken into consideration.
- When processing blocks download phase is skipped when it's detected
  that everything is already downloaded.
2022-11-11 14:54:44 +01:00
Francisco Gindre 9607f0cefa - [#610] Release 0.17.0-beta.rc1 2022-11-07 14:21:22 -03:00
Francisco Gindre 59dc56c9b4 Merge branch 'feature/zip-316-and-latest-upstream' into release/0.17.0-alpha.3 2022-11-01 15:09:39 -03:00
Francisco Gindre a12666e932 [#602] Improve error logging for InitializerError and RustWeldingError 2022-11-01 13:57:42 -03:00
Francisco Gindre c0f1aa959a Fix SendViewController `viewDidLoad` 2022-10-31 20:45:58 -03:00
Francisco Gindre a0d02c4d57 PR Suggestions
`XCAsyncTestCase` does not work as intended but `wait {}` does what
we need.

restored `needsToStartScanningWhenStopped` as a Bool var on actor

fixed Enhancement test
2022-10-31 20:17:21 -03:00
Francisco Gindre 01d85f5748 Fix compiler errors 2022-10-31 09:57:10 -03:00
Francisco Gindre dbd4b8934c Merge branch 'master' into merge-master 2022-10-31 09:34:38 -03:00
Michal Fousek 571166cc8a [#579] Fix database lock
Closes #579

This is what happens when database was locked:
- App is syncing.
- While rustBackend.scanBlocks(…) is in progress app goes to background. Background task is launched and
  ECC works for some time in background.
- When background task is finished synchronizer.stop() is called.
- Inside CompactBlockProcessor.stop()is called. In this method cancelableTask is canceled and state is
  set to stopped no matter what. And this is the problem. Because even when canceled is called rust
  scanning isn’t canceled. But state doesn’t care about it. And next time when app is launched new
  synchronisation is started and now you have processing code running in two threads causing DB lock.

So I moved setting state to stopped to processNewBlocks method. It's set
after the work is canceled and really stopped. When doing this there is
still one problem. When app goes to foreground new sync is not started
and current one is just stopped.
    So I added new variable and condition. And when this variable is set
to true new syncing process starts when the previous one is canceled.
This should cover all the problems.
2022-10-31 09:57:32 +01:00
Francisco Gindre b5d659e8a8 Fix various tests and deleted some that are not useful anymore
Closes #588
Closes #584
Closes #589
Closes #591
2022-10-27 20:09:08 -03:00
Lukas Korba cfe71d5da2
[#523] Make a CompactBlockProcessor an Actor (#565)
- Sample app refactored for the processor being an actor
- tests refactored as well
- dark side tests fixed
- utilities separated to new file
- synchronizer's start and stop are no longer in async context
- updating the UI for the scan fixed
2022-10-27 12:51:38 +02:00
Francisco Gindre 1d97ad920b [#581] getTransparentBalanceForAccount error not handled 2022-10-26 20:26:37 -03:00
Francisco Gindre 545380b3e2 [#577] Fix: reduce batch size when reaching increased load part of the chain
Currently a 100 batch can take up to 3 minutes to scan. decrease the size of the batches when the increased load part of the chain is reached until a faster sync is implemented

See related counterpart to remove it issue #576

Closes #577
2022-10-25 18:21:32 -03:00
Francisco Gindre 5de1b50b29 make Memo and MemoBytes parameters nullable so they can be omitted when sending to transparent receivers.
update libzcashlc
2022-10-24 22:49:04 -03:00