And also to GetBlockRange. This requires an updated version of zcashd
`getblock` RPC that returns these values. These values are written to
the compact block cache (/var/lib/lightwalletd/db/main/blocks), but of
course older cached compact blocks won't have these values. To get these
values into all cached blocks, shut down lightwalletd, remove that file,
and restart (with, of course, an updated version of zcashd running).
In PR 412 (darksidewallet fixes for tx v5), I added a failure condition
to the GetBlock gRPC based on a block being requested always being in
the cache. I believe my thinking was that since there is no asynchronous
independent zcashd, if it wasn't in the cache, it never would be. I
failed to take into account that the block ingestor takes time to run,
and a test can run quickly enough that the block ingestor hasn't had
time to process the block. I reproduced the problem by commenting out
the starting of the block ingestor.
I think that error condition can be removed, which is what this commit
does. I tested by leaving the block ingestor commented out, so that
GetBlock has to use the RPC interface to get the block, and it works.
See issue 397. If the block cache is still populating (lightwalletd is
syncing with zcashd), behave the same as if the cache was fully
populated, other than performance.
This turned out to affect only the GetLatestBlock() gRPC. Previously, it
would return the height and hash of the latest block in the cache. After
this commit, it queries zcashd using the getblockchaininfo, which
contains both of those values.
GetBlock() (and GetBlockRange()) already worked correctly; if the
requested block isn't in the cache, it requests it from zcashd.
Darkside test framework broke due to the V5 txid changes (issue 392).
This change enhances darksideRawRequest("getblock") to allow the
argument to be either a height or a block hash, rather than only a
height.
Fixes issue 408.
This bug was introduced by PR 393, which changed how txids are
determined. That PR changed each call to the zcash getblock call into a
pair of calls, the first to get the raw block data, the second to
retrieve the txids in the block. (Unfortunately, you can't get both in a
single getblock RPC.) But this ordering introduced a timing window in
which the block at the given height can change, if a reorg occurred
between the two calls.
This PR reorders the getblock calls, so that the first call gets the
transaction IDs, which also happens to return the block hash, so then
the second getblock call can specify the block hash, rather than the
height. This ensures that the two RPC calls return consistent data,
definitely the same block.
Now "go test ./..." passes, it had broken due to the V5 transaction
(NU5) changes in #392. I didn't have time to fix the tests at the time.
Also, this fixes a few lint problems.
This causes lightwalletd to discard cached blocks at the given height
and beyond. This in turn causes it to re-fetch those blocks from zcashd.
It's similar to --redownload, except that option discards all blocks
(equivalent to --sync-from-height 0, but the existing --redownload is
retained for compatibility).
This is mostly intended for testing. It's sometimes useful to force the
node to (re)sync some recent blocks, but redownloading all of them takes
around an hour.
This is a shortcut fix. Instead of replicating the zip244 transaction
logic that's implemented in librustzcash:
zcash_primitives/src/transaction/components/orchard.rs
cheat by calling getblock with verbose level 1, which prints the txid
of each transaction. This currently requires calling getblock twice,
once for the raw hex (as we have always done), and again to get the
transaction IDs. An easy improvement would be to add the raw hex to
verbosity 1 result. (Or have a new verbosity that shows both.)
Instead of the Sleep function pointer being a standalong global
variable, move it into a new Time struct, and add a Now function
pointer, so that time.Now() can be mocked. Time.Now() isn't used yet.
This will be cleaner if we need to mock more time-related functions in
the future.
This commit is based on adityapk00 streaming mempool interface but
avoids using goroutines, which are difficult to reason about.
Co-authored-by: Aditya Kulkarni <adityapk@gmail.com>
Add a new darksidewallet gRPC to add a utxo that can be returned by the
production GetAddressUtxos, for example:
grpcurl -plaintext -d '{"address":"t1g1HQJuwDoStGrYYLj8VhLu1J5igd8TNXV","txid":"1zjB42Z7FtwRZOBNMlTubCgx9l3dsZSqXxmWpuZXJto=","script": "dqkU8saQsCVS4mNwcByoGCtfOaHFaCiIrA==","valueZat": "3010000","height": "686773"}' localhost:9067 cash.z.wallet.sdk.rpc.DarksideStreamer/AddAddressUtxo
Then the following returns this entry:
grpcurl -plaintext -d '{"startHeight":0,"maxEntries":2,"addresses":["t1g1HQJuwDoStGrYYLj8VhLu1J5igd8TNXV"]}' localhost:9067 cash.z.wallet.sdk.rpc.CompactTxStreamer/GetAddressUtxos
You can also clear the list of entries:
grpcurl -plaintext localhost:9067 cash.z.wallet.sdk.rpc.DarksideStreamer/ClearAddressUtxo
PR 320 introduced a bug that causes the `blocks` and `lengths` database
files to be located one directory level higher than it they should be.
The bug doesn't cause any functional problem, it only makes the
lightwalletd do more work (re-download the block cache), and it also
makes it not possible to switch between testnet and mainnet.
This patch locates the database files back where they belong.
- improved corruption recovery (don't back up by just 1 block at a time)
- move darksidewallet gRPCs to their own .proto file
- this force-push removes my commits for reorg testing using SetState
Co-authored-by: Aditya Kulkarni <adityapk@gmail.com>
also to pick up changes to walletrpc/service.proto
(regenerate compact_formats.pb.go service.pb.go)
cd walletrpc
protoc compact_formats.proto --go_out=plugins=grpc:.
protoc service.proto --go_out=plugins=grpc:.
ran go mod tidy && go mod vendor
Jack Grigg
Larry Ruane
sasha
Steven Smith
Ben Wilson