In production mode, the `GetTreeState` gRPC supports block specification
by either height or block hash. But until now, darkside emulation only
supports height. This commit adds support for block hash.
It also allows entries to be deleted (darkside `RemoveTreeState`) by
block hash (not just height).
We also remove the `i < maxEntries` bound on returned results, as the
`limit` parameter already causes `zcashd` to bound its returned entries.
Closeszcash/lightwalletd#444.
And also to GetBlockRange. This requires an updated version of zcashd
`getblock` RPC that returns these values. These values are written to
the compact block cache (/var/lib/lightwalletd/db/main/blocks), but of
course older cached compact blocks won't have these values. To get these
values into all cached blocks, shut down lightwalletd, remove that file,
and restart (with, of course, an updated version of zcashd running).
In PR 412 (darksidewallet fixes for tx v5), I added a failure condition
to the GetBlock gRPC based on a block being requested always being in
the cache. I believe my thinking was that since there is no asynchronous
independent zcashd, if it wasn't in the cache, it never would be. I
failed to take into account that the block ingestor takes time to run,
and a test can run quickly enough that the block ingestor hasn't had
time to process the block. I reproduced the problem by commenting out
the starting of the block ingestor.
I think that error condition can be removed, which is what this commit
does. I tested by leaving the block ingestor commented out, so that
GetBlock has to use the RPC interface to get the block, and it works.
See issue 397. If the block cache is still populating (lightwalletd is
syncing with zcashd), behave the same as if the cache was fully
populated, other than performance.
This turned out to affect only the GetLatestBlock() gRPC. Previously, it
would return the height and hash of the latest block in the cache. After
this commit, it queries zcashd using the getblockchaininfo, which
contains both of those values.
GetBlock() (and GetBlockRange()) already worked correctly; if the
requested block isn't in the cache, it requests it from zcashd.
If the darkside "getblock" RPC handler is given a block hash as argument
(rather than a height), it was deserializing all active blocks
sequentially to find the one with the given hash. This is slow if there
is a large of blocks.
I considered the most general solution of adding a block hash map to
darkside so that any block could be looked up by its hash, but that
turns out to be a lot of effort. So instead, maintain a one-block cache
because a block is always looked up by hash immediately after being
looked up by height (this was changed in PR 412).
This commit adds support for adding/removing/clearing TreeState
structs for lightwalletd to return as if they were TreeState
messages requested to Zcashd for a given height.
Closes#390
Darkside test framework broke due to the V5 txid changes (issue 392).
This change enhances darksideRawRequest("getblock") to allow the
argument to be either a height or a block hash, rather than only a
height.
Fixes issue 408.
This bug was introduced by PR 393, which changed how txids are
determined. That PR changed each call to the zcash getblock call into a
pair of calls, the first to get the raw block data, the second to
retrieve the txids in the block. (Unfortunately, you can't get both in a
single getblock RPC.) But this ordering introduced a timing window in
which the block at the given height can change, if a reorg occurred
between the two calls.
This PR reorders the getblock calls, so that the first call gets the
transaction IDs, which also happens to return the block hash, so then
the second getblock call can specify the block hash, rather than the
height. This ensures that the two RPC calls return consistent data,
definitely the same block.
This commit extends The darkside's stub zcashd rpc handler to handle the
verbose form of `getblock` (that returns the txids of all transactions
in the block) and the rest of the darkside code has to set the txids in
the blocks and individual transactions it returns. Since lightwalletd
doesn't yet support V5 txid calculation, we use the old one, so it's not
accurate for V5 but that doesn't matter (during darkside wallet testing,
the code under test can't detect incorrect txids).
Now "go test ./..." passes, it had broken due to the V5 transaction
(NU5) changes in #392. I didn't have time to fix the tests at the time.
Also, this fixes a few lint problems.
It's better not to have the vendor directory; it was needed before
modules worked well in Go. These days, the `go.mod` and `go.sum` files
nail down the exact versions of each dependency as well as the vendor
directory did. There are many modern Go projects, such as
docker/compose, that don't have a vendor directory.
So don't run `go vendor`, or if you do, don't git-add the files that
are downloaded.
Another advantage of removing the vendor directory is that the repo is
only 9% of its former size.
Another thing I did as part of this commit is to remove all the
`require` lines in `go.mod` and then ran `go tidy`. This repopulated the
require lines but with the latest versions. This may fix#378 but I'm
not sure (because I still see `gopkg.in/yaml.v2 v2.4.0 // indirect`
in `go.mod`.
This causes lightwalletd to discard cached blocks at the given height
and beyond. This in turn causes it to re-fetch those blocks from zcashd.
It's similar to --redownload, except that option discards all blocks
(equivalent to --sync-from-height 0, but the existing --redownload is
retained for compatibility).
This is mostly intended for testing. It's sometimes useful to force the
node to (re)sync some recent blocks, but redownloading all of them takes
around an hour.
This is a shortcut fix. Instead of replicating the zip244 transaction
logic that's implemented in librustzcash:
zcash_primitives/src/transaction/components/orchard.rs
cheat by calling getblock with verbose level 1, which prints the txid
of each transaction. This currently requires calling getblock twice,
once for the raw hex (as we have always done), and again to get the
transaction IDs. An easy improvement would be to add the raw hex to
verbosity 1 result. (Or have a new verbosity that shows both.)
oscar-pepper
dependabot[bot]
Larry Ruane
ebfull
Jack Grigg
Yasser
Dimitris Apostolou
JG
Francisco Gindre
sasha
Steven Smith