This also modifies the internal `common.ZcashdRpcReplyGetrawtransaction`
type to ensure that the reinterpretation of the `-1` height value from
a `getrawtransaction` response is not platform-dependent.
This has been broken for a long time. If, when starting up, lightwalletd
discovers that the backend node (zebrad or zcashd) has not yet synced to
the Sapling activation height (419200 for mainnet, 280000 for testnet),
it should wait until that happens, then sync along with the backend
node. But instead, lightwalletd gets stuck indefinitely. The only way
to escape the hang is to stop and restart lightwalletd.
This commit fixes the problem; the block ingestor retries every 2
minutes (which was always the intention), rather than trying once and
then giving up.
This fixes issue #480. Enabling this option decreases the performance of
gRPCs GetBlock and GetBlockRange, but conserves disk space (the size of
the database is currently around 17GB).
Note, specifying this option will also delete the existing cache (so if
you start again without --nocache, the cache will need to be recreated).
Reduce the startup time from a couple of minutes to a few seconds (or
less) by removing the deserialization of all compact blocks in the disk
cache. This is an alternative to PR 482, see comment there for details.
No functional changes.
As pointed out in lightwalletd/pull/469, the errors package we're using
has been deprecated, so this commit updates the code to use "errors"
instead of "github.com/pkg/errors". Same with "io/ioutil", which has
been superceded by "os".
In production mode, the `GetTreeState` gRPC supports block specification
by either height or block hash. But until now, darkside emulation only
supports height. This commit adds support for block hash.
It also allows entries to be deleted (darkside `RemoveTreeState`) by
block hash (not just height).
And also to GetBlockRange. This requires an updated version of zcashd
`getblock` RPC that returns these values. These values are written to
the compact block cache (/var/lib/lightwalletd/db/main/blocks), but of
course older cached compact blocks won't have these values. To get these
values into all cached blocks, shut down lightwalletd, remove that file,
and restart (with, of course, an updated version of zcashd running).
In PR 412 (darksidewallet fixes for tx v5), I added a failure condition
to the GetBlock gRPC based on a block being requested always being in
the cache. I believe my thinking was that since there is no asynchronous
independent zcashd, if it wasn't in the cache, it never would be. I
failed to take into account that the block ingestor takes time to run,
and a test can run quickly enough that the block ingestor hasn't had
time to process the block. I reproduced the problem by commenting out
the starting of the block ingestor.
I think that error condition can be removed, which is what this commit
does. I tested by leaving the block ingestor commented out, so that
GetBlock has to use the RPC interface to get the block, and it works.
See issue 397. If the block cache is still populating (lightwalletd is
syncing with zcashd), behave the same as if the cache was fully
populated, other than performance.
This turned out to affect only the GetLatestBlock() gRPC. Previously, it
would return the height and hash of the latest block in the cache. After
this commit, it queries zcashd using the getblockchaininfo, which
contains both of those values.
GetBlock() (and GetBlockRange()) already worked correctly; if the
requested block isn't in the cache, it requests it from zcashd.
If the darkside "getblock" RPC handler is given a block hash as argument
(rather than a height), it was deserializing all active blocks
sequentially to find the one with the given hash. This is slow if there
is a large of blocks.
I considered the most general solution of adding a block hash map to
darkside so that any block could be looked up by its hash, but that
turns out to be a lot of effort. So instead, maintain a one-block cache
because a block is always looked up by hash immediately after being
looked up by height (this was changed in PR 412).
This commit adds support for adding/removing/clearing TreeState
structs for lightwalletd to return as if they were TreeState
messages requested to Zcashd for a given height.
Closes#390
Darkside test framework broke due to the V5 txid changes (issue 392).
This change enhances darksideRawRequest("getblock") to allow the
argument to be either a height or a block hash, rather than only a
height.
Fixes issue 408.
This bug was introduced by PR 393, which changed how txids are
determined. That PR changed each call to the zcash getblock call into a
pair of calls, the first to get the raw block data, the second to
retrieve the txids in the block. (Unfortunately, you can't get both in a
single getblock RPC.) But this ordering introduced a timing window in
which the block at the given height can change, if a reorg occurred
between the two calls.
This PR reorders the getblock calls, so that the first call gets the
transaction IDs, which also happens to return the block hash, so then
the second getblock call can specify the block hash, rather than the
height. This ensures that the two RPC calls return consistent data,
definitely the same block.
This commit extends The darkside's stub zcashd rpc handler to handle the
verbose form of `getblock` (that returns the txids of all transactions
in the block) and the rest of the darkside code has to set the txids in
the blocks and individual transactions it returns. Since lightwalletd
doesn't yet support V5 txid calculation, we use the old one, so it's not
accurate for V5 but that doesn't matter (during darkside wallet testing,
the code under test can't detect incorrect txids).
Now "go test ./..." passes, it had broken due to the V5 transaction
(NU5) changes in #392. I didn't have time to fix the tests at the time.
Also, this fixes a few lint problems.
This causes lightwalletd to discard cached blocks at the given height
and beyond. This in turn causes it to re-fetch those blocks from zcashd.
It's similar to --redownload, except that option discards all blocks
(equivalent to --sync-from-height 0, but the existing --redownload is
retained for compatibility).
This is mostly intended for testing. It's sometimes useful to force the
node to (re)sync some recent blocks, but redownloading all of them takes
around an hour.
This is a shortcut fix. Instead of replicating the zip244 transaction
logic that's implemented in librustzcash:
zcash_primitives/src/transaction/components/orchard.rs
cheat by calling getblock with verbose level 1, which prints the txid
of each transaction. This currently requires calling getblock twice,
once for the raw hex (as we have always done), and again to get the
transaction IDs. An easy improvement would be to add the raw hex to
verbosity 1 result. (Or have a new verbosity that shows both.)
Instead of the Sleep function pointer being a standalong global
variable, move it into a new Time struct, and add a Now function
pointer, so that time.Now() can be mocked. Time.Now() isn't used yet.
This will be cleaner if we need to mock more time-related functions in
the future.
This commit is based on adityapk00 streaming mempool interface but
avoids using goroutines, which are difficult to reason about.
Co-authored-by: Aditya Kulkarni <adityapk@gmail.com>
Add a new darksidewallet gRPC to add a utxo that can be returned by the
production GetAddressUtxos, for example:
grpcurl -plaintext -d '{"address":"t1g1HQJuwDoStGrYYLj8VhLu1J5igd8TNXV","txid":"1zjB42Z7FtwRZOBNMlTubCgx9l3dsZSqXxmWpuZXJto=","script": "dqkU8saQsCVS4mNwcByoGCtfOaHFaCiIrA==","valueZat": "3010000","height": "686773"}' localhost:9067 cash.z.wallet.sdk.rpc.DarksideStreamer/AddAddressUtxo
Then the following returns this entry:
grpcurl -plaintext -d '{"startHeight":0,"maxEntries":2,"addresses":["t1g1HQJuwDoStGrYYLj8VhLu1J5igd8TNXV"]}' localhost:9067 cash.z.wallet.sdk.rpc.CompactTxStreamer/GetAddressUtxos
You can also clear the list of entries:
grpcurl -plaintext localhost:9067 cash.z.wallet.sdk.rpc.DarksideStreamer/ClearAddressUtxo
wellweek
jimmycathy
Emerson Hall
Kris Nuttycombe
Daira-Emma Hopwood
Larry Ruane
Jack Grigg
Francisco Gindre
sasha
Steven Smith
Aditya Kulkarni