* node: Fix issue where transfers that were loaded from the DB did not add
a flow-cancel transfer on the TargetChain
Flow-canceling is done in the `ProcessMsgForTime` loop when a new
message occurs. However, this was not done when a node restarted and
reloaded transfers from the past 24 hours. As a result it was possible
for the node to calculate a result that showed that the outgoing
transfers for an emitter chain exceeded the daily limit. In effect this
is true but only with the condition that there was incoming flow to
allow this to happen. This appeared to violate an invariant and so the
node did not start properly.
node: Add unit tests when reloading flow cancel transactions from the
database
node: fix lint errors in governor_test.go
* node: Add a command-line flag to enable or disable flow-canceling on restart
Added a command-line flag to enable or disable flow-canceling when
starting the node. This should allow Guardians to disable flow canceling
in the case of future bugs or during a security incident. This should
prevent the need to rollback to earlier Guardian versions. (@mdulin2 )
* node: Use deterministic iteration order over chains when changing Governor state
- Adds a field that stores a sorted slice of chain IDs to the governor.
- Use this field to iterate in a determinstic order when performing
actions that change the state of the Governor
- This should help Guardians reach a more similar view of the Governor
in scenarios where iteration order might impact whether a transfer is
queued. (This is relevant especially in the case of Flow Canceling)
- Cases where only a single VAA is being modified were not changed.
Iteration order should not matter here and determinstic order may
may worse for performance when searching for a particular element.
* node: Fix tokenEntry when checking flow cancel for pending transfers
(Squash and merge bug fix from PR #4001)
Similar to a previous issue in the function `ProcessMsgForTime`, the
tokenEntry was not being generated properly.
This should result in queued "small transfers" being able to flow cancel
when they are released from the queue.
Also adds a comment on the CheckedInt64 function to indicate what its
error states mean and when they occur.
Add comments and change variable names for governor_monitoring
- Add function comments to explain what they do and what their error
states mean
- Adds governor logging to error cases
- Change variable names in publishStatus function. `value` was used
first to indicate the "governor usage" and then reused to indicate the
remaining available notional value for a chain. This refactor tries to
make it clear that these are different concepts
Add unit test for flow cancelling when a pending transfer is
released
- Add a unit test to ensure that, when a pending transfer is released,
it also does flow-cancelling on the TargetChain (previously we had a
bug here)
- Add documentation for CheckPendingForTime to clarify that it has
side-effects
* node: Modify error handling for CheckPending method in the Governor
Previous rollouts of the Flow Cancel feature contained issues when
calculating the Governor usage when usage was near the daily limit. This
caused an invariant to be violated. However, this was propagated to the
processor code and resulted in the processor restarting the entire
process. Instead, the Governor should simply fail-closed and report that
there is no remaining capacity, causing further VAAs to be queued until
the usage diminishes over time.
The circumstances leading to the invariant violations are not addressed
in this commit. Instead this commit reworks the way errors are handled
by the CheckPending, making careful choices about when the process
should or should not be killed.
- Change "invariant" error handling: instead of causing the process to
die, log an error and skip further for a single chain while allowing
processing for other chains to continue
- Remove 'invariant error' in TrimAndSumValueForChain as it can occur
somewhat regularly with the addition of the flow cancel feature
- Return dailyLimit in error condition rather than 0 so that future
transfers will be queued
- Do not cap the sum returned from TrimAndSumValueForChain: instead
allow it to exceed the daily limit.
- Modify unit tests to reflect this
- Add unit tests for overflow/underflow scenarios in the TrimAndSumValue
functions
- Change other less severe error cases to log warnings instead of
returning errors.
- Generally prevent flow-cancel related issues from affecting normal
Governor operations. Instead the flow cancel transfers should simply
not be populated and thus result in "GovernorV1" behavior.
- Add documentation to CheckPendingForTime to explain the dangers of
returning an error
- Reword error messages to be more precise and include more relevant
fields. Add documentation explaining when the process should and
should not die
* node: Add additional metrics for Governor status
Modify the monitoring code and protobuf files to make the status of the
Governor more legible when flow-canceling is enabled. This can be
consumed by Wormhole Dashboard to better reflect the effects of flow
cancelling.
On the level of the Governor:
- whether the Guardian has enabled flow cancel or not
On the level of the Governor's emitters, reports 24h metrics for:
- net value that has moved across the chain
- total outgoing amount
- total incoming flow cancel amount
Currently big transfers are not accounted for as they do not affect the
Governor's capacity. (They are always queued.)
* node: Add new flow cancel parameter to Governor in tests
* node: goimports formatting
* node: Bug fix in changes to governor monitoring
- Fix issue where stats weren't being populated unless flow cancel was
enabled
- Fix wrong return value used in unit test
- Fix typo in proto variable name
- Move sorting outside of a for loop for efficiency
- Restore unit test that was deleted in the process of rebasing
* node: address prealloc lint error in governor code
* node: Fix "generated proto differs from committed proto"
* node: Fix bug in chainIds allocation
- This resolves a mistake with allocating the chainIds in the governor
initialization that causes nil entries in the slice.
- Add unit tests to ensure that the chainIds slice matches the chains
map
- Add unit test to ensure that TrimAndSumValueForChain checks for a nil
pointer to avoid panics
* node: Fix returning nil on err in governor_test.go
* node: Cleanup comments in governor code
* node: fix governor comment
* node: enable flow cancel in governor_monitoring tests
* node: Add flow cancel information to p2p heartbeat features
* node: Remove outdated comment from governor
* node: Upgrade logs to Error from Warn when reloading transfers from
database
* node: Enable flow cancel in check_query test function
* node: Cleanup comments and redundant code in governor
* node: Refactor how the flow cancel token list gets populated
- Only populate the flow cancel tokens list once
- Change default behavior to use an empty flow cancel assets list, rather
than first populating the list and then clearing it
- Refactor the logic around enabling the flow cancel token field for
governed assets. Now it only executes if flow cancel is enabled,
rather than operating over an empty slice when flow cancel is disabled
- Modify devnet/testnet configs so that they are responsible for
returning the correct list of flow cancelling assets
* node: Add unit test for flow cancel feature flag
* node: Move new Governor status proto fields from Emitter to Chain
* node: lint governor_monitoring
---------
Co-authored-by: Maxwell Dulin <strikeout@maxwells-mbp.lan>
* Update ChainGovernorResetReleaseTimerRequest protobuf message
* Add numDays argument to governor-reset-release-timer command
* Update governor backend to support numDays argument
* Address review comments
* Add test for resetReleaseTimerForTime()'s numDays parameter
* Address review comments
* Add adminrpc test for ChainGovernorResetReleaseTimer
* Replace hardcoded upper boundaries with maxResetReleaseTimerDays
* Update governor whitepaper to reflect the new argument
* Added default value to governor whitepaper
---------
Co-authored-by: Jason Matthyser <jason@asymmetric.re>
* Add ibc-composability-mw to gateway along with updates to node for
ibc-composability-mw and ibc-translator.
* Move governance action to new GatewayModule
---------
Co-authored-by: Nikhil Suri <nikhilsuri@comcast.net>
* gRelayer: surrounding files
* modification to get compilation
* restore devnet
* remove generic relayer docker
* remove wait for relayer engine
* keep build time 20
* sh -> bash
* sh -> bash
* Remove comment
* bash -> sh
* Revert "bash -> sh"
This reverts commit 5c37e92fa1.
* bash->sh
* gRelayer: ethereum folder changes for generic-relayer-merge
* add eth-devnet
* Adds .github because workflow needs to install forge
* sdk-ci-tests need to install forge
* don't wait for nonexistent relayer engine
* update package.json and package-lock.json
* Remove unnecessary types from package.json
* ts-node
* gRelayer: ethereum folder changes for generic-relayer-merge
* sdk-ci-tests need to install forge
* don't wait for nonexistent relayer engine
* update package.json and package-lock.json
* remove these changes
* Relayer: Natspec documentation in IWormholeRelayer (#3032)
* WIP
* Fixes
* Updated interfaces
* remove bash
* Forward uses same refund chain id and refund address (#3034)
* WIP
* Fixes
* Forward uses same refund chain id and refund address
* Updated interfaces
* Adds .github because workflow needs to install forge
* sdk-ci-tests need to install forge
* don't wait for nonexistent relayer engine
* SDK minus payload tests
* Rename sdk relayer folder and file
* modify index.ts
* modify path
* sdk-ci-tests need to install forge
* don't wait for nonexistent relayer engine
* Add three governance VAA actions for generic relayers
* demandOption and const
* Remove forge build warnings
* Add note to interface for resend
* Verify additional VAAs in SDK
* via-ir on unless in Tilt
* Correct IWormholeReceiver interface
* Wormhole message fee now part of quoteDeliveryPrice (#3043)
* Fix to PR 3043
* Remove compiler warning
* Remove files
* remove generic relayer docker
* Fix typo
* Relayer/address drew review (#3060)
* Fix typo in Create2Factory
* Add event for contract upgrades
* Prevent registering contract if it is already registered
* Prevent allowing unset chainId for default delivery provider governance VAA
* memory to calldata for external functions in WormholeRelayerSend
* continue memory to calldata for external functions
* Fix pricing in delivery provider
* Sanity check new default delivery provider isn't 0 address
* Don't save vaaKey as local variable
* cache the length of array rather than iterate every time for vaaKeys
* Replacing memory with calldata in few locations
* Remove stale file DeliveryProviderMessages
* Remove batch VAA sender script
* Remove batch VAA from WormholeSimulator
* Wait for a confirmation in deploy scripts
* remove unnecessary comments
* Fix Delivery Provider Pricing and add a test
* remove console logs
* Revert "continue memory to calldata for external functions"
This reverts commit f322afb6c0.
* Revert "memory to calldata for external functions in WormholeRelayerSend"
This reverts commit 42fcaad884.
* Revert "Don't save vaaKey as local variable"
This reverts commit a9172379c5.
* Revert "cache the length of array rather than iterate every time for vaaKeys"
This reverts commit d61380a9b0.
* Revert "Replacing memory with calldata in few locations"
This reverts commit 94e47b6e72.
* Revert "Fix typo in Create2Factory"
This reverts commit a9f7bdf461.
* Update contract addresses for via-ir
* Update register chain test to only do one registration
* Slight improvements to delivery provider implementation
* typed errors for delivery provider
* Update SDK to have via-ir devnet address
* Fix test
* enable VIA-IR in CI and not in Tilt
* Fix chain id
* get register chain test to work
* correct contract address for via ir
* update sdk consts for via ir address
* base 32 address
* merge
* -f to -r
* relay provider -> delivery provider
* fix await
* Readme changes
* sdk: add new governance VAA for IbcReceiverUpdateChainConnection
* Enforce connectionId to be 64 bytes, add tests in rust sdk
* Update PrependBufferBytesFixed to LeftPadBytes, add template function + command for IBC governance VAA
* Add >64 length check for ibcReceiverUpdateChainConnectionConnectionId in runIbcReceiverUpdateChainConnectionTemplate command
* Update naming of governance VAA to reflect new mapping of channelId -> chainId
* Add TargetChainID to admin commands
* Node: Add IBC update channel to admin verify cmd
---------
Co-authored-by: Bruce Riley <briley@jumptrading.com>
* Sei testnet support
* Set contract addresses
* node/pkg/watchers: add RunWithScissors
* SDK tweaks
* node: revert "node/pkg/watchers: add RunWithScissors" (#2620)
This reverts commit 9b8bed4dbf.
* Make Sei use IBC watcher
---------
Co-authored-by: Paul Noel <panoel007@gmail.com>
Co-authored-by: Paul Noel <35237584+panoel@users.noreply.github.com>
* node: inject wormchain cosmwasm governance messages
* Use nested hash for defense in depth
* Use keccak.Reset() instead of creating new hash objects
* fix msg_server_wasmd_test
* Updated based on jynnantonix comments
* Check return value of binary.Write
* Include actual error in binary.Write panic case
* Governor publish gossip
Change-Id: I2b8b1ea84a0c411101a7027acd3a27a6d6464d59
* Update the config publish time
Change-Id: Ic6abf84befb1c20756da2ff66b15a8325dc46067
* Not setting value on enqueued VAAs correctly
Change-Id: I9fd3a5d8fc574f8382125445fa688efdae45b88c
* Publish at most 20 VAAs, not 20 per chain
Change-Id: Ic9dff99c59ee89d57fd79158844a1fe1a0003112
* Switch to using signed messages
Change-Id: I66cddc7477cd477aa77bdadfc346b588f2ae645b
* Publish status only once per minute
Change-Id: I972fb0cf868e89c6f74ae4441471a55df389f4dd
* Minor comment change
Change-Id: I0d3e443cbec7edd282f89c1a5cce5d5ec8776d55
* Add command to purge pythnet VAAs
* Add test for purging a single emitter address
* Fix lint error
* Using the wrong delete primative
Change-Id: I80d5294c17279d4e49220d81807e5964a5591721
bruce-riley
John Saigle
Jason Matthyser
Evan Gray
Csongor Kiss
Kevin Peters
Ben Guidarelli
Nikhil Suri
Paul Noel
Steve
derpy-duck
Conor Patrick
marcus-jump
Hendrik Hofstadt
Josh Siegel
Justin Schuldt
Bruce Riley