wormhole/relayer/spy_relayer/src/main.ts

135 lines
4.0 KiB
TypeScript
Raw Normal View History

Spy relayer cleanup (#1015) * initial spy-relayer * Update spy_relayer Dockerfile * added example mainnet config files * split out private keys into its own ENV variable * Update spy relayer supportedChains.json To remove the `walletPrivateKey` entries. All of the private keys have been split out into their own json file. * fixed evm private key env parse * missing solana accounts report 0 balance, rather than error * wallet address is logged in debug * spy_relayer: enabled prometheus default metrics Also set a prefix of `relayer_` * spy_relayer: updates to the prometheus bits * Use a single metric registry * Use a simpler metric name and add labels for individual wallets * spy_relayer: human readable app mode in the metrics [ listener | relayer | both ] * spy_relayer: unify metrics * remove the collection of default metrics * hardcode the `spy_relayer_` prefix on all custom metrics * fixed dep arrays, nullable terra token/balance info * attempt stack debug * debug pullTerraBalance * provider http or ws * update sdk * logging for tokenAddress is 0 * fix foreign address calc * fix calcLocalAddressesTerra * relayer/spy_relayer: update prometheus helpers Add / url handler for the ingress-gce stupid load balancer that doesn't support custom url healthchecks unless you make a BackendConfig custom resource definition. * logging refinement * use chain name in prometheus * adjust retry timeout calculation * spy_relayer: update prometheus bits * improved error handling * relayer ui improvements * prep sdk release * use latest sdk, manual redeem button * relaying ux improvements * gas price fix * shortened terra success log * use gh base relayer list * fix prometheus urls * Update prometheus metric name * only show TPS warning on mainnet * show relayer fee in source preview * fix unwrap check * add native bool to balance metric * logging improvements * add feeRecipientAddress to redeemOnSolana * gather solana fees * remove relayer ws support * add nativeCurrencySymbol to ChainConfigInfo * fix solana native symbol * demoteWorking option, logger contexts * scoped logging * bridge_ui: unwrap native * add evm wallet monitor test * solana vaa parsing fix * add monitorRedis * make Jeff's brain happy * log demoting keys * register redisQueue metric * human readable redisQueue metric * fix timestamp inconsistency * use scopedLogger for the first level of workers * pull wallet balances in parallel * more scoped logging * pick a solana fee * moving keys log improvement * update eth gas calculations based on recent txs * use postVaaSolanaWithRetry * split success and failures by chain * fix using terraCoin * check prom every 10s * batch getting evm token balances * batch calcLocalAddressesEVM * debug worker logging * log retry number * support Polygon? * reset status on demotion * enhance! * update avax fee Co-authored-by: Chase Moran <chasemoran45@gmail.com> Co-authored-by: Kevin Peters <kpeters@jumptrading.com> Co-authored-by: Evan Gray <battledingo@gmail.com>
2022-03-28 20:39:08 -07:00
//This has to run first so that the process variables are set up when the other modules are instantiated.
require("./helpers/loadConfig");
import { setDefaultWasm } from "@certusone/wormhole-sdk/lib/cjs/solana/wasm";
import { getCommonEnvironment } from "./configureEnv";
import { getLogger } from "./helpers/logHelper";
import { PromHelper, PromMode } from "./helpers/promHelpers";
import * as redisHelper from "./helpers/redisHelper";
import * as restListener from "./listener/rest_listen";
import * as spyListener from "./listener/spy_listen";
import * as relayWorker from "./relayer/relay_worker";
import * as walletMonitor from "./monitor";
Spy relayer cleanup (#1015) * initial spy-relayer * Update spy_relayer Dockerfile * added example mainnet config files * split out private keys into its own ENV variable * Update spy relayer supportedChains.json To remove the `walletPrivateKey` entries. All of the private keys have been split out into their own json file. * fixed evm private key env parse * missing solana accounts report 0 balance, rather than error * wallet address is logged in debug * spy_relayer: enabled prometheus default metrics Also set a prefix of `relayer_` * spy_relayer: updates to the prometheus bits * Use a single metric registry * Use a simpler metric name and add labels for individual wallets * spy_relayer: human readable app mode in the metrics [ listener | relayer | both ] * spy_relayer: unify metrics * remove the collection of default metrics * hardcode the `spy_relayer_` prefix on all custom metrics * fixed dep arrays, nullable terra token/balance info * attempt stack debug * debug pullTerraBalance * provider http or ws * update sdk * logging for tokenAddress is 0 * fix foreign address calc * fix calcLocalAddressesTerra * relayer/spy_relayer: update prometheus helpers Add / url handler for the ingress-gce stupid load balancer that doesn't support custom url healthchecks unless you make a BackendConfig custom resource definition. * logging refinement * use chain name in prometheus * adjust retry timeout calculation * spy_relayer: update prometheus bits * improved error handling * relayer ui improvements * prep sdk release * use latest sdk, manual redeem button * relaying ux improvements * gas price fix * shortened terra success log * use gh base relayer list * fix prometheus urls * Update prometheus metric name * only show TPS warning on mainnet * show relayer fee in source preview * fix unwrap check * add native bool to balance metric * logging improvements * add feeRecipientAddress to redeemOnSolana * gather solana fees * remove relayer ws support * add nativeCurrencySymbol to ChainConfigInfo * fix solana native symbol * demoteWorking option, logger contexts * scoped logging * bridge_ui: unwrap native * add evm wallet monitor test * solana vaa parsing fix * add monitorRedis * make Jeff's brain happy * log demoting keys * register redisQueue metric * human readable redisQueue metric * fix timestamp inconsistency * use scopedLogger for the first level of workers * pull wallet balances in parallel * more scoped logging * pick a solana fee * moving keys log improvement * update eth gas calculations based on recent txs * use postVaaSolanaWithRetry * split success and failures by chain * fix using terraCoin * check prom every 10s * batch getting evm token balances * batch calcLocalAddressesEVM * debug worker logging * log retry number * support Polygon? * reset status on demotion * enhance! * update avax fee Co-authored-by: Chase Moran <chasemoran45@gmail.com> Co-authored-by: Kevin Peters <kpeters@jumptrading.com> Co-authored-by: Evan Gray <battledingo@gmail.com>
2022-03-28 20:39:08 -07:00
const ARG_LISTEN_ONLY = "--listen_only";
const ARG_RELAY_ONLY = "--relay_only";
const ARG_WALLET_MONITOR_ONLY = "--wallet_monitor_only";
const ONLY_ONE_ARG_ERROR_MSG = `May only specify one of ${ARG_LISTEN_ONLY}, ${ARG_RELAY_ONLY}, or ${ARG_WALLET_MONITOR_ONLY}`;
const ONLY_ONE_ARG_ERROR_RESULT = `Multiple args found of ${ARG_LISTEN_ONLY}, ${ARG_RELAY_ONLY}, ${ARG_WALLET_MONITOR_ONLY}`;
Spy relayer cleanup (#1015) * initial spy-relayer * Update spy_relayer Dockerfile * added example mainnet config files * split out private keys into its own ENV variable * Update spy relayer supportedChains.json To remove the `walletPrivateKey` entries. All of the private keys have been split out into their own json file. * fixed evm private key env parse * missing solana accounts report 0 balance, rather than error * wallet address is logged in debug * spy_relayer: enabled prometheus default metrics Also set a prefix of `relayer_` * spy_relayer: updates to the prometheus bits * Use a single metric registry * Use a simpler metric name and add labels for individual wallets * spy_relayer: human readable app mode in the metrics [ listener | relayer | both ] * spy_relayer: unify metrics * remove the collection of default metrics * hardcode the `spy_relayer_` prefix on all custom metrics * fixed dep arrays, nullable terra token/balance info * attempt stack debug * debug pullTerraBalance * provider http or ws * update sdk * logging for tokenAddress is 0 * fix foreign address calc * fix calcLocalAddressesTerra * relayer/spy_relayer: update prometheus helpers Add / url handler for the ingress-gce stupid load balancer that doesn't support custom url healthchecks unless you make a BackendConfig custom resource definition. * logging refinement * use chain name in prometheus * adjust retry timeout calculation * spy_relayer: update prometheus bits * improved error handling * relayer ui improvements * prep sdk release * use latest sdk, manual redeem button * relaying ux improvements * gas price fix * shortened terra success log * use gh base relayer list * fix prometheus urls * Update prometheus metric name * only show TPS warning on mainnet * show relayer fee in source preview * fix unwrap check * add native bool to balance metric * logging improvements * add feeRecipientAddress to redeemOnSolana * gather solana fees * remove relayer ws support * add nativeCurrencySymbol to ChainConfigInfo * fix solana native symbol * demoteWorking option, logger contexts * scoped logging * bridge_ui: unwrap native * add evm wallet monitor test * solana vaa parsing fix * add monitorRedis * make Jeff's brain happy * log demoting keys * register redisQueue metric * human readable redisQueue metric * fix timestamp inconsistency * use scopedLogger for the first level of workers * pull wallet balances in parallel * more scoped logging * pick a solana fee * moving keys log improvement * update eth gas calculations based on recent txs * use postVaaSolanaWithRetry * split success and failures by chain * fix using terraCoin * check prom every 10s * batch getting evm token balances * batch calcLocalAddressesEVM * debug worker logging * log retry number * support Polygon? * reset status on demotion * enhance! * update avax fee Co-authored-by: Chase Moran <chasemoran45@gmail.com> Co-authored-by: Kevin Peters <kpeters@jumptrading.com> Co-authored-by: Evan Gray <battledingo@gmail.com>
2022-03-28 20:39:08 -07:00
setDefaultWasm("node");
const logger = getLogger();
// Load the relay config data.
let runListen: boolean = true;
Modular relayer support (#1266) * Add the default backend for the relayer Start to slowly split things out. * Configure the backends when configuring the env * First stab at the pluggable listener backend * Update relayer example (and test) config * relayer: JIT backend * relayer: walletMonitor remove useless function * relayer: update worker init() funcs It is silly to accept a boolean in thing.init() vs just not running init if you don't want to run thing. * relayer: remove env property from Listener It is verified in init() in spy_listener.ts and rest_listen.ts, so it can be deferred to not require the import. * relayer: clean up the main bits Only try to init() something when it is actually supposed to run per the cli flags. * spy_relayer: more descriptive var name in main * spy_relayer: Update relay worker with more docs * spy_relayer: add targetChainName to WorkerInfo This makes it easier to pretty print the chain name in the logs without having to look the name up. * spy_relayer: update logs and use the backend * Use the Relayer interface's process() method in place of processVaa() * Update the logs to include the chain name in the worker and auditor threads * spy_relayer: remove processRequest() It has been moved to the process() method of the default Relayer backend. * spy_relayer: start fleshing out the Relayer default * spy_relayer: fix a logic bug in checkQueue() Co-authored-by: @swimricky * spy_relayer: update TokenBridgeRelayer.process() * Remove some extra logic * Actually use the ChainId type since the id of 0 is in the sdk now * spy_signature: add Relayer.runAuditor() The auditor code is payload specific and needs to be with the backend. * spy_relayer: move Relayer.runAuditor() Make it part of the backend since the backend has all of the payload specific logic into it. * spy_relayer: move relay() --> Relayer.relay() The actual relaying is part of the backend, so make it so. * spy_relayer: add Relayer.runAuditor() * spy_relayer: no more deprecated hexToNativeString * spy_relayer: implement Relayer.targetChainId() This is used for finding workable items in the incoming queue to toss into the working queue. * spy_relayer: remove relay.ts The relay() function was moved to Relayer.relay() * spy_relayer: more uses of deprecated hexToNativeString() * spy_relayer: remove unused import * spy_relayer: review feedback from @bruce-riley * Fix some spelling tyops * Simplify some logging * Simplify a conditional for the payload version check * spy_relayer: misc fixes and code clean up * Fixed integration tests * Added launch.json file for debugging in VS Code * Updated to latest wormhole SDK * Backup queue uses same key as redis * Added Terra Classic flag * Throttle potential infinite loop in audit thread * Fixed spy service connection leak Co-authored-by: Evan Gray <battledingo@gmail.com> Co-authored-by: Kevin Peters <kpeters@jumptrading.com>
2022-08-05 07:20:36 -07:00
let runRelayWorker: boolean = true;
Spy relayer cleanup (#1015) * initial spy-relayer * Update spy_relayer Dockerfile * added example mainnet config files * split out private keys into its own ENV variable * Update spy relayer supportedChains.json To remove the `walletPrivateKey` entries. All of the private keys have been split out into their own json file. * fixed evm private key env parse * missing solana accounts report 0 balance, rather than error * wallet address is logged in debug * spy_relayer: enabled prometheus default metrics Also set a prefix of `relayer_` * spy_relayer: updates to the prometheus bits * Use a single metric registry * Use a simpler metric name and add labels for individual wallets * spy_relayer: human readable app mode in the metrics [ listener | relayer | both ] * spy_relayer: unify metrics * remove the collection of default metrics * hardcode the `spy_relayer_` prefix on all custom metrics * fixed dep arrays, nullable terra token/balance info * attempt stack debug * debug pullTerraBalance * provider http or ws * update sdk * logging for tokenAddress is 0 * fix foreign address calc * fix calcLocalAddressesTerra * relayer/spy_relayer: update prometheus helpers Add / url handler for the ingress-gce stupid load balancer that doesn't support custom url healthchecks unless you make a BackendConfig custom resource definition. * logging refinement * use chain name in prometheus * adjust retry timeout calculation * spy_relayer: update prometheus bits * improved error handling * relayer ui improvements * prep sdk release * use latest sdk, manual redeem button * relaying ux improvements * gas price fix * shortened terra success log * use gh base relayer list * fix prometheus urls * Update prometheus metric name * only show TPS warning on mainnet * show relayer fee in source preview * fix unwrap check * add native bool to balance metric * logging improvements * add feeRecipientAddress to redeemOnSolana * gather solana fees * remove relayer ws support * add nativeCurrencySymbol to ChainConfigInfo * fix solana native symbol * demoteWorking option, logger contexts * scoped logging * bridge_ui: unwrap native * add evm wallet monitor test * solana vaa parsing fix * add monitorRedis * make Jeff's brain happy * log demoting keys * register redisQueue metric * human readable redisQueue metric * fix timestamp inconsistency * use scopedLogger for the first level of workers * pull wallet balances in parallel * more scoped logging * pick a solana fee * moving keys log improvement * update eth gas calculations based on recent txs * use postVaaSolanaWithRetry * split success and failures by chain * fix using terraCoin * check prom every 10s * batch getting evm token balances * batch calcLocalAddressesEVM * debug worker logging * log retry number * support Polygon? * reset status on demotion * enhance! * update avax fee Co-authored-by: Chase Moran <chasemoran45@gmail.com> Co-authored-by: Kevin Peters <kpeters@jumptrading.com> Co-authored-by: Evan Gray <battledingo@gmail.com>
2022-03-28 20:39:08 -07:00
let runRest: boolean = true;
let runWalletMonitor: boolean = true;
Spy relayer cleanup (#1015) * initial spy-relayer * Update spy_relayer Dockerfile * added example mainnet config files * split out private keys into its own ENV variable * Update spy relayer supportedChains.json To remove the `walletPrivateKey` entries. All of the private keys have been split out into their own json file. * fixed evm private key env parse * missing solana accounts report 0 balance, rather than error * wallet address is logged in debug * spy_relayer: enabled prometheus default metrics Also set a prefix of `relayer_` * spy_relayer: updates to the prometheus bits * Use a single metric registry * Use a simpler metric name and add labels for individual wallets * spy_relayer: human readable app mode in the metrics [ listener | relayer | both ] * spy_relayer: unify metrics * remove the collection of default metrics * hardcode the `spy_relayer_` prefix on all custom metrics * fixed dep arrays, nullable terra token/balance info * attempt stack debug * debug pullTerraBalance * provider http or ws * update sdk * logging for tokenAddress is 0 * fix foreign address calc * fix calcLocalAddressesTerra * relayer/spy_relayer: update prometheus helpers Add / url handler for the ingress-gce stupid load balancer that doesn't support custom url healthchecks unless you make a BackendConfig custom resource definition. * logging refinement * use chain name in prometheus * adjust retry timeout calculation * spy_relayer: update prometheus bits * improved error handling * relayer ui improvements * prep sdk release * use latest sdk, manual redeem button * relaying ux improvements * gas price fix * shortened terra success log * use gh base relayer list * fix prometheus urls * Update prometheus metric name * only show TPS warning on mainnet * show relayer fee in source preview * fix unwrap check * add native bool to balance metric * logging improvements * add feeRecipientAddress to redeemOnSolana * gather solana fees * remove relayer ws support * add nativeCurrencySymbol to ChainConfigInfo * fix solana native symbol * demoteWorking option, logger contexts * scoped logging * bridge_ui: unwrap native * add evm wallet monitor test * solana vaa parsing fix * add monitorRedis * make Jeff's brain happy * log demoting keys * register redisQueue metric * human readable redisQueue metric * fix timestamp inconsistency * use scopedLogger for the first level of workers * pull wallet balances in parallel * more scoped logging * pick a solana fee * moving keys log improvement * update eth gas calculations based on recent txs * use postVaaSolanaWithRetry * split success and failures by chain * fix using terraCoin * check prom every 10s * batch getting evm token balances * batch calcLocalAddressesEVM * debug worker logging * log retry number * support Polygon? * reset status on demotion * enhance! * update avax fee Co-authored-by: Chase Moran <chasemoran45@gmail.com> Co-authored-by: Kevin Peters <kpeters@jumptrading.com> Co-authored-by: Evan Gray <battledingo@gmail.com>
2022-03-28 20:39:08 -07:00
let foundOne: boolean = false;
let error: string = "";
for (let idx = 0; idx < process.argv.length; ++idx) {
if (process.argv[idx] === ARG_LISTEN_ONLY) {
Spy relayer cleanup (#1015) * initial spy-relayer * Update spy_relayer Dockerfile * added example mainnet config files * split out private keys into its own ENV variable * Update spy relayer supportedChains.json To remove the `walletPrivateKey` entries. All of the private keys have been split out into their own json file. * fixed evm private key env parse * missing solana accounts report 0 balance, rather than error * wallet address is logged in debug * spy_relayer: enabled prometheus default metrics Also set a prefix of `relayer_` * spy_relayer: updates to the prometheus bits * Use a single metric registry * Use a simpler metric name and add labels for individual wallets * spy_relayer: human readable app mode in the metrics [ listener | relayer | both ] * spy_relayer: unify metrics * remove the collection of default metrics * hardcode the `spy_relayer_` prefix on all custom metrics * fixed dep arrays, nullable terra token/balance info * attempt stack debug * debug pullTerraBalance * provider http or ws * update sdk * logging for tokenAddress is 0 * fix foreign address calc * fix calcLocalAddressesTerra * relayer/spy_relayer: update prometheus helpers Add / url handler for the ingress-gce stupid load balancer that doesn't support custom url healthchecks unless you make a BackendConfig custom resource definition. * logging refinement * use chain name in prometheus * adjust retry timeout calculation * spy_relayer: update prometheus bits * improved error handling * relayer ui improvements * prep sdk release * use latest sdk, manual redeem button * relaying ux improvements * gas price fix * shortened terra success log * use gh base relayer list * fix prometheus urls * Update prometheus metric name * only show TPS warning on mainnet * show relayer fee in source preview * fix unwrap check * add native bool to balance metric * logging improvements * add feeRecipientAddress to redeemOnSolana * gather solana fees * remove relayer ws support * add nativeCurrencySymbol to ChainConfigInfo * fix solana native symbol * demoteWorking option, logger contexts * scoped logging * bridge_ui: unwrap native * add evm wallet monitor test * solana vaa parsing fix * add monitorRedis * make Jeff's brain happy * log demoting keys * register redisQueue metric * human readable redisQueue metric * fix timestamp inconsistency * use scopedLogger for the first level of workers * pull wallet balances in parallel * more scoped logging * pick a solana fee * moving keys log improvement * update eth gas calculations based on recent txs * use postVaaSolanaWithRetry * split success and failures by chain * fix using terraCoin * check prom every 10s * batch getting evm token balances * batch calcLocalAddressesEVM * debug worker logging * log retry number * support Polygon? * reset status on demotion * enhance! * update avax fee Co-authored-by: Chase Moran <chasemoran45@gmail.com> Co-authored-by: Kevin Peters <kpeters@jumptrading.com> Co-authored-by: Evan Gray <battledingo@gmail.com>
2022-03-28 20:39:08 -07:00
if (foundOne) {
logger.error(ONLY_ONE_ARG_ERROR_MSG);
error = ONLY_ONE_ARG_ERROR_RESULT;
Spy relayer cleanup (#1015) * initial spy-relayer * Update spy_relayer Dockerfile * added example mainnet config files * split out private keys into its own ENV variable * Update spy relayer supportedChains.json To remove the `walletPrivateKey` entries. All of the private keys have been split out into their own json file. * fixed evm private key env parse * missing solana accounts report 0 balance, rather than error * wallet address is logged in debug * spy_relayer: enabled prometheus default metrics Also set a prefix of `relayer_` * spy_relayer: updates to the prometheus bits * Use a single metric registry * Use a simpler metric name and add labels for individual wallets * spy_relayer: human readable app mode in the metrics [ listener | relayer | both ] * spy_relayer: unify metrics * remove the collection of default metrics * hardcode the `spy_relayer_` prefix on all custom metrics * fixed dep arrays, nullable terra token/balance info * attempt stack debug * debug pullTerraBalance * provider http or ws * update sdk * logging for tokenAddress is 0 * fix foreign address calc * fix calcLocalAddressesTerra * relayer/spy_relayer: update prometheus helpers Add / url handler for the ingress-gce stupid load balancer that doesn't support custom url healthchecks unless you make a BackendConfig custom resource definition. * logging refinement * use chain name in prometheus * adjust retry timeout calculation * spy_relayer: update prometheus bits * improved error handling * relayer ui improvements * prep sdk release * use latest sdk, manual redeem button * relaying ux improvements * gas price fix * shortened terra success log * use gh base relayer list * fix prometheus urls * Update prometheus metric name * only show TPS warning on mainnet * show relayer fee in source preview * fix unwrap check * add native bool to balance metric * logging improvements * add feeRecipientAddress to redeemOnSolana * gather solana fees * remove relayer ws support * add nativeCurrencySymbol to ChainConfigInfo * fix solana native symbol * demoteWorking option, logger contexts * scoped logging * bridge_ui: unwrap native * add evm wallet monitor test * solana vaa parsing fix * add monitorRedis * make Jeff's brain happy * log demoting keys * register redisQueue metric * human readable redisQueue metric * fix timestamp inconsistency * use scopedLogger for the first level of workers * pull wallet balances in parallel * more scoped logging * pick a solana fee * moving keys log improvement * update eth gas calculations based on recent txs * use postVaaSolanaWithRetry * split success and failures by chain * fix using terraCoin * check prom every 10s * batch getting evm token balances * batch calcLocalAddressesEVM * debug worker logging * log retry number * support Polygon? * reset status on demotion * enhance! * update avax fee Co-authored-by: Chase Moran <chasemoran45@gmail.com> Co-authored-by: Kevin Peters <kpeters@jumptrading.com> Co-authored-by: Evan Gray <battledingo@gmail.com>
2022-03-28 20:39:08 -07:00
break;
}
logger.info("spy_relay is running in listen only mode");
Modular relayer support (#1266) * Add the default backend for the relayer Start to slowly split things out. * Configure the backends when configuring the env * First stab at the pluggable listener backend * Update relayer example (and test) config * relayer: JIT backend * relayer: walletMonitor remove useless function * relayer: update worker init() funcs It is silly to accept a boolean in thing.init() vs just not running init if you don't want to run thing. * relayer: remove env property from Listener It is verified in init() in spy_listener.ts and rest_listen.ts, so it can be deferred to not require the import. * relayer: clean up the main bits Only try to init() something when it is actually supposed to run per the cli flags. * spy_relayer: more descriptive var name in main * spy_relayer: Update relay worker with more docs * spy_relayer: add targetChainName to WorkerInfo This makes it easier to pretty print the chain name in the logs without having to look the name up. * spy_relayer: update logs and use the backend * Use the Relayer interface's process() method in place of processVaa() * Update the logs to include the chain name in the worker and auditor threads * spy_relayer: remove processRequest() It has been moved to the process() method of the default Relayer backend. * spy_relayer: start fleshing out the Relayer default * spy_relayer: fix a logic bug in checkQueue() Co-authored-by: @swimricky * spy_relayer: update TokenBridgeRelayer.process() * Remove some extra logic * Actually use the ChainId type since the id of 0 is in the sdk now * spy_signature: add Relayer.runAuditor() The auditor code is payload specific and needs to be with the backend. * spy_relayer: move Relayer.runAuditor() Make it part of the backend since the backend has all of the payload specific logic into it. * spy_relayer: move relay() --> Relayer.relay() The actual relaying is part of the backend, so make it so. * spy_relayer: add Relayer.runAuditor() * spy_relayer: no more deprecated hexToNativeString * spy_relayer: implement Relayer.targetChainId() This is used for finding workable items in the incoming queue to toss into the working queue. * spy_relayer: remove relay.ts The relay() function was moved to Relayer.relay() * spy_relayer: more uses of deprecated hexToNativeString() * spy_relayer: remove unused import * spy_relayer: review feedback from @bruce-riley * Fix some spelling tyops * Simplify some logging * Simplify a conditional for the payload version check * spy_relayer: misc fixes and code clean up * Fixed integration tests * Added launch.json file for debugging in VS Code * Updated to latest wormhole SDK * Backup queue uses same key as redis * Added Terra Classic flag * Throttle potential infinite loop in audit thread * Fixed spy service connection leak Co-authored-by: Evan Gray <battledingo@gmail.com> Co-authored-by: Kevin Peters <kpeters@jumptrading.com>
2022-08-05 07:20:36 -07:00
runRelayWorker = false;
runWalletMonitor = false;
Spy relayer cleanup (#1015) * initial spy-relayer * Update spy_relayer Dockerfile * added example mainnet config files * split out private keys into its own ENV variable * Update spy relayer supportedChains.json To remove the `walletPrivateKey` entries. All of the private keys have been split out into their own json file. * fixed evm private key env parse * missing solana accounts report 0 balance, rather than error * wallet address is logged in debug * spy_relayer: enabled prometheus default metrics Also set a prefix of `relayer_` * spy_relayer: updates to the prometheus bits * Use a single metric registry * Use a simpler metric name and add labels for individual wallets * spy_relayer: human readable app mode in the metrics [ listener | relayer | both ] * spy_relayer: unify metrics * remove the collection of default metrics * hardcode the `spy_relayer_` prefix on all custom metrics * fixed dep arrays, nullable terra token/balance info * attempt stack debug * debug pullTerraBalance * provider http or ws * update sdk * logging for tokenAddress is 0 * fix foreign address calc * fix calcLocalAddressesTerra * relayer/spy_relayer: update prometheus helpers Add / url handler for the ingress-gce stupid load balancer that doesn't support custom url healthchecks unless you make a BackendConfig custom resource definition. * logging refinement * use chain name in prometheus * adjust retry timeout calculation * spy_relayer: update prometheus bits * improved error handling * relayer ui improvements * prep sdk release * use latest sdk, manual redeem button * relaying ux improvements * gas price fix * shortened terra success log * use gh base relayer list * fix prometheus urls * Update prometheus metric name * only show TPS warning on mainnet * show relayer fee in source preview * fix unwrap check * add native bool to balance metric * logging improvements * add feeRecipientAddress to redeemOnSolana * gather solana fees * remove relayer ws support * add nativeCurrencySymbol to ChainConfigInfo * fix solana native symbol * demoteWorking option, logger contexts * scoped logging * bridge_ui: unwrap native * add evm wallet monitor test * solana vaa parsing fix * add monitorRedis * make Jeff's brain happy * log demoting keys * register redisQueue metric * human readable redisQueue metric * fix timestamp inconsistency * use scopedLogger for the first level of workers * pull wallet balances in parallel * more scoped logging * pick a solana fee * moving keys log improvement * update eth gas calculations based on recent txs * use postVaaSolanaWithRetry * split success and failures by chain * fix using terraCoin * check prom every 10s * batch getting evm token balances * batch calcLocalAddressesEVM * debug worker logging * log retry number * support Polygon? * reset status on demotion * enhance! * update avax fee Co-authored-by: Chase Moran <chasemoran45@gmail.com> Co-authored-by: Kevin Peters <kpeters@jumptrading.com> Co-authored-by: Evan Gray <battledingo@gmail.com>
2022-03-28 20:39:08 -07:00
foundOne = true;
}
if (process.argv[idx] === ARG_RELAY_ONLY) {
Spy relayer cleanup (#1015) * initial spy-relayer * Update spy_relayer Dockerfile * added example mainnet config files * split out private keys into its own ENV variable * Update spy relayer supportedChains.json To remove the `walletPrivateKey` entries. All of the private keys have been split out into their own json file. * fixed evm private key env parse * missing solana accounts report 0 balance, rather than error * wallet address is logged in debug * spy_relayer: enabled prometheus default metrics Also set a prefix of `relayer_` * spy_relayer: updates to the prometheus bits * Use a single metric registry * Use a simpler metric name and add labels for individual wallets * spy_relayer: human readable app mode in the metrics [ listener | relayer | both ] * spy_relayer: unify metrics * remove the collection of default metrics * hardcode the `spy_relayer_` prefix on all custom metrics * fixed dep arrays, nullable terra token/balance info * attempt stack debug * debug pullTerraBalance * provider http or ws * update sdk * logging for tokenAddress is 0 * fix foreign address calc * fix calcLocalAddressesTerra * relayer/spy_relayer: update prometheus helpers Add / url handler for the ingress-gce stupid load balancer that doesn't support custom url healthchecks unless you make a BackendConfig custom resource definition. * logging refinement * use chain name in prometheus * adjust retry timeout calculation * spy_relayer: update prometheus bits * improved error handling * relayer ui improvements * prep sdk release * use latest sdk, manual redeem button * relaying ux improvements * gas price fix * shortened terra success log * use gh base relayer list * fix prometheus urls * Update prometheus metric name * only show TPS warning on mainnet * show relayer fee in source preview * fix unwrap check * add native bool to balance metric * logging improvements * add feeRecipientAddress to redeemOnSolana * gather solana fees * remove relayer ws support * add nativeCurrencySymbol to ChainConfigInfo * fix solana native symbol * demoteWorking option, logger contexts * scoped logging * bridge_ui: unwrap native * add evm wallet monitor test * solana vaa parsing fix * add monitorRedis * make Jeff's brain happy * log demoting keys * register redisQueue metric * human readable redisQueue metric * fix timestamp inconsistency * use scopedLogger for the first level of workers * pull wallet balances in parallel * more scoped logging * pick a solana fee * moving keys log improvement * update eth gas calculations based on recent txs * use postVaaSolanaWithRetry * split success and failures by chain * fix using terraCoin * check prom every 10s * batch getting evm token balances * batch calcLocalAddressesEVM * debug worker logging * log retry number * support Polygon? * reset status on demotion * enhance! * update avax fee Co-authored-by: Chase Moran <chasemoran45@gmail.com> Co-authored-by: Kevin Peters <kpeters@jumptrading.com> Co-authored-by: Evan Gray <battledingo@gmail.com>
2022-03-28 20:39:08 -07:00
if (foundOne) {
logger.error(ONLY_ONE_ARG_ERROR_MSG);
error = ONLY_ONE_ARG_ERROR_RESULT;
Spy relayer cleanup (#1015) * initial spy-relayer * Update spy_relayer Dockerfile * added example mainnet config files * split out private keys into its own ENV variable * Update spy relayer supportedChains.json To remove the `walletPrivateKey` entries. All of the private keys have been split out into their own json file. * fixed evm private key env parse * missing solana accounts report 0 balance, rather than error * wallet address is logged in debug * spy_relayer: enabled prometheus default metrics Also set a prefix of `relayer_` * spy_relayer: updates to the prometheus bits * Use a single metric registry * Use a simpler metric name and add labels for individual wallets * spy_relayer: human readable app mode in the metrics [ listener | relayer | both ] * spy_relayer: unify metrics * remove the collection of default metrics * hardcode the `spy_relayer_` prefix on all custom metrics * fixed dep arrays, nullable terra token/balance info * attempt stack debug * debug pullTerraBalance * provider http or ws * update sdk * logging for tokenAddress is 0 * fix foreign address calc * fix calcLocalAddressesTerra * relayer/spy_relayer: update prometheus helpers Add / url handler for the ingress-gce stupid load balancer that doesn't support custom url healthchecks unless you make a BackendConfig custom resource definition. * logging refinement * use chain name in prometheus * adjust retry timeout calculation * spy_relayer: update prometheus bits * improved error handling * relayer ui improvements * prep sdk release * use latest sdk, manual redeem button * relaying ux improvements * gas price fix * shortened terra success log * use gh base relayer list * fix prometheus urls * Update prometheus metric name * only show TPS warning on mainnet * show relayer fee in source preview * fix unwrap check * add native bool to balance metric * logging improvements * add feeRecipientAddress to redeemOnSolana * gather solana fees * remove relayer ws support * add nativeCurrencySymbol to ChainConfigInfo * fix solana native symbol * demoteWorking option, logger contexts * scoped logging * bridge_ui: unwrap native * add evm wallet monitor test * solana vaa parsing fix * add monitorRedis * make Jeff's brain happy * log demoting keys * register redisQueue metric * human readable redisQueue metric * fix timestamp inconsistency * use scopedLogger for the first level of workers * pull wallet balances in parallel * more scoped logging * pick a solana fee * moving keys log improvement * update eth gas calculations based on recent txs * use postVaaSolanaWithRetry * split success and failures by chain * fix using terraCoin * check prom every 10s * batch getting evm token balances * batch calcLocalAddressesEVM * debug worker logging * log retry number * support Polygon? * reset status on demotion * enhance! * update avax fee Co-authored-by: Chase Moran <chasemoran45@gmail.com> Co-authored-by: Kevin Peters <kpeters@jumptrading.com> Co-authored-by: Evan Gray <battledingo@gmail.com>
2022-03-28 20:39:08 -07:00
break;
}
logger.info("spy_relay is running in relay only mode");
runListen = false;
runRest = false;
runWalletMonitor = false;
foundOne = true;
}
if (process.argv[idx] === ARG_WALLET_MONITOR_ONLY) {
if (foundOne) {
logger.error(ONLY_ONE_ARG_ERROR_MSG);
error = ONLY_ONE_ARG_ERROR_RESULT;
break;
}
logger.info("spy_relay is running in wallet monitor only mode");
runListen = false;
runRest = false;
Modular relayer support (#1266) * Add the default backend for the relayer Start to slowly split things out. * Configure the backends when configuring the env * First stab at the pluggable listener backend * Update relayer example (and test) config * relayer: JIT backend * relayer: walletMonitor remove useless function * relayer: update worker init() funcs It is silly to accept a boolean in thing.init() vs just not running init if you don't want to run thing. * relayer: remove env property from Listener It is verified in init() in spy_listener.ts and rest_listen.ts, so it can be deferred to not require the import. * relayer: clean up the main bits Only try to init() something when it is actually supposed to run per the cli flags. * spy_relayer: more descriptive var name in main * spy_relayer: Update relay worker with more docs * spy_relayer: add targetChainName to WorkerInfo This makes it easier to pretty print the chain name in the logs without having to look the name up. * spy_relayer: update logs and use the backend * Use the Relayer interface's process() method in place of processVaa() * Update the logs to include the chain name in the worker and auditor threads * spy_relayer: remove processRequest() It has been moved to the process() method of the default Relayer backend. * spy_relayer: start fleshing out the Relayer default * spy_relayer: fix a logic bug in checkQueue() Co-authored-by: @swimricky * spy_relayer: update TokenBridgeRelayer.process() * Remove some extra logic * Actually use the ChainId type since the id of 0 is in the sdk now * spy_signature: add Relayer.runAuditor() The auditor code is payload specific and needs to be with the backend. * spy_relayer: move Relayer.runAuditor() Make it part of the backend since the backend has all of the payload specific logic into it. * spy_relayer: move relay() --> Relayer.relay() The actual relaying is part of the backend, so make it so. * spy_relayer: add Relayer.runAuditor() * spy_relayer: no more deprecated hexToNativeString * spy_relayer: implement Relayer.targetChainId() This is used for finding workable items in the incoming queue to toss into the working queue. * spy_relayer: remove relay.ts The relay() function was moved to Relayer.relay() * spy_relayer: more uses of deprecated hexToNativeString() * spy_relayer: remove unused import * spy_relayer: review feedback from @bruce-riley * Fix some spelling tyops * Simplify some logging * Simplify a conditional for the payload version check * spy_relayer: misc fixes and code clean up * Fixed integration tests * Added launch.json file for debugging in VS Code * Updated to latest wormhole SDK * Backup queue uses same key as redis * Added Terra Classic flag * Throttle potential infinite loop in audit thread * Fixed spy service connection leak Co-authored-by: Evan Gray <battledingo@gmail.com> Co-authored-by: Kevin Peters <kpeters@jumptrading.com>
2022-08-05 07:20:36 -07:00
runRelayWorker = false;
Spy relayer cleanup (#1015) * initial spy-relayer * Update spy_relayer Dockerfile * added example mainnet config files * split out private keys into its own ENV variable * Update spy relayer supportedChains.json To remove the `walletPrivateKey` entries. All of the private keys have been split out into their own json file. * fixed evm private key env parse * missing solana accounts report 0 balance, rather than error * wallet address is logged in debug * spy_relayer: enabled prometheus default metrics Also set a prefix of `relayer_` * spy_relayer: updates to the prometheus bits * Use a single metric registry * Use a simpler metric name and add labels for individual wallets * spy_relayer: human readable app mode in the metrics [ listener | relayer | both ] * spy_relayer: unify metrics * remove the collection of default metrics * hardcode the `spy_relayer_` prefix on all custom metrics * fixed dep arrays, nullable terra token/balance info * attempt stack debug * debug pullTerraBalance * provider http or ws * update sdk * logging for tokenAddress is 0 * fix foreign address calc * fix calcLocalAddressesTerra * relayer/spy_relayer: update prometheus helpers Add / url handler for the ingress-gce stupid load balancer that doesn't support custom url healthchecks unless you make a BackendConfig custom resource definition. * logging refinement * use chain name in prometheus * adjust retry timeout calculation * spy_relayer: update prometheus bits * improved error handling * relayer ui improvements * prep sdk release * use latest sdk, manual redeem button * relaying ux improvements * gas price fix * shortened terra success log * use gh base relayer list * fix prometheus urls * Update prometheus metric name * only show TPS warning on mainnet * show relayer fee in source preview * fix unwrap check * add native bool to balance metric * logging improvements * add feeRecipientAddress to redeemOnSolana * gather solana fees * remove relayer ws support * add nativeCurrencySymbol to ChainConfigInfo * fix solana native symbol * demoteWorking option, logger contexts * scoped logging * bridge_ui: unwrap native * add evm wallet monitor test * solana vaa parsing fix * add monitorRedis * make Jeff's brain happy * log demoting keys * register redisQueue metric * human readable redisQueue metric * fix timestamp inconsistency * use scopedLogger for the first level of workers * pull wallet balances in parallel * more scoped logging * pick a solana fee * moving keys log improvement * update eth gas calculations based on recent txs * use postVaaSolanaWithRetry * split success and failures by chain * fix using terraCoin * check prom every 10s * batch getting evm token balances * batch calcLocalAddressesEVM * debug worker logging * log retry number * support Polygon? * reset status on demotion * enhance! * update avax fee Co-authored-by: Chase Moran <chasemoran45@gmail.com> Co-authored-by: Kevin Peters <kpeters@jumptrading.com> Co-authored-by: Evan Gray <battledingo@gmail.com>
2022-03-28 20:39:08 -07:00
foundOne = true;
}
}
if (!foundOne) {
logger.info("spy_relay is running both the listener and relayer");
}
Modular relayer support (#1266) * Add the default backend for the relayer Start to slowly split things out. * Configure the backends when configuring the env * First stab at the pluggable listener backend * Update relayer example (and test) config * relayer: JIT backend * relayer: walletMonitor remove useless function * relayer: update worker init() funcs It is silly to accept a boolean in thing.init() vs just not running init if you don't want to run thing. * relayer: remove env property from Listener It is verified in init() in spy_listener.ts and rest_listen.ts, so it can be deferred to not require the import. * relayer: clean up the main bits Only try to init() something when it is actually supposed to run per the cli flags. * spy_relayer: more descriptive var name in main * spy_relayer: Update relay worker with more docs * spy_relayer: add targetChainName to WorkerInfo This makes it easier to pretty print the chain name in the logs without having to look the name up. * spy_relayer: update logs and use the backend * Use the Relayer interface's process() method in place of processVaa() * Update the logs to include the chain name in the worker and auditor threads * spy_relayer: remove processRequest() It has been moved to the process() method of the default Relayer backend. * spy_relayer: start fleshing out the Relayer default * spy_relayer: fix a logic bug in checkQueue() Co-authored-by: @swimricky * spy_relayer: update TokenBridgeRelayer.process() * Remove some extra logic * Actually use the ChainId type since the id of 0 is in the sdk now * spy_signature: add Relayer.runAuditor() The auditor code is payload specific and needs to be with the backend. * spy_relayer: move Relayer.runAuditor() Make it part of the backend since the backend has all of the payload specific logic into it. * spy_relayer: move relay() --> Relayer.relay() The actual relaying is part of the backend, so make it so. * spy_relayer: add Relayer.runAuditor() * spy_relayer: no more deprecated hexToNativeString * spy_relayer: implement Relayer.targetChainId() This is used for finding workable items in the incoming queue to toss into the working queue. * spy_relayer: remove relay.ts The relay() function was moved to Relayer.relay() * spy_relayer: more uses of deprecated hexToNativeString() * spy_relayer: remove unused import * spy_relayer: review feedback from @bruce-riley * Fix some spelling tyops * Simplify some logging * Simplify a conditional for the payload version check * spy_relayer: misc fixes and code clean up * Fixed integration tests * Added launch.json file for debugging in VS Code * Updated to latest wormhole SDK * Backup queue uses same key as redis * Added Terra Classic flag * Throttle potential infinite loop in audit thread * Fixed spy service connection leak Co-authored-by: Evan Gray <battledingo@gmail.com> Co-authored-by: Kevin Peters <kpeters@jumptrading.com>
2022-08-05 07:20:36 -07:00
const runAll: boolean = runListen && runRelayWorker && runWalletMonitor;
if (runListen && !spyListener.init()) {
process.exit(1);
}
Spy relayer cleanup (#1015) * initial spy-relayer * Update spy_relayer Dockerfile * added example mainnet config files * split out private keys into its own ENV variable * Update spy relayer supportedChains.json To remove the `walletPrivateKey` entries. All of the private keys have been split out into their own json file. * fixed evm private key env parse * missing solana accounts report 0 balance, rather than error * wallet address is logged in debug * spy_relayer: enabled prometheus default metrics Also set a prefix of `relayer_` * spy_relayer: updates to the prometheus bits * Use a single metric registry * Use a simpler metric name and add labels for individual wallets * spy_relayer: human readable app mode in the metrics [ listener | relayer | both ] * spy_relayer: unify metrics * remove the collection of default metrics * hardcode the `spy_relayer_` prefix on all custom metrics * fixed dep arrays, nullable terra token/balance info * attempt stack debug * debug pullTerraBalance * provider http or ws * update sdk * logging for tokenAddress is 0 * fix foreign address calc * fix calcLocalAddressesTerra * relayer/spy_relayer: update prometheus helpers Add / url handler for the ingress-gce stupid load balancer that doesn't support custom url healthchecks unless you make a BackendConfig custom resource definition. * logging refinement * use chain name in prometheus * adjust retry timeout calculation * spy_relayer: update prometheus bits * improved error handling * relayer ui improvements * prep sdk release * use latest sdk, manual redeem button * relaying ux improvements * gas price fix * shortened terra success log * use gh base relayer list * fix prometheus urls * Update prometheus metric name * only show TPS warning on mainnet * show relayer fee in source preview * fix unwrap check * add native bool to balance metric * logging improvements * add feeRecipientAddress to redeemOnSolana * gather solana fees * remove relayer ws support * add nativeCurrencySymbol to ChainConfigInfo * fix solana native symbol * demoteWorking option, logger contexts * scoped logging * bridge_ui: unwrap native * add evm wallet monitor test * solana vaa parsing fix * add monitorRedis * make Jeff's brain happy * log demoting keys * register redisQueue metric * human readable redisQueue metric * fix timestamp inconsistency * use scopedLogger for the first level of workers * pull wallet balances in parallel * more scoped logging * pick a solana fee * moving keys log improvement * update eth gas calculations based on recent txs * use postVaaSolanaWithRetry * split success and failures by chain * fix using terraCoin * check prom every 10s * batch getting evm token balances * batch calcLocalAddressesEVM * debug worker logging * log retry number * support Polygon? * reset status on demotion * enhance! * update avax fee Co-authored-by: Chase Moran <chasemoran45@gmail.com> Co-authored-by: Kevin Peters <kpeters@jumptrading.com> Co-authored-by: Evan Gray <battledingo@gmail.com>
2022-03-28 20:39:08 -07:00
Modular relayer support (#1266) * Add the default backend for the relayer Start to slowly split things out. * Configure the backends when configuring the env * First stab at the pluggable listener backend * Update relayer example (and test) config * relayer: JIT backend * relayer: walletMonitor remove useless function * relayer: update worker init() funcs It is silly to accept a boolean in thing.init() vs just not running init if you don't want to run thing. * relayer: remove env property from Listener It is verified in init() in spy_listener.ts and rest_listen.ts, so it can be deferred to not require the import. * relayer: clean up the main bits Only try to init() something when it is actually supposed to run per the cli flags. * spy_relayer: more descriptive var name in main * spy_relayer: Update relay worker with more docs * spy_relayer: add targetChainName to WorkerInfo This makes it easier to pretty print the chain name in the logs without having to look the name up. * spy_relayer: update logs and use the backend * Use the Relayer interface's process() method in place of processVaa() * Update the logs to include the chain name in the worker and auditor threads * spy_relayer: remove processRequest() It has been moved to the process() method of the default Relayer backend. * spy_relayer: start fleshing out the Relayer default * spy_relayer: fix a logic bug in checkQueue() Co-authored-by: @swimricky * spy_relayer: update TokenBridgeRelayer.process() * Remove some extra logic * Actually use the ChainId type since the id of 0 is in the sdk now * spy_signature: add Relayer.runAuditor() The auditor code is payload specific and needs to be with the backend. * spy_relayer: move Relayer.runAuditor() Make it part of the backend since the backend has all of the payload specific logic into it. * spy_relayer: move relay() --> Relayer.relay() The actual relaying is part of the backend, so make it so. * spy_relayer: add Relayer.runAuditor() * spy_relayer: no more deprecated hexToNativeString * spy_relayer: implement Relayer.targetChainId() This is used for finding workable items in the incoming queue to toss into the working queue. * spy_relayer: remove relay.ts The relay() function was moved to Relayer.relay() * spy_relayer: more uses of deprecated hexToNativeString() * spy_relayer: remove unused import * spy_relayer: review feedback from @bruce-riley * Fix some spelling tyops * Simplify some logging * Simplify a conditional for the payload version check * spy_relayer: misc fixes and code clean up * Fixed integration tests * Added launch.json file for debugging in VS Code * Updated to latest wormhole SDK * Backup queue uses same key as redis * Added Terra Classic flag * Throttle potential infinite loop in audit thread * Fixed spy service connection leak Co-authored-by: Evan Gray <battledingo@gmail.com> Co-authored-by: Kevin Peters <kpeters@jumptrading.com>
2022-08-05 07:20:36 -07:00
if (runRelayWorker && !relayWorker.init()) {
process.exit(1);
}
Spy relayer cleanup (#1015) * initial spy-relayer * Update spy_relayer Dockerfile * added example mainnet config files * split out private keys into its own ENV variable * Update spy relayer supportedChains.json To remove the `walletPrivateKey` entries. All of the private keys have been split out into their own json file. * fixed evm private key env parse * missing solana accounts report 0 balance, rather than error * wallet address is logged in debug * spy_relayer: enabled prometheus default metrics Also set a prefix of `relayer_` * spy_relayer: updates to the prometheus bits * Use a single metric registry * Use a simpler metric name and add labels for individual wallets * spy_relayer: human readable app mode in the metrics [ listener | relayer | both ] * spy_relayer: unify metrics * remove the collection of default metrics * hardcode the `spy_relayer_` prefix on all custom metrics * fixed dep arrays, nullable terra token/balance info * attempt stack debug * debug pullTerraBalance * provider http or ws * update sdk * logging for tokenAddress is 0 * fix foreign address calc * fix calcLocalAddressesTerra * relayer/spy_relayer: update prometheus helpers Add / url handler for the ingress-gce stupid load balancer that doesn't support custom url healthchecks unless you make a BackendConfig custom resource definition. * logging refinement * use chain name in prometheus * adjust retry timeout calculation * spy_relayer: update prometheus bits * improved error handling * relayer ui improvements * prep sdk release * use latest sdk, manual redeem button * relaying ux improvements * gas price fix * shortened terra success log * use gh base relayer list * fix prometheus urls * Update prometheus metric name * only show TPS warning on mainnet * show relayer fee in source preview * fix unwrap check * add native bool to balance metric * logging improvements * add feeRecipientAddress to redeemOnSolana * gather solana fees * remove relayer ws support * add nativeCurrencySymbol to ChainConfigInfo * fix solana native symbol * demoteWorking option, logger contexts * scoped logging * bridge_ui: unwrap native * add evm wallet monitor test * solana vaa parsing fix * add monitorRedis * make Jeff's brain happy * log demoting keys * register redisQueue metric * human readable redisQueue metric * fix timestamp inconsistency * use scopedLogger for the first level of workers * pull wallet balances in parallel * more scoped logging * pick a solana fee * moving keys log improvement * update eth gas calculations based on recent txs * use postVaaSolanaWithRetry * split success and failures by chain * fix using terraCoin * check prom every 10s * batch getting evm token balances * batch calcLocalAddressesEVM * debug worker logging * log retry number * support Polygon? * reset status on demotion * enhance! * update avax fee Co-authored-by: Chase Moran <chasemoran45@gmail.com> Co-authored-by: Kevin Peters <kpeters@jumptrading.com> Co-authored-by: Evan Gray <battledingo@gmail.com>
2022-03-28 20:39:08 -07:00
Modular relayer support (#1266) * Add the default backend for the relayer Start to slowly split things out. * Configure the backends when configuring the env * First stab at the pluggable listener backend * Update relayer example (and test) config * relayer: JIT backend * relayer: walletMonitor remove useless function * relayer: update worker init() funcs It is silly to accept a boolean in thing.init() vs just not running init if you don't want to run thing. * relayer: remove env property from Listener It is verified in init() in spy_listener.ts and rest_listen.ts, so it can be deferred to not require the import. * relayer: clean up the main bits Only try to init() something when it is actually supposed to run per the cli flags. * spy_relayer: more descriptive var name in main * spy_relayer: Update relay worker with more docs * spy_relayer: add targetChainName to WorkerInfo This makes it easier to pretty print the chain name in the logs without having to look the name up. * spy_relayer: update logs and use the backend * Use the Relayer interface's process() method in place of processVaa() * Update the logs to include the chain name in the worker and auditor threads * spy_relayer: remove processRequest() It has been moved to the process() method of the default Relayer backend. * spy_relayer: start fleshing out the Relayer default * spy_relayer: fix a logic bug in checkQueue() Co-authored-by: @swimricky * spy_relayer: update TokenBridgeRelayer.process() * Remove some extra logic * Actually use the ChainId type since the id of 0 is in the sdk now * spy_signature: add Relayer.runAuditor() The auditor code is payload specific and needs to be with the backend. * spy_relayer: move Relayer.runAuditor() Make it part of the backend since the backend has all of the payload specific logic into it. * spy_relayer: move relay() --> Relayer.relay() The actual relaying is part of the backend, so make it so. * spy_relayer: add Relayer.runAuditor() * spy_relayer: no more deprecated hexToNativeString * spy_relayer: implement Relayer.targetChainId() This is used for finding workable items in the incoming queue to toss into the working queue. * spy_relayer: remove relay.ts The relay() function was moved to Relayer.relay() * spy_relayer: more uses of deprecated hexToNativeString() * spy_relayer: remove unused import * spy_relayer: review feedback from @bruce-riley * Fix some spelling tyops * Simplify some logging * Simplify a conditional for the payload version check * spy_relayer: misc fixes and code clean up * Fixed integration tests * Added launch.json file for debugging in VS Code * Updated to latest wormhole SDK * Backup queue uses same key as redis * Added Terra Classic flag * Throttle potential infinite loop in audit thread * Fixed spy service connection leak Co-authored-by: Evan Gray <battledingo@gmail.com> Co-authored-by: Kevin Peters <kpeters@jumptrading.com>
2022-08-05 07:20:36 -07:00
if (runRest && !restListener.init()) {
process.exit(1);
}
Spy relayer cleanup (#1015) * initial spy-relayer * Update spy_relayer Dockerfile * added example mainnet config files * split out private keys into its own ENV variable * Update spy relayer supportedChains.json To remove the `walletPrivateKey` entries. All of the private keys have been split out into their own json file. * fixed evm private key env parse * missing solana accounts report 0 balance, rather than error * wallet address is logged in debug * spy_relayer: enabled prometheus default metrics Also set a prefix of `relayer_` * spy_relayer: updates to the prometheus bits * Use a single metric registry * Use a simpler metric name and add labels for individual wallets * spy_relayer: human readable app mode in the metrics [ listener | relayer | both ] * spy_relayer: unify metrics * remove the collection of default metrics * hardcode the `spy_relayer_` prefix on all custom metrics * fixed dep arrays, nullable terra token/balance info * attempt stack debug * debug pullTerraBalance * provider http or ws * update sdk * logging for tokenAddress is 0 * fix foreign address calc * fix calcLocalAddressesTerra * relayer/spy_relayer: update prometheus helpers Add / url handler for the ingress-gce stupid load balancer that doesn't support custom url healthchecks unless you make a BackendConfig custom resource definition. * logging refinement * use chain name in prometheus * adjust retry timeout calculation * spy_relayer: update prometheus bits * improved error handling * relayer ui improvements * prep sdk release * use latest sdk, manual redeem button * relaying ux improvements * gas price fix * shortened terra success log * use gh base relayer list * fix prometheus urls * Update prometheus metric name * only show TPS warning on mainnet * show relayer fee in source preview * fix unwrap check * add native bool to balance metric * logging improvements * add feeRecipientAddress to redeemOnSolana * gather solana fees * remove relayer ws support * add nativeCurrencySymbol to ChainConfigInfo * fix solana native symbol * demoteWorking option, logger contexts * scoped logging * bridge_ui: unwrap native * add evm wallet monitor test * solana vaa parsing fix * add monitorRedis * make Jeff's brain happy * log demoting keys * register redisQueue metric * human readable redisQueue metric * fix timestamp inconsistency * use scopedLogger for the first level of workers * pull wallet balances in parallel * more scoped logging * pick a solana fee * moving keys log improvement * update eth gas calculations based on recent txs * use postVaaSolanaWithRetry * split success and failures by chain * fix using terraCoin * check prom every 10s * batch getting evm token balances * batch calcLocalAddressesEVM * debug worker logging * log retry number * support Polygon? * reset status on demotion * enhance! * update avax fee Co-authored-by: Chase Moran <chasemoran45@gmail.com> Co-authored-by: Kevin Peters <kpeters@jumptrading.com> Co-authored-by: Evan Gray <battledingo@gmail.com>
2022-03-28 20:39:08 -07:00
Modular relayer support (#1266) * Add the default backend for the relayer Start to slowly split things out. * Configure the backends when configuring the env * First stab at the pluggable listener backend * Update relayer example (and test) config * relayer: JIT backend * relayer: walletMonitor remove useless function * relayer: update worker init() funcs It is silly to accept a boolean in thing.init() vs just not running init if you don't want to run thing. * relayer: remove env property from Listener It is verified in init() in spy_listener.ts and rest_listen.ts, so it can be deferred to not require the import. * relayer: clean up the main bits Only try to init() something when it is actually supposed to run per the cli flags. * spy_relayer: more descriptive var name in main * spy_relayer: Update relay worker with more docs * spy_relayer: add targetChainName to WorkerInfo This makes it easier to pretty print the chain name in the logs without having to look the name up. * spy_relayer: update logs and use the backend * Use the Relayer interface's process() method in place of processVaa() * Update the logs to include the chain name in the worker and auditor threads * spy_relayer: remove processRequest() It has been moved to the process() method of the default Relayer backend. * spy_relayer: start fleshing out the Relayer default * spy_relayer: fix a logic bug in checkQueue() Co-authored-by: @swimricky * spy_relayer: update TokenBridgeRelayer.process() * Remove some extra logic * Actually use the ChainId type since the id of 0 is in the sdk now * spy_signature: add Relayer.runAuditor() The auditor code is payload specific and needs to be with the backend. * spy_relayer: move Relayer.runAuditor() Make it part of the backend since the backend has all of the payload specific logic into it. * spy_relayer: move relay() --> Relayer.relay() The actual relaying is part of the backend, so make it so. * spy_relayer: add Relayer.runAuditor() * spy_relayer: no more deprecated hexToNativeString * spy_relayer: implement Relayer.targetChainId() This is used for finding workable items in the incoming queue to toss into the working queue. * spy_relayer: remove relay.ts The relay() function was moved to Relayer.relay() * spy_relayer: more uses of deprecated hexToNativeString() * spy_relayer: remove unused import * spy_relayer: review feedback from @bruce-riley * Fix some spelling tyops * Simplify some logging * Simplify a conditional for the payload version check * spy_relayer: misc fixes and code clean up * Fixed integration tests * Added launch.json file for debugging in VS Code * Updated to latest wormhole SDK * Backup queue uses same key as redis * Added Terra Classic flag * Throttle potential infinite loop in audit thread * Fixed spy service connection leak Co-authored-by: Evan Gray <battledingo@gmail.com> Co-authored-by: Kevin Peters <kpeters@jumptrading.com>
2022-08-05 07:20:36 -07:00
if (runWalletMonitor && !walletMonitor.init()) {
process.exit(1);
}
Spy relayer cleanup (#1015) * initial spy-relayer * Update spy_relayer Dockerfile * added example mainnet config files * split out private keys into its own ENV variable * Update spy relayer supportedChains.json To remove the `walletPrivateKey` entries. All of the private keys have been split out into their own json file. * fixed evm private key env parse * missing solana accounts report 0 balance, rather than error * wallet address is logged in debug * spy_relayer: enabled prometheus default metrics Also set a prefix of `relayer_` * spy_relayer: updates to the prometheus bits * Use a single metric registry * Use a simpler metric name and add labels for individual wallets * spy_relayer: human readable app mode in the metrics [ listener | relayer | both ] * spy_relayer: unify metrics * remove the collection of default metrics * hardcode the `spy_relayer_` prefix on all custom metrics * fixed dep arrays, nullable terra token/balance info * attempt stack debug * debug pullTerraBalance * provider http or ws * update sdk * logging for tokenAddress is 0 * fix foreign address calc * fix calcLocalAddressesTerra * relayer/spy_relayer: update prometheus helpers Add / url handler for the ingress-gce stupid load balancer that doesn't support custom url healthchecks unless you make a BackendConfig custom resource definition. * logging refinement * use chain name in prometheus * adjust retry timeout calculation * spy_relayer: update prometheus bits * improved error handling * relayer ui improvements * prep sdk release * use latest sdk, manual redeem button * relaying ux improvements * gas price fix * shortened terra success log * use gh base relayer list * fix prometheus urls * Update prometheus metric name * only show TPS warning on mainnet * show relayer fee in source preview * fix unwrap check * add native bool to balance metric * logging improvements * add feeRecipientAddress to redeemOnSolana * gather solana fees * remove relayer ws support * add nativeCurrencySymbol to ChainConfigInfo * fix solana native symbol * demoteWorking option, logger contexts * scoped logging * bridge_ui: unwrap native * add evm wallet monitor test * solana vaa parsing fix * add monitorRedis * make Jeff's brain happy * log demoting keys * register redisQueue metric * human readable redisQueue metric * fix timestamp inconsistency * use scopedLogger for the first level of workers * pull wallet balances in parallel * more scoped logging * pick a solana fee * moving keys log improvement * update eth gas calculations based on recent txs * use postVaaSolanaWithRetry * split success and failures by chain * fix using terraCoin * check prom every 10s * batch getting evm token balances * batch calcLocalAddressesEVM * debug worker logging * log retry number * support Polygon? * reset status on demotion * enhance! * update avax fee Co-authored-by: Chase Moran <chasemoran45@gmail.com> Co-authored-by: Kevin Peters <kpeters@jumptrading.com> Co-authored-by: Evan Gray <battledingo@gmail.com>
2022-03-28 20:39:08 -07:00
Modular relayer support (#1266) * Add the default backend for the relayer Start to slowly split things out. * Configure the backends when configuring the env * First stab at the pluggable listener backend * Update relayer example (and test) config * relayer: JIT backend * relayer: walletMonitor remove useless function * relayer: update worker init() funcs It is silly to accept a boolean in thing.init() vs just not running init if you don't want to run thing. * relayer: remove env property from Listener It is verified in init() in spy_listener.ts and rest_listen.ts, so it can be deferred to not require the import. * relayer: clean up the main bits Only try to init() something when it is actually supposed to run per the cli flags. * spy_relayer: more descriptive var name in main * spy_relayer: Update relay worker with more docs * spy_relayer: add targetChainName to WorkerInfo This makes it easier to pretty print the chain name in the logs without having to look the name up. * spy_relayer: update logs and use the backend * Use the Relayer interface's process() method in place of processVaa() * Update the logs to include the chain name in the worker and auditor threads * spy_relayer: remove processRequest() It has been moved to the process() method of the default Relayer backend. * spy_relayer: start fleshing out the Relayer default * spy_relayer: fix a logic bug in checkQueue() Co-authored-by: @swimricky * spy_relayer: update TokenBridgeRelayer.process() * Remove some extra logic * Actually use the ChainId type since the id of 0 is in the sdk now * spy_signature: add Relayer.runAuditor() The auditor code is payload specific and needs to be with the backend. * spy_relayer: move Relayer.runAuditor() Make it part of the backend since the backend has all of the payload specific logic into it. * spy_relayer: move relay() --> Relayer.relay() The actual relaying is part of the backend, so make it so. * spy_relayer: add Relayer.runAuditor() * spy_relayer: no more deprecated hexToNativeString * spy_relayer: implement Relayer.targetChainId() This is used for finding workable items in the incoming queue to toss into the working queue. * spy_relayer: remove relay.ts The relay() function was moved to Relayer.relay() * spy_relayer: more uses of deprecated hexToNativeString() * spy_relayer: remove unused import * spy_relayer: review feedback from @bruce-riley * Fix some spelling tyops * Simplify some logging * Simplify a conditional for the payload version check * spy_relayer: misc fixes and code clean up * Fixed integration tests * Added launch.json file for debugging in VS Code * Updated to latest wormhole SDK * Backup queue uses same key as redis * Added Terra Classic flag * Throttle potential infinite loop in audit thread * Fixed spy service connection leak Co-authored-by: Evan Gray <battledingo@gmail.com> Co-authored-by: Kevin Peters <kpeters@jumptrading.com>
2022-08-05 07:20:36 -07:00
if (error) {
logger.error(error);
process.exit(1);
}
const commonEnv = getCommonEnvironment();
const { promPort, readinessPort } = commonEnv;
logger.info("prometheus client listening on port " + promPort);
let promClient: PromHelper;
if (runAll) {
promClient = new PromHelper("spy_relay", promPort, PromMode.All);
} else if (runListen) {
promClient = new PromHelper("spy_relay", promPort, PromMode.Listen);
} else if (runRelayWorker) {
promClient = new PromHelper("spy_relay", promPort, PromMode.Relay);
} else if (runWalletMonitor) {
promClient = new PromHelper("spy_relay", promPort, PromMode.WalletMonitor);
Spy relayer cleanup (#1015) * initial spy-relayer * Update spy_relayer Dockerfile * added example mainnet config files * split out private keys into its own ENV variable * Update spy relayer supportedChains.json To remove the `walletPrivateKey` entries. All of the private keys have been split out into their own json file. * fixed evm private key env parse * missing solana accounts report 0 balance, rather than error * wallet address is logged in debug * spy_relayer: enabled prometheus default metrics Also set a prefix of `relayer_` * spy_relayer: updates to the prometheus bits * Use a single metric registry * Use a simpler metric name and add labels for individual wallets * spy_relayer: human readable app mode in the metrics [ listener | relayer | both ] * spy_relayer: unify metrics * remove the collection of default metrics * hardcode the `spy_relayer_` prefix on all custom metrics * fixed dep arrays, nullable terra token/balance info * attempt stack debug * debug pullTerraBalance * provider http or ws * update sdk * logging for tokenAddress is 0 * fix foreign address calc * fix calcLocalAddressesTerra * relayer/spy_relayer: update prometheus helpers Add / url handler for the ingress-gce stupid load balancer that doesn't support custom url healthchecks unless you make a BackendConfig custom resource definition. * logging refinement * use chain name in prometheus * adjust retry timeout calculation * spy_relayer: update prometheus bits * improved error handling * relayer ui improvements * prep sdk release * use latest sdk, manual redeem button * relaying ux improvements * gas price fix * shortened terra success log * use gh base relayer list * fix prometheus urls * Update prometheus metric name * only show TPS warning on mainnet * show relayer fee in source preview * fix unwrap check * add native bool to balance metric * logging improvements * add feeRecipientAddress to redeemOnSolana * gather solana fees * remove relayer ws support * add nativeCurrencySymbol to ChainConfigInfo * fix solana native symbol * demoteWorking option, logger contexts * scoped logging * bridge_ui: unwrap native * add evm wallet monitor test * solana vaa parsing fix * add monitorRedis * make Jeff's brain happy * log demoting keys * register redisQueue metric * human readable redisQueue metric * fix timestamp inconsistency * use scopedLogger for the first level of workers * pull wallet balances in parallel * more scoped logging * pick a solana fee * moving keys log improvement * update eth gas calculations based on recent txs * use postVaaSolanaWithRetry * split success and failures by chain * fix using terraCoin * check prom every 10s * batch getting evm token balances * batch calcLocalAddressesEVM * debug worker logging * log retry number * support Polygon? * reset status on demotion * enhance! * update avax fee Co-authored-by: Chase Moran <chasemoran45@gmail.com> Co-authored-by: Kevin Peters <kpeters@jumptrading.com> Co-authored-by: Evan Gray <battledingo@gmail.com>
2022-03-28 20:39:08 -07:00
} else {
Modular relayer support (#1266) * Add the default backend for the relayer Start to slowly split things out. * Configure the backends when configuring the env * First stab at the pluggable listener backend * Update relayer example (and test) config * relayer: JIT backend * relayer: walletMonitor remove useless function * relayer: update worker init() funcs It is silly to accept a boolean in thing.init() vs just not running init if you don't want to run thing. * relayer: remove env property from Listener It is verified in init() in spy_listener.ts and rest_listen.ts, so it can be deferred to not require the import. * relayer: clean up the main bits Only try to init() something when it is actually supposed to run per the cli flags. * spy_relayer: more descriptive var name in main * spy_relayer: Update relay worker with more docs * spy_relayer: add targetChainName to WorkerInfo This makes it easier to pretty print the chain name in the logs without having to look the name up. * spy_relayer: update logs and use the backend * Use the Relayer interface's process() method in place of processVaa() * Update the logs to include the chain name in the worker and auditor threads * spy_relayer: remove processRequest() It has been moved to the process() method of the default Relayer backend. * spy_relayer: start fleshing out the Relayer default * spy_relayer: fix a logic bug in checkQueue() Co-authored-by: @swimricky * spy_relayer: update TokenBridgeRelayer.process() * Remove some extra logic * Actually use the ChainId type since the id of 0 is in the sdk now * spy_signature: add Relayer.runAuditor() The auditor code is payload specific and needs to be with the backend. * spy_relayer: move Relayer.runAuditor() Make it part of the backend since the backend has all of the payload specific logic into it. * spy_relayer: move relay() --> Relayer.relay() The actual relaying is part of the backend, so make it so. * spy_relayer: add Relayer.runAuditor() * spy_relayer: no more deprecated hexToNativeString * spy_relayer: implement Relayer.targetChainId() This is used for finding workable items in the incoming queue to toss into the working queue. * spy_relayer: remove relay.ts The relay() function was moved to Relayer.relay() * spy_relayer: more uses of deprecated hexToNativeString() * spy_relayer: remove unused import * spy_relayer: review feedback from @bruce-riley * Fix some spelling tyops * Simplify some logging * Simplify a conditional for the payload version check * spy_relayer: misc fixes and code clean up * Fixed integration tests * Added launch.json file for debugging in VS Code * Updated to latest wormhole SDK * Backup queue uses same key as redis * Added Terra Classic flag * Throttle potential infinite loop in audit thread * Fixed spy service connection leak Co-authored-by: Evan Gray <battledingo@gmail.com> Co-authored-by: Kevin Peters <kpeters@jumptrading.com>
2022-08-05 07:20:36 -07:00
logger.error("Invalid run mode for Prometheus");
promClient = new PromHelper("spy_relay", promPort, PromMode.All);
}
redisHelper.init(promClient);
if (runListen) spyListener.run(promClient);
if (runRelayWorker) relayWorker.run(promClient);
if (runRest) restListener.run();
if (runWalletMonitor) walletMonitor.run(promClient);
if (readinessPort) {
const Net = require("net");
const readinessServer = new Net.Server();
readinessServer.listen(readinessPort, function () {
logger.info("listening for readiness requests on port " + readinessPort);
});
readinessServer.on("connection", function (socket: any) {
//logger.debug("readiness connection");
});
Spy relayer cleanup (#1015) * initial spy-relayer * Update spy_relayer Dockerfile * added example mainnet config files * split out private keys into its own ENV variable * Update spy relayer supportedChains.json To remove the `walletPrivateKey` entries. All of the private keys have been split out into their own json file. * fixed evm private key env parse * missing solana accounts report 0 balance, rather than error * wallet address is logged in debug * spy_relayer: enabled prometheus default metrics Also set a prefix of `relayer_` * spy_relayer: updates to the prometheus bits * Use a single metric registry * Use a simpler metric name and add labels for individual wallets * spy_relayer: human readable app mode in the metrics [ listener | relayer | both ] * spy_relayer: unify metrics * remove the collection of default metrics * hardcode the `spy_relayer_` prefix on all custom metrics * fixed dep arrays, nullable terra token/balance info * attempt stack debug * debug pullTerraBalance * provider http or ws * update sdk * logging for tokenAddress is 0 * fix foreign address calc * fix calcLocalAddressesTerra * relayer/spy_relayer: update prometheus helpers Add / url handler for the ingress-gce stupid load balancer that doesn't support custom url healthchecks unless you make a BackendConfig custom resource definition. * logging refinement * use chain name in prometheus * adjust retry timeout calculation * spy_relayer: update prometheus bits * improved error handling * relayer ui improvements * prep sdk release * use latest sdk, manual redeem button * relaying ux improvements * gas price fix * shortened terra success log * use gh base relayer list * fix prometheus urls * Update prometheus metric name * only show TPS warning on mainnet * show relayer fee in source preview * fix unwrap check * add native bool to balance metric * logging improvements * add feeRecipientAddress to redeemOnSolana * gather solana fees * remove relayer ws support * add nativeCurrencySymbol to ChainConfigInfo * fix solana native symbol * demoteWorking option, logger contexts * scoped logging * bridge_ui: unwrap native * add evm wallet monitor test * solana vaa parsing fix * add monitorRedis * make Jeff's brain happy * log demoting keys * register redisQueue metric * human readable redisQueue metric * fix timestamp inconsistency * use scopedLogger for the first level of workers * pull wallet balances in parallel * more scoped logging * pick a solana fee * moving keys log improvement * update eth gas calculations based on recent txs * use postVaaSolanaWithRetry * split success and failures by chain * fix using terraCoin * check prom every 10s * batch getting evm token balances * batch calcLocalAddressesEVM * debug worker logging * log retry number * support Polygon? * reset status on demotion * enhance! * update avax fee Co-authored-by: Chase Moran <chasemoran45@gmail.com> Co-authored-by: Kevin Peters <kpeters@jumptrading.com> Co-authored-by: Evan Gray <battledingo@gmail.com>
2022-03-28 20:39:08 -07:00
}