wormhole-explorer/blockchain-watcher
Julian c15fa976a6
[Blockchain Watcher] Feature 798/enable evm karura chain (#831)
* adding gen-relayer watcher code to branch

* folder rename

* adding some new  abstract types for processor redesign

* added first cut of new process manager logic

* large refactoring to environment configuration

* Add basic default config

* prettier run

* modifying event handler interface

* abstract handler typing changes

* Blockchain watcher: adding domain and infra layers (#786)

* reorg domain-infra

* watch evm blocks action

* adding evm log parser

* wider prettier

* renaming watch action

* adding doc

* persist latest metadata

* gh action for blockchain-watcher

* adding log-message-published mapper

* deps: remove peers and nodemon

* adding handler for LogMessagePublished

* added parser for log message published

---------

Co-authored-by: chase-45 <chasemoran45@gmail.com>

* Blockchain Watcher: ethereum -> sns implementation (#790)

* evm block repo implementation

* adding sns publisher

* adding external config and initial runner

* fix: start from latest if no fromBlock configured

* feat: add dryRun option

* fix: handler filtering and eth_getLogs filter payload

* local e2e

* actual sns arns for testnet

* smaller docker image

* deployment changes

* [Blockchain Watcher] Apply missing formatting for log message published (#791)

* Apply missing formatting for log message published

* keep extracting

* emitterAddress -> emitter

* using configured default winston logger

* wait for new block

* adding basic metrics

* staging mainnet config

* send hexa block number

* simpler log format

* group entities in different files

* [Blockchain watcher] Adding light observability (#793)

* using configured default winston logger

* wait for new block

* adding basic metrics

* staging mainnet config

* send hexa block number

* simpler log format

* better error logs

* wait when no block available

* grouping watchers

* adding 429 handling http client

* abstracting polling iteration

* Load jobs dinamically

* changing deployment strategy to jobs per pod

* group actions per platform

* [Blockchain Watcher] Add StartJob action + rate limit handling (#799)

* using configured default winston logger

* wait for new block

* adding basic metrics

* staging mainnet config

* send hexa block number

* simpler log format

* group entities in different files

* grouping watchers

* adding 429 handling http client

* abstracting polling iteration

* Load jobs dinamically

* changing deployment strategy to jobs per pod

* adding poll solana txs action

* adding solana repo implementation

* solana: mapping to logmsgpublished

* fixing solana mapper test

* updating readme

* adding some metrics

* adding solana worker

* adding finality param to get solana block

* minor changes - using confirmed for solana log extraction

* updating version

* add solana env var

* [Blockchain Watcher] Feature-813/fix-coverage-report (#815)

* Fix coverage report

* Run prettier

* Rollback global coverage value

* Set global coverage value to 55

* Add github PR template

* Run prettier

* Rename github template file

---------

Co-authored-by: Julian Merlo <julianmerlo@MacBook-Pro-de-Julian.local>

* private solana urls as secret

* adding prod cfg files

* Change repository name and cretae error foulder (#819)

* Change repository name and cretae error foulder

* Rename SolanaFailure file

* Rename SolanaFailure file

---------

Co-authored-by: Julian Merlo <julianmerlo@MacBook-Pro-de-Julian.local>

* passing node_options for solana

* adding missing envs templates

* cru

* ratelimited solana client + increase coverage

* [Blockchain Watcher] Feature 817/add hexagonal prefix (#824)

* Put hexadecimal prefix

* Create id variable

* Add error log

* Add error log

* Remove bigint in getBlock params

---------

Co-authored-by: Julian Merlo <julianmerlo@MacBook-Pro-de-Julian.local>

* fix never ending solana jobs

* adding job counter metric

* solana: add retry handling for 429s

* cleanup

* remove duplicated tests

* [Blockchain watcher] feature-798/enable-evm-karura-chain

* [Blockchain watcher] feature-798/enable-evm-karura-chain

* Add karura job in mainnet

* Mapped karura addresses

* Improve address tesnet

* Delete from block

* Resolve comment in PR, mapped chainId into EvmLog entity

* Revert address change

* Mapped mainet address

* Create chains constants

* Imporve config mock

* Resolve comment

* Mapped chainId into repository

* Run prettier

* Change commitment status for karura

* Run prettier

---------

Co-authored-by: chase-45 <chasemoran45@gmail.com>
Co-authored-by: matias martinez <matias@xlabs.xyz>
Co-authored-by: Matías Martínez <131624652+mat1asm@users.noreply.github.com>
Co-authored-by: Julian Merlo <julianmerlo@MacBook-Pro-de-Julian.local>
2023-11-30 12:43:37 -03:00
..
.github [Blockchain Watcher] Emit LogMessagePublished for Solana (#809) 2023-11-30 12:05:43 -03:00
config [Blockchain Watcher] Feature 798/enable evm karura chain (#831) 2023-11-30 12:43:37 -03:00
docs [Blockchain Watcher] Initial version (#806) 2023-11-28 16:00:45 -03:00
src [Blockchain Watcher] Feature 798/enable evm karura chain (#831) 2023-11-30 12:43:37 -03:00
test [Blockchain Watcher] Feature 798/enable evm karura chain (#831) 2023-11-30 12:43:37 -03:00
.dockerignore [Blockchain Watcher] Initial version (#806) 2023-11-28 16:00:45 -03:00
.gitignore [Blockchain Watcher] Initial version (#806) 2023-11-28 16:00:45 -03:00
.prettierrc.json [Blockchain Watcher] Initial version (#806) 2023-11-28 16:00:45 -03:00
Dockerfile [Blockchain Watcher] Initial version (#806) 2023-11-28 16:00:45 -03:00
README.md [Blockchain Watcher] Emit LogMessagePublished for Solana (#809) 2023-11-30 12:05:43 -03:00
TODO.md [Blockchain Watcher] Initial version (#806) 2023-11-28 16:00:45 -03:00
jest.config.js [Blockchain Watcher] Emit LogMessagePublished for Solana (#809) 2023-11-30 12:05:43 -03:00
package-lock.json [Blockchain Watcher] Emit LogMessagePublished for Solana (#809) 2023-11-30 12:05:43 -03:00
package.json [Blockchain Watcher] Emit LogMessagePublished for Solana (#809) 2023-11-30 12:05:43 -03:00
tsconfig.json [Blockchain Watcher] Initial version (#806) 2023-11-28 16:00:45 -03:00

README.md

Explorer Blockchain Watcher

The purpose of this process is to watch all Wormhole connected blockchains for events in Wormhole ecosystem contracts, and then produce database records for these events, and other important associated data.

Installation

run npm ci in the root of the project run npm dev in the root of the project

Deployment

This process is meant to be deployed as a docker container. The dockerfile is located in the root of the project. As of today, we have a one to one relationship from job to pod. So we statically define jobs as a config map that each pod then loads. See ../deploy/blockchain-watcher/workers for an example of how deployment works.

To deploy all workers for a given environment, run:

$  kubetpl render workers/* -i env/staging-testnet.env --syntax=go-template | kubectl apply -f -

Configuration

Configuration is loaded from files in config directory. There is a default file, and then a file for each environment. The environment is set by the NODE_ENV environment variable. If NODE_ENV is not set, the default file is used.

Some values may be overriden by using environment variables. See config/custom-environment-variables.json for a list of these variables.

$ NODE_ENV=staging LOG_LEVEL=debug npm run dev

When running locally, you need to configure one or more jobs. By default, jobs are read from metadata-repo/jobs/jobs.json.

Example:

{ "jobs": [
  {
    "id": "poll-log-message-published-ethereum",
    "chain": "ethereum",
    "source": {
      "action": "PollEvmLogs",
      "config": {
        "fromBlock": "10012499",
        "blockBatchSize": 100,
        "commitment": "latest",
        "interval": 15000,
        "addresses": ["0x706abc4E45D419950511e474C7B9Ed348A4a716c"],
        "chain": "ethereum",
        "topics": []
      }
    },
    "handlers": [
      {
        "action": "HandleEvmLogs",
        "target": "sns",
        "mapper": "evmLogMessagePublishedMapper",
        "config": {
          "abi": "event LogMessagePublished(address indexed sender, uint64 sequence, uint32 nonce, bytes payload, uint8 consistencyLevel)",
          "filter": {
            "addresses": ["0x706abc4E45D419950511e474C7B9Ed348A4a716c"],
            "topics": ["0x6eb224fb001ed210e379b335e35efe88672a8ce935d981a6896b27ffdf52a3b2"]
          }
        }
      }
    ]
  }
]}

Usage & Modification

Currently, jobs are read and loaded based on a JSON file. Each job has a source, and one or more handlers. Each handler has an action, a mapper and a target. For example, you can choose to use PollEvmLogs as an action and HandleEvmLogs as a handler. Fot this handler you need to set a mapper like evmLogMessagePublishedMapper. The target can be sns, or a fake one if dryRun is enabled.