Commit Graph

67 Commits

Author SHA1 Message Date
walker-16 7545acb77e
Feature/handle governor fly event processor (#1436)
* changes in fly component to send governor status events

* split processor in vaaProcessor and governor processor

* add governor processsor v1

* Add endpoint in tx-tracker to calculate txHash for a vaa id

* fly-event-processor integration with new tx-tracker endpoint and refactor

* Add governor vaas endpoint in api

* api, fix amount data type in governor vaas endpoint

* fly-event-processor normalize emitter and txHash

* fly-event-processor fix nodeGovernorVaa id

* fly-event-processor control array not empty in insert/delete many operation

* add index in nodeGovernorVaas collection by vaaId

* add prometheus metrics

* add tx-tracker url for fly-event-processor deployment

* Add sns attributes into sns messages

Co-authored-by: walker-16 <agpazos85@gmail.com>

* fix governor vaa endpoint empty response

---------

Co-authored-by: Fernando Torres <fert1335@gmail.com>
2024-05-31 15:38:21 -03:00
ftocal d4b63be148
Fix go dependencies (#1448)
* Update dependencies in tx-tracker and common

Co-authored-by: walker-16 <agpazos85@gmail.com>

* Update dependencies for analytics, fly-event-processor, jobs, notional, parser and pipeline

Co-authored-by: walker-16 <agpazos85@gmail.com>

---------

Co-authored-by: walker-16 <agpazos85@gmail.com>
2024-05-28 11:30:41 -03:00
walker-16 f354af6d96
1407 update sdk and modify tx tracker to support xlayer (#1440)
* Add support to xlayer in tx-tracker

* update commons chainIDs methods
2024-05-28 10:28:11 -03:00
walker-16 ccc7e558c4
add polygon amoy to tx-tracker (#1373)
Co-authored-by: ftocal <fert1335@gmail.com>
2024-05-13 12:11:55 -03:00
walker-16 b8e3f43333
1301 create a backgroundrecurrent process which checks which duplicate vaas txs actually reached finality on the blockchain (#1359)
* fly-event-processor initial commit

* Handle duplicated vaa event in fly-event-processor
Add deployment for fly-event-processor
Send duplicated vaa event from fly

Co-authored-by: walker-16 <agpazos85@gmail.com>

---------

Co-authored-by: Fernando Torres <fert1335@gmail.com>
2024-05-07 16:38:16 -03:00
walker-16 90b818884e
Add blast to tx-tracker (#1367)
Co-authored-by: ftocal <fert1335@gmail.com>
2024-04-29 17:39:58 -03:00
ftocal 688a0d0f71
Store duplicate vaas on a secondary collection (#1308)
* Add support to scroll in tx-tracker

* Handle duplicated vaa in fly

Co-authored-by: walker-16 <agpazos85@gmail.com>

* Modify tx-tracker and pipeline to support duplicated vaas
Co-authored-by: ftocal <fert1335@gmail.com>

* Add isDuplicated flag in vaas and operations endpoints
Add new endpoint to find duplicated vaas by vaa id
Co-authored-by: walker-16 <agpazos85@gmail.com>

* Add metrics for duplicated vaas

Co-authored-by: walker-16 <agpazos85@gmail.com>

* Update node dependencies in spy

Co-authored-by: walker-16 <agpazos85@gmail.com>

* Fix promethes metrics for missing tx-hash in tx-tracker

Co-authored-by: walker-16 <agpazos85@gmail.com>

---------

Co-authored-by: Agustin Pazos <agpazos85@gmail.com>
2024-04-22 14:26:07 -03:00
walker-16 8dc2b5ac5f
Normalize txHash destination in tx-tracker component (#1274)
Co-authored-by: ftocal <fert1335@gmail.com>
2024-04-08 14:05:48 -03:00
walker-16 04da0b3e4d
Add wormchain injective integration (#1260)
* Add wormchain injective integration

* Add metrics and logger for unknown wormchain

* Add wormchain rpc config
Co-authored-by: ftocal <fert1335@gmail.com>

* Add evmos, kujira, osmosis to wormchainRpcProviders
Co-authored-by: ftocal <fert1335@gmail.com>

* Add backfiller for tx-tracker with emitterChainId, emitterAddress and range time parameters

Co-authored-by: walker-16 <agpazos85@gmail.com>

---------

Co-authored-by: Fernando Torres <fert1335@gmail.com>
2024-04-05 15:24:21 -03:00
ftocal adb7231074
Add processed message metrics in pipeline components (#1239)
Co-authored-by: walker-16 <agpazos85@gmail.com>
2024-03-25 15:21:10 -03:00
walker-16 cc2ca592e2
Add metrics in tx-tracker to count destination tx inserted (#1233) 2024-03-22 11:30:46 -03:00
walker-16 bc3110f3d4
Modify intialize to support fallback url and rate limit (#1093)
* Modify intialize to support fallback url and rate limit

* Add fallback and rate limit config for all the chains

* remove unused WormchainTxDetail struct

* Add pool of items to common

* Add rpc pool to tx-tracker

* Rename chain url to baseUrl

* add fallback to wormchain chains

* update tx-tracker fetchone to use rpc pool

* Modify backfiller to use rpc pool

* fix merge to main run files

* Add rpc pool to controller

* remove comments

* fix pool sort function

* Add rpc pool to all the chains

* Fix algorand, aptos, sui integrations

* Fix pool solana

* remove api_sei3 old implementation

* Add metrics to get success/error rpc call

* Add rpc field in call rpc metric

* Fix common go.mod and remove unused if condition

* Update go.mod tx-tracker

* Create rpc count prometheus metrics

* Add fallback to insert originTx if rcp fail

* Check if the transaction doesn't exists

* Modify in tx-tracker already processed method

* fix wormchain rpc

* Add rpc settings as json file

* Enable overwrite to stress test

* Add rpc provider as k8s secret

* Add metrics to get vaa retry processing
Co-authored-by: ftocal fert1335@gmail.com

* Add worker pool in consumer for tx-tracker

* fix config by env

* Set consumer workers in 1

---------

Co-authored-by: Fernando Torres <fert1335@gmail.com>
2024-03-19 15:47:43 -03:00
ftocal a6cbcc7c55
Add backfiller for pipeline (#1221) 2024-03-15 16:44:24 -03:00
walker-16 e7e191b8fc
#1142: Fix solana transaction source address (#1182) 2024-03-07 11:21:26 -03:00
ftocal 3dba12b1f5
Fix aws-sdk dependencies (#1162)
Co-authored-by: walker-16 <agpazos85@gmail.com>
2024-02-28 16:33:30 -03:00
ftocal a882355c05
Add transfer prices backfiller in analytics (#1153)
* Add transfer prices backfiller in analytics
Save notional prices in s3 bucket

* Code review updates
2024-02-27 15:17:59 -03:00
ftocal dc1117f6f5
Add backfiller for tx-tracker (#1132) 2024-02-20 09:58:37 -03:00
ftocal 278cd50dcb
Fix txHash in solana with job (#1076)
* Remove filter solanas and aptos from method getVaasToMigrate

* Fix origin tx in solana
Add k8s service for tx-tracker

---------

Co-authored-by: Agustin Pazos <agpazos85@gmail.com>
2024-01-31 19:41:41 -03:00
walker-16 fb0573c160
migrate vaa to globaltransaction origintx (#940)
* migrate vaa to globaltransaction origintx

* Add deploy configuration for job to migrate vaas to originTx

* Add option to run migration process by range of date

* Update go dependencies for jobs

* Fix kind of job in deployment

---------

Co-authored-by: Fernando Torres <fert1335@gmail.com>
2024-01-30 12:19:31 -03:00
ftocal 6145453fd8
Fix enable transer redeemed (#1040) 2024-01-24 15:56:43 -03:00
ftocal 49f9bedd29
Handle transfer-redeemed event from blockchain-watcher (#1039) 2024-01-24 14:51:04 -03:00
ftocal d3c1c52b79
Add support to sepolia chains (#1001)
Co-authored-by: walker-16 <agpazos85@gmail.com>
2024-01-22 11:01:18 -03:00
walker-16 9cb5769c06
Update wormhole sdk to version v0.0.0-20240109172745-cc0cd9fc5229 in … (#998)
Update wormhole sdk to version v0.0.0-20240109172745-cc0cd9fc5229 in all the components
2024-01-17 12:36:54 -03:00
ftocal 27bf188149
Delete unhandled messages from sqs (#987)
Add more log
2024-01-15 14:11:41 -03:00
ftocal e21b836455
Add handler for redeem events from blockchain-watcher (#986) 2024-01-15 10:22:05 -03:00
ftocal 7992f76e56
Improve tx tracker logic without internal retries (#890)
* Improve tx tracker logic without internal retries

Co-authored-by: walker-16 <agpazos85@gmail.com>

* Add message attribute in pipeline sns

Co-authored-by: walker-16 <agpazos85@gmail.com>

---------

Co-authored-by: walker-16 <agpazos85@gmail.com>
2023-12-12 15:46:00 -03:00
ftocal 6edd3c4711
Ignore vaa from near in tx-tracker (#883)
Co-authored-by: walker-16 <agpazos85@gmail.com>
2023-12-11 10:16:06 -03:00
ftocal cbdd72ec9c
Process missing txHash in tx-tracker (#866)
Improve event processing in tx-tracker
Add new prometheus metrics
Handle vaa without txHash from gossip flow
Fix for already message processed
Change rpc endpoints and rate limit for tx-tracker

Co-authored-by: walker-16 <agpazos85@gmail.com>
2023-12-07 11:06:48 -03:00
ftocal af5bd3017f
Handle native tx hash for solana when receive event from blockchain-watcher (#851) 2023-12-04 16:29:38 -03:00
ftocal 2488da74aa
Add support to handle multiple signature for an address in solana (#850)
Co-authored-by: walker-16 <agpazos85@gmail.com>
2023-12-04 15:30:16 -03:00
ftocal 2836f4ffc2
Fix of an intermittent failure to obtain a solana transaction (#840) 2023-12-01 14:15:19 -03:00
ftocal 7c467f5267
Integrating blockchain watcher data into mongodb (#820)
* Handle log-message-published generated by blockchain-watcher in parser, tx-tracker y analytics

* Add deployment for staging-testnet environment
Improve logs

* Add operation endpoints

---------

Co-authored-by: Agustin Pazos <agpazos85@gmail.com>
2023-11-27 12:31:35 -03:00
ftocal 11b162ca54
Add Sei support in tx-tracker (#800)
* Add Sei support in tx-tracker

* Code review updates
2023-11-15 12:11:24 -03:00
walker-16 e5282ffb16
normalize osmosis txhash (#777)
Co-authored-by: ftocal <fert1335@gmail.com>
2023-10-30 11:23:37 -03:00
walker-16 f4fe630623
Add methods to identify kujira & evmos (#729)
* Add methods to identify kujira & evmos

* Add fetch methods to kajira and evmos

* fix fetch kujira and evmos methods

* Add support to search vaas gateway chain txHash

* Add deploy config to tx-tracker for evmos & kujira
2023-10-26 19:34:00 -03:00
walker-16 b46ba72d6b
Update source and destination channel for osmosis testnet (#758) 2023-10-25 17:05:55 -03:00
walker-16 217c6f28be
Fix [api] get transactions by address and add mongo query monitor (#707)
Fix api endpoint get transactions by address, add mongo query monitor
2023-09-25 16:50:16 -03:00
ftocal 5e3adeb4ad
Retrieve chain and sender of wormchain originated vaas (#678)
* Add additional information for osmosis transaction through wormchain

Co-authored-by: walker-16 <agpazos85@gmail.com>

* Modify tx-tracker deployment

Co-authored-by: walker-16 <agpazos85@gmail.com>

---------

Co-authored-by: walker-16 <agpazos85@gmail.com>
2023-09-04 15:17:23 -03:00
agodnic 802fc00434
Add documentation for the `tx-tracker` service (#642)
### Description

This pull request adds a README.md file explaining how the `tx-tracker` service works.
2023-08-18 10:33:36 -03:00
agodnic 3c7bab3f8a
Remove duplicated code: MongoDB connection handling (#590)
### Description

This pull request removes duplicated code related to MongoDB connection/disconnection attempts. This code was copied across all 8 microservices.

The functionality is now unified under the `common/dbutil` package.
2023-08-07 16:05:08 -03:00
agodnic d8bb9c46f6
`tx-tracker`: add support for the "Base" blockchain (#568)
### Description

Tracking issue: https://github.com/wormhole-foundation/wormhole-explorer/issues/569

This pull request adds support for the "Base" blockchain in different parts of the codebase:
* The functions `domain.TranslateEmitterAddress`, `domain.EncodeTrxHashByChainID` and `domain.DecodeNativeAddressToHex`.
* The `tx-tracker` service: it now connects to a Base RPC node to fetch origin transaction metadata.
2023-07-20 16:37:21 -03:00
walker-16 abb5baf24e
add metrics to tx-tracker (#556) 2023-07-19 09:04:41 -03:00
agodnic 44e236fc61
Update tx tracker backfiller (#540)
### Description

Related issues: https://github.com/wormhole-foundation/wormhole-explorer/issues/525, https://github.com/wormhole-foundation/wormhole-explorer/issues/524

Final updates before running the tx-tracker backfiller on staging and production.
2023-07-18 12:00:04 -03:00
agodnic 61d070e859
Fix tx tracker performance issues (#537)
### Description

The `tx-tracker` service is failing to process messages at the speed we need. This pull request is an attempt at fixing the issue.
2023-07-14 10:39:09 -03:00
agodnic 1ef666322c
Fix missing `txHash` for chains 1 and 22 (#530)
### Summary

This pull request changes the retry logic in the `tx-tracker`.

When failing to process a transaction, the associated SQS message will be left to expire and will reappear after `visibilityTimeout` seconds. As a side effect, there will be several retries before giving up.
2023-07-13 12:26:38 -03:00
agodnic 0a854b590d
Improve performance in the `tx-tracker` service (#519)
### Description

This pull request implements an improvement in the processing logic of the `tx-tracker` service to avoid processing a message more than once.
2023-07-11 16:31:45 -03:00
agodnic 93e1a72409
Improvements in the `tx-tracker` service (#517)
### Description

This pull request implements improvements in the `tx-tracker` service:
* Adjust SQS visibilityTimeout.
* Process PythNet messages concurrently (this will make it easier for the service to catch up if there are a lot of messages in the input queue).
* Add more context information to error messages.
2023-07-06 15:26:18 -03:00
agodnic 57f8878979
Adjust log message severity in `tx-tracker` (#511) 2023-07-06 10:26:16 -03:00
agodnic 9e4aa45305
[tx-tracker] Add support for Terra, Injective, Acala and Karura (#502)
### Description

Tracking issue: https://github.com/wormhole-foundation/wormhole-explorer/issues/499

This pull request adds support in the `tx-tracker` service for the blockchains Terra, Injective, Acala and Karura.

Also, deployment settings were updated with reasonable defaults.
2023-07-05 14:53:00 -03:00
agodnic 8ad4d6276f
Add retry logic to the `tx-tracker` service (#480)
### Summary

Tracking issue: https://github.com/wormhole-foundation/wormhole-explorer/issues/415

When the `tx-tracker` service processes a VAA, sometimes the transaction is not finalized (e.g.:for chains with slow finality, VAAs with low consistency level, out-of-sync nodes). This problem was observed on chains like Solana, Ethereum and Polygon, but could have happened on other chains.

This pull request adds the logic needed to wait and retry processing the VAA, hoping that the originating transaction reaches finality.

Also, the `tx-tracker` service now uses a worker pool to process transactions, which will lead to higher throughput. This will benefit the WormholeScan UI, specially in the case when a large number of VAAs are emitted in a short period of time.
2023-07-04 15:25:08 -03:00