Commit Graph

58 Commits

Author SHA1 Message Date
ftocal adb7231074
Add processed message metrics in pipeline components (#1239)
Co-authored-by: walker-16 <agpazos85@gmail.com>
2024-03-25 15:21:10 -03:00
walker-16 cc2ca592e2
Add metrics in tx-tracker to count destination tx inserted (#1233) 2024-03-22 11:30:46 -03:00
walker-16 bc3110f3d4
Modify intialize to support fallback url and rate limit (#1093)
* Modify intialize to support fallback url and rate limit

* Add fallback and rate limit config for all the chains

* remove unused WormchainTxDetail struct

* Add pool of items to common

* Add rpc pool to tx-tracker

* Rename chain url to baseUrl

* add fallback to wormchain chains

* update tx-tracker fetchone to use rpc pool

* Modify backfiller to use rpc pool

* fix merge to main run files

* Add rpc pool to controller

* remove comments

* fix pool sort function

* Add rpc pool to all the chains

* Fix algorand, aptos, sui integrations

* Fix pool solana

* remove api_sei3 old implementation

* Add metrics to get success/error rpc call

* Add rpc field in call rpc metric

* Fix common go.mod and remove unused if condition

* Update go.mod tx-tracker

* Create rpc count prometheus metrics

* Add fallback to insert originTx if rcp fail

* Check if the transaction doesn't exists

* Modify in tx-tracker already processed method

* fix wormchain rpc

* Add rpc settings as json file

* Enable overwrite to stress test

* Add rpc provider as k8s secret

* Add metrics to get vaa retry processing
Co-authored-by: ftocal fert1335@gmail.com

* Add worker pool in consumer for tx-tracker

* fix config by env

* Set consumer workers in 1

---------

Co-authored-by: Fernando Torres <fert1335@gmail.com>
2024-03-19 15:47:43 -03:00
ftocal a6cbcc7c55
Add backfiller for pipeline (#1221) 2024-03-15 16:44:24 -03:00
walker-16 e7e191b8fc
#1142: Fix solana transaction source address (#1182) 2024-03-07 11:21:26 -03:00
ftocal 3dba12b1f5
Fix aws-sdk dependencies (#1162)
Co-authored-by: walker-16 <agpazos85@gmail.com>
2024-02-28 16:33:30 -03:00
ftocal a882355c05
Add transfer prices backfiller in analytics (#1153)
* Add transfer prices backfiller in analytics
Save notional prices in s3 bucket

* Code review updates
2024-02-27 15:17:59 -03:00
ftocal dc1117f6f5
Add backfiller for tx-tracker (#1132) 2024-02-20 09:58:37 -03:00
ftocal 278cd50dcb
Fix txHash in solana with job (#1076)
* Remove filter solanas and aptos from method getVaasToMigrate

* Fix origin tx in solana
Add k8s service for tx-tracker

---------

Co-authored-by: Agustin Pazos <agpazos85@gmail.com>
2024-01-31 19:41:41 -03:00
walker-16 fb0573c160
migrate vaa to globaltransaction origintx (#940)
* migrate vaa to globaltransaction origintx

* Add deploy configuration for job to migrate vaas to originTx

* Add option to run migration process by range of date

* Update go dependencies for jobs

* Fix kind of job in deployment

---------

Co-authored-by: Fernando Torres <fert1335@gmail.com>
2024-01-30 12:19:31 -03:00
ftocal 6145453fd8
Fix enable transer redeemed (#1040) 2024-01-24 15:56:43 -03:00
ftocal 49f9bedd29
Handle transfer-redeemed event from blockchain-watcher (#1039) 2024-01-24 14:51:04 -03:00
ftocal d3c1c52b79
Add support to sepolia chains (#1001)
Co-authored-by: walker-16 <agpazos85@gmail.com>
2024-01-22 11:01:18 -03:00
walker-16 9cb5769c06
Update wormhole sdk to version v0.0.0-20240109172745-cc0cd9fc5229 in … (#998)
Update wormhole sdk to version v0.0.0-20240109172745-cc0cd9fc5229 in all the components
2024-01-17 12:36:54 -03:00
ftocal 27bf188149
Delete unhandled messages from sqs (#987)
Add more log
2024-01-15 14:11:41 -03:00
ftocal e21b836455
Add handler for redeem events from blockchain-watcher (#986) 2024-01-15 10:22:05 -03:00
ftocal 7992f76e56
Improve tx tracker logic without internal retries (#890)
* Improve tx tracker logic without internal retries

Co-authored-by: walker-16 <agpazos85@gmail.com>

* Add message attribute in pipeline sns

Co-authored-by: walker-16 <agpazos85@gmail.com>

---------

Co-authored-by: walker-16 <agpazos85@gmail.com>
2023-12-12 15:46:00 -03:00
ftocal 6edd3c4711
Ignore vaa from near in tx-tracker (#883)
Co-authored-by: walker-16 <agpazos85@gmail.com>
2023-12-11 10:16:06 -03:00
ftocal cbdd72ec9c
Process missing txHash in tx-tracker (#866)
Improve event processing in tx-tracker
Add new prometheus metrics
Handle vaa without txHash from gossip flow
Fix for already message processed
Change rpc endpoints and rate limit for tx-tracker

Co-authored-by: walker-16 <agpazos85@gmail.com>
2023-12-07 11:06:48 -03:00
ftocal af5bd3017f
Handle native tx hash for solana when receive event from blockchain-watcher (#851) 2023-12-04 16:29:38 -03:00
ftocal 2488da74aa
Add support to handle multiple signature for an address in solana (#850)
Co-authored-by: walker-16 <agpazos85@gmail.com>
2023-12-04 15:30:16 -03:00
ftocal 2836f4ffc2
Fix of an intermittent failure to obtain a solana transaction (#840) 2023-12-01 14:15:19 -03:00
ftocal 7c467f5267
Integrating blockchain watcher data into mongodb (#820)
* Handle log-message-published generated by blockchain-watcher in parser, tx-tracker y analytics

* Add deployment for staging-testnet environment
Improve logs

* Add operation endpoints

---------

Co-authored-by: Agustin Pazos <agpazos85@gmail.com>
2023-11-27 12:31:35 -03:00
ftocal 11b162ca54
Add Sei support in tx-tracker (#800)
* Add Sei support in tx-tracker

* Code review updates
2023-11-15 12:11:24 -03:00
walker-16 e5282ffb16
normalize osmosis txhash (#777)
Co-authored-by: ftocal <fert1335@gmail.com>
2023-10-30 11:23:37 -03:00
walker-16 f4fe630623
Add methods to identify kujira & evmos (#729)
* Add methods to identify kujira & evmos

* Add fetch methods to kajira and evmos

* fix fetch kujira and evmos methods

* Add support to search vaas gateway chain txHash

* Add deploy config to tx-tracker for evmos & kujira
2023-10-26 19:34:00 -03:00
walker-16 b46ba72d6b
Update source and destination channel for osmosis testnet (#758) 2023-10-25 17:05:55 -03:00
walker-16 217c6f28be
Fix [api] get transactions by address and add mongo query monitor (#707)
Fix api endpoint get transactions by address, add mongo query monitor
2023-09-25 16:50:16 -03:00
ftocal 5e3adeb4ad
Retrieve chain and sender of wormchain originated vaas (#678)
* Add additional information for osmosis transaction through wormchain

Co-authored-by: walker-16 <agpazos85@gmail.com>

* Modify tx-tracker deployment

Co-authored-by: walker-16 <agpazos85@gmail.com>

---------

Co-authored-by: walker-16 <agpazos85@gmail.com>
2023-09-04 15:17:23 -03:00
agodnic 802fc00434
Add documentation for the `tx-tracker` service (#642)
### Description

This pull request adds a README.md file explaining how the `tx-tracker` service works.
2023-08-18 10:33:36 -03:00
agodnic 3c7bab3f8a
Remove duplicated code: MongoDB connection handling (#590)
### Description

This pull request removes duplicated code related to MongoDB connection/disconnection attempts. This code was copied across all 8 microservices.

The functionality is now unified under the `common/dbutil` package.
2023-08-07 16:05:08 -03:00
agodnic d8bb9c46f6
`tx-tracker`: add support for the "Base" blockchain (#568)
### Description

Tracking issue: https://github.com/wormhole-foundation/wormhole-explorer/issues/569

This pull request adds support for the "Base" blockchain in different parts of the codebase:
* The functions `domain.TranslateEmitterAddress`, `domain.EncodeTrxHashByChainID` and `domain.DecodeNativeAddressToHex`.
* The `tx-tracker` service: it now connects to a Base RPC node to fetch origin transaction metadata.
2023-07-20 16:37:21 -03:00
walker-16 abb5baf24e
add metrics to tx-tracker (#556) 2023-07-19 09:04:41 -03:00
agodnic 44e236fc61
Update tx tracker backfiller (#540)
### Description

Related issues: https://github.com/wormhole-foundation/wormhole-explorer/issues/525, https://github.com/wormhole-foundation/wormhole-explorer/issues/524

Final updates before running the tx-tracker backfiller on staging and production.
2023-07-18 12:00:04 -03:00
agodnic 61d070e859
Fix tx tracker performance issues (#537)
### Description

The `tx-tracker` service is failing to process messages at the speed we need. This pull request is an attempt at fixing the issue.
2023-07-14 10:39:09 -03:00
agodnic 1ef666322c
Fix missing `txHash` for chains 1 and 22 (#530)
### Summary

This pull request changes the retry logic in the `tx-tracker`.

When failing to process a transaction, the associated SQS message will be left to expire and will reappear after `visibilityTimeout` seconds. As a side effect, there will be several retries before giving up.
2023-07-13 12:26:38 -03:00
agodnic 0a854b590d
Improve performance in the `tx-tracker` service (#519)
### Description

This pull request implements an improvement in the processing logic of the `tx-tracker` service to avoid processing a message more than once.
2023-07-11 16:31:45 -03:00
agodnic 93e1a72409
Improvements in the `tx-tracker` service (#517)
### Description

This pull request implements improvements in the `tx-tracker` service:
* Adjust SQS visibilityTimeout.
* Process PythNet messages concurrently (this will make it easier for the service to catch up if there are a lot of messages in the input queue).
* Add more context information to error messages.
2023-07-06 15:26:18 -03:00
agodnic 57f8878979
Adjust log message severity in `tx-tracker` (#511) 2023-07-06 10:26:16 -03:00
agodnic 9e4aa45305
[tx-tracker] Add support for Terra, Injective, Acala and Karura (#502)
### Description

Tracking issue: https://github.com/wormhole-foundation/wormhole-explorer/issues/499

This pull request adds support in the `tx-tracker` service for the blockchains Terra, Injective, Acala and Karura.

Also, deployment settings were updated with reasonable defaults.
2023-07-05 14:53:00 -03:00
agodnic 8ad4d6276f
Add retry logic to the `tx-tracker` service (#480)
### Summary

Tracking issue: https://github.com/wormhole-foundation/wormhole-explorer/issues/415

When the `tx-tracker` service processes a VAA, sometimes the transaction is not finalized (e.g.:for chains with slow finality, VAAs with low consistency level, out-of-sync nodes). This problem was observed on chains like Solana, Ethereum and Polygon, but could have happened on other chains.

This pull request adds the logic needed to wait and retry processing the VAA, hoping that the originating transaction reaches finality.

Also, the `tx-tracker` service now uses a worker pool to process transactions, which will lead to higher throughput. This will benefit the WormholeScan UI, specially in the case when a large number of VAAs are emitted in a short period of time.
2023-07-04 15:25:08 -03:00
agodnic eae49e2e9c
Add support for Oasis and Algorand in the `tx-tracker` service (#455)
### Description

This pull request adds support for Oasis and Algorand in the `tx-tracker` service.

With this change, the sender addresses for those chains will become available in the UI.
2023-06-23 22:21:34 -03:00
agodnic 4b4b6f61ca
Add support for cosmos chains in `tx-tracker` (#426)
### Description

Tracking issue: https://github.com/wormhole-foundation/wormhole-explorer/issues/418

This pull request modifies the `tx-tracker` service to support `ChainIDTerra2` and `ChainIDXpla`. In particular, this will make sender addresses from these blockchains available for the Wormhole Scan UI.

Support for `ChainIDTerra` was left out due to lack of working public RPC nodes (`https://lcd.terra.dev` doesn't seem to be functioning correctly at the moment).
2023-06-21 15:47:12 -03:00
agodnic 50c1d479a6
Add support for Moonbeam and Klaytn in the `tx-tracker` service (#422)
### Description

This pull request modifies the `tx-tracker` service to support the Moonbeam and Klaytn blockchains.

In particular, this will make sender addresses for these particular blockchains available for the Wormhole Scan UI.
2023-06-20 09:37:06 -03:00
agodnic 3f823c51ac
Add support for Aptos in the `tx-tracker` service (#421)
### Description

This pull request adds support for Aptos in the `tx-tracker` service.

This will lead to improvements for the wormhole Scan UI:
1. The transaction hash that was being previously displayed for Aptos VAAs was incorrect. This pull request fixes the issue.
2. The sender addresses for Aptos VAAs will now become available for for the UI.
2023-06-16 17:47:28 -03:00
agodnic bf3be056b4
Add support for Sui in the `tx-tracker` service (#416)
### Description

This pull request modifies the `tx-tracker` service to support the Sui blockchain.

In particular, this will make the Sui sender addresses available for the Wormhole Scan UI.
2023-06-15 21:42:46 -03:00
ftocal 9381695d37
Fix txHash encoding by chain (#410)
Add txHash encondig backfiller
Handle txHash base58 encoding for solana in tx-tracker
Add temporary field _originTxHash in vaas and vaaIdTxHash collections by backup
Add cobra to fly backfiller

Co-authored-by: walker-16 <agpazos85@gmail.com>
2023-06-15 09:57:39 -03:00
agodnic 19743322e4
Retrieve sender address for each transaction (#408)
### Summary

Tracking issue: https://github.com/wormhole-foundation/wormhole-explorer/issues/404

The WormholeScan UI needs to display the sender address for each token bridge VAA. This pull request modifies the `tx-tracker` service to obtain that information for Solana and eight EVM chains.

The transaction sender will become accessible through the following endpoints:
* `GET /api/v1/global-tx/{chain}/{emitter}/{seq}`: field `originTx.from`.
* `GET /api/v1/transactions`: field `originAddress`.
In both cases, the field is nullable (i.e.: sometimes it may not be available due to eventual consistency or internal errors)
2023-06-14 17:46:53 -03:00
ftocal 7fb3857a97
297 contract watchertx trackeranalyticfly change access to use iam role (#306)
* Use aws role to access sqs

Co-authored-by: walker-16 <agpazos85@gmail.com>

* Add ServiceAccount in deployment

Co-authored-by: walker-16 <agpazos85@gmail.com>

---------

Co-authored-by: walker-16 <agpazos85@gmail.com>
2023-05-10 15:49:40 -03:00
agodnic 09870f3d65
[tx-tracker / api] Avoid duplicating data (#256)
### Summary
Tracking issue: https://github.com/wormhole-foundation/wormhole-explorer/issues/254

For all chains except Solana, the data in `globalTransactions.sourceTx` was somewhat duplicated, since we already had that data in the `vaas` collection.
This pull request changes the `tx-tracker` service so that no redundant data will be written, and also changes the API to look up the data from the `vaas` collection instead.

The responses from the endpoint `/api/v1/global-tx` will no longer contain the field `originTx.from`.
2023-04-25 15:34:29 -03:00