Commit Graph

86 Commits

Author SHA1 Message Date
Mariano 6ab6824d82
[1520] Retrieve the feeUSD from coingecko in tx-tracker [WIP] (#1541)
* start implementing retrieving the feeUSD from coingecko

* add missing pricesApi in the initialization of tx-tracker service

* add missing initialization

* add missing cfg for coingecko properties

* fix solana in url

* replace coingecko api call with notional redis client

* add native token list in commons

* add decimal places

* change feeUSD to Raw.GasPrice

* add check for mainnet

* replace with sdk const for more coupling

* replace pricesApi with notionalCache

* undo changes on pricesApi

* undo more stuff

* remove lines

* remove unused timestamp

* remove unused props

* remove

* fix indent

* fix import

* change

* add cache configs

* change token_address to native

* fixes

* fix BNB coingeckoID

* fix polygon coingecko_id

* change to lowercase

* adjust type FeeDoc in operations endpoint

* change Blast native token

* pr review changes

* code-review fixes

* comment wormchain

* rename var

* add comment to wormchain gas token map entry
2024-07-15 15:43:53 -03:00
ftocal 65108a0338
1284 enhance operations endpoint to include gas costs in origin and destination chains (#1516)
* Modify fetchEvmTx to call eth_getTransactionReceipt rpc

* Add calculate evm fee

* modify repository to add feeDetail in the originTx upsert

* Add fee calculate for solana

* Add fee in operations endpoint
Update swagger documentation

Co-authored-by: walker-16 <agpazos85@gmail.com>

* add support fee for destinationTx in tx-tracker

* fix empty calculated fee

---------

Co-authored-by: Agustin Pazos <agpazos85@gmail.com>
2024-07-03 14:48:27 -03:00
ftocal 0bfd9e554b
Add vaa processing speed metrics in fly, analytics, parser and tx-tracker (#1531)
Co-authored-by: walker-16 <agpazos85@gmail.com>
2024-07-03 11:23:54 -03:00
Mariano 9d743e7f5a
[#1495] Add NTT Protocol Data to /api/v1/protocols/stats Endpoint (#1502)
* use new measurement for protocol stats and add NTT

* reorder consts
2024-06-26 14:31:42 -03:00
Martin Picco 2646baa9f1
Add NTT tokens to manual list (#1504) 2024-06-25 10:24:00 -03:00
ftocal dabe8172da
Add Ethena sUSDe token (#1491) 2024-06-14 09:58:36 -03:00
Mariano 97b9217b6b
Create backfiller for influx measurement vaa_volume_v3 (#1475)
* backfiller for vaa_volume_v3

* remove printf
2024-06-12 15:37:51 -03:00
ftocal 1856765a90
Add Ondo and JitoSOL tokens (#1483)
Update generated_mainnet_tokens.go
2024-06-11 10:57:08 -03:00
Martin Picco 1fee8b4f1e
Add USDC.e fantom token to manual tokens config (#1423) 2024-06-11 09:48:52 -03:00
walker-16 45575fb060
Add support to mantle in tx-tracker and common (#1441) 2024-06-04 18:18:19 -03:00
ftocal e366f04765
Find by txHash in observations endpoint (#1450)
Add txHash query parameter in observations endpoint
Add job to add nativeTxHash in observations collection except for solana, aptos and wormchain
Add encoded txHash in obvervations collection
2024-06-04 12:12:44 -03:00
walker-16 7545acb77e
Feature/handle governor fly event processor (#1436)
* changes in fly component to send governor status events

* split processor in vaaProcessor and governor processor

* add governor processsor v1

* Add endpoint in tx-tracker to calculate txHash for a vaa id

* fly-event-processor integration with new tx-tracker endpoint and refactor

* Add governor vaas endpoint in api

* api, fix amount data type in governor vaas endpoint

* fly-event-processor normalize emitter and txHash

* fly-event-processor fix nodeGovernorVaa id

* fly-event-processor control array not empty in insert/delete many operation

* add index in nodeGovernorVaas collection by vaaId

* add prometheus metrics

* add tx-tracker url for fly-event-processor deployment

* Add sns attributes into sns messages

Co-authored-by: walker-16 <agpazos85@gmail.com>

* fix governor vaa endpoint empty response

---------

Co-authored-by: Fernando Torres <fert1335@gmail.com>
2024-05-31 15:38:21 -03:00
ftocal d4b63be148
Fix go dependencies (#1448)
* Update dependencies in tx-tracker and common

Co-authored-by: walker-16 <agpazos85@gmail.com>

* Update dependencies for analytics, fly-event-processor, jobs, notional, parser and pipeline

Co-authored-by: walker-16 <agpazos85@gmail.com>

---------

Co-authored-by: walker-16 <agpazos85@gmail.com>
2024-05-28 11:30:41 -03:00
walker-16 f354af6d96
1407 update sdk and modify tx tracker to support xlayer (#1440)
* Add support to xlayer in tx-tracker

* update commons chainIDs methods
2024-05-28 10:28:11 -03:00
ftocal 4809dbda21
Update last guadian set automatically from eth contract (#1414)
* Update last guadian set automatically from eth contract

* Continue testing for an intermittent failure
2024-05-27 11:01:16 -03:00
ftocal f48b64e780
Add BORG token to the token list for ethereum and solana (#1413)
Co-authored-by: walker-16 <agpazos85@gmail.com>
2024-05-14 11:32:46 -03:00
walker-16 ccc7e558c4
add polygon amoy to tx-tracker (#1373)
Co-authored-by: ftocal <fert1335@gmail.com>
2024-05-13 12:11:55 -03:00
walker-16 b8e3f43333
1301 create a backgroundrecurrent process which checks which duplicate vaas txs actually reached finality on the blockchain (#1359)
* fly-event-processor initial commit

* Handle duplicated vaa event in fly-event-processor
Add deployment for fly-event-processor
Send duplicated vaa event from fly

Co-authored-by: walker-16 <agpazos85@gmail.com>

---------

Co-authored-by: Fernando Torres <fert1335@gmail.com>
2024-05-07 16:38:16 -03:00
ftocal cc707539ef
Handle tx hash for blast and scroll in common (#1384)
Co-authored-by: walker-16 <agpazos85@gmail.com>
2024-05-07 11:27:11 -03:00
ftocal 9d280f7d5f
Add W token for ethereum, arbitrum, optimism and base (#1358)
Co-authored-by: walker-16 <agpazos85@gmail.com>
2024-04-25 17:06:14 -03:00
ftocal 688a0d0f71
Store duplicate vaas on a secondary collection (#1308)
* Add support to scroll in tx-tracker

* Handle duplicated vaa in fly

Co-authored-by: walker-16 <agpazos85@gmail.com>

* Modify tx-tracker and pipeline to support duplicated vaas
Co-authored-by: ftocal <fert1335@gmail.com>

* Add isDuplicated flag in vaas and operations endpoints
Add new endpoint to find duplicated vaas by vaa id
Co-authored-by: walker-16 <agpazos85@gmail.com>

* Add metrics for duplicated vaas

Co-authored-by: walker-16 <agpazos85@gmail.com>

* Update node dependencies in spy

Co-authored-by: walker-16 <agpazos85@gmail.com>

* Fix promethes metrics for missing tx-hash in tx-tracker

Co-authored-by: walker-16 <agpazos85@gmail.com>

---------

Co-authored-by: Agustin Pazos <agpazos85@gmail.com>
2024-04-22 14:26:07 -03:00
walker-16 8dc2b5ac5f
Normalize txHash destination in tx-tracker component (#1274)
Co-authored-by: ftocal <fert1335@gmail.com>
2024-04-08 14:05:48 -03:00
walker-16 04da0b3e4d
Add wormchain injective integration (#1260)
* Add wormchain injective integration

* Add metrics and logger for unknown wormchain

* Add wormchain rpc config
Co-authored-by: ftocal <fert1335@gmail.com>

* Add evmos, kujira, osmosis to wormchainRpcProviders
Co-authored-by: ftocal <fert1335@gmail.com>

* Add backfiller for tx-tracker with emitterChainId, emitterAddress and range time parameters

Co-authored-by: walker-16 <agpazos85@gmail.com>

---------

Co-authored-by: Fernando Torres <fert1335@gmail.com>
2024-04-05 15:24:21 -03:00
ftocal 3d60517757
Update list of available tokens (#1269)
Co-authored-by: walker-16 <agpazos85@gmail.com>
2024-04-02 15:56:34 -03:00
ftocal f4390b8436
Handle unproccessable response from vaa-payload-parser (#1250)
Add retry in analytics metrics

Co-authored-by: walker-16 <agpazos85@gmail.com>
2024-03-26 11:53:11 -03:00
ftocal b19bc16dfb
Update list of available tokens (#1246)
* Update list of available tokens

Co-authored-by: walker-16 <agpazos85@gmail.com>

* Handle too many requests in notional jobs

---------

Co-authored-by: walker-16 <agpazos85@gmail.com>
2024-03-25 15:17:26 -03:00
walker-16 bc3110f3d4
Modify intialize to support fallback url and rate limit (#1093)
* Modify intialize to support fallback url and rate limit

* Add fallback and rate limit config for all the chains

* remove unused WormchainTxDetail struct

* Add pool of items to common

* Add rpc pool to tx-tracker

* Rename chain url to baseUrl

* add fallback to wormchain chains

* update tx-tracker fetchone to use rpc pool

* Modify backfiller to use rpc pool

* fix merge to main run files

* Add rpc pool to controller

* remove comments

* fix pool sort function

* Add rpc pool to all the chains

* Fix algorand, aptos, sui integrations

* Fix pool solana

* remove api_sei3 old implementation

* Add metrics to get success/error rpc call

* Add rpc field in call rpc metric

* Fix common go.mod and remove unused if condition

* Update go.mod tx-tracker

* Create rpc count prometheus metrics

* Add fallback to insert originTx if rcp fail

* Check if the transaction doesn't exists

* Modify in tx-tracker already processed method

* fix wormchain rpc

* Add rpc settings as json file

* Enable overwrite to stress test

* Add rpc provider as k8s secret

* Add metrics to get vaa retry processing
Co-authored-by: ftocal fert1335@gmail.com

* Add worker pool in consumer for tx-tracker

* fix config by env

* Set consumer workers in 1

---------

Co-authored-by: Fernando Torres <fert1335@gmail.com>
2024-03-19 15:47:43 -03:00
Mariano faa8e38b2e
[ISSUE-1213] Refactor of top-contributors stats and activity job (#1215)
* start refactor of stats and activity job

 change approach for collecting stats and activity metrics

 multiple stuff

fix unfinished merging conflicts

 changes

 update

Delete .run/wormhole-explorer-api.run.xml

 add unit-tesdt

 add unit-test for mayan and allbrdige

add protocols-stats-1h job

 add new job

 change stats query

fix query

 start refactor of stats and activity job

 change approach for collecting stats and activity metrics

 multiple stuff

fix unfinished merging conflicts

 changes

 update

Delete .run/wormhole-explorer-api.run.xml

 add unit-tesdt

 add unit-test for mayan and allbrdige

add protocols-stats-1h job

 add new job

 change stats query

fix query

* remove temp backfiller

* fix unit-tests

* remove useless dbconsts

* remove useless consts

* fix some renaming

* fix unit-tests

* fix tvl portal_token_bridge
2024-03-18 10:21:42 -03:00
ftocal a6cbcc7c55
Add backfiller for pipeline (#1221) 2024-03-15 16:44:24 -03:00
Mariano 4009482dcc
[ISSUE-1089] Collect CCTP - Metrics (#1165)
Add cctp and portal_token_bridge stats to protocols-stats endpoint

changes

 indent

more changes on the script

 new working approach on influx task

 indent script

 tested insert

 add 2 versions of the script

multiple changes

 add 1day task

 add logic to retrieve internal protocols

 remove unecessary code

 readd empty script

fix  unit-tests and measurement namings

 fix queries

 fix alignment

rename function task

fix names

improvements on influx task

 add .run config to gitignore

add .run to gitignore

fix task and rename

working api

 multiple things

Delete .run/wormscan api.run.xml

Delete analytics/scripts/test_query.flux

wip

 multiple fixes

 fix test

wip

 fix queries

fix unit-test due to query changes
2024-03-11 17:33:36 -03:00
ftocal 08c9427606
Fix parsedPayload use as any type for different kind of messages (#1169)
Co-authored-by: walker-16 <agpazos85@gmail.com>
2024-03-04 16:49:54 -03:00
Mariano 6bbfa7bf23
Add cache to protocols stats endpoints (/protocols/stats) (#1149)
* add cache to protocols stats endpoint

 add mock for cache

add unit test case for cache miss

 change from to current

* add configs

* add missing config for api-service.yaml

* add cache ttl for staging-mainnet
2024-03-04 15:55:53 -03:00
ftocal 3dba12b1f5
Fix aws-sdk dependencies (#1162)
Co-authored-by: walker-16 <agpazos85@gmail.com>
2024-02-28 16:33:30 -03:00
ftocal a882355c05
Add transfer prices backfiller in analytics (#1153)
* Add transfer prices backfiller in analytics
Save notional prices in s3 bucket

* Code review updates
2024-02-27 15:17:59 -03:00
Mariano 175abbb11a
[Issue:1052] Create job for fetching contributor stats and storing in db (#1144)
* [Issue:1052] Create job for fetching contributor stats and storing in db

revert unnecessary changes on api/handlers/stats

revert changes in go.mod and go.sum

revert change in go.work

add schedule for contributors stats job

change response parsing order

changes due to draft-pr review

 move on with contributors activity implementation

change to every hour

fix typo

 change contributor stats implementation to do a single write transaction

 normalize to UTC contributors activity timestamp

 add cronjob schedule for contributors

[Issue:1052][Part 2] Create endpoint to expose contributors stats and activities  (#1123)

* add endpoint for retrieving stats and activity

* remove model.go file and move types to service file

* add unit tests to contributors service

* integrate new contributors controller

* fix more stuff

 fix unit-tests

 changes due to pr review

 fix query

 fix unit-tests

fix total_value_secure

 move constantes to common pkg

remove extra changes

rename contributor to protocols

 finish renames

Changes for deployment

adjust different response types from different protocols contributors

 fix controller test

 big refactor in activty job and stats job since protocols are returning different formats

api responding fine

 remove uneccessary generics

 target dbconsts

 fix

Delete deploy/common/env/staging-mainnet.env

undo unwanted changes

 readd staging-mainnet.env

 fix unit-tests

add missing protocols_stats/activity_version

 remove property protocols_json

fix JOB_ID env var in protocols-activity.yaml

fix typos in env vars configs

 change tu numbers

 changes due to own review

 add new line

* add swagger docs
2024-02-22 14:58:45 -03:00
ftocal 7d5a3bbe52
Improve fly reliability (#1094)
Add sqs queue for observations
Add tx hash strategy for vaas
Refactoring to organize the construction
2024-02-06 21:00:45 -03:00
ftocal c4707fbb69
Fix vaa parse endpoint (#1078) 2024-02-01 12:01:24 -03:00
ftocal d8aeb72cb6
Fix token address from native token address in vaa-payload-parser response (#1066)
Update supported tokens

Co-authored-by: walker-16 <agpazos85@gmail.com>
2024-02-01 11:26:51 -03:00
walker-16 fb0573c160
migrate vaa to globaltransaction origintx (#940)
* migrate vaa to globaltransaction origintx

* Add deploy configuration for job to migrate vaas to originTx

* Add option to run migration process by range of date

* Update go dependencies for jobs

* Fix kind of job in deployment

---------

Co-authored-by: Fernando Torres <fert1335@gmail.com>
2024-01-30 12:19:31 -03:00
ftocal 49f9bedd29
Handle transfer-redeemed event from blockchain-watcher (#1039) 2024-01-24 14:51:04 -03:00
ftocal d3c1c52b79
Add support to sepolia chains (#1001)
Co-authored-by: walker-16 <agpazos85@gmail.com>
2024-01-22 11:01:18 -03:00
walker-16 9cb5769c06
Update wormhole sdk to version v0.0.0-20240109172745-cc0cd9fc5229 in … (#998)
Update wormhole sdk to version v0.0.0-20240109172745-cc0cd9fc5229 in all the components
2024-01-17 12:36:54 -03:00
ftocal 27bf188149
Delete unhandled messages from sqs (#987)
Add more log
2024-01-15 14:11:41 -03:00
ftocal e21b836455
Add handler for redeem events from blockchain-watcher (#986) 2024-01-15 10:22:05 -03:00
ftocal d7391f6ba1
Create api notional with historical prices (#931)
Add historical prices job
Change transfer report job
2024-01-02 10:38:03 -03:00
ftocal af5bd3017f
Handle native tx hash for solana when receive event from blockchain-watcher (#851) 2023-12-04 16:29:38 -03:00
ftocal 695fd0dcd4
Add supported tokens for testnet (#810)
Update supported tokens for mainnet
2023-11-28 10:16:40 -03:00
ftocal 7c467f5267
Integrating blockchain watcher data into mongodb (#820)
* Handle log-message-published generated by blockchain-watcher in parser, tx-tracker y analytics

* Add deployment for staging-testnet environment
Improve logs

* Add operation endpoints

---------

Co-authored-by: Agustin Pazos <agpazos85@gmail.com>
2023-11-27 12:31:35 -03:00
ftocal 28fc5198b0
New job for transfer report (#756)
Co-authored-by: walker-16 <agpazos85@gmail.com>
2023-10-20 18:30:34 -03:00
ftocal 91bf42971b
Use Redis channel in spy and fly (#753) 2023-10-18 11:18:32 -03:00