Commit Graph

96 Commits

Author SHA1 Message Date
ftocal e366f04765
Find by txHash in observations endpoint (#1450)
Add txHash query parameter in observations endpoint
Add job to add nativeTxHash in observations collection except for solana, aptos and wormchain
Add encoded txHash in obvervations collection
2024-06-04 12:12:44 -03:00
walker-16 7545acb77e
Feature/handle governor fly event processor (#1436)
* changes in fly component to send governor status events

* split processor in vaaProcessor and governor processor

* add governor processsor v1

* Add endpoint in tx-tracker to calculate txHash for a vaa id

* fly-event-processor integration with new tx-tracker endpoint and refactor

* Add governor vaas endpoint in api

* api, fix amount data type in governor vaas endpoint

* fly-event-processor normalize emitter and txHash

* fly-event-processor fix nodeGovernorVaa id

* fly-event-processor control array not empty in insert/delete many operation

* add index in nodeGovernorVaas collection by vaaId

* add prometheus metrics

* add tx-tracker url for fly-event-processor deployment

* Add sns attributes into sns messages

Co-authored-by: walker-16 <agpazos85@gmail.com>

* fix governor vaa endpoint empty response

---------

Co-authored-by: Fernando Torres <fert1335@gmail.com>
2024-05-31 15:38:21 -03:00
ftocal 4809dbda21
Update last guadian set automatically from eth contract (#1414)
* Update last guadian set automatically from eth contract

* Continue testing for an intermittent failure
2024-05-27 11:01:16 -03:00
walker-16 b8e3f43333
1301 create a backgroundrecurrent process which checks which duplicate vaas txs actually reached finality on the blockchain (#1359)
* fly-event-processor initial commit

* Handle duplicated vaa event in fly-event-processor
Add deployment for fly-event-processor
Send duplicated vaa event from fly

Co-authored-by: walker-16 <agpazos85@gmail.com>

---------

Co-authored-by: Fernando Torres <fert1335@gmail.com>
2024-05-07 16:38:16 -03:00
ftocal 688a0d0f71
Store duplicate vaas on a secondary collection (#1308)
* Add support to scroll in tx-tracker

* Handle duplicated vaa in fly

Co-authored-by: walker-16 <agpazos85@gmail.com>

* Modify tx-tracker and pipeline to support duplicated vaas
Co-authored-by: ftocal <fert1335@gmail.com>

* Add isDuplicated flag in vaas and operations endpoints
Add new endpoint to find duplicated vaas by vaa id
Co-authored-by: walker-16 <agpazos85@gmail.com>

* Add metrics for duplicated vaas

Co-authored-by: walker-16 <agpazos85@gmail.com>

* Update node dependencies in spy

Co-authored-by: walker-16 <agpazos85@gmail.com>

* Fix promethes metrics for missing tx-hash in tx-tracker

Co-authored-by: walker-16 <agpazos85@gmail.com>

---------

Co-authored-by: Agustin Pazos <agpazos85@gmail.com>
2024-04-22 14:26:07 -03:00
ftocal b34beb63df
Add new version of guardian set in fly (#1311)
Co-authored-by: walker-16 <agpazos85@gmail.com>
2024-04-16 19:49:08 -03:00
Mariano e06057acfd
[ISSUE-1220] Add filters to /operations endpoint (#1262)
* start

 add search by chain

 change chainId query param handling

 change condition

 only from chain

change query

 add filter by appId

 add payload type for operations query

add logs

 add log for error in mongodb call

 add more logs and recover to find possible panic

change type to float64

add more logs for troubleshooting

 add more logs for troubleshooting payloadType query param

add another defer

* change query

* add exclusiveAppId,sourceChain and targetChain

* unify search criteria

* change queryies

* combine query params filters

* change implementation of sourceChain and targetChain

* insert filtering by chain and by appid as stages in aggregation pipeline

* fix appIds matching condition

* move query to a separate pipeline which starts from parsedVAA

* adjust query by appId

* add matching also for standardizedProperties

* change

* try using  instead of

* simplify query

* add queryLoggging and remove other parts of the query to troubleshooting

* working

* add index creation

* update swagger docs

* tweak index performance and fix timestamp on parsedVaa collection

* start

 add search by chain

 change chainId query param handling

 change condition

 only from chain

change query

 add filter by appId

 add payload type for operations query

add logs

 add log for error in mongodb call

 add more logs and recover to find possible panic

change type to float64

add more logs for troubleshooting

 add more logs for troubleshooting payloadType query param

add another defer

* change query

* add exclusiveAppId,sourceChain and targetChain

* unify search criteria

* change queryies

* combine query params filters

* change implementation of sourceChain and targetChain

* insert filtering by chain and by appid as stages in aggregation pipeline

* fix appIds matching condition

* move query to a separate pipeline which starts from parsedVAA

* adjust query by appId

* add matching also for standardizedProperties

* change

* try using  instead of

* simplify query

* add queryLoggging and remove other parts of the query to troubleshooting

* working

* add index creation

* update swagger docs

* tweak index performance and fix timestamp on parsedVaa collection

* add lookup for globaltransactions
2024-04-09 09:57:08 -03:00
ftocal 67d0e970f0
Increment observation tx hash cache expiration (#1184)
* Increment observation tx hash cache expiration

* Add cache dedup for pyth vaas

* Increment observation dedup cache

* wip
2024-03-12 12:23:29 -03:00
ftocal 2bec07b037
Add improvements in fly (#1183)
Add metrics for observation by guardian
Configurable cache
Add metrics for cache
Fallback for vaas publication
2024-03-06 11:59:09 -03:00
ftocal 98a6264739
Changes to support ci/cd workflow in staging environment (#1166)
Co-authored-by: walker-16 <agpazos85@gmail.com>
2024-02-29 11:14:31 -03:00
walker-16 4760c820c3
808 enhance operationsaddress to provide information for all address formats (#1145)
* Add index in globalTransactions by originTx.attribute.value.originAddress

* Add in the operation by address endpoint support to wormchain chains
2024-02-23 10:32:26 -03:00
ftocal c9bc9660f6
Remove redis in txHash store (#1112) 2024-02-08 12:10:13 -03:00
ftocal 408851ff1f
Save observation in mongo as fallback when the publishing fails in sqs queue (#1106)
Co-authored-by: walker-16 <agpazos85@gmail.com>
2024-02-07 17:02:57 -03:00
walker-16 c8665d68f5
Fix globalTransactions index in fly (#1104)
Co-authored-by: ftocal fert1335@gmail.com
2024-02-07 12:30:29 -03:00
ftocal 7d5a3bbe52
Improve fly reliability (#1094)
Add sqs queue for observations
Add tx hash strategy for vaas
Refactoring to organize the construction
2024-02-06 21:00:45 -03:00
walker-16 90f615dd0d
modify boostrap to fix quic-v1 version (#996) 2024-02-01 11:23:30 -03:00
walker-16 8c5e9e5353
Feature/improve operation endpoint performance (#1064)
* Change query to sort by timestamp and id

* Add maximun size to pageSize in operation endpoint

* Add missing indexes for mongodb

* improve search txHash for nested wormchain txHash

---------

Co-authored-by: Fernando Torres <fert1335@gmail.com>
2024-02-01 11:21:34 -03:00
walker-16 fb0573c160
migrate vaa to globaltransaction origintx (#940)
* migrate vaa to globaltransaction origintx

* Add deploy configuration for job to migrate vaas to originTx

* Add option to run migration process by range of date

* Update go dependencies for jobs

* Fix kind of job in deployment

---------

Co-authored-by: Fernando Torres <fert1335@gmail.com>
2024-01-30 12:19:31 -03:00
ftocal fbf5809241
Increase vaa splitter buffer channel (#985)
Co-authored-by: walker-16 <agpazos85@gmail.com>
2024-01-12 20:31:56 -03:00
ftocal d8d1d76a1e
Add quic-v1 support in mainnet (#967)
Co-authored-by: walker-16 <agpazos85@gmail.com>
2024-01-11 12:03:58 -03:00
ftocal 076338f63c
Add quic-v1 support in testnet (#927)
Co-authored-by: walker-16 <agpazos85@gmail.com>
2023-12-20 15:10:30 -03:00
ftocal c23909348a
Add bootstrap peers in mainnet environment (#925)
Co-authored-by: walker-16 <agpazos85@gmail.com>
2023-12-19 15:56:40 -03:00
ftocal 1c075e3745
Add worker pool to process observations from gossip network (#909)
Co-authored-by: walker-16 <agpazos85@gmail.com>
2023-12-18 10:41:07 -03:00
ftocal 7c467f5267
Integrating blockchain watcher data into mongodb (#820)
* Handle log-message-published generated by blockchain-watcher in parser, tx-tracker y analytics

* Add deployment for staging-testnet environment
Improve logs

* Add operation endpoints

---------

Co-authored-by: Agustin Pazos <agpazos85@gmail.com>
2023-11-27 12:31:35 -03:00
ftocal 91bf42971b
Use Redis channel in spy and fly (#753) 2023-10-18 11:18:32 -03:00
walker-16 955d69e099
Modify testnet p2p bootstrap to add xlabs (#745) 2023-10-11 14:35:39 -03:00
walker-16 217c6f28be
Fix [api] get transactions by address and add mongo query monitor (#707)
Fix api endpoint get transactions by address, add mongo query monitor
2023-09-25 16:50:16 -03:00
ftocal 8b74d76f30
Update wormhole node dependency version in fly (#654)
Co-authored-by: walker-16 <agpazos85@gmail.com>
2023-08-24 15:16:43 -03:00
ftocal 503a9da5d3
UDP buffer configuration in fly (#660)
Co-authored-by: walker-16 <agpazos85@gmail.com>
2023-08-23 10:19:16 -03:00
ftocal be7e6670b0
Add alert for guardain no activity (#609)
Co-authored-by: walker-16 <agpazos85@gmail.com>
2023-08-08 11:11:35 -03:00
agodnic 3c7bab3f8a
Remove duplicated code: MongoDB connection handling (#590)
### Description

This pull request removes duplicated code related to MongoDB connection/disconnection attempts. This code was copied across all 8 microservices.

The functionality is now unified under the `common/dbutil` package.
2023-08-07 16:05:08 -03:00
walker-16 75fffe3ef6
add warning alert and metrics to fly component (#576)
* Add unknown guardian set index alert

* Add prometheus metrics to check if exists observations without txhash

* Add max sequence cache error prometheus metrics
2023-07-25 15:39:27 -03:00
ftocal 94307b849a
[fly] Missing observations (#570)
Add environment variables for buffer channels (observations y vaas)
Increment size of buffer channels for observations and vaas
Align k8s resources in fly

Co-authored-by: walker-16 <agpazos85@gmail.com>
2023-07-24 11:24:39 -03:00
walker-16 c4a55bd6e2
Fix alert prefix with new enviroment values (#509) 2023-07-06 11:20:18 -03:00
ftocal 4e8b43a598
Fix max-sequece key in fly (#491) 2023-07-03 15:58:34 -03:00
ftocal fc28528e0d
Fix cache and metrics environments in fly (#490) 2023-07-03 15:10:52 -03:00
walker-16 b07d569edf
Add pipeline prometheus metrics and opsgenie alerts (#471)
* Add pipeline prometheus metrics

* Add alerts & metrics env vars for pipeline

* fix pipeline env vars

* Fix enviroment in fly and pipeline

* Add alert client to pipeline

* Add critical alerts to pipeline component

* fix pipeline deploy

* fix ALERTS_ENABLED env var

* Add prometheus annotation metrics

* configurate prometheus in fiber

* fix vaaSendNotifications metrics counter

* Fix NewWatcher
2023-07-03 11:42:44 -03:00
walker-16 38944cafd3
Fix env vars Environment (#476) 2023-06-29 15:07:07 -03:00
gipsh 2306d103d8
321 replace redis to elasticcache (#467)
* redis prefix support for caches

* fly support for prefix

* unit tests

* redis prefix for notional cache updater

* fix test

* fix tests

* use redis-prfix from config map
2023-06-28 15:04:36 -03:00
walker-16 1ba76b5c85
Fix METRIC_ENABLED to METRICS_ENABLED in fly component (#469) 2023-06-28 13:07:45 -03:00
walker-16 914580cfdc
Add critical opsgenie alert to fly (#463) 2023-06-27 12:17:33 -03:00
walker-16 471a6dad4d
fix prometheus field name from GuardianNode to guardian_node (#460)
Co-authored-by: ftocal <fert1335@gmail.com>
2023-06-26 16:25:28 -03:00
walker-16 737405054a
446 create prometheus metrics (#457)
* Add vaa metrics

* Add dummy metrics and observations metrics

* Add heartbeat, governorConfig, governorStatus metrics

* fix go.sum

* Add deploy en vars METRICS_ENABLED to fly component

Co-authored-by: ftocal <fert1335@gmail.com>

---------

Co-authored-by: Fernando Torres <fert1335@gmail.com>
2023-06-26 12:47:22 -03:00
walker-16 1c67efc3d5
Add deploy alert env var to fly + fix message prefix (#456) 2023-06-26 11:47:23 -03:00
walker-16 35b7d7b46b
remove unnecessary env var using opsgenie team api-key (#454)
remove unnecessary env var using opsgenie team
2023-06-23 11:08:57 -03:00
walker-16 25b891f6d2
Add opsgenie client + error saving vaa alert in fly component (#442) 2023-06-22 18:59:23 -03:00
ftocal 9381695d37
Fix txHash encoding by chain (#410)
Add txHash encondig backfiller
Handle txHash base58 encoding for solana in tx-tracker
Add temporary field _originTxHash in vaas and vaaIdTxHash collections by backup
Add cobra to fly backfiller

Co-authored-by: walker-16 <agpazos85@gmail.com>
2023-06-15 09:57:39 -03:00
gipsh 3867a2eb24
remove 0x prefix and lowercase the hash (#257)
Co-authored-by: gipsh <gipsh@gmail.com>
2023-05-10 17:34:24 -03:00
ftocal 7fb3857a97
297 contract watchertx trackeranalyticfly change access to use iam role (#306)
* Use aws role to access sqs

Co-authored-by: walker-16 <agpazos85@gmail.com>

* Add ServiceAccount in deployment

Co-authored-by: walker-16 <agpazos85@gmail.com>

---------

Co-authored-by: walker-16 <agpazos85@gmail.com>
2023-05-10 15:49:40 -03:00
ftocal 58e2879068
Filter vaa and observation for pyth message in testnet (#263) 2023-04-25 16:23:07 -03:00