Commit Graph

92 Commits

Author SHA1 Message Date
ftocal 688a0d0f71
Store duplicate vaas on a secondary collection (#1308)
* Add support to scroll in tx-tracker

* Handle duplicated vaa in fly

Co-authored-by: walker-16 <agpazos85@gmail.com>

* Modify tx-tracker and pipeline to support duplicated vaas
Co-authored-by: ftocal <fert1335@gmail.com>

* Add isDuplicated flag in vaas and operations endpoints
Add new endpoint to find duplicated vaas by vaa id
Co-authored-by: walker-16 <agpazos85@gmail.com>

* Add metrics for duplicated vaas

Co-authored-by: walker-16 <agpazos85@gmail.com>

* Update node dependencies in spy

Co-authored-by: walker-16 <agpazos85@gmail.com>

* Fix promethes metrics for missing tx-hash in tx-tracker

Co-authored-by: walker-16 <agpazos85@gmail.com>

---------

Co-authored-by: Agustin Pazos <agpazos85@gmail.com>
2024-04-22 14:26:07 -03:00
ftocal b34beb63df
Add new version of guardian set in fly (#1311)
Co-authored-by: walker-16 <agpazos85@gmail.com>
2024-04-16 19:49:08 -03:00
Mariano e06057acfd
[ISSUE-1220] Add filters to /operations endpoint (#1262)
* start

 add search by chain

 change chainId query param handling

 change condition

 only from chain

change query

 add filter by appId

 add payload type for operations query

add logs

 add log for error in mongodb call

 add more logs and recover to find possible panic

change type to float64

add more logs for troubleshooting

 add more logs for troubleshooting payloadType query param

add another defer

* change query

* add exclusiveAppId,sourceChain and targetChain

* unify search criteria

* change queryies

* combine query params filters

* change implementation of sourceChain and targetChain

* insert filtering by chain and by appid as stages in aggregation pipeline

* fix appIds matching condition

* move query to a separate pipeline which starts from parsedVAA

* adjust query by appId

* add matching also for standardizedProperties

* change

* try using  instead of

* simplify query

* add queryLoggging and remove other parts of the query to troubleshooting

* working

* add index creation

* update swagger docs

* tweak index performance and fix timestamp on parsedVaa collection

* start

 add search by chain

 change chainId query param handling

 change condition

 only from chain

change query

 add filter by appId

 add payload type for operations query

add logs

 add log for error in mongodb call

 add more logs and recover to find possible panic

change type to float64

add more logs for troubleshooting

 add more logs for troubleshooting payloadType query param

add another defer

* change query

* add exclusiveAppId,sourceChain and targetChain

* unify search criteria

* change queryies

* combine query params filters

* change implementation of sourceChain and targetChain

* insert filtering by chain and by appid as stages in aggregation pipeline

* fix appIds matching condition

* move query to a separate pipeline which starts from parsedVAA

* adjust query by appId

* add matching also for standardizedProperties

* change

* try using  instead of

* simplify query

* add queryLoggging and remove other parts of the query to troubleshooting

* working

* add index creation

* update swagger docs

* tweak index performance and fix timestamp on parsedVaa collection

* add lookup for globaltransactions
2024-04-09 09:57:08 -03:00
ftocal 67d0e970f0
Increment observation tx hash cache expiration (#1184)
* Increment observation tx hash cache expiration

* Add cache dedup for pyth vaas

* Increment observation dedup cache

* wip
2024-03-12 12:23:29 -03:00
ftocal 2bec07b037
Add improvements in fly (#1183)
Add metrics for observation by guardian
Configurable cache
Add metrics for cache
Fallback for vaas publication
2024-03-06 11:59:09 -03:00
ftocal 98a6264739
Changes to support ci/cd workflow in staging environment (#1166)
Co-authored-by: walker-16 <agpazos85@gmail.com>
2024-02-29 11:14:31 -03:00
walker-16 4760c820c3
808 enhance operationsaddress to provide information for all address formats (#1145)
* Add index in globalTransactions by originTx.attribute.value.originAddress

* Add in the operation by address endpoint support to wormchain chains
2024-02-23 10:32:26 -03:00
ftocal c9bc9660f6
Remove redis in txHash store (#1112) 2024-02-08 12:10:13 -03:00
ftocal 408851ff1f
Save observation in mongo as fallback when the publishing fails in sqs queue (#1106)
Co-authored-by: walker-16 <agpazos85@gmail.com>
2024-02-07 17:02:57 -03:00
walker-16 c8665d68f5
Fix globalTransactions index in fly (#1104)
Co-authored-by: ftocal fert1335@gmail.com
2024-02-07 12:30:29 -03:00
ftocal 7d5a3bbe52
Improve fly reliability (#1094)
Add sqs queue for observations
Add tx hash strategy for vaas
Refactoring to organize the construction
2024-02-06 21:00:45 -03:00
walker-16 90f615dd0d
modify boostrap to fix quic-v1 version (#996) 2024-02-01 11:23:30 -03:00
walker-16 8c5e9e5353
Feature/improve operation endpoint performance (#1064)
* Change query to sort by timestamp and id

* Add maximun size to pageSize in operation endpoint

* Add missing indexes for mongodb

* improve search txHash for nested wormchain txHash

---------

Co-authored-by: Fernando Torres <fert1335@gmail.com>
2024-02-01 11:21:34 -03:00
walker-16 fb0573c160
migrate vaa to globaltransaction origintx (#940)
* migrate vaa to globaltransaction origintx

* Add deploy configuration for job to migrate vaas to originTx

* Add option to run migration process by range of date

* Update go dependencies for jobs

* Fix kind of job in deployment

---------

Co-authored-by: Fernando Torres <fert1335@gmail.com>
2024-01-30 12:19:31 -03:00
ftocal fbf5809241
Increase vaa splitter buffer channel (#985)
Co-authored-by: walker-16 <agpazos85@gmail.com>
2024-01-12 20:31:56 -03:00
ftocal d8d1d76a1e
Add quic-v1 support in mainnet (#967)
Co-authored-by: walker-16 <agpazos85@gmail.com>
2024-01-11 12:03:58 -03:00
ftocal 076338f63c
Add quic-v1 support in testnet (#927)
Co-authored-by: walker-16 <agpazos85@gmail.com>
2023-12-20 15:10:30 -03:00
ftocal c23909348a
Add bootstrap peers in mainnet environment (#925)
Co-authored-by: walker-16 <agpazos85@gmail.com>
2023-12-19 15:56:40 -03:00
ftocal 1c075e3745
Add worker pool to process observations from gossip network (#909)
Co-authored-by: walker-16 <agpazos85@gmail.com>
2023-12-18 10:41:07 -03:00
ftocal 7c467f5267
Integrating blockchain watcher data into mongodb (#820)
* Handle log-message-published generated by blockchain-watcher in parser, tx-tracker y analytics

* Add deployment for staging-testnet environment
Improve logs

* Add operation endpoints

---------

Co-authored-by: Agustin Pazos <agpazos85@gmail.com>
2023-11-27 12:31:35 -03:00
ftocal 91bf42971b
Use Redis channel in spy and fly (#753) 2023-10-18 11:18:32 -03:00
walker-16 955d69e099
Modify testnet p2p bootstrap to add xlabs (#745) 2023-10-11 14:35:39 -03:00
walker-16 217c6f28be
Fix [api] get transactions by address and add mongo query monitor (#707)
Fix api endpoint get transactions by address, add mongo query monitor
2023-09-25 16:50:16 -03:00
ftocal 8b74d76f30
Update wormhole node dependency version in fly (#654)
Co-authored-by: walker-16 <agpazos85@gmail.com>
2023-08-24 15:16:43 -03:00
ftocal 503a9da5d3
UDP buffer configuration in fly (#660)
Co-authored-by: walker-16 <agpazos85@gmail.com>
2023-08-23 10:19:16 -03:00
ftocal be7e6670b0
Add alert for guardain no activity (#609)
Co-authored-by: walker-16 <agpazos85@gmail.com>
2023-08-08 11:11:35 -03:00
agodnic 3c7bab3f8a
Remove duplicated code: MongoDB connection handling (#590)
### Description

This pull request removes duplicated code related to MongoDB connection/disconnection attempts. This code was copied across all 8 microservices.

The functionality is now unified under the `common/dbutil` package.
2023-08-07 16:05:08 -03:00
walker-16 75fffe3ef6
add warning alert and metrics to fly component (#576)
* Add unknown guardian set index alert

* Add prometheus metrics to check if exists observations without txhash

* Add max sequence cache error prometheus metrics
2023-07-25 15:39:27 -03:00
ftocal 94307b849a
[fly] Missing observations (#570)
Add environment variables for buffer channels (observations y vaas)
Increment size of buffer channels for observations and vaas
Align k8s resources in fly

Co-authored-by: walker-16 <agpazos85@gmail.com>
2023-07-24 11:24:39 -03:00
walker-16 c4a55bd6e2
Fix alert prefix with new enviroment values (#509) 2023-07-06 11:20:18 -03:00
ftocal 4e8b43a598
Fix max-sequece key in fly (#491) 2023-07-03 15:58:34 -03:00
ftocal fc28528e0d
Fix cache and metrics environments in fly (#490) 2023-07-03 15:10:52 -03:00
walker-16 b07d569edf
Add pipeline prometheus metrics and opsgenie alerts (#471)
* Add pipeline prometheus metrics

* Add alerts & metrics env vars for pipeline

* fix pipeline env vars

* Fix enviroment in fly and pipeline

* Add alert client to pipeline

* Add critical alerts to pipeline component

* fix pipeline deploy

* fix ALERTS_ENABLED env var

* Add prometheus annotation metrics

* configurate prometheus in fiber

* fix vaaSendNotifications metrics counter

* Fix NewWatcher
2023-07-03 11:42:44 -03:00
walker-16 38944cafd3
Fix env vars Environment (#476) 2023-06-29 15:07:07 -03:00
gipsh 2306d103d8
321 replace redis to elasticcache (#467)
* redis prefix support for caches

* fly support for prefix

* unit tests

* redis prefix for notional cache updater

* fix test

* fix tests

* use redis-prfix from config map
2023-06-28 15:04:36 -03:00
walker-16 1ba76b5c85
Fix METRIC_ENABLED to METRICS_ENABLED in fly component (#469) 2023-06-28 13:07:45 -03:00
walker-16 914580cfdc
Add critical opsgenie alert to fly (#463) 2023-06-27 12:17:33 -03:00
walker-16 471a6dad4d
fix prometheus field name from GuardianNode to guardian_node (#460)
Co-authored-by: ftocal <fert1335@gmail.com>
2023-06-26 16:25:28 -03:00
walker-16 737405054a
446 create prometheus metrics (#457)
* Add vaa metrics

* Add dummy metrics and observations metrics

* Add heartbeat, governorConfig, governorStatus metrics

* fix go.sum

* Add deploy en vars METRICS_ENABLED to fly component

Co-authored-by: ftocal <fert1335@gmail.com>

---------

Co-authored-by: Fernando Torres <fert1335@gmail.com>
2023-06-26 12:47:22 -03:00
walker-16 1c67efc3d5
Add deploy alert env var to fly + fix message prefix (#456) 2023-06-26 11:47:23 -03:00
walker-16 35b7d7b46b
remove unnecessary env var using opsgenie team api-key (#454)
remove unnecessary env var using opsgenie team
2023-06-23 11:08:57 -03:00
walker-16 25b891f6d2
Add opsgenie client + error saving vaa alert in fly component (#442) 2023-06-22 18:59:23 -03:00
ftocal 9381695d37
Fix txHash encoding by chain (#410)
Add txHash encondig backfiller
Handle txHash base58 encoding for solana in tx-tracker
Add temporary field _originTxHash in vaas and vaaIdTxHash collections by backup
Add cobra to fly backfiller

Co-authored-by: walker-16 <agpazos85@gmail.com>
2023-06-15 09:57:39 -03:00
gipsh 3867a2eb24
remove 0x prefix and lowercase the hash (#257)
Co-authored-by: gipsh <gipsh@gmail.com>
2023-05-10 17:34:24 -03:00
ftocal 7fb3857a97
297 contract watchertx trackeranalyticfly change access to use iam role (#306)
* Use aws role to access sqs

Co-authored-by: walker-16 <agpazos85@gmail.com>

* Add ServiceAccount in deployment

Co-authored-by: walker-16 <agpazos85@gmail.com>

---------

Co-authored-by: walker-16 <agpazos85@gmail.com>
2023-05-10 15:49:40 -03:00
ftocal 58e2879068
Filter vaa and observation for pyth message in testnet (#263) 2023-04-25 16:23:07 -03:00
ftocal e1298f54ee
logging as json (#208) 2023-03-23 15:36:50 -03:00
ftocal 824ba3c7f9
Add contract watcher component (#190)
* init contract-watcher

* Add processor and blockain watchers

* Add pagination and save last blocknumber procesed by chain

* Add processing by blocks

* Add contract-watcher deploy manifest

* Add endpoint to get globalTransactions by Id

* Add originTX to get globalTransactionById endpoint

* Add wait time for new blocks

* Add initial block for evm watcher

* Add rate limit for evm watcher

* Handle testnet environment and small fixes

* Update wormhole dependencies

* Fix api documentation for swagger

---------

Co-authored-by: Agustin Pazos <agpazos85@gmail.com>
2023-03-15 16:52:50 -03:00
ftocal 867f1c2962
[API/PARSER] Chain activity and last transaction endpoints (#175)
* Fix analytic vaa metric and close metric client

* Add analytic component manifest

* Add metrics for cross chain activity endpoint

* Feature/add endpoint get vaa count (#171)

* Add trasaction controller + service

* Init getLastTrx endpoint

* Add endpoint for chain activity

* Add configmap and secrets for influx
Change Dockerfiles to support common library

* Add vaa_count endpoint

* Add cross-chain activity endpoint

* Fix vaa_count endpoint

* Add influx to api-service manifest

* Make response and add docs for chain activity endpoint

* Fix api deploy

* Fix lasl-trx json name

* Fix chain activity endpoint

* Fix doc and endpoint route last-txs

---------

Co-authored-by: Agustin Pazos <agpazos85@gmail.com>
2023-03-07 16:25:42 -03:00
agodnic d51695999a
[FLY] Remove duplicated code (#172)
Remove duplicated constants, use the definitions in the `common` package instead.
2023-03-06 14:36:40 -03:00