Commit Graph

39 Commits

Author SHA1 Message Date
Mariano e06057acfd
[ISSUE-1220] Add filters to /operations endpoint (#1262)
* start

 add search by chain

 change chainId query param handling

 change condition

 only from chain

change query

 add filter by appId

 add payload type for operations query

add logs

 add log for error in mongodb call

 add more logs and recover to find possible panic

change type to float64

add more logs for troubleshooting

 add more logs for troubleshooting payloadType query param

add another defer

* change query

* add exclusiveAppId,sourceChain and targetChain

* unify search criteria

* change queryies

* combine query params filters

* change implementation of sourceChain and targetChain

* insert filtering by chain and by appid as stages in aggregation pipeline

* fix appIds matching condition

* move query to a separate pipeline which starts from parsedVAA

* adjust query by appId

* add matching also for standardizedProperties

* change

* try using  instead of

* simplify query

* add queryLoggging and remove other parts of the query to troubleshooting

* working

* add index creation

* update swagger docs

* tweak index performance and fix timestamp on parsedVaa collection

* start

 add search by chain

 change chainId query param handling

 change condition

 only from chain

change query

 add filter by appId

 add payload type for operations query

add logs

 add log for error in mongodb call

 add more logs and recover to find possible panic

change type to float64

add more logs for troubleshooting

 add more logs for troubleshooting payloadType query param

add another defer

* change query

* add exclusiveAppId,sourceChain and targetChain

* unify search criteria

* change queryies

* combine query params filters

* change implementation of sourceChain and targetChain

* insert filtering by chain and by appid as stages in aggregation pipeline

* fix appIds matching condition

* move query to a separate pipeline which starts from parsedVAA

* adjust query by appId

* add matching also for standardizedProperties

* change

* try using  instead of

* simplify query

* add queryLoggging and remove other parts of the query to troubleshooting

* working

* add index creation

* update swagger docs

* tweak index performance and fix timestamp on parsedVaa collection

* add lookup for globaltransactions
2024-04-09 09:57:08 -03:00
ftocal adb7231074
Add processed message metrics in pipeline components (#1239)
Co-authored-by: walker-16 <agpazos85@gmail.com>
2024-03-25 15:21:10 -03:00
ftocal f98d8f28ba
Add backfiller for parser with more parameters (#1197) 2024-03-19 12:28:10 -03:00
ftocal 98a6264739
Changes to support ci/cd workflow in staging environment (#1166)
Co-authored-by: walker-16 <agpazos85@gmail.com>
2024-02-29 11:14:31 -03:00
ftocal d3c1c52b79
Add support to sepolia chains (#1001)
Co-authored-by: walker-16 <agpazos85@gmail.com>
2024-01-22 11:01:18 -03:00
walker-16 9cb5769c06
Update wormhole sdk to version v0.0.0-20240109172745-cc0cd9fc5229 in … (#998)
Update wormhole sdk to version v0.0.0-20240109172745-cc0cd9fc5229 in all the components
2024-01-17 12:36:54 -03:00
ftocal 27bf188149
Delete unhandled messages from sqs (#987)
Add more log
2024-01-15 14:11:41 -03:00
ftocal e21b836455
Add handler for redeem events from blockchain-watcher (#986) 2024-01-15 10:22:05 -03:00
ftocal d7391f6ba1
Create api notional with historical prices (#931)
Add historical prices job
Change transfer report job
2024-01-02 10:38:03 -03:00
ftocal 695fd0dcd4
Add supported tokens for testnet (#810)
Update supported tokens for mainnet
2023-11-28 10:16:40 -03:00
ftocal 7c467f5267
Integrating blockchain watcher data into mongodb (#820)
* Handle log-message-published generated by blockchain-watcher in parser, tx-tracker y analytics

* Add deployment for staging-testnet environment
Improve logs

* Add operation endpoints

---------

Co-authored-by: Agustin Pazos <agpazos85@gmail.com>
2023-11-27 12:31:35 -03:00
walker-16 217c6f28be
Fix [api] get transactions by address and add mongo query monitor (#707)
Fix api endpoint get transactions by address, add mongo query monitor
2023-09-25 16:50:16 -03:00
walker-16 87fd9d15ef
Create an api endpoint to call vaa payload parser component (#596)
* move vaa payload parser to common

* Add proxy endpoint to vaa payload parser

* Modify api deployment

* fix docs

* Fix swagger documentation
2023-08-10 11:02:14 -03:00
ftocal 0d2b5bdfd7
Add index to parsedVaa collection (#616)
* Improve log to get latency for all endpoints except ready and health

Co-authored-by: walker-16 <agpazos85@gmail.com>

* Add index to parsedVaa collection

Co-authored-by: walker-16 <agpazos85@gmail.com>

---------

Co-authored-by: walker-16 <agpazos85@gmail.com>
2023-08-09 11:55:51 -03:00
agodnic 3c7bab3f8a
Remove duplicated code: MongoDB connection handling (#590)
### Description

This pull request removes duplicated code related to MongoDB connection/disconnection attempts. This code was copied across all 8 microservices.

The functionality is now unified under the `common/dbutil` package.
2023-08-07 16:05:08 -03:00
walker-16 4f1987c698
514 modify parser to support new vaa payload parser response (#521)
Parse amount from metadata fields
Modify parser client to ParseWithMetadata
Add support to new vaa payload parser parse endpoint

Co-authored-by: Fernando Torres <fert1335@gmail.com>
2023-07-12 12:51:17 -03:00
walker-16 c4a55bd6e2
Fix alert prefix with new enviroment values (#509) 2023-07-06 11:20:18 -03:00
walker-16 96ec327b74
Add opsgenie alertClient to parser (#496)
* Add opsgenie alertClient to parser

* Add metricsClient to processor

* Define critical opsgenie alert parser

* Add parser alert
2023-07-05 15:39:56 -03:00
agodnic 88089a6a0e
Update parser Makefile (#501)
### Description

* Remove unused Makefile target
* Put executable files in a `bin/` directory
2023-07-04 17:48:47 -03:00
ftocal b16f1739d0
Change to deploy prometheus metrics for parser and pipeline (#497) 2023-07-04 15:23:26 -03:00
walker-16 2a92a93eb4
475 parser create prometheus metrics (#484)
* Fix Environment env var in parser and deploy

* Add alert and metrics env vars config

* Define parser metrics

* Add parser prometheus annotations

* Fix metrics definitions

* Implement metrics in parser

* Fix phrometheus annotation from backfiller to service

* Add prometheus fiber
2023-07-03 15:46:47 -03:00
ftocal 5c2f91d935
Add backfiller command to process vaa saved in database (#349)
* Add endpoint to parse a vaa

* Add backfiller command to process vaa saved in database
2023-05-24 16:52:22 -03:00
ftocal 7fb3857a97
297 contract watchertx trackeranalyticfly change access to use iam role (#306)
* Use aws role to access sqs

Co-authored-by: walker-16 <agpazos85@gmail.com>

* Add ServiceAccount in deployment

Co-authored-by: walker-16 <agpazos85@gmail.com>

---------

Co-authored-by: walker-16 <agpazos85@gmail.com>
2023-05-10 15:49:40 -03:00
agodnic d9d49ec919
Add 24h volume to scorecards (#274)
### Summary

This pull request adds volume metrics to influxdb. Also, it adds the 24h volume metric to `GET /api/v1/scorecards`.

Tracking issues: https://github.com/wormhole-foundation/wormhole-explorer/issues/221, https://github.com/wormhole-foundation/wormhole-explorer/issues/280

### Changes:
* The `parser` service no longer generates metrics for influxdb. All metrics-related code was removed from that service, that code was moved to the analytics service instead.
* New volume metrics were added to the analytics service.
* The notional cache was modified to use token names (i.e.: ticker symbols) as keys instead of chain IDs.
* The notional cache reader was moved to the `common/client/cache` package.
* A little bit of duplicated code between the cache reader and writer was removed.
* A 24h volume metric was added to `GET /api/v1/scorecards`.
* A dictionary that stores token metadata was added under `common/domain/tokenbridge.go`. More tokens will be added to it in the near future.
2023-05-04 20:17:03 -03:00
agodnic d0cf9cc04b
[API] Fix issue in VAA search filters (#211)
### Summary

Searching for VAAs with the `txHash` and the `parsedPayload` parameters simultaneously wasn't working correctly. This commit fixes the problem.

Tracking issue: https://github.com/wormhole-foundation/wormhole-explorer/issues/210
2023-03-27 16:14:38 -03:00
ftocal e1298f54ee
logging as json (#208) 2023-03-23 15:36:50 -03:00
ftocal 867f1c2962
[API/PARSER] Chain activity and last transaction endpoints (#175)
* Fix analytic vaa metric and close metric client

* Add analytic component manifest

* Add metrics for cross chain activity endpoint

* Feature/add endpoint get vaa count (#171)

* Add trasaction controller + service

* Init getLastTrx endpoint

* Add endpoint for chain activity

* Add configmap and secrets for influx
Change Dockerfiles to support common library

* Add vaa_count endpoint

* Add cross-chain activity endpoint

* Fix vaa_count endpoint

* Add influx to api-service manifest

* Make response and add docs for chain activity endpoint

* Fix api deploy

* Fix lasl-trx json name

* Fix chain activity endpoint

* Fix doc and endpoint route last-txs

---------

Co-authored-by: Agustin Pazos <agpazos85@gmail.com>
2023-03-07 16:25:42 -03:00
agodnic c1dea6ea4d
[PARSER/PIPELINE] Update .gitignore (#174) 2023-03-06 14:35:53 -03:00
walker-16 cbb7956f90
Feature/refactor parser to use sns sqs (#148)
* Add README to configurate SNS and subscribe with parser FIFO SQS

* Remove producer and watcher, handler new queue vaa event message
2023-02-16 10:55:54 -03:00
ftocal 84e4fba5a1
Improve logs for health/ready endpoints (#133)
* Improve logs for health/ready endpoints in fly

* Improve logs for health/ready endpoints in spy

* Improve logs for health/ready endpoints in parser
2023-02-03 15:58:58 -03:00
walker-16 7506ff0bf5
Add pprof all components (#134)
* Fix Config pprof in fly

* Refactor pprof explorer parser component

* Add pprof to spy component

* Add pprof to api [api/v1, /v1]
2023-02-03 15:18:44 -03:00
ftocal 821071086d
Fix: parser memory leaks in sqs consumer (#122)
Co-authored-by: walker-16 <agpazos85@gmail.com>
2023-02-02 14:42:23 -03:00
gipsh dcea5b2332
pass context to missing funcs (#116)
add missiong context

Co-authored-by: gipsh <gipsh@gmail.com>
2023-02-01 06:43:07 -03:00
walker-16 9933fb0d9d
Change aws sdk to v2, to fix memory leak long pooling (#95) 2023-01-26 14:15:39 -03:00
gipsh f4f543fc95
create cicd pipeline for deployments to staging environment - step 1 (#71)
* add missing makefiles and fix deps

* add parser makefile and github action for PR

* fix warning

* fix printf

Co-authored-by: gipsh <gipsh@gmail.com>
2023-01-23 18:41:19 -03:00
walker-16 2283e84ed6
Enabled pprof parser component (#83) 2023-01-23 16:40:39 -03:00
ftocal 70bf3f1059
update sdk wormhole (#82) 2023-01-23 16:13:59 -03:00
ftocal 9c069aabc3
Add k8s deployment for parser (#63) 2023-01-19 12:42:18 -03:00
ftocal 303b731d60
Feature/add parser pipeline (#46)
* Add pipeline to parse vaa

* Add parser repository and start building pipeline publisher

* Add queue sqs and in-memory to pipeline

* Add ses publisher and queue sqs/memory

* Fix watcher query

* Fix type to marshall/unsmarshall chainID and emitterAddress

* Fix js execution is run many times

* Add comments

* Add comments and improvements

* Change VAA Parser implementation

* Fix and change integration-test vaa-payload-parser

* fix appID mongo field name

* Fix guardianSet version 3, add xlabs to position 8

Co-authored-by: Agustin Pazos <agpazos85@gmail.com>
2023-01-18 12:42:14 -03:00