Commit Graph

202 Commits

Author SHA1 Message Date
agodnic 37d6d42c19
Remove duplicated code in token database (#356)
### Summary

Tracking issue: https://github.com/wormhole-foundation/wormhole-explorer/issues/344

Before this pull request, there were two separate token databases (one being used by the InfluxDB backfiller, and another one in the `common/` module being used by the analytics service).

Having two different token databases resulted in inconsistencies, due to each of these databases containing different tokens.

This PR unifies those two databases into a single one, under the `common/` module.
2023-05-30 11:14:19 -03:00
agodnic 4aca29fe7d
Generate volume metrics for unknown tokens (#348)
### Summary
This pull request fixes an issue in which volume metrics were not being generated for unknown tokens (i.e.: tokens that are not present in our database).

Also, the function that generated volume metrics was modified in order to handle timestamp collisions.
2023-05-29 10:54:09 -03:00
walker-16 cb9532b5c1
Remove unused observation fields version and upadte docs (#355) 2023-05-29 10:53:52 -03:00
ftocal 5c2f91d935
Add backfiller command to process vaa saved in database (#349)
* Add endpoint to parse a vaa

* Add backfiller command to process vaa saved in database
2023-05-24 16:52:22 -03:00
agodnic 69251f136d
[API] Run scorecard queries concurrently (#339)
### Summary

Before pull request, in `GET /api/v1/scorecards`, the queries for each scorecard were being executed sequentially. This PR changes the handler to run all those queries concurrently.

Tracking issue: https://github.com/wormhole-foundation/wormhole-explorer/issues/336
2023-05-23 14:50:19 -03:00
ftocal fe574754eb
Fix volume when the metric is notional (#341) 2023-05-23 11:40:54 -03:00
agodnic 3035c3e1f2
Update InfluxDB backfiller (#335)
### Summary

This pull request adds the source code from https://github.com/XLabs/influx-backfiller to this repository.

Also, several changes have been made to the InfluxDB backfiller:
* Stop using floats to store/manipulate price data, use decimals or integers instead.
* Update the volume metric to be in sync with the rest of the codebase.
* Remove duplicated code between the influx backfiller and analytics modules.
* Remove a fair amount of unused code
2023-05-23 11:27:23 -03:00
walker-16 878b201259
fix globalTransaction empty id issue (#337) 2023-05-22 17:30:21 -03:00
ftocal 1a78481b08
Changes to deploy on production environment (#331)
Co-authored-by: walker-16 <agpazos85@gmail.com>
2023-05-19 13:39:16 -03:00
ftocal 63ae5a21a5
Add tvl envs in api deployment (#328) 2023-05-18 17:07:00 -03:00
agodnic ce72cf7463
[API] Add 24h messages to `GET /api/v1/scorecards` (#326)
### Summary

Tracking issue: https://github.com/wormhole-foundation/wormhole-explorer/issues/320

This pull request adds the `24h_messages` field to the response in `GET /api/v1/scorecards`. This field indicates the total number of VAAs emitted in the last 24 hours, including messages from PythNet.

Also, the analytics component has been updated to generate a new metric for this purpose. There is a task that summarizes data every 5 minutes to speed up query times. The data is retained for 24 hours only. For this particular metric, writes to InfluxDB are being batched to improve performance.
2023-05-18 11:14:36 -03:00
walker-16 3a445da161
288 tvl (#327)
* get tvl from json api

* Add tvl scorecards

---------

Co-authored-by: gipsh <gipsh@MaasNeoTek>
2023-05-17 15:04:17 -03:00
agodnic d75062ae50
Fix inaccurate volume in `GET /api/v1/scorecards` (#312)
The endpoint `GET /api/v1/scorecards` was returning an inaccurate number for the volume metric. This pull request fixes the issue.

Additionally, the response model of `GET /api/v1/top-assets-by-volume` has been changed as specified by @raop155:
```
{
  "assets": [
    {
      "symbol": "USDCet",
      "tokenChain": 1,
      "tokenAddress": "000000000000000000000000a0b86991c6218b36c1d19d4a2e9eb0ce3606eb48",
      "volume": "475173.83806883"
    }
  ]
}

```
2023-05-15 17:30:15 -03:00
walker-16 9d7484e97b
Add last-tx filter for valid configuration and fix doc (#319) 2023-05-15 16:59:54 -03:00
ftocal c2b94f6448
Add total tx count and total tx volume by portal bridge for scorecards (#314)
Co-authored-by: walker-16 <agpazos85@gmail.com>
2023-05-15 15:15:12 -03:00
agodnic c25ebcb6fc
New endpoint: top chain pairs by number of transfers (#307)
### Summary

Tracking issue: https://github.com/wormhole-foundation/wormhole-explorer/issues/275

This pull request implements the endpoint `GET /api/v1/top-chain-pairs-by-num-transfers`, which returns the chain pairs that have the most transfers.Internally, the endpoint uses data summarized daily to speed up query execution times.

This endpoint has a mandatory query parameter named timerange, which must be set to `7d`, `15d` or `30d`.
2023-05-12 13:05:18 -03:00
walker-16 25a675f99a
fix query from last-tx endpoint (#311)
Co-authored-by: ftocal <fert1335@gmail.com>
2023-05-12 10:54:29 -03:00
ftocal eb414c9f65
fix api deployment (#308) 2023-05-11 12:16:12 -03:00
ftocal 12b39bea59
Add summarization data for tx_count endpoint (#296)
* Add summarization data for tx_count endpoint

Co-authored-by: walker-16 <agpazos85@gmail.com>

* Add new buckets in deployment

Co-authored-by: walker-16 <agpazos85@gmail.com>

---------

Co-authored-by: walker-16 <agpazos85@gmail.com>
2023-05-10 18:18:32 -03:00
agodnic a21f40ed55
Get top assets by volume (#302)
### Summary

Tracking issue: https://github.com/wormhole-foundation/wormhole-explorer/issues/276

This pull request implements the endpoint `GET /api/v1/top-assets-by-volume`, which returns the assets that have the highest volume. Internally, the endpoint uses data summarized daily to speed up query execution times.

This endpoint has a mandatory query parameter named `timerange`, which must be set to `7d`, `15d` or `30d`.
2023-05-10 17:39:18 -03:00
gipsh 3867a2eb24
remove 0x prefix and lowercase the hash (#257)
Co-authored-by: gipsh <gipsh@gmail.com>
2023-05-10 17:34:24 -03:00
ftocal 7fb3857a97
297 contract watchertx trackeranalyticfly change access to use iam role (#306)
* Use aws role to access sqs

Co-authored-by: walker-16 <agpazos85@gmail.com>

* Add ServiceAccount in deployment

Co-authored-by: walker-16 <agpazos85@gmail.com>

---------

Co-authored-by: walker-16 <agpazos85@gmail.com>
2023-05-10 15:49:40 -03:00
walker-16 4ecf5983e6
Fix allways analytic push func Failed (#305) 2023-05-10 13:15:37 -03:00
agodnic a31d59822d
Add definitions to token dictionary (#292)
### Summary

In order to compute volume metrics for each token, we need metadata about the token that is not present in the VAAs (e.g.: decimals, symbol).

That information is statically defined in `common/domain/tokenbridge.go`. This pull request adds the most relevant tokens (i.e.: the tokens that contribute the most volume) to the existing definitions. It is very likely that more token definitions will be added in the future as needed.

The token metadata that was previously defined in the `notional` package (symbols, coingecko IDs) was also moved to `common/domain/tokenbridge.go`. 

Tracking issue https://github.com/wormhole-foundation/wormhole-explorer/issues/281
2023-05-08 17:51:18 -03:00
walker-16 e2f32c2883
Fix global tx to keep successful status (#294)
* Fix global tx to keep successful status

Co-authored-by: ftocal <fert1335@gmail.com>

* Increment evm retry from 5 to 10

---------

Co-authored-by: ftocal <fert1335@gmail.com>
2023-05-08 12:55:09 -03:00
agodnic d9d49ec919
Add 24h volume to scorecards (#274)
### Summary

This pull request adds volume metrics to influxdb. Also, it adds the 24h volume metric to `GET /api/v1/scorecards`.

Tracking issues: https://github.com/wormhole-foundation/wormhole-explorer/issues/221, https://github.com/wormhole-foundation/wormhole-explorer/issues/280

### Changes:
* The `parser` service no longer generates metrics for influxdb. All metrics-related code was removed from that service, that code was moved to the analytics service instead.
* New volume metrics were added to the analytics service.
* The notional cache was modified to use token names (i.e.: ticker symbols) as keys instead of chain IDs.
* The notional cache reader was moved to the `common/client/cache` package.
* A little bit of duplicated code between the cache reader and writer was removed.
* A 24h volume metric was added to `GET /api/v1/scorecards`.
* A dictionary that stores token metadata was added under `common/domain/tokenbridge.go`. More tokens will be added to it in the near future.
2023-05-04 20:17:03 -03:00
walker-16 fd51f0a819
Fix endpoint last-txs to fit portal explorer requirements (#286) 2023-05-03 14:01:55 -03:00
ftocal c0a3ed2ea9
Add moonbean deploy (#284) 2023-05-03 11:05:27 -03:00
walker-16 f79057a375
273 contract watcher fix status for evm standard (#278)
* Add client to eth transaction Receipt endpoint

* Only process evm transaction from the contractAddress

* Add function to get Status for evm and evm standard watcher

* Fix unkown status and retry count
2023-05-02 16:52:28 -03:00
ftocal 1e87d5f022
Fix status for solana (#277)
Add retry mechanism
2023-05-02 16:44:13 -03:00
agodnic 69ff71d440
Sort the output of `GET /api/v1/vaas` by timestamp (#267)
### Summary

The output of `GET /api/v1/vaas` was not being correctly sorted. This pull request changes the behavior so that the output will be sorted by the `timestamp` field.

Additionally, duplicated code was removed in the VAA code that handles queries.

Also, there was an unused query parameter `sortBy` which was being exposed but not used. Hence it was removed.

Tracking issue: https://github.com/wormhole-foundation/wormhole-explorer/issues/233
2023-04-28 10:18:07 -03:00
walker-16 1ea141ec95
fix pagination in governor notional available endpoint (#272) 2023-04-27 17:31:33 -03:00
walker-16 bae5f06afb
250 governornotionalavailable sending an invalid chainid returns invalid data (#270)
* Check supported chainID in endpoint governor available notional by chain

* check supported chainID in endpoint mx notional available by chainID
2023-04-27 13:56:18 -03:00
walker-16 c0ec64c749
fix moonbeam ratelimit (#269) 2023-04-27 11:17:31 -03:00
walker-16 45af329b56
Add moonbeam to contract-watcher mainent, testnet (#268) 2023-04-27 10:17:43 -03:00
ftocal a381bbdb8f
Fix in solana watcher when api endpoint fails (#266) 2023-04-26 16:58:30 -03:00
walker-16 459f74d3d2
change txhash pipeline retry to 5 (#264) 2023-04-25 17:19:00 -03:00
ftocal 58e2879068
Filter vaa and observation for pyth message in testnet (#263) 2023-04-25 16:23:07 -03:00
agodnic 09870f3d65
[tx-tracker / api] Avoid duplicating data (#256)
### Summary
Tracking issue: https://github.com/wormhole-foundation/wormhole-explorer/issues/254

For all chains except Solana, the data in `globalTransactions.sourceTx` was somewhat duplicated, since we already had that data in the `vaas` collection.
This pull request changes the `tx-tracker` service so that no redundant data will be written, and also changes the API to look up the data from the `vaas` collection instead.

The responses from the endpoint `/api/v1/global-tx` will no longer contain the field `originTx.from`.
2023-04-25 15:34:29 -03:00
walker-16 3ea4603baf
return empty array and not null in 200 empty repsonse governor limit … (#261)
return empty array and not null in 200 empty repsonse governor limit endpoint
2023-04-25 14:50:44 -03:00
ftocal c9bd4e0dbe
Add oasis support to contract watcher (#258) 2023-04-25 14:48:26 -03:00
walker-16 4a49d1f4cd
discard pyth message to get txhash in testnet (#260) 2023-04-25 12:04:47 -03:00
ftocal 4e0c077a63
Add aptos support to contract watcher (#255) 2023-04-24 12:02:19 -03:00
walker-16 f8b631c71f
Add avalanche to contract-watcher (#253) 2023-04-21 18:30:20 -03:00
walker-16 ffeb5927f8
Add notional cache client (#241)
* Add notional cache client

* gracefull shutdown pubsub and distributed cache
2023-04-20 17:02:25 -03:00
agodnic c8aba636e4
[API/ANALYTICS] Scorecards endpoint (#232)
### Summary

This pull request adds the `GET /api/v1/scorecards` endpoint, which is required by the wormscan frontend.

Most of the fields that this endpoint should return are being omitted because the data is not currently available on the backend. Those fields will be added iteratively as the data becomes available.

The current format of the response is:
```json
{
  "total_tx_count": "1300200",
  "24h_tx_count": "4200"
}
```

Tracking issue: https://github.com/wormhole-foundation/wormhole-explorer/issues/221

## Deployment details

In order to populate the `"total_tx_count"` metric, a task is needed in influxdb:

```
$ cat total-vaa-count.flux 
option task = {
	name: "Total number of emitted VAAs",
	every: 1m
}

from(bucket: "wormhole-explorer")
  |> range(start: 2018-01-01T00:00:00Z)
  |> filter(fn: (r) => r._measurement == "vaa_count")
  |> group(columns: ["_measurement"])
  |> set(key: "_measurement", value: "total_vaa_count")
  |> count()
  |> map(fn: (r) => ({r with _time: now()}))
  |> map(fn: (r) => ({r with _field: "total_vaa_count"}))
  |> to(bucket: "wormhole-explorer", org: "xlabs")

```
2023-04-20 16:01:10 -03:00
ftocal 18efc01460
Increase memory resources for contract-watcher (#248)
Increase memory resources for contract-watcher in staging and production environments
2023-04-20 12:54:30 -03:00
walker-16 ce83e1c3a6
Add gracefull shutdown to explorer api (#244) 2023-04-19 14:30:46 -03:00
ftocal 4f438ae188
Add notional asset lookup (#239)
* Add notional job

* fix coingecko integration

* Add deployment for notional jobs

---------

Co-authored-by: Agustin Pazos <agpazos85@gmail.com>
2023-04-18 12:09:31 -03:00
gipsh 3c597917f5
Improve txhash handler (#237)
* txHash separate handler

* remove unused code

* tests

* improve test and fix error

---------

Co-authored-by: gipsh <gipsh@gmail.com>
2023-04-17 10:42:28 -03:00