Commit Graph

431 Commits

Author SHA1 Message Date
gipsh 32e67a5824
Use new custom storage for redis (#425)
* allow prefix for redis keys on rate limiter

* use storage module from lib

---------

Co-authored-by: Hernan Gips <hernangips@OnoSendai.local>
2023-06-20 12:25:04 -03:00
ftocal 69885aed0c
Fix intermittent failure in x-chain-activity endpoint (#420)
* Replace start_time and end_time query parameters with fixed ranges at the endpoint of the chain activity
Add InfluxDB tasks for enpdoint chain-activity for each timeSpan
Add cache for chain-activity endpoint

* Code review updates
2023-06-20 10:34:20 -03:00
agodnic 50c1d479a6
Add support for Moonbeam and Klaytn in the `tx-tracker` service (#422)
### Description

This pull request modifies the `tx-tracker` service to support the Moonbeam and Klaytn blockchains.

In particular, this will make sender addresses for these particular blockchains available for the Wormhole Scan UI.
2023-06-20 09:37:06 -03:00
gipsh b490be0f5d
fix api doc basepath (#423)
fix api doc basepath

Co-authored-by: Hernan Gips <hernangips@OnoSendai.local>
2023-06-18 13:17:22 -03:00
agodnic 3f823c51ac
Add support for Aptos in the `tx-tracker` service (#421)
### Description

This pull request adds support for Aptos in the `tx-tracker` service.

This will lead to improvements for the wormhole Scan UI:
1. The transaction hash that was being previously displayed for Aptos VAAs was incorrect. This pull request fixes the issue.
2. The sender addresses for Aptos VAAs will now become available for for the UI.
2023-06-16 17:47:28 -03:00
gipsh ba3d1d9e61
Vanilla rate limiter (#414)
* update doc

* api def update

* first commit

* support redis, allow for private ips

* add deployment vars

---------

Co-authored-by: Hernan Gips <hernangips@OnoSendai.local>
2023-06-15 23:22:47 -03:00
agodnic bf3be056b4
Add support for Sui in the `tx-tracker` service (#416)
### Description

This pull request modifies the `tx-tracker` service to support the Sui blockchain.

In particular, this will make the Sui sender addresses available for the Wormhole Scan UI.
2023-06-15 21:42:46 -03:00
walker-16 97946d5a56
Issue [#446] remove duplicate data transaction history graph (#413) 2023-06-15 13:01:57 -03:00
ftocal 9381695d37
Fix txHash encoding by chain (#410)
Add txHash encondig backfiller
Handle txHash base58 encoding for solana in tx-tracker
Add temporary field _originTxHash in vaas and vaaIdTxHash collections by backup
Add cobra to fly backfiller

Co-authored-by: walker-16 <agpazos85@gmail.com>
2023-06-15 09:57:39 -03:00
agodnic 19743322e4
Retrieve sender address for each transaction (#408)
### Summary

Tracking issue: https://github.com/wormhole-foundation/wormhole-explorer/issues/404

The WormholeScan UI needs to display the sender address for each token bridge VAA. This pull request modifies the `tx-tracker` service to obtain that information for Solana and eight EVM chains.

The transaction sender will become accessible through the following endpoints:
* `GET /api/v1/global-tx/{chain}/{emitter}/{seq}`: field `originTx.from`.
* `GET /api/v1/transactions`: field `originAddress`.
In both cases, the field is nullable (i.e.: sometimes it may not be available due to eventual consistency or internal errors)
2023-06-14 17:46:53 -03:00
agodnic a0475ab17e
Add endpoint `GET /api/v1/transactions` (#388)
### Summary
Tracking issue: https://github.com/wormhole-foundation/wormhole-explorer/issues/385

This pull request implements a new endpoint, `GET /api/v1/transactions`, which will be consumed by the wormhole explorer UI.

The endpoint returns a paginated list of transactions, in which each element contains a brief overview of the transaction (ID, txHash, status, etc.).

It exposes offset-based pagination via the parameters `page` and `pageSize`. Also, results can be obtained for a specific address by using the `address` query parameter.

The response model looks like this:
```json
{
  "transactions": [
   {
    "id": "1/5ec18c34b47c63d17ab43b07b9b2319ea5ee2d163bce2e467000174e238c8e7f/12965",
    "timestamp": "2023-06-08T19:30:19Z",
    "txHash": "a302c4ab2d6b9a6003951d2e91f8fdbb83cfa20f6ffb588b95ef0290aab37066",
    "originChain": 1,
    "status": "ongoing"
  },
  {
    "id": "22/0000000000000000000000000000000000000000000000000000000000000001/18308",
    "timestamp": "2023-06-08T19:17:14Z",
    "txHash": "00000000000000000000000000000000000000000000000000000000000047e7",
    "originChain": 22,
    "destinationAddress": "0x00000000000000000000000067e8a40816a983fbe3294aaebd0cc2391815b86b",
    "destinationChain": 5,
    "tokenAmount": "0.12",
    "usdAmount": "0.12012",
    "symbol": "USDC",
    "status": "completed"
  },
  ...
  ]
}
```

### Limitations of the current implementation
1. Doesn't return the total number of results (this may result in a performance issue when we filter by address)
2. Can only filter by receiver address (we don't have sender information in the database yet)
2023-06-12 11:43:48 -03:00
agodnic d0436fa46d
Improve pipeline Makefile (#387)
### Description
Minor refactoring in the pipeline service's Makefile.

Changes:
* Move `pipeline` executable to the `pipeline/bin` directory.
* Remove unused targets from `pipeline/Makefile`.
2023-06-12 10:00:23 -03:00
walker-16 100e473c2f
Add celo to contract-watcher(token bridge, wormhole connect contracts) (#393) 2023-06-09 17:57:22 -03:00
walker-16 8679d9c211
Contract-watcher add support wormhole connect wrapper contract (#391) 2023-06-08 17:53:51 -03:00
ftocal 444c185f38
Ignore metric when emitter or destination chains are unset (#390) 2023-06-08 13:03:15 -03:00
ftocal b02d602b14
fix percentage for destinations when is notional (#389) 2023-06-08 10:38:34 -03:00
agodnic 598db1b876
Persist symbol and price for VAAs (#384)
### Description

Tracking issue: https://github.com/wormhole-foundation/wormhole-explorer/issues/377

This pull request modifies the analytics service so that it persists price information for a VAA in MongoDB. Data is stored only for token bridge VAAs, and only if the price is cached for the symbol being used.

A new collection `transferPrices` has been created, with the following document model:
```
{
  "_id": "22/0000000000000000000000000000000000000000000000000000000000000001/18087",
  "price": "0.999684",
  "symbol": "USDC",
  "timestamp": 2023-06-06T14:04:47.000+00:00
  "tokenAmount": "0.1",
  "usdAmount": "0.0999684"
}
```
2023-06-07 12:18:38 -03:00
ftocal b4c217e32c
Fix status for aptos blockchain (#386)
Fix pagination for backfilling
2023-06-06 18:04:34 -03:00
ftocal ec95642a9b
Fix vaa-count metric backfiller and Influx query for x-chain-activity endpoint (#382)
Fix vaa-count metric backfiller
Fix Influx query for x-chain-activity endpoint
2023-06-05 16:33:52 -03:00
agodnic 2a3e2f7f0b
Add log messages on analytics service startup (#368)
### Summary

Often times, the analytics service fails to start due to connectivity issues with other services (specifically redis).

Previously, no log information was being emitted, which made it hard to troubleshoot the exact reason for the service failing to start.

This pull request adds log messages when the service starts to make those cases easier to debug.
2023-06-01 17:02:50 -03:00
Ben Guidarelli 6cbdf36b2c
Fix url param name (#370)
Fix URL parameter name in swagger documentation
2023-06-01 17:02:11 -03:00
gipsh a835a220fe
Update doc (#375)
* update doc

* api def update

* update swagger doc

* update deps
2023-06-01 16:46:24 -03:00
ftocal 112ad51c69
Fix and refactor of analytics (#376)
Fix vaa-volume metric backfiller
Rename analytic to analytics
Move influx-backfiller as part of analytics
2023-06-01 16:32:00 -03:00
agodnic e7a5f0c83a
Fix an infinite loop in redis cache client (#369)
When initializing the notional cache, if there was more than one page of results, the client would run into an infinite loop.

This issue was causing several services to stall on startup in the staging environment (API, analytics, etc).
2023-05-31 16:25:16 -03:00
agodnic 0ec1cb86e2
Use decimals instead of floats in price cache (#362)
## Description

Previously, the notional cache was using the `float64` type to manipulate price data. Since floating point types can't represent price data accurately, this commit changes the codebase to use a lossless representation (i.e.: `decimal.Decimal`).
2023-05-31 16:24:40 -03:00
ftocal 06d6950ad3
Add backfiller command to process a chain (#364)
* Add backfiller command to process a chain

* Add deployment for solana backfiller
2023-05-31 10:29:47 -03:00
walker-16 c510df7948
Add endpoint to get token data by chainId and token_address (#365) 2023-05-31 10:29:16 -03:00
agodnic 9d33d82789
Add log messages on API startup (#361)
### Summary

Often times, the API service fails to start due to connectivity issues with other services (e.g.: redis).

Previously, no log information was being emitted, which made it hard to troubleshoot the exact reason for the service failing to start.

This pull request adds log messages when the service starts to make those cases easier to debug.
2023-05-30 11:37:10 -03:00
agodnic 9e2d3b445d
[API] Fix sorting order in observation endpoints (#357)
### Summary

This pull request fixes an issue in which all observation-related endpoints were returning results sorted by `id` (https://github.com/wormhole-foundation/wormhole-explorer/issues/358). The expected behavior is to sort results based on descending timestamp order instead.

Also, swagger docs were updated to document this behavior.
2023-05-30 11:35:30 -03:00
agodnic 37d6d42c19
Remove duplicated code in token database (#356)
### Summary

Tracking issue: https://github.com/wormhole-foundation/wormhole-explorer/issues/344

Before this pull request, there were two separate token databases (one being used by the InfluxDB backfiller, and another one in the `common/` module being used by the analytics service).

Having two different token databases resulted in inconsistencies, due to each of these databases containing different tokens.

This PR unifies those two databases into a single one, under the `common/` module.
2023-05-30 11:14:19 -03:00
agodnic 4aca29fe7d
Generate volume metrics for unknown tokens (#348)
### Summary
This pull request fixes an issue in which volume metrics were not being generated for unknown tokens (i.e.: tokens that are not present in our database).

Also, the function that generated volume metrics was modified in order to handle timestamp collisions.
2023-05-29 10:54:09 -03:00
walker-16 cb9532b5c1
Remove unused observation fields version and upadte docs (#355) 2023-05-29 10:53:52 -03:00
ftocal 5c2f91d935
Add backfiller command to process vaa saved in database (#349)
* Add endpoint to parse a vaa

* Add backfiller command to process vaa saved in database
2023-05-24 16:52:22 -03:00
agodnic 69251f136d
[API] Run scorecard queries concurrently (#339)
### Summary

Before pull request, in `GET /api/v1/scorecards`, the queries for each scorecard were being executed sequentially. This PR changes the handler to run all those queries concurrently.

Tracking issue: https://github.com/wormhole-foundation/wormhole-explorer/issues/336
2023-05-23 14:50:19 -03:00
ftocal fe574754eb
Fix volume when the metric is notional (#341) 2023-05-23 11:40:54 -03:00
agodnic 3035c3e1f2
Update InfluxDB backfiller (#335)
### Summary

This pull request adds the source code from https://github.com/XLabs/influx-backfiller to this repository.

Also, several changes have been made to the InfluxDB backfiller:
* Stop using floats to store/manipulate price data, use decimals or integers instead.
* Update the volume metric to be in sync with the rest of the codebase.
* Remove duplicated code between the influx backfiller and analytics modules.
* Remove a fair amount of unused code
2023-05-23 11:27:23 -03:00
walker-16 878b201259
fix globalTransaction empty id issue (#337) 2023-05-22 17:30:21 -03:00
ftocal 1a78481b08
Changes to deploy on production environment (#331)
Co-authored-by: walker-16 <agpazos85@gmail.com>
2023-05-19 13:39:16 -03:00
ftocal 63ae5a21a5
Add tvl envs in api deployment (#328) 2023-05-18 17:07:00 -03:00
agodnic ce72cf7463
[API] Add 24h messages to `GET /api/v1/scorecards` (#326)
### Summary

Tracking issue: https://github.com/wormhole-foundation/wormhole-explorer/issues/320

This pull request adds the `24h_messages` field to the response in `GET /api/v1/scorecards`. This field indicates the total number of VAAs emitted in the last 24 hours, including messages from PythNet.

Also, the analytics component has been updated to generate a new metric for this purpose. There is a task that summarizes data every 5 minutes to speed up query times. The data is retained for 24 hours only. For this particular metric, writes to InfluxDB are being batched to improve performance.
2023-05-18 11:14:36 -03:00
walker-16 3a445da161
288 tvl (#327)
* get tvl from json api

* Add tvl scorecards

---------

Co-authored-by: gipsh <gipsh@MaasNeoTek>
2023-05-17 15:04:17 -03:00
agodnic d75062ae50
Fix inaccurate volume in `GET /api/v1/scorecards` (#312)
The endpoint `GET /api/v1/scorecards` was returning an inaccurate number for the volume metric. This pull request fixes the issue.

Additionally, the response model of `GET /api/v1/top-assets-by-volume` has been changed as specified by @raop155:
```
{
  "assets": [
    {
      "symbol": "USDCet",
      "tokenChain": 1,
      "tokenAddress": "000000000000000000000000a0b86991c6218b36c1d19d4a2e9eb0ce3606eb48",
      "volume": "475173.83806883"
    }
  ]
}

```
2023-05-15 17:30:15 -03:00
walker-16 9d7484e97b
Add last-tx filter for valid configuration and fix doc (#319) 2023-05-15 16:59:54 -03:00
ftocal c2b94f6448
Add total tx count and total tx volume by portal bridge for scorecards (#314)
Co-authored-by: walker-16 <agpazos85@gmail.com>
2023-05-15 15:15:12 -03:00
agodnic c25ebcb6fc
New endpoint: top chain pairs by number of transfers (#307)
### Summary

Tracking issue: https://github.com/wormhole-foundation/wormhole-explorer/issues/275

This pull request implements the endpoint `GET /api/v1/top-chain-pairs-by-num-transfers`, which returns the chain pairs that have the most transfers.Internally, the endpoint uses data summarized daily to speed up query execution times.

This endpoint has a mandatory query parameter named timerange, which must be set to `7d`, `15d` or `30d`.
2023-05-12 13:05:18 -03:00
walker-16 25a675f99a
fix query from last-tx endpoint (#311)
Co-authored-by: ftocal <fert1335@gmail.com>
2023-05-12 10:54:29 -03:00
ftocal eb414c9f65
fix api deployment (#308) 2023-05-11 12:16:12 -03:00
ftocal 12b39bea59
Add summarization data for tx_count endpoint (#296)
* Add summarization data for tx_count endpoint

Co-authored-by: walker-16 <agpazos85@gmail.com>

* Add new buckets in deployment

Co-authored-by: walker-16 <agpazos85@gmail.com>

---------

Co-authored-by: walker-16 <agpazos85@gmail.com>
2023-05-10 18:18:32 -03:00
agodnic a21f40ed55
Get top assets by volume (#302)
### Summary

Tracking issue: https://github.com/wormhole-foundation/wormhole-explorer/issues/276

This pull request implements the endpoint `GET /api/v1/top-assets-by-volume`, which returns the assets that have the highest volume. Internally, the endpoint uses data summarized daily to speed up query execution times.

This endpoint has a mandatory query parameter named `timerange`, which must be set to `7d`, `15d` or `30d`.
2023-05-10 17:39:18 -03:00
gipsh 3867a2eb24
remove 0x prefix and lowercase the hash (#257)
Co-authored-by: gipsh <gipsh@gmail.com>
2023-05-10 17:34:24 -03:00