For Solana and Aptos, the transaction hash returned by `GET /api/v1/vaas` is incorrect (currently it's the value transmitted in the gossip network, which is not the real transaction hash).
This pull request replaces it with the real transaction hash.
* Replace start_time and end_time query parameters with fixed ranges at the endpoint of the chain activity
Add InfluxDB tasks for enpdoint chain-activity for each timeSpan
Add cache for chain-activity endpoint
* Code review updates
### Description
This pull request adds support for Aptos in the `tx-tracker` service.
This will lead to improvements for the wormhole Scan UI:
1. The transaction hash that was being previously displayed for Aptos VAAs was incorrect. This pull request fixes the issue.
2. The sender addresses for Aptos VAAs will now become available for for the UI.
Add txHash encondig backfiller
Handle txHash base58 encoding for solana in tx-tracker
Add temporary field _originTxHash in vaas and vaaIdTxHash collections by backup
Add cobra to fly backfiller
Co-authored-by: walker-16 <agpazos85@gmail.com>
### Summary
Tracking issue: https://github.com/wormhole-foundation/wormhole-explorer/issues/404
The WormholeScan UI needs to display the sender address for each token bridge VAA. This pull request modifies the `tx-tracker` service to obtain that information for Solana and eight EVM chains.
The transaction sender will become accessible through the following endpoints:
* `GET /api/v1/global-tx/{chain}/{emitter}/{seq}`: field `originTx.from`.
* `GET /api/v1/transactions`: field `originAddress`.
In both cases, the field is nullable (i.e.: sometimes it may not be available due to eventual consistency or internal errors)
### Summary
Tracking issue: https://github.com/wormhole-foundation/wormhole-explorer/issues/385
This pull request implements a new endpoint, `GET /api/v1/transactions`, which will be consumed by the wormhole explorer UI.
The endpoint returns a paginated list of transactions, in which each element contains a brief overview of the transaction (ID, txHash, status, etc.).
It exposes offset-based pagination via the parameters `page` and `pageSize`. Also, results can be obtained for a specific address by using the `address` query parameter.
The response model looks like this:
```json
{
"transactions": [
{
"id": "1/5ec18c34b47c63d17ab43b07b9b2319ea5ee2d163bce2e467000174e238c8e7f/12965",
"timestamp": "2023-06-08T19:30:19Z",
"txHash": "a302c4ab2d6b9a6003951d2e91f8fdbb83cfa20f6ffb588b95ef0290aab37066",
"originChain": 1,
"status": "ongoing"
},
{
"id": "22/0000000000000000000000000000000000000000000000000000000000000001/18308",
"timestamp": "2023-06-08T19:17:14Z",
"txHash": "00000000000000000000000000000000000000000000000000000000000047e7",
"originChain": 22,
"destinationAddress": "0x00000000000000000000000067e8a40816a983fbe3294aaebd0cc2391815b86b",
"destinationChain": 5,
"tokenAmount": "0.12",
"usdAmount": "0.12012",
"symbol": "USDC",
"status": "completed"
},
...
]
}
```
### Limitations of the current implementation
1. Doesn't return the total number of results (this may result in a performance issue when we filter by address)
2. Can only filter by receiver address (we don't have sender information in the database yet)
### Summary
Often times, the API service fails to start due to connectivity issues with other services (e.g.: redis).
Previously, no log information was being emitted, which made it hard to troubleshoot the exact reason for the service failing to start.
This pull request adds log messages when the service starts to make those cases easier to debug.
### Summary
This pull request fixes an issue in which all observation-related endpoints were returning results sorted by `id` (https://github.com/wormhole-foundation/wormhole-explorer/issues/358). The expected behavior is to sort results based on descending timestamp order instead.
Also, swagger docs were updated to document this behavior.
### Summary
Tracking issue: https://github.com/wormhole-foundation/wormhole-explorer/issues/344
Before this pull request, there were two separate token databases (one being used by the InfluxDB backfiller, and another one in the `common/` module being used by the analytics service).
Having two different token databases resulted in inconsistencies, due to each of these databases containing different tokens.
This PR unifies those two databases into a single one, under the `common/` module.
### Summary
This pull request fixes an issue in which volume metrics were not being generated for unknown tokens (i.e.: tokens that are not present in our database).
Also, the function that generated volume metrics was modified in order to handle timestamp collisions.
### Summary
Before pull request, in `GET /api/v1/scorecards`, the queries for each scorecard were being executed sequentially. This PR changes the handler to run all those queries concurrently.
Tracking issue: https://github.com/wormhole-foundation/wormhole-explorer/issues/336
### Summary
Tracking issue: https://github.com/wormhole-foundation/wormhole-explorer/issues/320
This pull request adds the `24h_messages` field to the response in `GET /api/v1/scorecards`. This field indicates the total number of VAAs emitted in the last 24 hours, including messages from PythNet.
Also, the analytics component has been updated to generate a new metric for this purpose. There is a task that summarizes data every 5 minutes to speed up query times. The data is retained for 24 hours only. For this particular metric, writes to InfluxDB are being batched to improve performance.
The endpoint `GET /api/v1/scorecards` was returning an inaccurate number for the volume metric. This pull request fixes the issue.
Additionally, the response model of `GET /api/v1/top-assets-by-volume` has been changed as specified by @raop155:
```
{
"assets": [
{
"symbol": "USDCet",
"tokenChain": 1,
"tokenAddress": "000000000000000000000000a0b86991c6218b36c1d19d4a2e9eb0ce3606eb48",
"volume": "475173.83806883"
}
]
}
```
### Summary
Tracking issue: https://github.com/wormhole-foundation/wormhole-explorer/issues/275
This pull request implements the endpoint `GET /api/v1/top-chain-pairs-by-num-transfers`, which returns the chain pairs that have the most transfers.Internally, the endpoint uses data summarized daily to speed up query execution times.
This endpoint has a mandatory query parameter named timerange, which must be set to `7d`, `15d` or `30d`.
### Summary
Tracking issue: https://github.com/wormhole-foundation/wormhole-explorer/issues/276
This pull request implements the endpoint `GET /api/v1/top-assets-by-volume`, which returns the assets that have the highest volume. Internally, the endpoint uses data summarized daily to speed up query execution times.
This endpoint has a mandatory query parameter named `timerange`, which must be set to `7d`, `15d` or `30d`.
### Summary
In order to compute volume metrics for each token, we need metadata about the token that is not present in the VAAs (e.g.: decimals, symbol).
That information is statically defined in `common/domain/tokenbridge.go`. This pull request adds the most relevant tokens (i.e.: the tokens that contribute the most volume) to the existing definitions. It is very likely that more token definitions will be added in the future as needed.
The token metadata that was previously defined in the `notional` package (symbols, coingecko IDs) was also moved to `common/domain/tokenbridge.go`.
Tracking issue https://github.com/wormhole-foundation/wormhole-explorer/issues/281
### Summary
This pull request adds volume metrics to influxdb. Also, it adds the 24h volume metric to `GET /api/v1/scorecards`.
Tracking issues: https://github.com/wormhole-foundation/wormhole-explorer/issues/221, https://github.com/wormhole-foundation/wormhole-explorer/issues/280
### Changes:
* The `parser` service no longer generates metrics for influxdb. All metrics-related code was removed from that service, that code was moved to the analytics service instead.
* New volume metrics were added to the analytics service.
* The notional cache was modified to use token names (i.e.: ticker symbols) as keys instead of chain IDs.
* The notional cache reader was moved to the `common/client/cache` package.
* A little bit of duplicated code between the cache reader and writer was removed.
* A 24h volume metric was added to `GET /api/v1/scorecards`.
* A dictionary that stores token metadata was added under `common/domain/tokenbridge.go`. More tokens will be added to it in the near future.
### Summary
The output of `GET /api/v1/vaas` was not being correctly sorted. This pull request changes the behavior so that the output will be sorted by the `timestamp` field.
Additionally, duplicated code was removed in the VAA code that handles queries.
Also, there was an unused query parameter `sortBy` which was being exposed but not used. Hence it was removed.
Tracking issue: https://github.com/wormhole-foundation/wormhole-explorer/issues/233
### Summary
Tracking issue: https://github.com/wormhole-foundation/wormhole-explorer/issues/254
For all chains except Solana, the data in `globalTransactions.sourceTx` was somewhat duplicated, since we already had that data in the `vaas` collection.
This pull request changes the `tx-tracker` service so that no redundant data will be written, and also changes the API to look up the data from the `vaas` collection instead.
The responses from the endpoint `/api/v1/global-tx` will no longer contain the field `originTx.from`.
### Summary
This pull request adds the `GET /api/v1/scorecards` endpoint, which is required by the wormscan frontend.
Most of the fields that this endpoint should return are being omitted because the data is not currently available on the backend. Those fields will be added iteratively as the data becomes available.
The current format of the response is:
```json
{
"total_tx_count": "1300200",
"24h_tx_count": "4200"
}
```
Tracking issue: https://github.com/wormhole-foundation/wormhole-explorer/issues/221
## Deployment details
In order to populate the `"total_tx_count"` metric, a task is needed in influxdb:
```
$ cat total-vaa-count.flux
option task = {
name: "Total number of emitted VAAs",
every: 1m
}
from(bucket: "wormhole-explorer")
|> range(start: 2018-01-01T00:00:00Z)
|> filter(fn: (r) => r._measurement == "vaa_count")
|> group(columns: ["_measurement"])
|> set(key: "_measurement", value: "total_vaa_count")
|> count()
|> map(fn: (r) => ({r with _time: now()}))
|> map(fn: (r) => ({r with _field: "total_vaa_count"}))
|> to(bucket: "wormhole-explorer", org: "xlabs")
```
### Summary
This pull request adds the functionality to search for VAAs using Solana tx hashes, e.g.:
`GET /api/v1/vaas?txHash=2qnNNBQQQ152sGkQ1Rnq2Kdr48LvdN4oXwnTWyEmJqYYWFusLWxbwhpGS5KTXH1bKKx9gg9BLZMKd3g1cXiy7QMz`.
Also, the VAA objects returned by the API now have a new field `nativeTxHash`, which contains the tx hash that generated the VAA in the native format of the emitter chain.
Tracking issues: https://github.com/wormhole-foundation/wormhole-explorer/issues/212, https://github.com/wormhole-foundation/wormhole-explorer/issues/214
### Testing guidelines
Endpoints affected:
* `GET /api/v1/vaas` now supports Solana tx hashes by using the `?txHash=` query parameter. This search is limited to VAAs created by the portal token bridge.
* Some VAAs returned by the API should have a new field `nativeTxHash`. All VAAs related to the portal token bridge should have this field set.
* An internal refactoring affected pretty much all routes under `GET /api/v1/vaas*`, which should be re-tested.
* In the response of `/api/v1/global-tx/`, the field `originTx.signer` has been renamed to `originTx.from`
### Summary
Several endpoints related to VAAs and observations returned `null` when no results were found.
This commit changes the behavior to return `[]` instead, which is more ergonomic for users.
Endpoints affected:
- `GET /api/v1/observations/:chain`
- `GET /api/v1/observations/:chain/:emitter`
- `GET /api/v1/observations/:chain/:emitter/:sequence`
- `GET /api/v1/observations/:chain/:emitter/:sequence/:signer/:hash`
- `GET /api/v1/vaas/{chain_id}`
- `GET /api/v1/vaas/{chain_id}/{emitter}`
- `GET /api/v1/vaas/{chain_id}/{emitter}/{seq}`
Fix governor endpoints
A few governor endpoints were handling addresses incorrectly,
which resulted in errors.
* `GET /api/v1/governor/config/{governor_address}`
* `GET /api/v1/governor/status/{governor_address}`
This commit fixes the issue and adds doc comments to prevent the
same caveat from happening again in the future.
### Summary
This PR enables passing emitter addresses in Solana format when the `chainId` parameter is set to 1. If `chainId` is set to a different value, the API will expect a regular Wormhole address.
Tracking issue: https://github.com/wormhole-foundation/wormhole-explorer/issues/189
### Endpoints affected
- `GET /api/v1/global-tx/{chain_id}/{emitter}/{seq}`
- `GET /api/v1/observations/:chain/:emitter`
- `GET /api/v1/observations/:chain/:emitter/:sequence`
- `GET /api/v1/observations/:chain/:emitter/:sequence/:signer/:hash`
- `GET /api/v1/vaas/{chain_id}/{emitter}`
- `GET /api/v1/vaas/{chain_id}/{emitter}/{seq}`
### Example
These two calls should be equivalent:
- `GET /api/v1/vaas/1/ec7372995d5cc8732397fb0ad35c0121e0eaa90d26f828a534cab54391b3a4f5` (address in Wormhole format)
- `GET /api/v1/vaas/1/Gv1KWf8DT1jKv5pKBmGaTmVszqa56Xn8YGx2Pg7i7qAk` (the same address, in Solana format)
* init contract-watcher
* Add processor and blockain watchers
* Add pagination and save last blocknumber procesed by chain
* Add processing by blocks
* Add contract-watcher deploy manifest
* Add endpoint to get globalTransactions by Id
* Add originTX to get globalTransactionById endpoint
* Add wait time for new blocks
* Add initial block for evm watcher
* Add rate limit for evm watcher
* Handle testnet environment and small fixes
* Update wormhole dependencies
* Fix api documentation for swagger
---------
Co-authored-by: Agustin Pazos <agpazos85@gmail.com>
### Summary
Context: https://github.com/wormhole-foundation/wormhole-explorer/issues/154
This PR modifies all endpoints that receive an emitter/guardian address to accept a wider range of formats.
After this pull request, all of these are equivalent:
* `0x000000000000000000000000f890982f9310df57d00f659cf4fd87e65aded8d7`
* `000000000000000000000000f890982f9310df57d00f659cf4fd87e65aded8d7`
* `0xf890982f9310df57d00f659cf4fd87e65aded8d7`
* `f890982f9310df57d00f659cf4fd87e65aded8d7`
### Testing plan
* Added unit tests for the parsing code.
* Tested manually a few of the affected endpoints.
### Summary
* Swagger: remove `sortOrder` query parameter from several endpoints in which it didn't make sense.
* Swagger: remove query params `page` and `pageSize` from endpoints that return a single object.
* Sort the output of `GET /api/v1/governor/config` by ascending id.
* Refactor: remove duplicated code, format code for readability.
The API's swagger documentation exposes a route
`GET /api/v1/governor/max_available/:chain`,
but instead it should be
`GET /api/v1/governor/notional/max_available/:chain`.
### Summary
Changes to `GET /api/v1/governor/limit`:
* Fix the behavior of the parameters `page` and `pageSize` (which previously were being ignored).
* When no results are found, return an HTTP status code of 200 and set the response to an empty array.
* Remove the `sortOrder` query parameter.
The rationale for removing the `sortOrder` parameter is that sorting chains by ascending/descending chain ID doesn't seem to be a useful operation.
### Summary
In some cases, the query parameters for a MongoDB aggregation pipeline were not being initialized.
This led to errors in `GET /api/v1/vaas/{chain}/{emitter}/{seq}`.
Other endpoints could also have been affected.
Previously, when calling `GET /api/v1/governor/enqueued_vaas`, it returned a status code of 404 when there were no enqueued VAAs.
Now, the endpoint returns a status code of 200, and the response body contains an empty array.
### Summary
* On all endpoints, fix the behavior of the `page` parameter (which was previously being ignored).
* On `GET /api/v1/vaas`, fix the behavior of the `sortBy` parameter (which wasn't working when `parsedPayload=true`).
* For all endpoints, validate the query parameters `page`, `pageNumber` and `sortOrder`.
* Return descriptive errors when pagination-related parameters happen to be invalid (`page`, `pageSize`, `sortOrder`).
* Validate guardian addresses in query params.
### Testing plan
All parameters involved were manually tested.
### Summary
Split `/v1` and `/api/v1` controllers into different packages (respectively `api/routes/guardian` and `api/routes/wormscan`).
Still need to move services/repositories/models according to the new package layout.
### Summary
Updating from swaggo v1.8.9 to v1.8.10 caused issues with the generated swagger file.
The issue was solved by moving the handler for `GET /swagger.json` outside the `/doc` directory.
* add vaaId and txHash relation in vaaIdTxHash collection
* add txHash in vaas collection
* add txHash in response vaa and as filter in find vaas endpoints
* Add pipeline to parse vaa
* Add parser repository and start building pipeline publisher
* Add queue sqs and in-memory to pipeline
* Add ses publisher and queue sqs/memory
* Fix watcher query
* Fix type to marshall/unsmarshall chainID and emitterAddress
* Fix js execution is run many times
* Add comments
* Add comments and improvements
* Change VAA Parser implementation
* Fix and change integration-test vaa-payload-parser
* fix appID mongo field name
* Fix guardianSet version 3, add xlabs to position 8
Co-authored-by: Agustin Pazos <agpazos85@gmail.com>
* Add use of sequence cache in API
* Add sequence cache in fly
* Deploy for API
* Improve use cache in API
* Remove sequence cache in fly for pythnet
Co-authored-by: Fernando Torres <fert1335@gmail.com>
agodnic
gipsh
ftocal
walker-16
Ben Guidarelli
gipsh
Gabriel Zimmermann