* add 2 new tasks for collecting chain activity every day and hour
* making progress
* change query 2
* add query by month and year
* changes on task
* more changes
* change to 1d
* add 1d
* fix query
* adjust queryies
* change the way the query is executed
* changes on query
* making more progress
* fix per year query
* add a second group of tasks for downsampling
* add app_id
* update swagger docs
* optimize new tasks
* fix W
* fix W
* start using the new measurement
* change endpoint signature
* update endpoint name
* fix indents
* code review changes
* remove unnecessary break
* Add support to scroll in tx-tracker
* Handle duplicated vaa in fly
Co-authored-by: walker-16 <agpazos85@gmail.com>
* Modify tx-tracker and pipeline to support duplicated vaas
Co-authored-by: ftocal <fert1335@gmail.com>
* Add isDuplicated flag in vaas and operations endpoints
Add new endpoint to find duplicated vaas by vaa id
Co-authored-by: walker-16 <agpazos85@gmail.com>
* Add metrics for duplicated vaas
Co-authored-by: walker-16 <agpazos85@gmail.com>
* Update node dependencies in spy
Co-authored-by: walker-16 <agpazos85@gmail.com>
* Fix promethes metrics for missing tx-hash in tx-tracker
Co-authored-by: walker-16 <agpazos85@gmail.com>
---------
Co-authored-by: Agustin Pazos <agpazos85@gmail.com>
* start
add search by chain
change chainId query param handling
change condition
only from chain
change query
add filter by appId
add payload type for operations query
add logs
add log for error in mongodb call
add more logs and recover to find possible panic
change type to float64
add more logs for troubleshooting
add more logs for troubleshooting payloadType query param
add another defer
* change query
* add exclusiveAppId,sourceChain and targetChain
* unify search criteria
* change queryies
* combine query params filters
* change implementation of sourceChain and targetChain
* insert filtering by chain and by appid as stages in aggregation pipeline
* fix appIds matching condition
* move query to a separate pipeline which starts from parsedVAA
* adjust query by appId
* add matching also for standardizedProperties
* change
* try using instead of
* simplify query
* add queryLoggging and remove other parts of the query to troubleshooting
* working
* add index creation
* update swagger docs
* tweak index performance and fix timestamp on parsedVaa collection
* start
add search by chain
change chainId query param handling
change condition
only from chain
change query
add filter by appId
add payload type for operations query
add logs
add log for error in mongodb call
add more logs and recover to find possible panic
change type to float64
add more logs for troubleshooting
add more logs for troubleshooting payloadType query param
add another defer
* change query
* add exclusiveAppId,sourceChain and targetChain
* unify search criteria
* change queryies
* combine query params filters
* change implementation of sourceChain and targetChain
* insert filtering by chain and by appid as stages in aggregation pipeline
* fix appIds matching condition
* move query to a separate pipeline which starts from parsedVAA
* adjust query by appId
* add matching also for standardizedProperties
* change
* try using instead of
* simplify query
* add queryLoggging and remove other parts of the query to troubleshooting
* working
* add index creation
* update swagger docs
* tweak index performance and fix timestamp on parsedVaa collection
* add lookup for globaltransactions
Add cctp and portal_token_bridge stats to protocols-stats endpoint
changes
indent
more changes on the script
new working approach on influx task
indent script
tested insert
add 2 versions of the script
multiple changes
add 1day task
add logic to retrieve internal protocols
remove unecessary code
readd empty script
fix unit-tests and measurement namings
fix queries
fix alignment
rename function task
fix names
improvements on influx task
add .run config to gitignore
add .run to gitignore
fix task and rename
working api
multiple things
Delete .run/wormscan api.run.xml
Delete analytics/scripts/test_query.flux
wip
multiple fixes
fix test
wip
fix queries
fix unit-test due to query changes
* [Issue:1052] Create job for fetching contributor stats and storing in db
revert unnecessary changes on api/handlers/stats
revert changes in go.mod and go.sum
revert change in go.work
add schedule for contributors stats job
change response parsing order
changes due to draft-pr review
move on with contributors activity implementation
change to every hour
fix typo
change contributor stats implementation to do a single write transaction
normalize to UTC contributors activity timestamp
add cronjob schedule for contributors
[Issue:1052][Part 2] Create endpoint to expose contributors stats and activities (#1123)
* add endpoint for retrieving stats and activity
* remove model.go file and move types to service file
* add unit tests to contributors service
* integrate new contributors controller
* fix more stuff
fix unit-tests
changes due to pr review
fix query
fix unit-tests
fix total_value_secure
move constantes to common pkg
remove extra changes
rename contributor to protocols
finish renames
Changes for deployment
adjust different response types from different protocols contributors
fix controller test
big refactor in activty job and stats job since protocols are returning different formats
api responding fine
remove uneccessary generics
target dbconsts
fix
Delete deploy/common/env/staging-mainnet.env
undo unwanted changes
readd staging-mainnet.env
fix unit-tests
add missing protocols_stats/activity_version
remove property protocols_json
fix JOB_ID env var in protocols-activity.yaml
fix typos in env vars configs
change tu numbers
changes due to own review
add new line
* add swagger docs
* Change query to sort by timestamp and id
* Add maximun size to pageSize in operation endpoint
* Add missing indexes for mongodb
* improve search txHash for nested wormchain txHash
---------
Co-authored-by: Fernando Torres <fert1335@gmail.com>
### Description
This pull request adds the parameter `toChain` to the endpoint `GET /api/v1/vaas/{emitterChain}/{emitterAddress}`.
Other VAA-related endpoints do not support this parameter.
Additionally, for performance reasons, a composite index must be created in MongoDB: `db.parsedVaa.createIndex({"emitterChain": -1, "emitterAddr": -1, "rawStandardizedProperties.toChain": -1, "indexedAt": -1})`
### Description
Before this pull request, when calling `GET /api/v1/vaas?txHash={h}` for a hash that has multiple VAAs associated with it, we were just returning one. After this pull request, the API should return all of the VAAs associated with that transaction hash.
Tracking issue: https://github.com/wormhole-foundation/wormhole-explorer/issues/563
### Summary
This pull request modifies the API service to expose two endpoints for the Wormhole Scan UI:
* `GET /api/v1/transactions`: data needed to render the transactions list page.
* `GET /api/v1/transactions/{chain}/{id}/{sequence}`: data needed to render the transaction detail page.
### Description
Tracking issue: https://github.com/wormhole-foundation/wormhole-explorer/issues/451.
The Wormhole Scan UI needs to provide a link to the emitter address for each VAA.
This pull request adds a field containing decoded emitter addresses in the following endpoints:
* `GET /api/v1/transactions`: field `emitterNativeAddress`
* `GET /api/v1/vaas*`: field `emiterNativeAddr`
* Replace start_time and end_time query parameters with fixed ranges at the endpoint of the chain activity
Add InfluxDB tasks for enpdoint chain-activity for each timeSpan
Add cache for chain-activity endpoint
* Code review updates
### Description
This pull request adds support for Aptos in the `tx-tracker` service.
This will lead to improvements for the wormhole Scan UI:
1. The transaction hash that was being previously displayed for Aptos VAAs was incorrect. This pull request fixes the issue.
2. The sender addresses for Aptos VAAs will now become available for for the UI.
### Summary
Tracking issue: https://github.com/wormhole-foundation/wormhole-explorer/issues/404
The WormholeScan UI needs to display the sender address for each token bridge VAA. This pull request modifies the `tx-tracker` service to obtain that information for Solana and eight EVM chains.
The transaction sender will become accessible through the following endpoints:
* `GET /api/v1/global-tx/{chain}/{emitter}/{seq}`: field `originTx.from`.
* `GET /api/v1/transactions`: field `originAddress`.
In both cases, the field is nullable (i.e.: sometimes it may not be available due to eventual consistency or internal errors)
### Summary
Tracking issue: https://github.com/wormhole-foundation/wormhole-explorer/issues/385
This pull request implements a new endpoint, `GET /api/v1/transactions`, which will be consumed by the wormhole explorer UI.
The endpoint returns a paginated list of transactions, in which each element contains a brief overview of the transaction (ID, txHash, status, etc.).
It exposes offset-based pagination via the parameters `page` and `pageSize`. Also, results can be obtained for a specific address by using the `address` query parameter.
The response model looks like this:
```json
{
"transactions": [
{
"id": "1/5ec18c34b47c63d17ab43b07b9b2319ea5ee2d163bce2e467000174e238c8e7f/12965",
"timestamp": "2023-06-08T19:30:19Z",
"txHash": "a302c4ab2d6b9a6003951d2e91f8fdbb83cfa20f6ffb588b95ef0290aab37066",
"originChain": 1,
"status": "ongoing"
},
{
"id": "22/0000000000000000000000000000000000000000000000000000000000000001/18308",
"timestamp": "2023-06-08T19:17:14Z",
"txHash": "00000000000000000000000000000000000000000000000000000000000047e7",
"originChain": 22,
"destinationAddress": "0x00000000000000000000000067e8a40816a983fbe3294aaebd0cc2391815b86b",
"destinationChain": 5,
"tokenAmount": "0.12",
"usdAmount": "0.12012",
"symbol": "USDC",
"status": "completed"
},
...
]
}
```
### Limitations of the current implementation
1. Doesn't return the total number of results (this may result in a performance issue when we filter by address)
2. Can only filter by receiver address (we don't have sender information in the database yet)
### Summary
This pull request fixes an issue in which all observation-related endpoints were returning results sorted by `id` (https://github.com/wormhole-foundation/wormhole-explorer/issues/358). The expected behavior is to sort results based on descending timestamp order instead.
Also, swagger docs were updated to document this behavior.
### Summary
Tracking issue: https://github.com/wormhole-foundation/wormhole-explorer/issues/344
Before this pull request, there were two separate token databases (one being used by the InfluxDB backfiller, and another one in the `common/` module being used by the analytics service).
Having two different token databases resulted in inconsistencies, due to each of these databases containing different tokens.
This PR unifies those two databases into a single one, under the `common/` module.
### Summary
Tracking issue: https://github.com/wormhole-foundation/wormhole-explorer/issues/320
This pull request adds the `24h_messages` field to the response in `GET /api/v1/scorecards`. This field indicates the total number of VAAs emitted in the last 24 hours, including messages from PythNet.
Also, the analytics component has been updated to generate a new metric for this purpose. There is a task that summarizes data every 5 minutes to speed up query times. The data is retained for 24 hours only. For this particular metric, writes to InfluxDB are being batched to improve performance.
The endpoint `GET /api/v1/scorecards` was returning an inaccurate number for the volume metric. This pull request fixes the issue.
Additionally, the response model of `GET /api/v1/top-assets-by-volume` has been changed as specified by @raop155:
```
{
"assets": [
{
"symbol": "USDCet",
"tokenChain": 1,
"tokenAddress": "000000000000000000000000a0b86991c6218b36c1d19d4a2e9eb0ce3606eb48",
"volume": "475173.83806883"
}
]
}
```
### Summary
Tracking issue: https://github.com/wormhole-foundation/wormhole-explorer/issues/275
This pull request implements the endpoint `GET /api/v1/top-chain-pairs-by-num-transfers`, which returns the chain pairs that have the most transfers.Internally, the endpoint uses data summarized daily to speed up query execution times.
This endpoint has a mandatory query parameter named timerange, which must be set to `7d`, `15d` or `30d`.
### Summary
Tracking issue: https://github.com/wormhole-foundation/wormhole-explorer/issues/276
This pull request implements the endpoint `GET /api/v1/top-assets-by-volume`, which returns the assets that have the highest volume. Internally, the endpoint uses data summarized daily to speed up query execution times.
This endpoint has a mandatory query parameter named `timerange`, which must be set to `7d`, `15d` or `30d`.
### Summary
This pull request adds volume metrics to influxdb. Also, it adds the 24h volume metric to `GET /api/v1/scorecards`.
Tracking issues: https://github.com/wormhole-foundation/wormhole-explorer/issues/221, https://github.com/wormhole-foundation/wormhole-explorer/issues/280
### Changes:
* The `parser` service no longer generates metrics for influxdb. All metrics-related code was removed from that service, that code was moved to the analytics service instead.
* New volume metrics were added to the analytics service.
* The notional cache was modified to use token names (i.e.: ticker symbols) as keys instead of chain IDs.
* The notional cache reader was moved to the `common/client/cache` package.
* A little bit of duplicated code between the cache reader and writer was removed.
* A 24h volume metric was added to `GET /api/v1/scorecards`.
* A dictionary that stores token metadata was added under `common/domain/tokenbridge.go`. More tokens will be added to it in the near future.
Mariano
ftocal
walker-16
Ben Guidarelli
gipsh
agodnic
gipsh