wormhole-explorer/parser
ftocal adb7231074
Add processed message metrics in pipeline components (#1239)
Co-authored-by: walker-16 <agpazos85@gmail.com>
2024-03-25 15:21:10 -03:00
..
bin Update parser Makefile (#501) 2023-07-04 17:48:47 -03:00
cmd Add backfiller for parser with more parameters (#1197) 2024-03-19 12:28:10 -03:00
config Add backfiller for parser with more parameters (#1197) 2024-03-19 12:28:10 -03:00
consumer Add processed message metrics in pipeline components (#1239) 2024-03-25 15:21:10 -03:00
http Integrating blockchain watcher data into mongodb (#820) 2023-11-27 12:31:35 -03:00
internal Add processed message metrics in pipeline components (#1239) 2024-03-25 15:21:10 -03:00
migration Add index to parsedVaa collection (#616) 2023-08-09 11:55:51 -03:00
parser Create an api endpoint to call vaa payload parser component (#596) 2023-08-10 11:02:14 -03:00
processor Add supported tokens for testnet (#810) 2023-11-28 10:16:40 -03:00
queue Add processed message metrics in pipeline components (#1239) 2024-03-25 15:21:10 -03:00
.env.example Feature/add parser pipeline (#46) 2023-01-18 12:42:14 -03:00
.gitignore Update parser Makefile (#501) 2023-07-04 17:48:47 -03:00
Dockerfile [API/PARSER] Chain activity and last transaction endpoints (#175) 2023-03-07 16:25:42 -03:00
Makefile Update parser Makefile (#501) 2023-07-04 17:48:47 -03:00
README.md Add backfiller command to process vaa saved in database (#349) 2023-05-24 16:52:22 -03:00
go.mod Changes to support ci/cd workflow in staging environment (#1166) 2024-02-29 11:14:31 -03:00
go.sum Changes to support ci/cd workflow in staging environment (#1166) 2024-02-29 11:14:31 -03:00

README.md

Parser

This component is in charge of parsing the VAA payload and persists it.

VAA parsing is delegated to external service so that users can add custom parsers in the external service without affecting this service.

Usage

Service

parser service

Backfiller

parser backfiller [flags]

Command-line arguments

  • --end-time string maximum VAA timestamp to process (default now)
  • --log-level string log level (default "INFO")
  • --mongo-database string mongo database
  • --mongo-uri string mongo connection
  • --page-size int VAA payload parser timeout (default 100)
  • --start-time string minimum VAA timestamp to process (default "1970-01-01T00:00:00Z")
  • --vaa-payload-parser-timeout int maximum waiting time in call to VAA payload service in second (default 10)
  • --vaa-payload-parser-url string VAA payload parser service URL

Running parser as service with localstack

Here are some aws commands to configure localstack with the necessary resources

Config sns topic

aws --profile localstack --endpoint-url=http://localhost:4566 sns create-topic --name vaas-pipeline.fifo  --attributes FifoTopic=true,ContentBasedDeduplication=false

Config SQS FIFO with dead letter queue localstack

aws --profile localstack --endpoint-url=http://localhost:4566 sqs create-queue --queue-name=wormhole-vaa-parser-dlq-queue.fifo --attributes "FifoQueue=true"
aws --profile localstack --endpoint-url=http://localhost:4566 sqs create-queue --queue-name=wormhole-vaa-parser-queue.fifo --attributes FifoQueue=true,MessageRetentionPeriod=3600,ReceiveMessageWaitTimeSeconds=5,VisibilityTimeout=20,RedrivePolicy="\"{\\\"deadLetterTargetArn\\\":\\\"arn:aws:sqs:us-east-1:000000000000:wormhole-vaa-parser-dlq-queue.fifo\\\",\\\"maxReceiveCount\\\":\\\"2\\\"}\""

Subscribe SQS FIFO to vaas-pipeline.fifo topic

aws --profile localstack --endpoint-url=http://localhost:4566 sns subscribe --topic-arn arn:aws:sns:us-east-1:000000000000:vaas-pipeline.fifo --protocol sqs --notification-endpoint http://localhost:4566/000000000000/wormhole-vaa-parser-queue.fifo

Check message in the dead letter queue localstack

aws --profile localstack --endpoint-url=http://localhost:4566 sqs receive-message --queue-url=http://localhost:4566/000000000000/wormhole-vaa-parser-dlq-queue.fifo