wormhole-explorer/parser
ftocal 7c467f5267
Integrating blockchain watcher data into mongodb (#820)
* Handle log-message-published generated by blockchain-watcher in parser, tx-tracker y analytics

* Add deployment for staging-testnet environment
Improve logs

* Add operation endpoints

---------

Co-authored-by: Agustin Pazos <agpazos85@gmail.com>
2023-11-27 12:31:35 -03:00
..
bin
cmd Integrating blockchain watcher data into mongodb (#820) 2023-11-27 12:31:35 -03:00
config Integrating blockchain watcher data into mongodb (#820) 2023-11-27 12:31:35 -03:00
consumer Integrating blockchain watcher data into mongodb (#820) 2023-11-27 12:31:35 -03:00
http Integrating blockchain watcher data into mongodb (#820) 2023-11-27 12:31:35 -03:00
internal Remove duplicated code: MongoDB connection handling (#590) 2023-08-07 16:05:08 -03:00
migration Add index to parsedVaa collection (#616) 2023-08-09 11:55:51 -03:00
parser Create an api endpoint to call vaa payload parser component (#596) 2023-08-10 11:02:14 -03:00
processor Integrating blockchain watcher data into mongodb (#820) 2023-11-27 12:31:35 -03:00
queue Integrating blockchain watcher data into mongodb (#820) 2023-11-27 12:31:35 -03:00
.env.example
.gitignore
Dockerfile
Makefile
README.md
go.mod Integrating blockchain watcher data into mongodb (#820) 2023-11-27 12:31:35 -03:00
go.sum Integrating blockchain watcher data into mongodb (#820) 2023-11-27 12:31:35 -03:00

README.md

Parser

This component is in charge of parsing the VAA payload and persists it.

VAA parsing is delegated to external service so that users can add custom parsers in the external service without affecting this service.

Usage

Service

parser service

Backfiller

parser backfiller [flags]

Command-line arguments

  • --end-time string maximum VAA timestamp to process (default now)
  • --log-level string log level (default "INFO")
  • --mongo-database string mongo database
  • --mongo-uri string mongo connection
  • --page-size int VAA payload parser timeout (default 100)
  • --start-time string minimum VAA timestamp to process (default "1970-01-01T00:00:00Z")
  • --vaa-payload-parser-timeout int maximum waiting time in call to VAA payload service in second (default 10)
  • --vaa-payload-parser-url string VAA payload parser service URL

Running parser as service with localstack

Here are some aws commands to configure localstack with the necessary resources

Config sns topic

aws --profile localstack --endpoint-url=http://localhost:4566 sns create-topic --name vaas-pipeline.fifo  --attributes FifoTopic=true,ContentBasedDeduplication=false

Config SQS FIFO with dead letter queue localstack

aws --profile localstack --endpoint-url=http://localhost:4566 sqs create-queue --queue-name=wormhole-vaa-parser-dlq-queue.fifo --attributes "FifoQueue=true"
aws --profile localstack --endpoint-url=http://localhost:4566 sqs create-queue --queue-name=wormhole-vaa-parser-queue.fifo --attributes FifoQueue=true,MessageRetentionPeriod=3600,ReceiveMessageWaitTimeSeconds=5,VisibilityTimeout=20,RedrivePolicy="\"{\\\"deadLetterTargetArn\\\":\\\"arn:aws:sqs:us-east-1:000000000000:wormhole-vaa-parser-dlq-queue.fifo\\\",\\\"maxReceiveCount\\\":\\\"2\\\"}\""

Subscribe SQS FIFO to vaas-pipeline.fifo topic

aws --profile localstack --endpoint-url=http://localhost:4566 sns subscribe --topic-arn arn:aws:sns:us-east-1:000000000000:vaas-pipeline.fifo --protocol sqs --notification-endpoint http://localhost:4566/000000000000/wormhole-vaa-parser-queue.fifo

Check message in the dead letter queue localstack

aws --profile localstack --endpoint-url=http://localhost:4566 sqs receive-message --queue-url=http://localhost:4566/000000000000/wormhole-vaa-parser-dlq-queue.fifo