Commit Graph

117 Commits

Author SHA1 Message Date
Groovie | Mango a981ff4b2b
build amt_latest tx array with latest at leftmost position (#51) 2024-01-31 10:39:29 +01:00
godmodegalactus 65b27ce930
Adding metrics for gSFA old and new 2024-01-29 18:19:22 +01:00
godmodegalactus 797f638f44
Adding metrics to save block info 2024-01-29 15:33:05 +01:00
godmodegalactus a9aebba6b5
Fixing build issues 2024-01-29 15:11:16 +01:00
godmodegalactus 343bb70428
Fix minor bug 2024-01-29 15:09:03 +01:00
godmodegalactus f0e37643c5
Adding more metrics, and having multiple postgres sessions 2024-01-29 14:51:21 +01:00
godmodegalactus 681b042ba1
Optimize account saving 2024-01-29 14:14:07 +01:00
godmodegalactus e8cebe89fd
Increasing limit for number of blocks that could be saved 2024-01-29 12:45:13 +01:00
godmodegalactus 92bf5f84d2
Limiting the number of heavy accounts in the block 2024-01-29 12:03:28 +01:00
godmodegalactus 295277fbf0
Reverting adding transaction data 2024-01-29 11:54:33 +01:00
godmodegalactus 8b37b3e2d9
Adding more metrics 2024-01-29 11:29:16 +01:00
galactus 829cc7952a
Alt processing 2024 01 26 (#50)
* Banking stage sidecar deadlock (#48)

* Properly loading the ALTs

* Adding a timeout for ALTs task list

* More optimization for ALTs

* Adding timeout for getMA

* Avoid RwLocks because they cause deadlocks

* Fixing the deadlock

* cargo fmt and minor changes

* Fixing up metrics

* minor changes

* Solving issue with accounts metric

* Avoid waiting alts while processing block

* Fixing the metrics, minor optimizations

* fix double insertion of transaction accounts
2024-01-29 10:54:11 +01:00
Groovie | Mango 566a107145
removed FK contraint on error_code (#49) 2024-01-27 09:35:10 +01:00
godmodegalactus 64d607155c
Creating seperate task to save transaction accounts 2024-01-22 17:48:34 +01:00
Groovie | Mango abbf805726
reduce LIMIT_LATEST_TXS_PER_ACCOUNT to 100 (#43) 2024-01-11 18:15:38 +01:00
Groovie | Mango fbfef4db06
slow sqlfunction (#42)
* rework array_dedup_append function

* array_dedup_append: implemented faster version

uses nested array remove for merge+deduplication
2024-01-11 17:39:44 +01:00
Groovie | Mango fd99ea1f0a
simplify insertinto queries (#41)
* remove extra subquery

* handle block already exist case

* remove trailing semicolons
2024-01-10 16:56:35 +01:00
Groovie | Mango f5f9c99375
multiple write dbsessions (#40)
* postgres connection handling:

- use dedicated connections
- set work_mem

* serialize writes

* brush send_block_info_to_buffer logging

* remove sleep

* brush log

* cleanu
2024-01-10 16:39:05 +01:00
Groovie | Mango f0727de608
log metrics for postgres write statements (#39)
* add debug logs for query execution

* use tokio instant

* words
2024-01-10 14:13:19 +01:00
godmodegalactus 527fe545c1
resubscribe banking stage errors every 10 mins incase of inactivity 2024-01-09 15:35:15 +01:00
Groovie | Mango 5c3edc836f
add table with latest 1000 txs per acc_id (#38)
* wip

* use fancy postgres method

* cleanup

* fix logs

* format

* simpler function

* Revert "simpler function"

This reverts commit 57120b9ce8.

* fix sql function which did not deduplicate

* fix initial tx array being null
2024-01-09 15:12:03 +01:00
godmodegalactus 77ae2a38a2
Making missing sig into char(88) 2024-01-08 16:14:02 +01:00
godmodegalactus c8c59135f8
Reverting sigature to char(88) and account key to char(44) 2024-01-05 17:08:23 +01:00
Groovie | Mango 050bbbbb76
Fix/fly cleanup2 (#36)
* clippy+fmt

* add run-service-and-cleanup.sh to docker
2024-01-05 11:16:27 +01:00
Groovie | Mango f877a3a0a9
run cleanupjob on fly (#35)
* clippy+fmt

* poor-man's integration of cleanup job

- runs every 5 hours
- deletes all but 2mn slots

* change slots back
2024-01-05 11:12:44 +01:00
Groovie | Mango 7a3480e044
Fix/cleanupdb delete amb (#31)
* clippy+fmt

* cleanup also from amb table
2024-01-05 11:06:47 +01:00
galactus 6b0e2d269f
Loading all ALTS in bulk, saving stats of priority fees by cus (#34)
* Loading all ALTS in bulk, saving stats of priority fees by cus

* cargo fmt
2024-01-03 11:43:55 +01:00
godmodegalactus 3a783316b3
Copying alts from build and using it in docker file 2023-12-26 22:25:24 +01:00
godmodegalactus c2e5ccae34
Update docker file with location of alts 2023-12-26 17:56:17 +01:00
godmodegalactus 739c8d2a67
Preloading ALTs from a predetermined list 2023-12-26 14:40:33 +01:00
godmodegalactus ff0d1ebc8c
disabling alts 2023-12-24 10:31:45 +01:00
godmodegalactus af07d199dd
Loading all alts before starting 2023-12-24 10:21:53 +01:00
godmodegalactus 31da6d2b5b
Reverting ATLs and changing signature and account key to text 2023-12-24 10:07:49 +01:00
godmodegalactus 921dce82bd
Changing back signature type to char(88) and account_key to char(44) 2023-12-22 21:42:48 +01:00
godmodegalactus a47f4254aa
making temp tables back char 88 and char 44 2023-12-22 21:03:17 +01:00
godmodegalactus c79801ebeb
disable alts 2023-12-22 18:00:20 +01:00
godmodegalactus bf15a3cd14
Avoid deserializing alts all the time 2023-12-22 16:42:46 +01:00
Groovie | Mango c51d8f87b5
Improve/use text insteadof varchar char (#33)
* clippy+fmt

* use varchar for signature+account_address

* cluster amt table

* Postgres: use TEXT everywhere
2023-12-22 15:24:06 +01:00
godmodegalactus 481c1e2d06
Updating prometheus counter name 2023-12-22 15:02:29 +01:00
godmodegalactus dda7329282
Adding ALTs counter 2023-12-22 15:01:06 +01:00
Groovie | Mango 04b1a061e2
Feature/cleanupdb binary fly (#30)
* clippy+fmt

* add cleanupdb to Dockerfile
2023-12-21 16:37:38 +01:00
Groovie | Mango 6309a0048f
clippy+fmt (#29) 2023-12-21 15:11:06 +01:00
Groovie | Mango c74d9ab3e5
Feature/cleanup job (#28)
cleanup job for database
2023-12-21 14:39:18 +01:00
galactus 1076e22523
Adding support for address lookup tables (#26)
* Adding support for address lookup tables

* Updating code for ATLS

* Changes after groovies comments
2023-12-21 11:20:55 +01:00
galactus 2e59fb8148
Adding the unnecessary btree map (#27) 2023-12-19 13:22:10 +01:00
godmodegalactus 7a0e8a6084
Revert "Banking stage subscribe also to have slots notifications (#25)"
This reverts commit a11b235010.
2023-12-15 21:51:54 +01:00
galactus a11b235010
Banking stage subscribe also to have slots notifications (#25) 2023-12-15 19:34:36 +01:00
galactus 3449190818
exit process (#24) 2023-12-15 12:26:50 +01:00
galactus 7015110f90
Restarting sidecar if blocktasks reach too high (#23) 2023-12-15 12:19:40 +01:00
galactus e30a2d5935
Adding timeout of 30s to resubscribe to blocks (#22) 2023-12-14 11:53:09 +01:00