Commit Graph

103 Commits

Author SHA1 Message Date
GroovieGermanikus 333df8d3ab
array_dedup_append: implemented faster version
uses nested array remove for merge+deduplication
2024-01-10 20:11:14 +01:00
GroovieGermanikus 82e679af8c
rework array_dedup_append function 2024-01-10 17:47:26 +01:00
Groovie | Mango fd99ea1f0a
simplify insertinto queries (#41)
* remove extra subquery

* handle block already exist case

* remove trailing semicolons
2024-01-10 16:56:35 +01:00
Groovie | Mango f5f9c99375
multiple write dbsessions (#40)
* postgres connection handling:

- use dedicated connections
- set work_mem

* serialize writes

* brush send_block_info_to_buffer logging

* remove sleep

* brush log

* cleanu
2024-01-10 16:39:05 +01:00
Groovie | Mango f0727de608
log metrics for postgres write statements (#39)
* add debug logs for query execution

* use tokio instant

* words
2024-01-10 14:13:19 +01:00
godmodegalactus 527fe545c1
resubscribe banking stage errors every 10 mins incase of inactivity 2024-01-09 15:35:15 +01:00
Groovie | Mango 5c3edc836f
add table with latest 1000 txs per acc_id (#38)
* wip

* use fancy postgres method

* cleanup

* fix logs

* format

* simpler function

* Revert "simpler function"

This reverts commit 57120b9ce8.

* fix sql function which did not deduplicate

* fix initial tx array being null
2024-01-09 15:12:03 +01:00
godmodegalactus 77ae2a38a2
Making missing sig into char(88) 2024-01-08 16:14:02 +01:00
godmodegalactus c8c59135f8
Reverting sigature to char(88) and account key to char(44) 2024-01-05 17:08:23 +01:00
Groovie | Mango 050bbbbb76
Fix/fly cleanup2 (#36)
* clippy+fmt

* add run-service-and-cleanup.sh to docker
2024-01-05 11:16:27 +01:00
Groovie | Mango f877a3a0a9
run cleanupjob on fly (#35)
* clippy+fmt

* poor-man's integration of cleanup job

- runs every 5 hours
- deletes all but 2mn slots

* change slots back
2024-01-05 11:12:44 +01:00
Groovie | Mango 7a3480e044
Fix/cleanupdb delete amb (#31)
* clippy+fmt

* cleanup also from amb table
2024-01-05 11:06:47 +01:00
galactus 6b0e2d269f
Loading all ALTS in bulk, saving stats of priority fees by cus (#34)
* Loading all ALTS in bulk, saving stats of priority fees by cus

* cargo fmt
2024-01-03 11:43:55 +01:00
godmodegalactus 3a783316b3
Copying alts from build and using it in docker file 2023-12-26 22:25:24 +01:00
godmodegalactus c2e5ccae34
Update docker file with location of alts 2023-12-26 17:56:17 +01:00
godmodegalactus 739c8d2a67
Preloading ALTs from a predetermined list 2023-12-26 14:40:33 +01:00
godmodegalactus ff0d1ebc8c
disabling alts 2023-12-24 10:31:45 +01:00
godmodegalactus af07d199dd
Loading all alts before starting 2023-12-24 10:21:53 +01:00
godmodegalactus 31da6d2b5b
Reverting ATLs and changing signature and account key to text 2023-12-24 10:07:49 +01:00
godmodegalactus 921dce82bd
Changing back signature type to char(88) and account_key to char(44) 2023-12-22 21:42:48 +01:00
godmodegalactus a47f4254aa
making temp tables back char 88 and char 44 2023-12-22 21:03:17 +01:00
godmodegalactus c79801ebeb
disable alts 2023-12-22 18:00:20 +01:00
godmodegalactus bf15a3cd14
Avoid deserializing alts all the time 2023-12-22 16:42:46 +01:00
Groovie | Mango c51d8f87b5
Improve/use text insteadof varchar char (#33)
* clippy+fmt

* use varchar for signature+account_address

* cluster amt table

* Postgres: use TEXT everywhere
2023-12-22 15:24:06 +01:00
godmodegalactus 481c1e2d06
Updating prometheus counter name 2023-12-22 15:02:29 +01:00
godmodegalactus dda7329282
Adding ALTs counter 2023-12-22 15:01:06 +01:00
Groovie | Mango 04b1a061e2
Feature/cleanupdb binary fly (#30)
* clippy+fmt

* add cleanupdb to Dockerfile
2023-12-21 16:37:38 +01:00
Groovie | Mango 6309a0048f
clippy+fmt (#29) 2023-12-21 15:11:06 +01:00
Groovie | Mango c74d9ab3e5
Feature/cleanup job (#28)
cleanup job for database
2023-12-21 14:39:18 +01:00
galactus 1076e22523
Adding support for address lookup tables (#26)
* Adding support for address lookup tables

* Updating code for ATLS

* Changes after groovies comments
2023-12-21 11:20:55 +01:00
galactus 2e59fb8148
Adding the unnecessary btree map (#27) 2023-12-19 13:22:10 +01:00
godmodegalactus 7a0e8a6084
Revert "Banking stage subscribe also to have slots notifications (#25)"
This reverts commit a11b235010.
2023-12-15 21:51:54 +01:00
galactus a11b235010
Banking stage subscribe also to have slots notifications (#25) 2023-12-15 19:34:36 +01:00
galactus 3449190818
exit process (#24) 2023-12-15 12:26:50 +01:00
galactus 7015110f90
Restarting sidecar if blocktasks reach too high (#23) 2023-12-15 12:19:40 +01:00
galactus e30a2d5935
Adding timeout of 30s to resubscribe to blocks (#22) 2023-12-14 11:53:09 +01:00
Groovie | Mango a862c18da5
add index transaction_slot(slot) (#21) 2023-12-13 14:12:02 +01:00
Groovie | Mango 30a640434a
fix migration (#19)
- add missing schema prefix
- add missing index on blockhash
2023-12-13 10:01:42 +01:00
galactus 88cfaeb56d
Fixing sidecar (#20)
* Revert "CHecking stream for timeouts. (#18)"

This reverts commit d7969a8087.

* Revert "Restarting on geyser stream error (#17)"

This reverts commit 21ab222099.

* fixing minor error
2023-12-12 11:15:38 +01:00
galactus 49f519df0b
Changing schema and adding accounts data (#10)
* chaning schema

fixing the schema

updating postgres code

Adding transaction slots into postgres

saving account used by transactions

saving block data

* Saving all the block info

completing block save

Optimizing postgres saving of data

fixing schema after groovies review

Schema changes after groovies review

Fixing that is_writable is always true

* adding a new index on slot
2023-12-12 09:58:56 +01:00
galactus d7969a8087
CHecking stream for timeouts. (#18) 2023-12-09 21:50:02 +01:00
galactus 21ab222099
Restarting on geyser stream error (#17) 2023-12-09 19:47:19 +01:00
galactus d5da3bae02
making postgres errors into panics (#16) 2023-12-08 14:47:00 +01:00
galactus c59f849e10
Adding grpc x token (#15) 2023-12-06 15:37:37 +00:00
galactus c0d16a416b
avoid updating transaction infos on all slots (#14) 2023-12-06 11:57:36 +01:00
galactus 5eae4abec3
optimizing the maps (#13) 2023-12-06 11:20:20 +01:00
galactus f6608e5c1c
filtering transactions without status (#12) 2023-12-05 22:17:46 +01:00
galactus 63edab3060
chaning primary key on transaction infos table (#11) 2023-12-05 19:11:18 +01:00
galactus 80283c5d48
Making the blocks parsing optional, so that triton can launch side car without blocks parsing (#9) 2023-12-05 15:42:58 +01:00
galactus bae8b36465
removing verbose transactions (#8) 2023-12-04 21:49:36 +01:00