Commit Graph

43 Commits

Author SHA1 Message Date
GroovieGermanikus 333df8d3ab
array_dedup_append: implemented faster version
uses nested array remove for merge+deduplication
2024-01-10 20:11:14 +01:00
Groovie | Mango fd99ea1f0a
simplify insertinto queries (#41)
* remove extra subquery

* handle block already exist case

* remove trailing semicolons
2024-01-10 16:56:35 +01:00
Groovie | Mango f5f9c99375
multiple write dbsessions (#40)
* postgres connection handling:

- use dedicated connections
- set work_mem

* serialize writes

* brush send_block_info_to_buffer logging

* remove sleep

* brush log

* cleanu
2024-01-10 16:39:05 +01:00
Groovie | Mango f0727de608
log metrics for postgres write statements (#39)
* add debug logs for query execution

* use tokio instant

* words
2024-01-10 14:13:19 +01:00
godmodegalactus 527fe545c1
resubscribe banking stage errors every 10 mins incase of inactivity 2024-01-09 15:35:15 +01:00
Groovie | Mango 5c3edc836f
add table with latest 1000 txs per acc_id (#38)
* wip

* use fancy postgres method

* cleanup

* fix logs

* format

* simpler function

* Revert "simpler function"

This reverts commit 57120b9ce8.

* fix sql function which did not deduplicate

* fix initial tx array being null
2024-01-09 15:12:03 +01:00
godmodegalactus 77ae2a38a2
Making missing sig into char(88) 2024-01-08 16:14:02 +01:00
godmodegalactus c8c59135f8
Reverting sigature to char(88) and account key to char(44) 2024-01-05 17:08:23 +01:00
Groovie | Mango 7a3480e044
Fix/cleanupdb delete amb (#31)
* clippy+fmt

* cleanup also from amb table
2024-01-05 11:06:47 +01:00
godmodegalactus 31da6d2b5b
Reverting ATLs and changing signature and account key to text 2023-12-24 10:07:49 +01:00
godmodegalactus a47f4254aa
making temp tables back char 88 and char 44 2023-12-22 21:03:17 +01:00
godmodegalactus bf15a3cd14
Avoid deserializing alts all the time 2023-12-22 16:42:46 +01:00
Groovie | Mango c51d8f87b5
Improve/use text insteadof varchar char (#33)
* clippy+fmt

* use varchar for signature+account_address

* cluster amt table

* Postgres: use TEXT everywhere
2023-12-22 15:24:06 +01:00
Groovie | Mango 6309a0048f
clippy+fmt (#29) 2023-12-21 15:11:06 +01:00
Groovie | Mango c74d9ab3e5
Feature/cleanup job (#28)
cleanup job for database
2023-12-21 14:39:18 +01:00
galactus 1076e22523
Adding support for address lookup tables (#26)
* Adding support for address lookup tables

* Updating code for ATLS

* Changes after groovies comments
2023-12-21 11:20:55 +01:00
galactus 2e59fb8148
Adding the unnecessary btree map (#27) 2023-12-19 13:22:10 +01:00
galactus 3449190818
exit process (#24) 2023-12-15 12:26:50 +01:00
galactus e30a2d5935
Adding timeout of 30s to resubscribe to blocks (#22) 2023-12-14 11:53:09 +01:00
galactus 88cfaeb56d
Fixing sidecar (#20)
* Revert "CHecking stream for timeouts. (#18)"

This reverts commit d7969a8087.

* Revert "Restarting on geyser stream error (#17)"

This reverts commit 21ab222099.

* fixing minor error
2023-12-12 11:15:38 +01:00
galactus 49f519df0b
Changing schema and adding accounts data (#10)
* chaning schema

fixing the schema

updating postgres code

Adding transaction slots into postgres

saving account used by transactions

saving block data

* Saving all the block info

completing block save

Optimizing postgres saving of data

fixing schema after groovies review

Schema changes after groovies review

Fixing that is_writable is always true

* adding a new index on slot
2023-12-12 09:58:56 +01:00
galactus 21ab222099
Restarting on geyser stream error (#17) 2023-12-09 19:47:19 +01:00
galactus d5da3bae02
making postgres errors into panics (#16) 2023-12-08 14:47:00 +01:00
galactus 5eae4abec3
optimizing the maps (#13) 2023-12-06 11:20:20 +01:00
galactus 63edab3060
chaning primary key on transaction infos table (#11) 2023-12-05 19:11:18 +01:00
galactus ebf7135005
Panic on postgres disconnection (#6) 2023-12-04 15:29:58 +01:00
galactus fbf2e16988
Adding supp info (#5)
* adding prioritization fees with write lock accounts

* adding supplimentary infos

* moving to multiple geyser endpoints, using copyin instead of insert

* mincor changes

* adding missing column in transaction info

* minor bug fixes and fmt

* restart block subscription on geyser

* resubsribing to banking errors notification on failure
2023-12-01 11:08:27 +00:00
godmodegalactus 3ec0a4372d
Saving accounts used in json format 2023-11-24 13:39:48 +01:00
godmodegalactus c1a61ab0a6
get banking blocks from rpc 2023-11-24 11:41:19 +01:00
godmodegalactus 2e994bd2dc
Some fixes, saving read account, serializing data in json format 2023-11-23 16:23:55 +01:00
GroovieGermanikus 4f21a76966
Merge branch 'main' of github.com:blockworks-foundation/BankingStageErrorsTrackingSidecar 2023-11-23 14:28:51 +01:00
GroovieGermanikus ffd6d5ece8
log 2023-11-23 14:27:51 +01:00
godmodegalactus d8646f65d6
Adding postgres certificates and grpc token 2023-11-23 11:24:16 +01:00
GroovieGermanikus 1b0c4f25d9
add lifeness prometheus probe 2023-11-23 11:16:00 +01:00
Godmode Galactus 77b675a0d7
som minor changes 2023-10-26 13:48:27 +02:00
Godmode Galactus 070c3264ba
moving from custom types to text 2023-10-24 13:30:34 +02:00
Godmode Galactus fd8fa7fe05
making proper types 2023-10-21 18:40:19 +02:00
Godmode Galactus 5136b09c89
adding clap cli 2023-10-20 20:14:33 +02:00
Godmode Galactus 77ff31a547
adding accounts in banking stage notifications 2023-10-20 10:23:26 +02:00
Godmode Galactus 0e151a75ec
renaming timestamp and using changing timestamp type 2023-10-17 15:13:42 +02:00
Godmode Galactus 3cb4a874b9
saving some more transaction data and block_info 2023-10-17 12:11:54 +02:00
godmodegalactus 177e68a514
fixing postgres save 2023-10-02 15:23:32 +02:00
Godmode Galactus 8c029c19f3
Adding postgres 2023-10-02 10:51:38 +02:00