4238126f22
microbatch
2024-02-28 12:57:32 +01:00
96814a835a
TEST microbatch
2024-02-28 09:37:00 +01:00
1aea32786c
Revert "use one linear stream"
...
This reverts commit cf5a6717d2
2024-02-28 09:11:37 +01:00
cf5a6717d2
use one linear stream
2024-02-28 08:30:07 +01:00
c46f557a49
update table magic
2024-02-28 08:25:16 +01:00
25137f95cf
ddl for tablespace
2024-02-27 21:45:44 +01:00
250ccfab74
allow multiple tablespaces
2024-02-27 21:45:28 +01:00
9e4be4b691
fix SET
2024-02-27 21:28:33 +01:00
c9a95a1d25
fix compile
2024-02-27 21:08:30 +01:00
4bee01aa82
use mango_tempspace
2024-02-27 21:01:35 +01:00
36845d7c81
Revert "CAUTION: disable synchronous commit"
...
This reverts commit 4e0a8211ac
2024-02-27 20:48:09 +01:00
4e0a8211ac
CAUTION: disable synchronous commit
2024-02-27 16:02:49 +01:00
45d72fa10c
enable experimental deployment tag
2024-02-27 16:02:28 +01:00
1515ee2119
fix account_key vs key on insert ( #61 )
2024-02-24 19:08:41 +01:00
bab25959f6
postgres schema fixes after moving to self-operated PostgreSQL ( #57 )
...
* remove indices from migration.sql which here never created
* BREAKING: make transaction_id the primary key in transactions lookup table
* use varchar in postgres.rs
* add "NOT NULL" constraint wherever possible
* fix small accounts lookup table naming
* fly app for migration
see https://github.com/blockworks-foundation/BankingStageErrorsTrackingSidecar/issues/56
* autovacuum settings
* databse init script
* explain query_user role
* tune toast_tuple_target and comment on page size
* minor fixes
* dba queries
* jot down the fly machine config
* restore create schema
* tweak storage param for postgres 15
2024-02-24 18:29:52 +01:00
50dadc0071
fix percentiles missing if values are dense ( #60 )
2024-02-23 10:33:47 +01:00
2c51ea8abf
issue #58 : fix INSERT INTO deadlock by ordering data ( #59 )
...
* issue #58 : fix INSERT INTO deadlock by ordering data
* order transactions store in INSERT INTO batch to fix https://github.com/blockworks-foundation/BankingStageErrorsTrackingSidecar/issues/58
* log tx info batch size
* workaround for cargo-chef MSRV issue
2024-02-12 17:10:40 +01:00
9ca22c9a46
reduce LIMIT_LATEST_TXS_PER_ACCOUNT ( #54 )
2024-02-03 09:19:10 +01:00
99427a9253
Revert "partition accounts_map_transaction_latest table ( #47 )"
...
This reverts commit 7a577a43fa
2024-02-01 17:56:20 +01:00
a065914e62
Revert "build amt_latest tx array with latest at leftmost position ( #51 )"
...
This reverts commit a981ff4b2b
2024-02-01 17:55:38 +01:00
6383a8a24e
Revert "clear postgres session and use hardcoded temptable names ( #52 )" ( #53 )
...
This reverts commit 7afa8fc45f
2024-02-01 09:44:23 +01:00
7afa8fc45f
clear postgres session and use hardcoded temptable names ( #52 )
...
* clear pg session, redesign insert_accounts_for_transaction
* redesign insert_transactions_for_block
* redesign save_account_usage_in_block
* redesign create_transaction_ids
* redesign create_accounts_for_transaction
* narrow resources cleared in clear_session
2024-02-01 08:07:19 +01:00
7a577a43fa
partition accounts_map_transaction_latest table ( #47 )
2024-01-31 10:49:16 +01:00
a981ff4b2b
build amt_latest tx array with latest at leftmost position ( #51 )
2024-01-31 10:39:29 +01:00
65b27ce930
Adding metrics for gSFA old and new
2024-01-29 18:19:22 +01:00
797f638f44
Adding metrics to save block info
2024-01-29 15:33:05 +01:00
a9aebba6b5
Fixing build issues
2024-01-29 15:11:16 +01:00
343bb70428
Fix minor bug
2024-01-29 15:09:03 +01:00
f0e37643c5
Adding more metrics, and having multiple postgres sessions
2024-01-29 14:51:21 +01:00
681b042ba1
Optimize account saving
2024-01-29 14:14:07 +01:00
e8cebe89fd
Increasing limit for number of blocks that could be saved
2024-01-29 12:45:13 +01:00
92bf5f84d2
Limiting the number of heavy accounts in the block
2024-01-29 12:03:28 +01:00
295277fbf0
Reverting adding transaction data
2024-01-29 11:54:33 +01:00
8b37b3e2d9
Adding more metrics
2024-01-29 11:29:16 +01:00
829cc7952a
Alt processing 2024 01 26 ( #50 )
...
* Banking stage sidecar deadlock (#48 )
* Properly loading the ALTs
* Adding a timeout for ALTs task list
* More optimization for ALTs
* Adding timeout for getMA
* Avoid RwLocks because they cause deadlocks
* Fixing the deadlock
* cargo fmt and minor changes
* Fixing up metrics
* minor changes
* Solving issue with accounts metric
* Avoid waiting alts while processing block
* Fixing the metrics, minor optimizations
* fix double insertion of transaction accounts
2024-01-29 10:54:11 +01:00
566a107145
removed FK contraint on error_code ( #49 )
2024-01-27 09:35:10 +01:00
64d607155c
Creating seperate task to save transaction accounts
2024-01-22 17:48:34 +01:00
abbf805726
reduce LIMIT_LATEST_TXS_PER_ACCOUNT to 100 ( #43 )
2024-01-11 18:15:38 +01:00
fbfef4db06
slow sqlfunction ( #42 )
...
* rework array_dedup_append function
* array_dedup_append: implemented faster version
uses nested array remove for merge+deduplication
2024-01-11 17:39:44 +01:00
fd99ea1f0a
simplify insertinto queries ( #41 )
...
* remove extra subquery
* handle block already exist case
* remove trailing semicolons
2024-01-10 16:56:35 +01:00
f5f9c99375
multiple write dbsessions ( #40 )
...
* postgres connection handling:
- use dedicated connections
- set work_mem
* serialize writes
* brush send_block_info_to_buffer logging
* remove sleep
* brush log
* cleanu
2024-01-10 16:39:05 +01:00
f0727de608
log metrics for postgres write statements ( #39 )
...
* add debug logs for query execution
* use tokio instant
* words
2024-01-10 14:13:19 +01:00
527fe545c1
resubscribe banking stage errors every 10 mins incase of inactivity
2024-01-09 15:35:15 +01:00
5c3edc836f
add table with latest 1000 txs per acc_id ( #38 )
...
* wip
* use fancy postgres method
* cleanup
* fix logs
* format
* simpler function
* Revert "simpler function"
This reverts commit 57120b9ce8
2024-01-09 15:12:03 +01:00
77ae2a38a2
Making missing sig into char(88)
2024-01-08 16:14:02 +01:00
c8c59135f8
Reverting sigature to char(88) and account key to char(44)
2024-01-05 17:08:23 +01:00
050bbbbb76
Fix/fly cleanup2 ( #36 )
...
* clippy+fmt
* add run-service-and-cleanup.sh to docker
2024-01-05 11:16:27 +01:00
f877a3a0a9
run cleanupjob on fly ( #35 )
...
* clippy+fmt
* poor-man's integration of cleanup job
- runs every 5 hours
- deletes all but 2mn slots
* change slots back
2024-01-05 11:12:44 +01:00
7a3480e044
Fix/cleanupdb delete amb ( #31 )
...
* clippy+fmt
* cleanup also from amb table
2024-01-05 11:06:47 +01:00
6b0e2d269f
Loading all ALTS in bulk, saving stats of priority fees by cus ( #34 )
...
* Loading all ALTS in bulk, saving stats of priority fees by cus
* cargo fmt
2024-01-03 11:43:55 +01:00
GroovieGermanikus
Groovie | Mango
godmodegalactus
galactus