Commit Graph

207 Commits

Author SHA1 Message Date
Christian Kamm ebd392e628 ChainData: Overwrite data on the same key, slot, write version update 2023-03-17 10:41:19 +01:00
Christian Kamm 96ca006963 Typos and outdated comments 2023-03-16 17:15:15 +01:00
Christian Kamm 12e57ef765 ChainData: Remove the metrics into its separate thing
Until they're refactored away completely.
2023-03-16 17:10:34 +01:00
Christian Kamm 53eefa1395 ChainData: Port over fixes from v4 repo
See mango-v4:7c4d0529790e730820e9ed408310a3bca82493a0

1. Only the CreatedBank message contains the parent and often that was
   not the first message received. That confused the data structure,
   because it then considered a chain of slots alive that had only a
   single slot in it.
   Fixed by tracking as "best_chain" only the top slot that actually has
   a parent set.
2. OptimisticallyConfirm messages sometimes arrive before CreateBank,
   which would change the slot status from Confirmed back to Processed.
   Nothing dependend on confirmed vs processed, but it's been fixed
   anyway by making slot status only increase confirmation status.
2023-03-16 16:04:22 +01:00
Christian Kamm 0a41974aa5 cargo fmt 2023-03-16 15:31:37 +01:00
Christian Kamm 19ccf2ff19 Grpc: Use tls config if uri starts with https
Even if no explicit TlsConfig is set.
2023-03-16 15:23:08 +01:00
Christian Kamm d3fec673cf Separate out shared code into a "data-streams" crate 2023-03-16 14:58:52 +01:00
Maximilian Schneider 8ee6c6ac10 remove warnings 2023-02-18 20:41:40 +09:00
Maximilian Schneider ffe776b70d AccountWriteFilter generic router to process account writes for different modules in the same process 2023-02-07 17:22:21 +09:00
Maximilian Schneider c872d32943 add some basic throttling 2023-02-05 17:14:50 +09:00
Maximilian Schneider a0af15cb4a read crank keypair from config 2023-02-05 15:19:43 +09:00
Maximilian Schneider b9afc2c200 Merge branch 'v4' into max/crank 2023-02-05 10:53:02 +09:00
Riordan Panayides 956b5aa3dd Upgrade mango-v4 2023-02-05 01:43:18 +00:00
Maximilian Schneider 1c41e94e28 sort pubkeys 2023-02-04 02:27:37 +09:00
Maximilian Schneider 6287f1c344 actually spawn tasks 2023-02-04 02:15:55 +09:00
Maximilian Schneider 17ba55eac6 delete unused 2023-02-04 01:43:26 +09:00
Maximilian Schneider dc74c89626 serum crank implemented 2023-02-04 01:37:19 +09:00
Maximilian Schneider 0a09ad81ae implement mango v4 crank 2023-02-03 20:55:59 +09:00
Riordan Panayides 4b75b88a98 cargo fmt 2023-02-03 11:49:22 +00:00
Riordan Panayides ba4aa29f7f Update deps 2023-02-03 11:45:29 +00:00
Riordan Panayides 746174ce8e Orderbook updates
* Subscribe to accounts rather than program
* Process out of order writes correctly
* Update native to ui helpers
2023-02-03 11:44:46 +00:00
Riordan Panayides 5c8ebc53b1 Fills updates
* Unify fill event schema
* Change fill updates json
* Convert all native values to ui
* Add fills postgres target
2023-02-03 11:39:53 +00:00
Riordan Panayides 6dfe88ac15 Add deployment configs 2023-02-03 11:31:18 +00:00
Maximilian Schneider e22d30b9fd add blockhash poller & transaction sender 2023-02-03 19:45:22 +09:00
Maximilian Schneider 31149f7e0d Merge remote-tracking branch 'origin/v4' into max/crank 2023-02-03 16:22:35 +09:00
Maximilian Schneider 5e78f008f6 create crank skeleton 2023-02-03 16:15:58 +09:00
Riordan Panayides 1233cc58f3 Update chaindata metric names, finish serum events unified schema 2023-01-20 17:29:02 +00:00
Riordan Panayides 8d6b6f5f38 Fix serum event change detection 2023-01-20 16:50:39 +00:00
Riordan Panayides f88ec6a53c Add new common event schema 2023-01-20 16:50:19 +00:00
Riordan Panayides 3dc7ae1246 * Add chaindata metrics
* jemalloc for fills
* Reenable dropped fill processing
* Add gMA snapshot support
* Tidy up serum orderbook change detection
* cargo fmt
2023-01-20 14:52:01 +00:00
Riordan Panayides 0d41c04de5 Snapshot all program ids in fills 2023-01-19 02:44:54 +00:00
Maximilian Schneider 531a2c3d11 collect stats on nr of account writes & bytes store 2023-01-19 11:18:38 +09:00
Riordan Panayides 393e10c440 Disable serum fill change/drop detection, rename connector metrics 2023-01-18 19:39:53 +00:00
Riordan Panayides b89faa1ed0 Add fills service ping handling 2023-01-11 15:43:07 +00:00
Riordan Panayides c838c58ca6 Fix orderbook decimals 2023-01-11 15:41:19 +00:00
Riordan Panayides f45e614765 Update CD, dependencies, READMEs 2023-01-11 15:40:36 +00:00
Riordan Panayides 464946dc16 Switch mack to mango-v4/dev, add dedup queue metric, fix orderbook checkpoint 2023-01-05 20:18:56 +00:00
Riordan Panayides c2a942336c Set up fly deployments, upgrade mango-v4 2023-01-04 00:04:35 +00:00
Riordan Panayides 1b1063c5a7 Restore CI scripts 2023-01-04 00:03:16 +00:00
Riordan Panayides 8f19a20b91 Update mango-v4, try to use async client 2023-01-01 16:42:36 +00:00
Riordan Panayides 05f42c7352 Filter fill updates on subscription 2022-12-27 15:03:18 +00:00
Riordan Panayides 9f528603b0 Add subscription commands for fills 2022-12-27 14:57:50 +00:00
Riordan Panayides 7f0ddd3ac5 Fix serum fills 2022-12-24 14:43:43 +00:00
Riordan Panayides fad9f7bae3 Add Serum support for orderbook, fix pnl main 2022-12-24 12:49:17 +00:00
Riordan Panayides df562a3e42 Auto discover markets in fills feed 2022-12-16 11:03:21 +00:00
Riordan Panayides 49bcdeee77 Use new geyser plugin, add orderbook service 2022-12-16 11:00:42 +00:00
Riordan Panayides c00935bae7 Tidy up repo, upgrade solana/mango 2022-12-16 10:58:04 +00:00
Riordan Panayides 21f614b286 Update fills feed, disable connector-mango 2022-11-21 03:47:20 +00:00
Riordan Panayides cf26a885f2 Update to mango-v4, update fills feed 2022-11-16 14:57:50 +00:00
Riordan Panayides 2a15d96424 Add some basic pnl service metrics 2022-10-08 18:13:26 +01:00