Commit Graph

107 Commits

Author SHA1 Message Date
Maximilian Schneider b6912202bd
Refactoring, Fills Feed changes (#1)
lib:
- Move fill/orderbook filters out of shared lib into the services
- Add some common structs to shared lib
- Add libraries to fills/orderbook services

feeds:
- Add graceful exit handling
- Publish single perp fill event for both maker and taker
- Disable openbook fills processing
- Fix perp fill event quantity decimals
- Handle revoked fills in postgres
- Allow subscriptions to multiple and all markets, and accounts
- Add event queue head updates
2023-04-07 16:27:54 +01:00
Maximilian Schneider d4a0ff7602 switch to AGPL license 2023-03-25 15:58:32 +04:00
Christian Kamm 0e406af7f3 Drop more dependencies 2023-03-21 12:12:03 +01:00
Christian Kamm d2590c9174 Remove unneeded dependencies 2023-03-21 12:06:42 +01:00
Christian Kamm ede1f41088 Switch to yellowstone-grpc-proto
It's no longer needed to copy the grpc proto defs here and compile them
ourselves.
2023-03-21 11:26:47 +01:00
Christian Kamm 6731496fde This is now mango-fills
- mango-fills-connector: Is about getting data from rpc, websockets,
  geyser. Shall be reused in v4/client and other places.
- mango-fills-lib: is a library we'll probably remove
- service-* is the individual services
2023-03-17 12:57:11 +01:00
Christian Kamm 12e57ef765 ChainData: Remove the metrics into its separate thing
Until they're refactored away completely.
2023-03-16 17:10:34 +01:00
Christian Kamm 0a41974aa5 cargo fmt 2023-03-16 15:31:37 +01:00
Christian Kamm d3fec673cf Separate out shared code into a "data-streams" crate 2023-03-16 14:58:52 +01:00
Maximilian Schneider 8ee6c6ac10 remove warnings 2023-02-18 20:41:40 +09:00
Maximilian Schneider ffe776b70d AccountWriteFilter generic router to process account writes for different modules in the same process 2023-02-07 17:22:21 +09:00
Maximilian Schneider b9afc2c200 Merge branch 'v4' into max/crank 2023-02-05 10:53:02 +09:00
Riordan Panayides 956b5aa3dd Upgrade mango-v4 2023-02-05 01:43:18 +00:00
Riordan Panayides 4b75b88a98 cargo fmt 2023-02-03 11:49:22 +00:00
Riordan Panayides ba4aa29f7f Update deps 2023-02-03 11:45:29 +00:00
Riordan Panayides 746174ce8e Orderbook updates
* Subscribe to accounts rather than program
* Process out of order writes correctly
* Update native to ui helpers
2023-02-03 11:44:46 +00:00
Riordan Panayides 5c8ebc53b1 Fills updates
* Unify fill event schema
* Change fill updates json
* Convert all native values to ui
* Add fills postgres target
2023-02-03 11:39:53 +00:00
Maximilian Schneider 31149f7e0d Merge remote-tracking branch 'origin/v4' into max/crank 2023-02-03 16:22:35 +09:00
Maximilian Schneider 5e78f008f6 create crank skeleton 2023-02-03 16:15:58 +09:00
Riordan Panayides 1233cc58f3 Update chaindata metric names, finish serum events unified schema 2023-01-20 17:29:02 +00:00
Riordan Panayides 8d6b6f5f38 Fix serum event change detection 2023-01-20 16:50:39 +00:00
Riordan Panayides f88ec6a53c Add new common event schema 2023-01-20 16:50:19 +00:00
Riordan Panayides 3dc7ae1246 * Add chaindata metrics
* jemalloc for fills
* Reenable dropped fill processing
* Add gMA snapshot support
* Tidy up serum orderbook change detection
* cargo fmt
2023-01-20 14:52:01 +00:00
Riordan Panayides 0d41c04de5 Snapshot all program ids in fills 2023-01-19 02:44:54 +00:00
Maximilian Schneider 531a2c3d11 collect stats on nr of account writes & bytes store 2023-01-19 11:18:38 +09:00
Riordan Panayides 393e10c440 Disable serum fill change/drop detection, rename connector metrics 2023-01-18 19:39:53 +00:00
Riordan Panayides c838c58ca6 Fix orderbook decimals 2023-01-11 15:41:19 +00:00
Riordan Panayides 464946dc16 Switch mack to mango-v4/dev, add dedup queue metric, fix orderbook checkpoint 2023-01-05 20:18:56 +00:00
Riordan Panayides c2a942336c Set up fly deployments, upgrade mango-v4 2023-01-04 00:04:35 +00:00
Riordan Panayides 8f19a20b91 Update mango-v4, try to use async client 2023-01-01 16:42:36 +00:00
Riordan Panayides 7f0ddd3ac5 Fix serum fills 2022-12-24 14:43:43 +00:00
Riordan Panayides fad9f7bae3 Add Serum support for orderbook, fix pnl main 2022-12-24 12:49:17 +00:00
Riordan Panayides df562a3e42 Auto discover markets in fills feed 2022-12-16 11:03:21 +00:00
Riordan Panayides 49bcdeee77 Use new geyser plugin, add orderbook service 2022-12-16 11:00:42 +00:00
Riordan Panayides c00935bae7 Tidy up repo, upgrade solana/mango 2022-12-16 10:58:04 +00:00
Riordan Panayides 21f614b286 Update fills feed, disable connector-mango 2022-11-21 03:47:20 +00:00
Riordan Panayides cf26a885f2 Update to mango-v4, update fills feed 2022-11-16 14:57:50 +00:00
Riordan Panayides 190528b722 Add process name label, fix some metric types 2022-10-08 12:57:47 +01:00
Riordan Panayides bd7f80f201 Sanitize metric names 2022-10-07 13:15:27 +01:00
Riordan Panayides e2729306b8 cargo fmt 2022-10-07 11:46:19 +01:00
Riordan Panayides 2881378b7f Refactor metrics 2022-10-07 11:44:53 +01:00
Riordan Panayides 1d1f462cf6 Rename count metrics 2022-10-06 01:33:07 +01:00
Riordan Panayides d14b4081aa cargo fmt 2022-10-05 23:53:20 +01:00
Riordan Panayides dbf28b74be Merge branch 'master' of github.com:blockworks-foundation/solana-accountsdb-connector 2022-10-05 23:50:28 +01:00
Riordan Panayides ae7ffc1dcb Add prometheus metrics to fills 2022-10-05 23:50:21 +01:00
waterquarks 1bebc2f6bc
Aggregate t event metrics (#3)
* Aggregate t event metrics

* Misc

* Misc
2022-09-21 16:45:55 +02:00
riordanp 2f09d0a8f2
Add docker build and fly deployment actions (#2)
* Test dockerfile with caching

* Fix action

* Remove tags

* Use sccache release

* Use max mode

* idk

* Try no runtime build

* Use cargo-chef

* Test rebuild

* Revert "Test rebuild"

This reverts commit d7e1654c63a0cc99a3898c1b17df0cddd13ac0bd.

* Publish image to GCR

* Trigger build

* Read config from env

* Fix typo

* Bind to correct address

* Lint

* Add fly config

* Switch trigger branch to main

* Use both common branch names
2022-09-20 23:01:24 +02:00
Riordan Panayides 3dfb430416 Add zstd decoding 2022-08-25 11:51:38 +01:00
Riordan Panayides 6b280445cc Try 1.10.35 2022-08-24 17:19:40 +01:00
Riordan Panayides f39fcdfba9 Fix lib for 1.10 compatibility 2022-08-24 13:47:16 +01:00