Commit Graph

84 Commits

Author SHA1 Message Date
Riordan Panayides 0d41c04de5 Snapshot all program ids in fills 2023-01-19 02:44:54 +00:00
Maximilian Schneider 531a2c3d11 collect stats on nr of account writes & bytes store 2023-01-19 11:18:38 +09:00
Riordan Panayides 393e10c440 Disable serum fill change/drop detection, rename connector metrics 2023-01-18 19:39:53 +00:00
Riordan Panayides c838c58ca6 Fix orderbook decimals 2023-01-11 15:41:19 +00:00
Riordan Panayides 464946dc16 Switch mack to mango-v4/dev, add dedup queue metric, fix orderbook checkpoint 2023-01-05 20:18:56 +00:00
Riordan Panayides c2a942336c Set up fly deployments, upgrade mango-v4 2023-01-04 00:04:35 +00:00
Riordan Panayides 8f19a20b91 Update mango-v4, try to use async client 2023-01-01 16:42:36 +00:00
Riordan Panayides 7f0ddd3ac5 Fix serum fills 2022-12-24 14:43:43 +00:00
Riordan Panayides fad9f7bae3 Add Serum support for orderbook, fix pnl main 2022-12-24 12:49:17 +00:00
Riordan Panayides df562a3e42 Auto discover markets in fills feed 2022-12-16 11:03:21 +00:00
Riordan Panayides 49bcdeee77 Use new geyser plugin, add orderbook service 2022-12-16 11:00:42 +00:00
Riordan Panayides c00935bae7 Tidy up repo, upgrade solana/mango 2022-12-16 10:58:04 +00:00
Riordan Panayides 21f614b286 Update fills feed, disable connector-mango 2022-11-21 03:47:20 +00:00
Riordan Panayides cf26a885f2 Update to mango-v4, update fills feed 2022-11-16 14:57:50 +00:00
Riordan Panayides 190528b722 Add process name label, fix some metric types 2022-10-08 12:57:47 +01:00
Riordan Panayides bd7f80f201 Sanitize metric names 2022-10-07 13:15:27 +01:00
Riordan Panayides e2729306b8 cargo fmt 2022-10-07 11:46:19 +01:00
Riordan Panayides 2881378b7f Refactor metrics 2022-10-07 11:44:53 +01:00
Riordan Panayides 1d1f462cf6 Rename count metrics 2022-10-06 01:33:07 +01:00
Riordan Panayides d14b4081aa cargo fmt 2022-10-05 23:53:20 +01:00
Riordan Panayides dbf28b74be Merge branch 'master' of github.com:blockworks-foundation/solana-accountsdb-connector 2022-10-05 23:50:28 +01:00
Riordan Panayides ae7ffc1dcb Add prometheus metrics to fills 2022-10-05 23:50:21 +01:00
waterquarks 1bebc2f6bc
Aggregate t event metrics (#3)
* Aggregate t event metrics

* Misc

* Misc
2022-09-21 16:45:55 +02:00
riordanp 2f09d0a8f2
Add docker build and fly deployment actions (#2)
* Test dockerfile with caching

* Fix action

* Remove tags

* Use sccache release

* Use max mode

* idk

* Try no runtime build

* Use cargo-chef

* Test rebuild

* Revert "Test rebuild"

This reverts commit d7e1654c63.

* Publish image to GCR

* Trigger build

* Read config from env

* Fix typo

* Bind to correct address

* Lint

* Add fly config

* Switch trigger branch to main

* Use both common branch names
2022-09-20 23:01:24 +02:00
Riordan Panayides 3dfb430416 Add zstd decoding 2022-08-25 11:51:38 +01:00
Riordan Panayides 6b280445cc Try 1.10.35 2022-08-24 17:19:40 +01:00
Riordan Panayides f39fcdfba9 Fix lib for 1.10 compatibility 2022-08-24 13:47:16 +01:00
Riordan Panayides 18fc4afd93 Bump ci rust version 2022-08-24 13:16:34 +01:00
Riordan Panayides 495b5371ec Update solana and mango deps 2022-08-24 12:56:50 +01:00
Maximilian Schneider 8dde08d377 service-fills: add slote and write version to fill update 2022-04-08 05:23:52 +02:00
Maximilian Schneider b41c094aba merge fill service 2022-03-28 20:01:16 +02:00
Christian Kamm c2f0c0ac05 Fix write_version sanity checking for 1.9
In 1.9, the write_version will no longer be strictly monotonic across
accounts.
2022-03-28 15:37:53 +02:00
Christian Kamm 1be0cc8fb6 pnl-service: first working version 2022-03-23 18:39:42 +01:00
Christian Kamm cdf6d54156 Config: Separate out source config
We want to be able to not have a postgres target
2022-03-23 08:51:17 +01:00
Christian Kamm 3beaac671d Add memory_target and chain_data
This will be useful for storing account data in memory.
2022-03-23 08:19:19 +01:00
Christian Kamm 42f07d58a6 Keep slot, write_version etc as u64, convert only for postgres
This makes the conversion to i64 something that's local to the postgres
target, and doesn't affect unrelated code.
2022-03-23 08:18:02 +01:00
Christian Kamm 83e9c54fc0 Fix talking to older plugins
The previous commit only looked like it worked due to stale generated
code.
2022-03-23 08:16:02 +01:00
Christian Kamm 2b1bf634c7 grpc proto: Change back to AccountsDb name to avoid incompatibility
Existing plugin deployments must continue to work with the new version
of the connector.
2022-03-22 12:54:38 +01:00
Linus Kendall b6cefddef2 Compatibility fixes for 1.9
- Renamed to geyser everywhere and updated the crates
- Changed FullRpcClient to AccountsDataClient
2022-03-22 08:36:59 +00:00
Christian Kamm f25c49d983 Configurable max account write queue size 2022-03-16 11:04:30 +01:00
Christian Kamm 80423fdac9 Track "is_selected" to denote closed/reused accounts 2022-01-28 10:12:01 +01:00
Christian Kamm 66266a3469 Update solana version to 1.8.14 2022-01-26 10:08:53 +01:00
Christian Kamm 185bf4f9cf Postgres: Fix account_write cleanup query
Avoid misbehaving in situations where there's an account write for a
slot that doesn't yet have an entry in the slots table by limiting the
slot numbers.
2022-01-20 15:28:16 +01:00
Christian Kamm dc80d8266c Process postgres cleanup jobs separately
Executing them for every rooted slot may overwhelm the database.
2022-01-20 14:49:51 +01:00
Christian Kamm cf26557f7d Update to solana 1.8.12 2022-01-12 12:00:06 +01:00
Christian Kamm afc08d8266 Fix metadata 2022-01-07 08:21:00 +01:00
Christian Kamm 7d8fd45de4 Add support for updating a monitoring table
That way connector interruptions can be seen directly form the target
database.
2022-01-06 12:54:35 +01:00
Christian Kamm b1f18c0b13 Fix data deletion not being enableable 2022-01-05 16:02:34 +01:00
Christian Kamm fb7e0d14e2 Update to solana 1.8.11 2022-01-04 18:39:48 +01:00
Christian Kamm 5b5eaba4ff Pipeline SQL inserts for better performance
Previously insert speed was primarily limited by roundtrip time to the
database since they were fully sequential in each connection. Now a full
batch of inserts is done per connection in parallel.

This could still be improved a lot. Currently each connection will fully
wait for all inserts to be done in the current batch before starting
another one.
2022-01-04 10:15:57 +01:00