Commit Graph

100 Commits

Author SHA1 Message Date
Christian Kamm 343b5408bc Postgres: Add option to allow invalid certificates 2021-11-14 13:14:19 +01:00
Christian Kamm c4bf05b1d2 Postgres: Use native-tls to allow TLS connections 2021-11-13 09:32:32 +01:00
Christian Kamm 5e3db3203a Readme: Add more motivation / context 2021-11-09 18:41:34 +01:00
Christian Kamm f1608409ca Documentation 2021-11-09 15:07:42 +01:00
Christian Kamm 71071d9b52 Add example plugin config file 2021-11-09 14:52:05 +01:00
Christian Kamm 456b19ea06 Add example SQL script for computing mango PnL 2021-11-09 14:50:08 +01:00
Christian Kamm d889b5ff5b Move postgres numeric support to separate file 2021-11-09 14:32:58 +01:00
Christian Kamm 3a0e2535f0 Use anyhow::Result over Result<T, anyhow::Error> 2021-11-09 14:27:09 +01:00
Christian Kamm 0cdee40851 Metrics cleanup 2021-11-09 14:23:55 +01:00
Christian Kamm 4d5f0c5d10 Basic metrics 2021-11-09 09:00:00 +01:00
Christian Kamm 029cbd2fb5 Allow multiple grpc sources 2021-11-08 15:48:50 +01:00
Christian Kamm f43aaf0f55 Don't process old account writes
This will help when receiving data from multiple validator nodes
2021-11-08 15:31:35 +01:00
Christian Kamm 35c0c5da23 Fix delayed snapshots
The snapshot was requested too early before.
2021-11-08 14:26:37 +01:00
Christian Kamm 11f4d6bd67 Use separate pubkey table 2021-11-08 14:10:48 +01:00
Christian Kamm a17f142875 Use enum for slot status 2021-11-08 12:44:01 +01:00
Christian Kamm 7f6e25e107 Wipe old slots from table, now nothing accumulates 2021-11-08 12:09:14 +01:00
Christian Kamm 47f8bb823d Remove unnecessary trailing backslashes from multiline string literals 2021-11-08 11:59:57 +01:00
Christian Kamm 95da45f597 Remove stale TODO comments 2021-11-08 11:44:37 +01:00
Christian Kamm 0424c3c9d6 grpc configurability 2021-11-08 11:42:22 +01:00
Christian Kamm d366581826 Make more postgres settings configurable 2021-11-08 11:30:17 +01:00
Christian Kamm d9cb30c2b3 Make parallel postgres connections configurable 2021-11-08 10:05:17 +01:00
Christian Kamm cbe9e16b62 Configurable program id 2021-11-08 09:57:56 +01:00
Christian Kamm e67dd6c235 Split into connector library, and raw and mango binaries 2021-11-08 09:45:41 +01:00
Christian Kamm 3088e8b071 Fix warnings and plugin shutdown 2021-11-07 11:32:57 +01:00
Christian Kamm 387d34e511 Fix potential missing data with initial snapshotting 2021-11-07 10:57:16 +01:00
Christian Kamm 9f7f72c3cb Encode addresses as base58 2021-11-05 15:19:47 +01:00
Christian Kamm 0ab99ac00b Fix psql numerics; dump mango group/cache 2021-11-05 10:46:29 +01:00
Christian Kamm df89f35aa8 Add sql scripts for creating views 2021-11-04 10:00:29 +01:00
Christian Kamm c2ac94a7b8 Add mango-specific account table 2021-11-04 09:57:55 +01:00
Christian Kamm bdd0e49ca8 Allow adding custom account tables 2021-11-03 18:56:40 +01:00
Christian Kamm d7b696a071 Fix postgres failure modes
- reconnect on fail
- critical error if postgres target can't be reached for long
- retry failing postgres actions a limited number of times
- exit if postgres connection seems gone for long or retries exhausted
- use async_channel instead of crossbeam_channel - needs to be async
2021-11-03 13:35:49 +01:00
Christian Kamm d51332ebec snapshotting for grpc 2021-11-02 14:51:38 +01:00
Christian Kamm 5e84fac5a9 Add configuration 2021-11-02 13:38:01 +01:00
Christian Kamm 16f8dad40d Use solana_logger for logging setup 2021-11-02 13:37:43 +01:00
Christian Kamm d8015daea6 Split connector into more files 2021-11-02 13:37:43 +01:00
Christian Kamm ce3f9e8a29 Postgres: Use a single query to update uncle status 2021-11-02 13:37:43 +01:00
Christian Kamm 0c1435f7fd switch to crossbeam channels 2021-11-02 13:37:43 +01:00
Christian Kamm 83cbc7e673 Split connector into multiple files 2021-11-02 13:37:39 +01:00
Christian Kamm 132f6bcf38 cargo fmt 2021-11-02 13:37:08 +01:00
Christian Kamm 8fe4795733 plugin: remove unused dependencies 2021-11-02 13:37:08 +01:00
Christian Kamm f3fdb58810 plugin cleanup
- new and changed json config
- shorten main type name to "Plugin"
- move all plugin data into a helper struct, because construction and
  initialization is separate
2021-11-02 13:37:08 +01:00
Christian Kamm 9fd1f4d25d Fix old-data deletion query 2021-11-02 13:37:08 +01:00
Christian Kamm cf1de61d4c gRPC: log connection retry errors 2021-11-02 13:37:08 +01:00
Christian Kamm ffd619f372 Update Cargo.lock 2021-11-02 13:37:08 +01:00
Christian Kamm 279e731d29 gRPC: unpack slot status correctly 2021-11-02 13:37:08 +01:00
Christian Kamm eab62d0599 gRPC: send regular ping
The idea was that slot updates would be really frequent and serve as a
ping. But during node startup there can be a long phase of no data.
2021-11-02 13:37:08 +01:00
Christian Kamm 6bb288e6b1 Create tokio runtime for the plugin 2021-11-02 13:37:08 +01:00
Christian Kamm 67f60beab1 Exit early when receiver stream closes 2021-11-02 13:37:08 +01:00
Christian Kamm 82d793832c Add readme 2021-11-02 13:37:08 +01:00
Christian Kamm 11ba681f9c Initial import from separate projects 2021-11-02 13:37:03 +01:00