Commit Graph

68 Commits

Author SHA1 Message Date
Christian Kamm a7a53dd6f7 Heroku: Have two running instances 2021-12-20 12:40:39 +01:00
Christian Kamm 1b885355fd Fix log messages for slot update retries 2021-12-20 11:28:53 +01:00
Christian Kamm 76e14d0300 Fix initial snapshot #1
Previously the initial snapshot could be returned for a discarded slot.
Now a finalized-commitment snapshot is retrieved.
2021-12-13 16:48:33 +01:00
Christian Kamm 17d6787bf1 Add TLS domain name argument 2021-11-30 17:15:41 +01:00
Christian Kamm 60ca647650 Do the slot updates before wiping non-rooted account writes 2021-11-26 11:59:23 +01:00
Christian Kamm c65baee04f If a slot becomes rooted, mark preceeding non-uncle slots rooted.
This fixes the _rooted views, which previously would skip data from
slots that were never actively confirmed to be rooted.
2021-11-26 11:54:08 +01:00
Christian Kamm a941f486ed Fix bug introduced by deleting older slots
Deleting older slots meant that older (but still most recent) account
writes that had their slot deleted would no longer show up in views.

Now the guarantee is that an account write for a missing slot that is
smaller than the top slot in the slots table is rooted.

To guarantee that, account writes for non-rooted slots that preceed
rooted slots are deleted now.
2021-11-25 10:38:55 +01:00
Christian Kamm 3cdbeddf8e Update solana version to 1.8.5 2021-11-19 11:59:59 +01:00
Christian Kamm e34ee0489c Update solana version to 1.8.3 2021-11-19 11:57:47 +01:00
Christian Kamm 2abe2cca03 Heroku: Add aptfile to pull in build-time dependencies 2021-11-19 11:32:45 +01:00
Christian Kamm c82caf58c8 Fix dependency on jsonrpc fork 2021-11-19 11:17:25 +01:00
Christian Kamm 3912f6142f Add setup for heroku 2021-11-19 10:44:30 +01:00
Christian Kamm b65815e4d4 gRPC: Optionally use TLS 2021-11-18 14:21:17 +01:00
Christian Kamm 4c9b782d1f Mango: Drop all views in drop_views.sql 2021-11-15 16:18:27 +01:00
Christian Kamm 5ef1a19300 Mango: Update views and PnL query 2021-11-15 16:15:18 +01:00
Christian Kamm dbfbae1a49 Delay snapshot more, add metric on slot insertion queue 2021-11-15 15:49:59 +01:00
Christian Kamm dde006243b Minor clippy fixes 2021-11-15 15:37:29 +01:00
Christian Kamm 8f6bd7bc22 Postgres: Allow parallel slot update connections
It would be nicer if many of the writes could happen on the same
connection. Separate connections are only necessary if transactions are
used.
2021-11-15 15:28:18 +01:00
Christian Kamm 343b5408bc Postgres: Add option to allow invalid certificates 2021-11-14 13:14:19 +01:00
Christian Kamm c4bf05b1d2 Postgres: Use native-tls to allow TLS connections 2021-11-13 09:32:32 +01:00
Christian Kamm 5e3db3203a Readme: Add more motivation / context 2021-11-09 18:41:34 +01:00
Christian Kamm f1608409ca Documentation 2021-11-09 15:07:42 +01:00
Christian Kamm 71071d9b52 Add example plugin config file 2021-11-09 14:52:05 +01:00
Christian Kamm 456b19ea06 Add example SQL script for computing mango PnL 2021-11-09 14:50:08 +01:00
Christian Kamm d889b5ff5b Move postgres numeric support to separate file 2021-11-09 14:32:58 +01:00
Christian Kamm 3a0e2535f0 Use anyhow::Result over Result<T, anyhow::Error> 2021-11-09 14:27:09 +01:00
Christian Kamm 0cdee40851 Metrics cleanup 2021-11-09 14:23:55 +01:00
Christian Kamm 4d5f0c5d10 Basic metrics 2021-11-09 09:00:00 +01:00
Christian Kamm 029cbd2fb5 Allow multiple grpc sources 2021-11-08 15:48:50 +01:00
Christian Kamm f43aaf0f55 Don't process old account writes
This will help when receiving data from multiple validator nodes
2021-11-08 15:31:35 +01:00
Christian Kamm 35c0c5da23 Fix delayed snapshots
The snapshot was requested too early before.
2021-11-08 14:26:37 +01:00
Christian Kamm 11f4d6bd67 Use separate pubkey table 2021-11-08 14:10:48 +01:00
Christian Kamm a17f142875 Use enum for slot status 2021-11-08 12:44:01 +01:00
Christian Kamm 7f6e25e107 Wipe old slots from table, now nothing accumulates 2021-11-08 12:09:14 +01:00
Christian Kamm 47f8bb823d Remove unnecessary trailing backslashes from multiline string literals 2021-11-08 11:59:57 +01:00
Christian Kamm 95da45f597 Remove stale TODO comments 2021-11-08 11:44:37 +01:00
Christian Kamm 0424c3c9d6 grpc configurability 2021-11-08 11:42:22 +01:00
Christian Kamm d366581826 Make more postgres settings configurable 2021-11-08 11:30:17 +01:00
Christian Kamm d9cb30c2b3 Make parallel postgres connections configurable 2021-11-08 10:05:17 +01:00
Christian Kamm cbe9e16b62 Configurable program id 2021-11-08 09:57:56 +01:00
Christian Kamm e67dd6c235 Split into connector library, and raw and mango binaries 2021-11-08 09:45:41 +01:00
Christian Kamm 3088e8b071 Fix warnings and plugin shutdown 2021-11-07 11:32:57 +01:00
Christian Kamm 387d34e511 Fix potential missing data with initial snapshotting 2021-11-07 10:57:16 +01:00
Christian Kamm 9f7f72c3cb Encode addresses as base58 2021-11-05 15:19:47 +01:00
Christian Kamm 0ab99ac00b Fix psql numerics; dump mango group/cache 2021-11-05 10:46:29 +01:00
Christian Kamm df89f35aa8 Add sql scripts for creating views 2021-11-04 10:00:29 +01:00
Christian Kamm c2ac94a7b8 Add mango-specific account table 2021-11-04 09:57:55 +01:00
Christian Kamm bdd0e49ca8 Allow adding custom account tables 2021-11-03 18:56:40 +01:00
Christian Kamm d7b696a071 Fix postgres failure modes
- reconnect on fail
- critical error if postgres target can't be reached for long
- retry failing postgres actions a limited number of times
- exit if postgres connection seems gone for long or retries exhausted
- use async_channel instead of crossbeam_channel - needs to be async
2021-11-03 13:35:49 +01:00
Christian Kamm d51332ebec snapshotting for grpc 2021-11-02 14:51:38 +01:00