Commit Graph

23 Commits

Author SHA1 Message Date
Christian Kamm f25c49d983 Configurable max account write queue size 2022-03-16 11:04:30 +01:00
Christian Kamm 80423fdac9 Track "is_selected" to denote closed/reused accounts 2022-01-28 10:12:01 +01:00
Christian Kamm dc80d8266c Process postgres cleanup jobs separately
Executing them for every rooted slot may overwhelm the database.
2022-01-20 14:49:51 +01:00
Christian Kamm 7d8fd45de4 Add support for updating a monitoring table
That way connector interruptions can be seen directly form the target
database.
2022-01-06 12:54:35 +01:00
Christian Kamm 5b5eaba4ff Pipeline SQL inserts for better performance
Previously insert speed was primarily limited by roundtrip time to the
database since they were fully sequential in each connection. Now a full
batch of inserts is done per connection in parallel.

This could still be improved a lot. Currently each connection will fully
wait for all inserts to be done in the current batch before starting
another one.
2022-01-04 10:15:57 +01:00
Christian Kamm d5395a3b75 Add config flag for deleting old data
Keeping it is very useful for testing.
2022-01-03 10:36:11 +01:00
Christian Kamm 17d6787bf1 Add TLS domain name argument 2021-11-30 17:15:41 +01:00
Christian Kamm b65815e4d4 gRPC: Optionally use TLS 2021-11-18 14:21:17 +01:00
Christian Kamm dde006243b Minor clippy fixes 2021-11-15 15:37:29 +01:00
Christian Kamm 8f6bd7bc22 Postgres: Allow parallel slot update connections
It would be nicer if many of the writes could happen on the same
connection. Separate connections are only necessary if transactions are
used.
2021-11-15 15:28:18 +01:00
Christian Kamm 343b5408bc Postgres: Add option to allow invalid certificates 2021-11-14 13:14:19 +01:00
Christian Kamm d889b5ff5b Move postgres numeric support to separate file 2021-11-09 14:32:58 +01:00
Christian Kamm 3a0e2535f0 Use anyhow::Result over Result<T, anyhow::Error> 2021-11-09 14:27:09 +01:00
Christian Kamm 4d5f0c5d10 Basic metrics 2021-11-09 09:00:00 +01:00
Christian Kamm 029cbd2fb5 Allow multiple grpc sources 2021-11-08 15:48:50 +01:00
Christian Kamm 11f4d6bd67 Use separate pubkey table 2021-11-08 14:10:48 +01:00
Christian Kamm a17f142875 Use enum for slot status 2021-11-08 12:44:01 +01:00
Christian Kamm 47f8bb823d Remove unnecessary trailing backslashes from multiline string literals 2021-11-08 11:59:57 +01:00
Christian Kamm 0424c3c9d6 grpc configurability 2021-11-08 11:42:22 +01:00
Christian Kamm d366581826 Make more postgres settings configurable 2021-11-08 11:30:17 +01:00
Christian Kamm d9cb30c2b3 Make parallel postgres connections configurable 2021-11-08 10:05:17 +01:00
Christian Kamm cbe9e16b62 Configurable program id 2021-11-08 09:57:56 +01:00
Christian Kamm e67dd6c235 Split into connector library, and raw and mango binaries 2021-11-08 09:45:41 +01:00