Commit Graph

1179 Commits

Author SHA1 Message Date
Groovie | Mango d10910e737
reducing yellowstone dependencies (#389)
* WIP - not compiling

* remove connect_hacked

* use reimported types

* cleanup

* fix fmt
2024-04-17 15:11:51 +02:00
Christian Kamm 2d614365e7
experiment: reduce number of source endpoints to 1 (#386)
Prediction is that this has no negative effect but reduces memory use a
lot.
2024-04-03 21:06:29 +02:00
Groovie | Mango b84e880961
remove async_channel and bring back tokio mpsc§ (#384) 2024-04-03 11:22:36 +02:00
godmodegalactus 90dbcaa9d8
Adding more prometheus metrics to track QUIC errors details 2024-04-03 10:51:50 +02:00
Groovie | Mango 69d7dbb123
optimize mapping of produced block (#382)
optimize mapping of produced block
2024-04-03 10:40:53 +02:00
godmodegalactus 06373c0844
Closing QUIC connection correctly, changing some QUIC params 2024-04-03 10:25:30 +02:00
godmodegalactus 6b8fe682ae
Revert "Close connection while dropping"
This reverts commit 5ef1391b9a.
2024-04-03 10:02:45 +02:00
godmodegalactus 5ef1391b9a
Close connection while dropping 2024-04-02 23:06:52 +02:00
godmodegalactus d0a2fe200b
Using default solana timeout for quic connection 2024-04-02 21:45:35 +02:00
Groovie | Mango 944a93114c
Bugfix,379 identity key pair loading must be optional (#381)
fix identity keypair loading logic

reverting to this logic
1. check if env variable IDENTITY is present
2. check if cli arg --identity-keypair is present
3. assume no identity
2024-04-02 15:52:06 +02:00
godmodegalactus 12a6832c56
Swapping notify channel with broadcast channel 2024-04-02 15:07:01 +02:00
godmodegalactus ca3fa46139
Using broadcast channels instead of notify 2024-04-02 14:48:54 +02:00
godmodegalactus 91cf06436a
Changing exit signal from notify to broadcast channel in grpc multiplexer 2024-04-02 14:45:41 +02:00
galactus 681334197f
Adding custom tpu send transaction example (#380)
* Adding an example for custom tpu send transaction

* Fixing the custom tpu example

* Optimizing SentTransactionInfo, and calculating TPS

* Reverting unwanted changes

* After groovies review
2024-04-02 14:26:54 +02:00
Lou-Kamades 4d7700145e
fix parsing of use_grpc env var (#378) 2024-04-02 10:21:46 +02:00
godmodegalactus 1e956a2b4f
Revert "Replacing notify by cancellation token to correctly shutdown tasks"
This reverts commit d1abc345cf.
2024-03-28 11:49:20 +01:00
godmodegalactus 02229060cf
Revert "Fix clippy fmt"
This reverts commit bac036ceaa.
2024-03-28 11:49:17 +01:00
godmodegalactus bac036ceaa
Fix clippy fmt 2024-03-28 09:12:12 +01:00
godmodegalactus d1abc345cf
Replacing notify by cancellation token to correctly shutdown tasks 2024-03-28 09:09:42 +01:00
Lou-Kamades 9926d2e5c1
Merge pull request #376 from blockworks-foundation/feature/benchrunner-tc1-tc2
benchrunner tc2 integration
2024-03-27 22:36:36 -05:00
Lou-Kamades bc096d165c
Merge pull request #375 from blockworks-foundation/lou/bench-helpers
refactor: use VersionedTransaction in benchmarks
2024-03-27 14:13:31 -04:00
GroovieGermanikus 9f73f8e4a0
add sql queries 2024-03-27 19:05:47 +01:00
GroovieGermanikus 1f502a3a57
cleanup 2024-03-27 19:00:35 +01:00
GroovieGermanikus f3657d7db1
wire up size_tx 2024-03-27 18:57:51 +01:00
GroovieGermanikus 8aca443d56
wire up confirmation_rate + bench1 2024-03-27 18:54:14 +01:00
Lou-Kamades 0105602e05
refactor: use VersionedTransaction in benchmarks 2024-03-27 12:53:50 -05:00
GroovieGermanikus 4a6a594873
split metric struct 2024-03-27 18:40:04 +01:00
GroovieGermanikus 2a87e2f8d7
permutate config + impl 2024-03-27 18:13:11 +01:00
GroovieGermanikus fd3fe80336
introduce traits 2024-03-27 18:00:10 +01:00
godmodegalactus dfe8d972eb
Porting back the exit notify change 2024-03-27 17:57:40 +01:00
GroovieGermanikus 4d5b9f23f3
move bench run+persist logic around 2024-03-27 17:41:56 +01:00
galactus 79748c5a58
Merge production main 2024 03 27 (#374)
* Use jemalloc

* Solving issue of finalized meta after processed block (#365)

* Solving issue of finalized meta after processed block

* Fixing the broken test

* Removing unwanted logs

* Add logs on block queues and reception time

* reduce level of block reception log

* Revert "Solving issue of finalized meta after processed block (#365)"

This reverts commit c09700fd79.

Production runs show more stable memory use but worse performance on
landing transactions.

* use yellowstone grpc with hacked-windowsize

* upgraded geyser-grpc-connector + increased timeout

* Fix block stream throughput problems

By temporarily pasting in a function to connect to block streams via a
more generously configured endpoint.

* Enabling unstable tokio on fly

* Fixing prioritization heap metrics (#370)

* Fixing prioritization heap metrics

* Fixing transaction sizes tests

* Correctly ending the block streams instead of aborting them

* Replacing exit signal with exit notification (#372)

* Replacing exit signal with exit notification

* Deprecitating nightly version

* Increase connection size

* Fixing issue with multiple notify channels

* Joining heap task instead of aborting

* fix postgres ssl algo issue - RC2-40-CBC  (#373)

* fix 373: openssl-legacy.cnf

* Remove replace console subscriber with tracing subscriber

---------

Co-authored-by: Christian Kamm <mail@ckamm.de>
Co-authored-by: GroovieGermanikus <groovie@mango.markets>
Co-authored-by: Groovie | Mango <95291500+grooviegermanikus@users.noreply.github.com>
2024-03-27 17:09:38 +01:00
GroovieGermanikus bd9d15ab3b
fix 373: openssl-legacy.cnf 2024-03-27 12:32:58 +01:00
GroovieGermanikus 21355098b4
cleanup: histogram 2024-03-27 12:23:42 +01:00
Groovie | Mango f5991d87e8
add histogram utils (#361) 2024-03-27 12:15:40 +01:00
Groovie | Mango a98e52fbd6
fix postgres ssl algo issue - RC2-40-CBC (#373) 2024-03-27 12:12:00 +01:00
Lou-Kamades 4f51c27a08
TC-1 (#359)
add benchnew with confirmation-slot and confirmation-rate (TC-1 and TC-2)
2024-03-27 11:45:30 +01:00
godmodegalactus 7d39a947ac
Minor changes, increasing default lite-rpc limits 2024-03-26 17:36:11 +01:00
godmodegalactus 41b193d10a
Fixing rpc account fly config 2024-03-26 15:37:20 +01:00
galactus 29bd6deae4
Fixing accounts on demand (#371)
* Fixing accounts on demand

* Adding missing file and renaming gprc to grpc

* Adding missing grpc_accounts_streaming
2024-03-26 14:32:50 +01:00
Groovie | Mango 3c994597b4
benchrunner service (#363)
benchrunner service
+ minor changes in bench code
2024-03-26 13:41:46 +01:00
galactus 087cc7f204
Merge prod main 2024 03 25 (#369)
* Use jemalloc

* Solving issue of finalized meta after processed block (#365)

* Solving issue of finalized meta after processed block

* Fixing the broken test

* Removing unwanted logs

* Add logs on block queues and reception time

* reduce level of block reception log

* Revert "Solving issue of finalized meta after processed block (#365)"

This reverts commit c09700fd79.

Production runs show more stable memory use but worse performance on
landing transactions.

* use yellowstone grpc with hacked-windowsize

* upgraded geyser-grpc-connector + increased timeout

* Fix block stream throughput problems

By temporarily pasting in a function to connect to block streams via a
more generously configured endpoint.

* Enabling unstable tokio on fly

* update to v0.10.3+yellowstone.1.12+solana.1.17.15-hacked-windowsize3

---------

Co-authored-by: Christian Kamm <mail@ckamm.de>
Co-authored-by: GroovieGermanikus <groovie@mango.markets>
2024-03-25 17:44:57 +01:00
godmodegalactus 229d051bce
Enabling unstable tokio on fly 2024-03-25 15:34:19 +01:00
galactus a12d79aee8
Getting finalized meta before processed main (#366)
* Use jemalloc

* Solving issue of finalized meta after processed block (#365)

* Solving issue of finalized meta after processed block

* Fixing the broken test

* Minor changes

---------

Co-authored-by: Christian Kamm <mail@ckamm.de>
2024-03-22 19:44:23 +01:00
Groovie | Mango cd9df11501
use geyser blockmeta to progress finalizedslot (#367)
introduce BlockInfo on a new BlockInfoStream; maps from yellowstone BlockMeta
fix: last_finalized_slot must progress unconditionally
2024-03-22 19:22:38 +01:00
galactus defdc20dd5
Getting finalized meta before processed main (#366)
* Use jemalloc

* Solving issue of finalized meta after processed block (#365)

* Solving issue of finalized meta after processed block

* Fixing the broken test

* Minor changes

---------

Co-authored-by: Christian Kamm <mail@ckamm.de>
2024-03-22 12:30:14 +01:00
GroovieGermanikus ae495bed53
Merge branch 'production'
using 1.17.15 geyser
2024-03-22 10:57:20 +01:00
galactus 7e6a1a4003
Reverting back the geyser version (#364) 2024-03-22 09:55:46 +01:00
godmodegalactus 5be4d2e755
Bug fix update finalized blockhash 2024-03-22 09:22:51 +01:00
godmodegalactus 9946abb12d
Minor performance fix 2024-03-21 18:43:29 +01:00