Commit Graph

94 Commits

Author SHA1 Message Date
Groovie | Mango d10910e737
reducing yellowstone dependencies (#389)
* WIP - not compiling

* remove connect_hacked

* use reimported types

* cleanup

* fix fmt
2024-04-17 15:11:51 +02:00
galactus 681334197f
Adding custom tpu send transaction example (#380)
* Adding an example for custom tpu send transaction

* Fixing the custom tpu example

* Optimizing SentTransactionInfo, and calculating TPS

* Reverting unwanted changes

* After groovies review
2024-04-02 14:26:54 +02:00
Lou-Kamades 4f51c27a08
TC-1 (#359)
add benchnew with confirmation-slot and confirmation-rate (TC-1 and TC-2)
2024-03-27 11:45:30 +01:00
galactus 29bd6deae4
Fixing accounts on demand (#371)
* Fixing accounts on demand

* Adding missing file and renaming gprc to grpc

* Adding missing grpc_accounts_streaming
2024-03-26 14:32:50 +01:00
Groovie | Mango cd9df11501
use geyser blockmeta to progress finalizedslot (#367)
introduce BlockInfo on a new BlockInfoStream; maps from yellowstone BlockMeta
fix: last_finalized_slot must progress unconditionally
2024-03-22 19:22:38 +01:00
galactus defdc20dd5
Getting finalized meta before processed main (#366)
* Use jemalloc

* Solving issue of finalized meta after processed block (#365)

* Solving issue of finalized meta after processed block

* Fixing the broken test

* Minor changes

---------

Co-authored-by: Christian Kamm <mail@ckamm.de>
2024-03-22 12:30:14 +01:00
GroovieGermanikus ae495bed53
Merge branch 'production'
using 1.17.15 geyser
2024-03-22 10:57:20 +01:00
galactus 7e6a1a4003
Reverting back the geyser version (#364) 2024-03-22 09:55:46 +01:00
godmodegalactus f219e5eee1
Minor updates to block information store 2024-03-21 16:44:17 +01:00
godmodegalactus 50b81135c3
Merge branch 'production' 2024-03-18 20:01:02 +01:00
godmodegalactus 85fcb6097e
Adding some more metrics for quic tpu, and changing some parameters 2024-03-18 17:25:52 +01:00
godmodegalactus 9be86919ba
Merge branch 'production' 2024-03-16 16:18:01 +01:00
godmodegalactus 9f2f1c44fc
Disable lite rpc restart when rpc is not reponding and we are using grpc 2024-03-16 09:11:34 +01:00
Groovie | Mango af2e41d6ac
fix error log on service issue (#360) 2024-03-15 12:27:23 +01:00
godmodegalactus 374777cabc
minor optimizations, checking blockheight and confirmation status at replay 2024-03-09 19:18:33 +01:00
galactus 1dcd3abcf2
Adding few metrics for accounts, getting accounts arguments from env (#341) 2024-02-28 17:08:27 +01:00
galactus 49bb7ff9f3
Implementing account on demand (#338)
* Implementing account on demand

* Resubscribing accounts after every sometime

* Changing account stream commitment to processed instead

* Fixing account on demand, and account subscription bugs

* Changing variable name for commitment from confirmed to processed

* Some account optimizations

* Adding get balance call

* Changes after groovies review.

* On demand accounts subscribe first then delete old subscription
2024-02-28 15:40:11 +01:00
galactus c25c44eb9a
Solving potential deadlock in accounts library related to get_mut on dashmap (#334) 2024-02-22 12:49:47 +01:00
galactus 0ab947b7d7
Updating jsonrpsee and fixing cors (#332)
* Updating json rpsee and fixing CORS

* Fixing cors and updating jsonrpsee and fixing a bug in address lookup table store

* Adding rpc errors, and fixing groovies comments

* for moment get performance data from rpc client
2024-02-21 14:55:44 +01:00
Groovie | Mango 2c42a536b6
Improve/geyser replace streams (#330)
* add env ENABLE_INSTRUMENT_TRACING to enable tracing

* add tracing dependency

* example tracings

* log recent_processed_blocks.len

* bound channels

* replace fastestwins with task version

* revert fake error

* fix meta stream

* disble almost all

* disable more

* keep slot channel up

* log memory usage using CAP

* log CAP memory usage

* add span tracing to mutliplex tasks

* merge

* fix import

* remove useless spans

* fix span logger

* log emitted data from multiplexer

* improve naming of mpsc from autoconnection task

* WIP: use task/mpsc autoconnect for full blocks

* use task/mpsc autoconnect for block meta too

* tag the channels

* add spans

* bump to use updated geyser-grpc-connector with mpsc

* remove unused functions

* extract constant

* split plugger for one and many downstreams

* clippy+fmt

* use task for slots too

* remove broadcast intermediate channel
2024-02-15 21:49:15 +01:00
galactus bcc3c37346
Accounts support (#327)
* Slitting rpc and pubsub services, adding few class and rpc, pubsub callbacks for accounts

* Implementing account storage and service writing some tests

* Chainging to interface, adding new tests

* Implementing accounts related rpc endpoints

* Adding tests for account filters

* Minor bug fixed and errors

* Fixing issue with get program accounts

* Implementing subscription

* use auto reconnect (#329)

* groovie comments part_1

* Fixing tests, fixing groovies comments

* Adding workspace to test filters, minor changes

---------

Co-authored-by: Groovie | Mango <95291500+grooviegermanikus@users.noreply.github.com>
2024-02-14 19:20:25 +01:00
Groovie | Mango 38da1ed987
Improvement/check block sequencing (#299)
use grpc multiplex and improve sequencing
2024-02-07 15:51:22 +01:00
galactus 196b400e15
Adding support for address lookup tables (#319)
* Initial implementation of address lookup tables

* Implementing fetching of address lookup tables

* Use address lookup tables to calculate account prio fees

* Adding some logs and renaming classes

* preloading all the alts correctly

* Changes after groovies review

* Fixing issues after rebase
2024-02-06 17:59:46 +01:00
Groovie | Mango fae4cecc0e
Improvement/saving blocks use lookup tables (#318)
block store basic implementation
2024-02-05 20:20:15 +01:00
galactus 4d5a21530c
Adding CLI argument to set the max number of connections with the leader (#317)
* Adding CLI argument to set the max number of connections with the leader

* Fixing env variable name
2024-02-05 14:06:24 +01:00
galactus 9e031902e3
Implementing missing endpoints (#316)
* Implementing missing endpoints

* Adding slot subscription, and implementing prio fee method

* Changes after groovie and anikets review
2024-02-02 16:42:42 +01:00
galactus 27b1e6ad16
Calculating prio fees by account (#310)
* Calculating readable and writable accounts for transaction and saving in transaction info, declaration of primary strutures

Implementing calculation of prioritization by accounts

Optimize calculation of prioritization fees stats (#301)

* Optimize calculation of prioritization fees stats

* comment on arrays in response

* Fix the calculation of supp

* Fix fmt

---------

Co-authored-by: GroovieGermanikus <groovie@mango.markets>

Implementing calculation of prioritization by accounts

Adding endpoint methods for prioritization fees by account

* Implement account stats subscription

* Fixing the serialization and deserialization of last

* Changes after groovies review
2024-02-01 18:49:51 +01:00
Groovie | Mango 7171b524f0
Feature/block priofees (#274)
* boilerplate

* add dummy endpoint

* simple logic with highes block

* endpoint works

* remove cu calculus

* log get_latest_block startup timing

* websocket subscription basics

* WIP

* propagate fees via channel

* cleanup

* use broadcast sender

* proper handling of broadcast channel issues

* rename stuff

* move to dedicated crate

* cleanup deps

* move data definition

* hide private types

* percentile math

* code format

* integrated stats by cu

* add cu stats

* restart inline asserts

* warn about DashMap access

* switch to BTreeMap

* remove vote transactions from calculus

* return vec

* array

* flat format

* reformat

* split arrays

* rename keys

* enable experimental tag

* code format

* more logging

* reset log level

* rename websocket method to blockPrioritizationFeesSubscribe

* HACK: use processed blocks

* add is_vote_transaction

* do not fail if fees goes down

* udp message size example

* imrpove 100 handling

* simplify inital sort

* Revert "HACK: use processed blocks"

This reverts commit ff17d9475c.

* add percentile test case

* add per block cu_consumed+tx_count(nonvote)

* fix supid cleanup bug

* add TxAggregateStats

* add by_cu test

* add alternative cu percentlie impl

* cleanup cu tests

* add check for step

* saturating_sub

* remove useless file

* add is_vote to history crate

* fix fmt+clippy (nightly)

* clippy

* clippy

* clippy

* clippy
2024-01-25 10:29:14 +01:00
galactus de26490e9c
Squash and merging all the commits from groovies branch groovie/post… (#300)
* Squash and merging all the commits from  groovies branch groovie/postgres_saving_blocksII

* Ingore tests which require postgres
2024-01-22 15:22:11 +01:00
galactus 1209a8ae4d
Increase finish quic timeout (#280) 2024-01-17 13:10:11 +01:00
galactus 4ed7eead8b
Adding prometheus metrics to follow quic states (#277) 2024-01-16 11:30:40 +01:00
galactus d91e2e9316
disable geyser stakes and votes for refactoring (#268) 2024-01-11 14:37:06 +01:00
musitdev 21d0cee332 set initial rpc schedule parameters 2024-01-08 13:19:23 +01:00
musitdev 032232f6f3 add rpc leader shedule loading if the bootstrap file is not present 2024-01-05 15:58:25 +01:00
musitdev 402ebd69b1 correct from PR remarks 2024-01-04 13:36:13 +01:00
GroovieGermanikus 772e2c9909
reset values for leader schedule cache 2024-01-04 11:12:45 +01:00
musitdev 77853a34ca rm trace comment 2024-01-04 09:56:55 +01:00
musitdev 2e7d86b896 rm trace comment 2024-01-04 09:25:46 +01:00
musitdev dbc152bb64 rebase from master 2024-01-03 16:50:16 +01:00
musitdev 5a24c2bb4d set stake_vote optional and integrate leader schedule and vote in send Tx 2024-01-03 10:25:45 +01:00
musitdev eadff4fce3 set inital schedule parameters 2024-01-03 10:21:07 +01:00
musitdev 93ab67db18 test leader and vote RPC calls 2024-01-03 10:21:06 +01:00
musitdev 69754a019a start get_leader_slot, update for get_vote_account 2024-01-03 10:20:18 +01:00
musitdev 3e4c81c567 Stake and vote management. Integration of the main loop in current architecture 2024-01-03 10:20:14 +01:00
Groovie | Mango 5cfb6de72a
grpc multiplexing integrate (#255)
allow multiple GRPC sources
2023-12-22 14:42:20 +01:00
Aniket Prajapati 3eecb89b81
simple config, env and cli (#247)
* simple config, env and cli

* fix comp

* init env first

* added env override

* postgres config

* fmt

* updated env in readme

* examples

* a
2023-11-27 14:24:59 +05:30
Godmode Galactus e6d2ab897e
Update grpc version to 0.11.0 so that we can use the latest triton service 2023-11-14 10:42:58 +01:00
Arrowana 15ffb44f6e allow x-token 2023-11-07 21:04:17 +11:00
Godmode Galactus e65ef4558b
restoring block polling with semaphore, some more changes related to processed slots and tpu manager 2023-10-25 20:54:19 +02:00
galactus f9dfcf4891
Merge pull request #223 from blockworks-foundation/epoch_integration
Epoch integration
2023-10-04 20:29:57 +02:00