Commit Graph

111 Commits

Author SHA1 Message Date
galactus 49bb7ff9f3
Implementing account on demand (#338)
* Implementing account on demand

* Resubscribing accounts after every sometime

* Changing account stream commitment to processed instead

* Fixing account on demand, and account subscription bugs

* Changing variable name for commitment from confirmed to processed

* Some account optimizations

* Adding get balance call

* Changes after groovies review.

* On demand accounts subscribe first then delete old subscription
2024-02-28 15:40:11 +01:00
galactus 064d47719e
Creating an example directory and adding example to be implemented (#340) 2024-02-26 20:43:02 +01:00
Groovie | Mango 855c0190eb
prepare new "util" crate (#333)
prepare new "util" crate
2024-02-22 16:54:42 +01:00
galactus 0ab947b7d7
Updating jsonrpsee and fixing cors (#332)
* Updating json rpsee and fixing CORS

* Fixing cors and updating jsonrpsee and fixing a bug in address lookup table store

* Adding rpc errors, and fixing groovies comments

* for moment get performance data from rpc client
2024-02-21 14:55:44 +01:00
Groovie | Mango 2c42a536b6
Improve/geyser replace streams (#330)
* add env ENABLE_INSTRUMENT_TRACING to enable tracing

* add tracing dependency

* example tracings

* log recent_processed_blocks.len

* bound channels

* replace fastestwins with task version

* revert fake error

* fix meta stream

* disble almost all

* disable more

* keep slot channel up

* log memory usage using CAP

* log CAP memory usage

* add span tracing to mutliplex tasks

* merge

* fix import

* remove useless spans

* fix span logger

* log emitted data from multiplexer

* improve naming of mpsc from autoconnection task

* WIP: use task/mpsc autoconnect for full blocks

* use task/mpsc autoconnect for block meta too

* tag the channels

* add spans

* bump to use updated geyser-grpc-connector with mpsc

* remove unused functions

* extract constant

* split plugger for one and many downstreams

* clippy+fmt

* use task for slots too

* remove broadcast intermediate channel
2024-02-15 21:49:15 +01:00
galactus bcc3c37346
Accounts support (#327)
* Slitting rpc and pubsub services, adding few class and rpc, pubsub callbacks for accounts

* Implementing account storage and service writing some tests

* Chainging to interface, adding new tests

* Implementing accounts related rpc endpoints

* Adding tests for account filters

* Minor bug fixed and errors

* Fixing issue with get program accounts

* Implementing subscription

* use auto reconnect (#329)

* groovie comments part_1

* Fixing tests, fixing groovies comments

* Adding workspace to test filters, minor changes

---------

Co-authored-by: Groovie | Mango <95291500+grooviegermanikus@users.noreply.github.com>
2024-02-14 19:20:25 +01:00
Groovie | Mango 38da1ed987
Improvement/check block sequencing (#299)
use grpc multiplex and improve sequencing
2024-02-07 15:51:22 +01:00
galactus 196b400e15
Adding support for address lookup tables (#319)
* Initial implementation of address lookup tables

* Implementing fetching of address lookup tables

* Use address lookup tables to calculate account prio fees

* Adding some logs and renaming classes

* preloading all the alts correctly

* Changes after groovies review

* Fixing issues after rebase
2024-02-06 17:59:46 +01:00
Groovie | Mango fae4cecc0e
Improvement/saving blocks use lookup tables (#318)
block store basic implementation
2024-02-05 20:20:15 +01:00
galactus 9e031902e3
Implementing missing endpoints (#316)
* Implementing missing endpoints

* Adding slot subscription, and implementing prio fee method

* Changes after groovie and anikets review
2024-02-02 16:42:42 +01:00
galactus 27b1e6ad16
Calculating prio fees by account (#310)
* Calculating readable and writable accounts for transaction and saving in transaction info, declaration of primary strutures

Implementing calculation of prioritization by accounts

Optimize calculation of prioritization fees stats (#301)

* Optimize calculation of prioritization fees stats

* comment on arrays in response

* Fix the calculation of supp

* Fix fmt

---------

Co-authored-by: GroovieGermanikus <groovie@mango.markets>

Implementing calculation of prioritization by accounts

Adding endpoint methods for prioritization fees by account

* Implement account stats subscription

* Fixing the serialization and deserialization of last

* Changes after groovies review
2024-02-01 18:49:51 +01:00
godmodegalactus a44923c334
Updating to version 0.2.4 2024-01-29 13:11:54 +01:00
Groovie | Mango 7171b524f0
Feature/block priofees (#274)
* boilerplate

* add dummy endpoint

* simple logic with highes block

* endpoint works

* remove cu calculus

* log get_latest_block startup timing

* websocket subscription basics

* WIP

* propagate fees via channel

* cleanup

* use broadcast sender

* proper handling of broadcast channel issues

* rename stuff

* move to dedicated crate

* cleanup deps

* move data definition

* hide private types

* percentile math

* code format

* integrated stats by cu

* add cu stats

* restart inline asserts

* warn about DashMap access

* switch to BTreeMap

* remove vote transactions from calculus

* return vec

* array

* flat format

* reformat

* split arrays

* rename keys

* enable experimental tag

* code format

* more logging

* reset log level

* rename websocket method to blockPrioritizationFeesSubscribe

* HACK: use processed blocks

* add is_vote_transaction

* do not fail if fees goes down

* udp message size example

* imrpove 100 handling

* simplify inital sort

* Revert "HACK: use processed blocks"

This reverts commit ff17d9475c.

* add percentile test case

* add per block cu_consumed+tx_count(nonvote)

* fix supid cleanup bug

* add TxAggregateStats

* add by_cu test

* add alternative cu percentlie impl

* cleanup cu tests

* add check for step

* saturating_sub

* remove useless file

* add is_vote to history crate

* fix fmt+clippy (nightly)

* clippy

* clippy

* clippy

* clippy
2024-01-25 10:29:14 +01:00
galactus de26490e9c
Squash and merging all the commits from groovies branch groovie/post… (#300)
* Squash and merging all the commits from  groovies branch groovie/postgres_saving_blocksII

* Ingore tests which require postgres
2024-01-22 15:22:11 +01:00
galactus e00683884a
Update geyser grpc connector commit (#289) 2024-01-18 15:53:02 +01:00
galactus ba77b0b389
Updating cargo file (#287)
* Updating cargo file

* Updating tag for grpc connector
2024-01-18 11:28:35 +01:00
galactus 6f61d89743
Fixing the performance issue with geyser multiplexing (#275) 2024-01-15 20:22:59 +01:00
galactus bea39d8c6a
Migrating to solana v1.17.15 (#273) 2024-01-15 18:43:48 +01:00
galactus d91e2e9316
disable geyser stakes and votes for refactoring (#268) 2024-01-11 14:37:06 +01:00
GroovieGermanikus ef5716932f
fix: panic on geyser close, multiplex bug
https://github.com/blockworks-foundation/geyser-grpc-connector/issues/3
2024-01-07 20:08:20 +01:00
musitdev dbc152bb64 rebase from master 2024-01-03 16:50:16 +01:00
musitdev dc37b865f6 update cargo.toml with workplace 2024-01-03 10:20:18 +01:00
musitdev 3e4c81c567 Stake and vote management. Integration of the main loop in current architecture 2024-01-03 10:20:14 +01:00
Groovie | Mango 5cfb6de72a
grpc multiplexing integrate (#255)
allow multiple GRPC sources
2023-12-22 14:42:20 +01:00
Aniket Prajapati 3eecb89b81
simple config, env and cli (#247)
* simple config, env and cli

* fix comp

* init env first

* added env override

* postgres config

* fmt

* updated env in readme

* examples

* a
2023-11-27 14:24:59 +05:30
Godmode Galactus e6d2ab897e
Update grpc version to 0.11.0 so that we can use the latest triton service 2023-11-14 10:42:58 +01:00
Godmode Galactus e025f91650
Adding some optimization 2023-10-25 10:37:20 +02:00
Steve 12c97b88d1
Merge pull request #228 from grooviegermanikus/groovie/disable-gso-flag
disable gso (generic segmentation offload)
2023-10-13 23:22:50 +02:00
GroovieGermanikus eb0fb9c6f4
cleanup cargo file 2023-10-09 17:45:08 +02:00
Godmode Galactus 14f26534a7
Saving block and transactions in postgres table 2023-10-05 14:52:55 +02:00
GroovieGermanikus d353569dbb
WIP: add DISABLE_GSO env flag 2023-10-04 20:44:52 +02:00
galactus f9dfcf4891
Merge pull request #223 from blockworks-foundation/epoch_integration
Epoch integration
2023-10-04 20:29:57 +02:00
godmodegalactus fc1810cc89
Removing postgres_session and moving it to history, start to implement saving blocks 2023-10-04 15:25:06 +02:00
musitdev c6aba8f396 Epoch integration 2023-10-04 11:28:18 +02:00
GroovieGermanikus e471ba638e bench: add support for large transaction sizes 2023-09-29 09:57:36 +02:00
godmodegalactus 7ef29e1af6
implementing history, block storage method
changes after groovies review
2023-09-21 15:39:25 +02:00
galactus ffc6a15e5b
Merge pull request #177 from blockworks-foundation/quic-proxy-gm-improvements
Quic proxy gm improvements
2023-09-05 09:19:07 +02:00
GroovieGermanikus 5b3c46b8a2 recreated Cargo.lock 2023-09-04 15:02:31 +02:00
GroovieGermanikus 34e2cff6b6 move keypair_loaded to core 2023-09-02 14:59:56 +02:00
Godmode Galactus a418fe7fc7
Refactoring the code from gyser branch and merging into master 2023-08-31 12:34:13 +02:00
GroovieGermanikus 87b8c077d0 cargo.lock 2023-07-29 11:18:20 +02:00
GroovieGermanikus 9e064001f8 extract integration test 2023-07-20 14:08:38 +02:00
GroovieGermanikus 0490e557de merge integration test branch into proxy branch 2023-07-20 13:45:54 +02:00
GroovieGermanikus 7c3f7bea74 refactor out tpu_quic_connection 2023-06-29 11:05:49 +02:00
GroovieGermanikus 87219da3e4 Cargo.lock 2023-06-23 09:01:17 +02:00
Godmode Galactus 1eefe06189
nit, changing the version of lite-rpc to 0.2.1 2023-06-16 14:21:58 +02:00
Godmode Galactus 478e65a93c
Adding more information to notifications 2023-06-13 17:04:16 +02:00
Godmode Galactus 3995d6ee39
removing dependency of jsonrpsee for core and services 2023-06-13 10:58:19 +02:00
Godmode Galactus 9c91c9528a
setting tokio to any for core and services and specifying version for lite-rpc 2023-06-13 10:58:19 +02:00
Godmode Galactus 3c31e25761
Adding licence and not publishing lite-rpc by default 2023-06-10 14:48:22 +02:00