Commit Graph

21 Commits

Author SHA1 Message Date
Richard Patel 4ddc64e2c0 bazel: add Go rules 2022-10-30 08:17:36 +01:00
Richard Patel 582a566dc5 blockstore: don't depend on exp/constraints
Gazelle hates exp/constraints for some inexplicable reason.

> build aborted: no such package '@org_golang_x_exp//constraints': BUILD file not found in directory 'constraints' of external repository @org_golang_x_exp.
2022-10-29 20:18:46 +02:00
Richard Patel 2f5954278c blockstore: simplify BlockWalkI.Entries()
Shredding is already implied by SlotMeta,
thus no need to return shreds and entries in the iterator.
2022-10-29 13:50:25 +02:00
Richard Patel 776c1e88f1 sort Go imports 2022-10-29 13:30:26 +02:00
Richard Patel 9a647f9aee cargen: use blockstore abstraction
Uses the blockstore.BlockWalkI abstraction introduced in 4644c3f
to decouple cargen from RocksDB.
2022-10-28 20:21:32 +00:00
Richard Patel 4644c3fa8e blockstore: split RocksDB-specific code into _rocks.go files
Splitting pure Go and Cgo code into separate Go files
allows other packages to depend on blockstore mocks.

This is useful for writing test scenarios for external
components that consume data from blockstore.
2022-10-28 18:31:15 +00:00
Leo 55c6b113af Make rocksdb optional and document build 2022-09-26 14:07:11 +02:00
Leo 0d5dd26722 radiance: optional build tag for rocksdb commands
Compiling the rocksdb binding isn't as straight-forward
as installing rocksdb headers:

$ go build ./...
# github.com/linxGnu/grocksdb
../../go/pkg/mod/github.com/linx!gnu/grocksdb@v1.7.7/cache.go:84:2: could not determine kind of name for C.rocksdb_lru_cache_options_set_num_shard_bits

Adding an optional build tag to compile radiance with rocksdb
support, otherwise remove the blockstore command. This keeps
everything else working with CGO_ENABLED=0.
2022-09-26 13:29:59 +02:00
Richard Patel bb94b7e6c0
blockstore: let BlockWalk iterate over rooted slots only 2022-09-25 15:43:57 +02:00
Richard Patel 5fb1dfa576
car: refactor into cargen 2022-09-25 15:35:03 +02:00
Richard Patel 6b5feeba9a
ipldgen: add IPLD ledger stream generator 2022-09-25 15:35:03 +02:00
Richard Patel 758ede5690 Move Go module to go.firedancer.io/radiance
Change-Id: I76845d19e864d6c7cd0fff77dbe63fc722bbf808
Reviewed-on: https://git.firedancer.io/c/radiance/+/131
2022-09-21 23:40:06 +00:00
gagliardetto f6a9e60f31
shred: improve DataShredsToEntries performance
* Don't use reflection for shred decoding
* Add shred decoding benchmark
2022-09-19 13:46:29 +02:00
Richard Patel bb6d4ce6f6
shred: skip empty shreds
Encountered in mainnet slot 102763321 data shred index 484
2022-09-18 01:49:01 +02:00
Richard Patel 0f7098a795 shred: support recent legacy shred
Some time along Solana mainnet-beta history,
the legacy data shred header was extended
from 86 bytes to 88 bytes.

We don't know when exactly this change has happened.
Or whether the shred version is self-describing.

For now, add a simple switch by slot number.
2022-09-18 01:21:42 +02:00
Richard Patel 0585231e9f
blockstore: add genesis deshred test 2022-09-17 20:42:34 +02:00
Richard Patel 047585136b
cmd/blockstore: add verify-data tool
Adds tool that scans over the ledger entries in a blockstore.
2022-09-17 18:51:24 +02:00
Richard Patel d7cd878bc9 blockstore: accurately encode entry batches as YAML
Updates blockstore dump YAML schema to reflect that
slices of shreds map to slices of entries.

Adds a new "entry_batches" wrapper list that annotates
each slice of entries with shred range and encoded size.
2022-09-11 08:56:03 +02:00
Richard Patel 3c69cf0eea blockstore: decode mainnet genesis shreds 2022-09-10 19:56:59 +02:00
Richard Patel 62357507ff blockstore: add shreds, slot meta client 2022-09-10 14:30:49 +02:00
Richard Patel 816ab66d28 blockstore: add package, add RocksDB client 2022-09-10 14:30:49 +02:00