Commit Graph

98 Commits

Author SHA1 Message Date
Richard Patel 2742cd8867 fix Go tests
- merkletree: fix broken test case
- fixtures: fix compatibility with Bazel
2022-10-30 08:19:30 +01:00
Richard Patel 4ddc64e2c0 bazel: add Go rules 2022-10-30 08:17:36 +01:00
Richard Patel ba4e9d4b0d radiance: add car dump command
Adds command to dump the first signature of all txs in a CAR.
2022-10-29 15:00:50 +02:00
Richard Patel 2f5954278c blockstore: simplify BlockWalkI.Entries()
Shredding is already implied by SlotMeta,
thus no need to return shreds and entries in the iterator.
2022-10-29 13:50:25 +02:00
Richard Patel 776c1e88f1 sort Go imports 2022-10-29 13:30:26 +02:00
Richard Patel 8e6cbec8d2 radiance: only build car create if RocksDB enabled 2022-10-28 20:24:46 +00:00
Richard Patel dc8246dcfd poh: move package from runtime/poh to poh, add tests 2022-10-20 01:02:47 +00:00
Richard Patel 5d7f351484 replay: mainnet epoch 0 PoH 🏁
- Implements transaction mixin
- Replays fine for a few thousand slots so far
2022-10-19 01:44:30 +00:00
Richard Patel 14594ded1b
replay: mainnet block 0 🏁 2022-10-18 16:48:39 -05:00
Leo 55c6b113af Make rocksdb optional and document build 2022-09-26 14:07:11 +02:00
Leo 0d5dd26722 radiance: optional build tag for rocksdb commands
Compiling the rocksdb binding isn't as straight-forward
as installing rocksdb headers:

$ go build ./...
# github.com/linxGnu/grocksdb
../../go/pkg/mod/github.com/linx!gnu/grocksdb@v1.7.7/cache.go:84:2: could not determine kind of name for C.rocksdb_lru_cache_options_set_num_shard_bits

Adding an optional build tag to compile radiance with rocksdb
support, otherwise remove the blockstore command. This keeps
everything else working with CGO_ENABLED=0.
2022-09-26 13:29:59 +02:00
Leo a7ce08b6fe Unfork quic-go and update it for Go 1.19 compat
Solana is now properly doing ALPN, so we no longer need the fork.
2022-09-26 13:27:23 +02:00
Richard Patel 480c66c744
cmd/car: add profiling support 2022-09-26 00:32:04 +02:00
Richard Patel 2d8b42f33a
cargen: improve godoc 2022-09-25 15:35:04 +02:00
Richard Patel 5fb1dfa576
car: refactor into cargen 2022-09-25 15:35:03 +02:00
Richard Patel dd176b872c
car: write blocks, not just transactions 2022-09-25 15:35:03 +02:00
Richard Patel 6b5feeba9a
ipldgen: add IPLD ledger stream generator 2022-09-25 15:35:03 +02:00
Richard Patel 2aaf87d454
cmd/blockstore: add yaml --roots flag 2022-09-24 18:40:08 +02:00
Richard Patel 34df1a0fd7
car: write transaction blocks 2022-09-24 17:40:57 +02:00
Richard Patel 119cef0fd1 car: add CARv1 writer 2022-09-24 15:01:17 +00:00
Richard Patel 7aea7df6fd car: allow multiplexer to seek to a slot 2022-09-24 13:43:54 +00:00
Richard Patel 31e05eec91 cmd/car: add blockstore multiplexer
Adds iterator that fuses multiple blockstores into a contiguous chain.
2022-09-23 21:04:19 +00:00
Richard Patel 758ede5690 Move Go module to go.firedancer.io/radiance
Change-Id: I76845d19e864d6c7cd0fff77dbe63fc722bbf808
Reviewed-on: https://git.firedancer.io/c/radiance/+/131
2022-09-21 23:40:06 +00:00
gagliardetto f6a9e60f31
shred: improve DataShredsToEntries performance
* Don't use reflection for shred decoding
* Add shred decoding benchmark
2022-09-19 13:46:29 +02:00
Richard Patel 41b9449a99
verify-data: decrease log verbosity of skipped slots 2022-09-18 01:55:51 +02:00
Richard Patel 13bbff84f3
cmd/blockstore/verify-data: add rate gauges 2022-09-18 01:39:02 +02:00
Richard Patel 907e65fb6c fixtures: make shred sort order natural
Sorry for the `git pull` jumpscare!
2022-09-18 01:21:42 +02:00
Richard Patel 0f7098a795 shred: support recent legacy shred
Some time along Solana mainnet-beta history,
the legacy data shred header was extended
from 86 bytes to 88 bytes.

We don't know when exactly this change has happened.
Or whether the shred version is self-describing.

For now, add a simple switch by slot number.
2022-09-18 01:21:42 +02:00
Richard Patel dfd7be865c
cmd/blockstore/verify-data: add --dump-sigs flag
Add flag to print sigs for scientific purposes
2022-09-17 23:36:42 +02:00
Richard Patel 64f4cbec93
cmd/blockstore/verify-data: log txn and byte count 2022-09-17 20:26:16 +02:00
Richard Patel 7eae6ce5ad
cmd/blockstore/verify-data: fix result output in tty mode
Fixes bug where results don't get printed
after progress bar finishes
2022-09-17 20:15:07 +02:00
Richard Patel 3405d76a1a
cmd/blockstore/verify-data: add progress bar, bug fixes
Adds fancy progress bar :3

Fixes deadlock bug
Fixes bug where workers read past their task end
2022-09-17 20:11:00 +02:00
Richard Patel 493c87d0da
cmd/blockstore: fix minor RocksDB iterator leak
Closes iterators after finishing.
Doesn't matter anyways because app exit drops the entire DB handle.
2022-09-17 18:54:48 +02:00
Richard Patel 047585136b
cmd/blockstore: add verify-data tool
Adds tool that scans over the ledger entries in a blockstore.
2022-09-17 18:51:24 +02:00
Pires 1c6b3857a0 cmd/solrays: remove pubsub RPC from allow list 2022-09-15 09:45:53 -05:00
Pires 200fb932b7 readme: normalize Markdown headers syntax 2022-09-15 09:45:53 -05:00
Pires 6db3220d26 cmd/solrays: move IsValidHostname to pkg/util 2022-09-15 09:45:53 -05:00
Pires 9b3f35d416 cmd/solrays: validate provided TLS hostname, if any. 2022-09-15 09:45:53 -05:00
Pires ff647037a8 cmd/solrays: remove bogus comment
`solrays` doesn't support multiple backends, yet. And I would argue that
if/when it does, the health of the proxy shouldn't be tied to the pool of RPC
nodes.
2022-09-15 09:45:53 -05:00
Pires 0a8d064997 docs: introduce generic build instructions
Also, minimally document `solrays`.
2022-09-15 09:45:53 -05:00
Pires 16a24848e6 cmd/solrays: revisit supported RPC list 2022-09-15 09:45:53 -05:00
Pires a54c60d889 cmd/solrays: fix go-systemd/v22 import 2022-09-15 09:45:53 -05:00
Richard Patel 3e87b452bd cmd/blockstore/yaml: print first and last slot 2022-09-12 14:04:07 +02:00
Leo 22cd485d9c cmd/solrays: import from internal repo 2022-09-11 21:06:23 +02:00
Richard Patel 0773c47d05 cmd/blockstore/yaml: accept --slots=all
Adds option to dump all slots
2022-09-11 09:07:16 +02:00
Richard Patel d7cd878bc9 blockstore: accurately encode entry batches as YAML
Updates blockstore dump YAML schema to reflect that
slices of shreds map to slices of entries.

Adds a new "entry_batches" wrapper list that annotates
each slice of entries with shred range and encoded size.
2022-09-11 08:56:03 +02:00
Richard Patel b902a03a05 cmd/radiance: add blockstore yaml command 2022-09-10 19:57:10 +02:00
Richard Patel 3c69cf0eea blockstore: decode mainnet genesis shreds 2022-09-10 19:56:59 +02:00
Richard Patel 02237cd4c5 cmd/radiance: add blockstore dumpshreds command 2022-09-10 14:30:49 +02:00
Richard Patel be154eae0f cmd: add IntRange parser util 2022-09-10 14:29:37 +02:00