Commit Graph

374 Commits

Author SHA1 Message Date
Richard Patel c9db39178f
add .car and RocksDB to gitignore 2022-09-25 15:35:04 +02:00
Richard Patel 2d8b42f33a
cargen: improve godoc 2022-09-25 15:35:04 +02:00
Richard Patel 473624867a
cargen: implement CAR file splitting 2022-09-25 15:35:03 +02:00
Richard Patel 5fb1dfa576
car: refactor into cargen 2022-09-25 15:35:03 +02:00
Richard Patel dd176b872c
car: write blocks, not just transactions 2022-09-25 15:35:03 +02:00
Richard Patel 6b5feeba9a
ipldgen: add IPLD ledger stream generator 2022-09-25 15:35:03 +02:00
Richard Patel 6ce807bb2e
generate.sh: run ipld gengo 2022-09-25 14:25:25 +02:00
Richard Patel d3b2c3e9a5
ipldsch: add ledger IPLD schema 2022-09-25 14:25:03 +02:00
Richard Patel 2aaf87d454
cmd/blockstore: add yaml --roots flag 2022-09-24 18:40:08 +02:00
Richard Patel 34df1a0fd7
car: write transaction blocks 2022-09-24 17:40:57 +02:00
Richard Patel 119cef0fd1 car: add CARv1 writer 2022-09-24 15:01:17 +00:00
Richard Patel 7aea7df6fd car: allow multiplexer to seek to a slot 2022-09-24 13:43:54 +00:00
Richard Patel 31e05eec91 cmd/car: add blockstore multiplexer
Adds iterator that fuses multiple blockstores into a contiguous chain.
2022-09-23 21:04:19 +00:00
Richard Patel b396aea76a remove Dependabot
Moving to Gerrit, which is compatible with Dependabot.

Change-Id: Ieb1e782d9f9aebfad03f9763687eafa686126e82
Reviewed-on: https://git.firedancer.io/c/radiance/+/145
2022-09-22 21:52:51 +00:00
Richard Patel 758ede5690 Move Go module to go.firedancer.io/radiance
Change-Id: I76845d19e864d6c7cd0fff77dbe63fc722bbf808
Reviewed-on: https://git.firedancer.io/c/radiance/+/131
2022-09-21 23:40:06 +00:00
Leo 8bbc0f1a75 Remove Git LFS 2022-09-19 16:21:37 +02:00
gagliardetto f6a9e60f31
shred: improve DataShredsToEntries performance
* Don't use reflection for shred decoding
* Add shred decoding benchmark
2022-09-19 13:46:29 +02:00
dependabot[bot] 869f92e798 build(deps): bump github.com/klauspost/compress from 1.15.9 to 1.15.10
Bumps [github.com/klauspost/compress](https://github.com/klauspost/compress) from 1.15.9 to 1.15.10.
- [Release notes](https://github.com/klauspost/compress/releases)
- [Changelog](https://github.com/klauspost/compress/blob/master/.goreleaser.yml)
- [Commits](https://github.com/klauspost/compress/compare/v1.15.9...v1.15.10)

---
updated-dependencies:
- dependency-name: github.com/klauspost/compress
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-09-19 12:34:27 +02:00
Richard Patel 8db4cbffeb
go mod tidy 2022-09-18 17:07:12 +02:00
Richard Patel 41b9449a99
verify-data: decrease log verbosity of skipped slots 2022-09-18 01:55:51 +02:00
Richard Patel bb6d4ce6f6
shred: skip empty shreds
Encountered in mainnet slot 102763321 data shred index 484
2022-09-18 01:49:01 +02:00
Richard Patel 13bbff84f3
cmd/blockstore/verify-data: add rate gauges 2022-09-18 01:39:02 +02:00
Richard Patel 907e65fb6c fixtures: make shred sort order natural
Sorry for the `git pull` jumpscare!
2022-09-18 01:21:42 +02:00
Richard Patel 0f7098a795 shred: support recent legacy shred
Some time along Solana mainnet-beta history,
the legacy data shred header was extended
from 86 bytes to 88 bytes.

We don't know when exactly this change has happened.
Or whether the shred version is self-describing.

For now, add a simple switch by slot number.
2022-09-18 01:21:42 +02:00
Richard Patel dfd7be865c
cmd/blockstore/verify-data: add --dump-sigs flag
Add flag to print sigs for scientific purposes
2022-09-17 23:36:42 +02:00
dependabot[bot] b2c06c4509 build(deps): bump github.com/twmb/franz-go from 1.6.0 to 1.7.1
Bumps [github.com/twmb/franz-go](https://github.com/twmb/franz-go) from 1.6.0 to 1.7.1.
- [Release notes](https://github.com/twmb/franz-go/releases)
- [Changelog](https://github.com/twmb/franz-go/blob/master/CHANGELOG.md)
- [Commits](https://github.com/twmb/franz-go/compare/v1.6.0...v1.7.1)

---
updated-dependencies:
- dependency-name: github.com/twmb/franz-go
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-09-17 22:40:03 +02:00
dependabot[bot] 85a3cdde16 build(deps): bump github.com/coreos/go-systemd/v22 from 22.3.2 to 22.4.0
Bumps [github.com/coreos/go-systemd/v22](https://github.com/coreos/go-systemd) from 22.3.2 to 22.4.0.
- [Release notes](https://github.com/coreos/go-systemd/releases)
- [Commits](https://github.com/coreos/go-systemd/compare/v22.3.2...v22.4.0)

---
updated-dependencies:
- dependency-name: github.com/coreos/go-systemd/v22
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-09-17 22:39:48 +02:00
Richard Patel 0585231e9f
blockstore: add genesis deshred test 2022-09-17 20:42:34 +02:00
Richard Patel 64f4cbec93
cmd/blockstore/verify-data: log txn and byte count 2022-09-17 20:26:16 +02:00
Richard Patel 7eae6ce5ad
cmd/blockstore/verify-data: fix result output in tty mode
Fixes bug where results don't get printed
after progress bar finishes
2022-09-17 20:15:07 +02:00
Richard Patel 3405d76a1a
cmd/blockstore/verify-data: add progress bar, bug fixes
Adds fancy progress bar :3

Fixes deadlock bug
Fixes bug where workers read past their task end
2022-09-17 20:11:00 +02:00
Richard Patel 493c87d0da
cmd/blockstore: fix minor RocksDB iterator leak
Closes iterators after finishing.
Doesn't matter anyways because app exit drops the entire DB handle.
2022-09-17 18:54:48 +02:00
Richard Patel 047585136b
cmd/blockstore: add verify-data tool
Adds tool that scans over the ledger entries in a blockstore.
2022-09-17 18:51:24 +02:00
Richard Patel b1052fb69a
genesis: support genesis.bin
Adds handwritten serde for mainnet genesis config.
(Binary doesn't properly support bincode-fixedint-le)
2022-09-17 16:47:47 +02:00
Richard Patel fd64bc1d6a
runtime: add core ledger data structures
Adds structs that appear in the GenesisConfig object

Adds handwritten serde for runtime structs because
binary doesn't properly support bincode-fixedint-le
2022-09-17 16:46:26 +02:00
Richard Patel 07c1b166a4
fixtures: add Open() function
Adds a method for streaming fixture files
2022-09-17 16:43:19 +02:00
Richard Patel 452d388b5c
archiveutil: add OpenTar() for `.tar`, `.tar.gz`, `.tar.bz2`
Adds utility function that reads tar streams
with optional compression handling.
2022-09-17 16:39:47 +02:00
Richard Patel c78470ade9
fixtures: add mainnet genesis 2022-09-17 16:27:42 +02:00
Leo e7ea00c4dc Be a little more explicit about lack of warranties 2022-09-15 20:30:36 +02:00
Leo 6e85173a43 Make README radiant again 2022-09-15 20:25:20 +02:00
Pires 684184f1ee gitignore: add missing newline to EOF 2022-09-15 09:45:53 -05:00
Pires ffd44286a8 pkg/util: Schroeder wants testify/assert
This will come in handy for sure very soon
2022-09-15 09:45:53 -05:00
Pires 1c6b3857a0 cmd/solrays: remove pubsub RPC from allow list 2022-09-15 09:45:53 -05:00
Pires 200fb932b7 readme: normalize Markdown headers syntax 2022-09-15 09:45:53 -05:00
Pires 6db3220d26 cmd/solrays: move IsValidHostname to pkg/util 2022-09-15 09:45:53 -05:00
Pires 9b3f35d416 cmd/solrays: validate provided TLS hostname, if any. 2022-09-15 09:45:53 -05:00
Pires ff647037a8 cmd/solrays: remove bogus comment
`solrays` doesn't support multiple backends, yet. And I would argue that
if/when it does, the health of the proxy shouldn't be tied to the pool of RPC
nodes.
2022-09-15 09:45:53 -05:00
Pires 0a8d064997 docs: introduce generic build instructions
Also, minimally document `solrays`.
2022-09-15 09:45:53 -05:00
Pires 16a24848e6 cmd/solrays: revisit supported RPC list 2022-09-15 09:45:53 -05:00
Pires a54c60d889 cmd/solrays: fix go-systemd/v22 import 2022-09-15 09:45:53 -05:00