Commit Graph

374 Commits

Author SHA1 Message Date
Richard Patel 2742cd8867 fix Go tests
- merkletree: fix broken test case
- fixtures: fix compatibility with Bazel
2022-10-30 08:19:30 +01:00
Richard Patel 4ddc64e2c0 bazel: add Go rules 2022-10-30 08:17:36 +01:00
Richard Patel 4976cfc518 bazel: build RocksDB with CMake
Also build grocksdb with `go_library`
2022-10-29 21:36:17 +02:00
Richard Patel 02d697c47e third_party: add Gazelle-generated Go module rules 2022-10-29 20:19:41 +02:00
Richard Patel bcf9eb553b genesis: fix test 2022-10-29 20:18:46 +02:00
Richard Patel 352517d3bb go.mod: upgrade golang.org packages 2022-10-29 20:18:46 +02:00
Richard Patel 582a566dc5 blockstore: don't depend on exp/constraints
Gazelle hates exp/constraints for some inexplicable reason.

> build aborted: no such package '@org_golang_x_exp//constraints': BUILD file not found in directory 'constraints' of external repository @org_golang_x_exp.
2022-10-29 20:18:46 +02:00
Richard Patel 98a105cbe0 third_party: add gflags v2.2.2, rocksdb v7.7.3 bazel repos 2022-10-29 20:18:46 +02:00
Richard Patel c38d2684df third_party: remove rocksdb submodule 2022-10-29 20:18:46 +02:00
Richard Patel 7be4ce9016 bazel: add rules_foreign_cc 0.9.0 2022-10-29 20:18:45 +02:00
Richard Patel bea73d4542 gitignore: add Bazel 2022-10-29 20:18:45 +02:00
Richard Patel 861d2a4aab go mod tidy 2022-10-29 20:18:45 +02:00
Richard Patel c78ee582ce bazel: add initial Gazelle integration
Gazelle is a build file generator for Go modules.
2022-10-29 20:18:45 +02:00
Richard Patel c6bbf95022 bazel: add Protobuf v3.21.9 2022-10-29 15:57:26 +02:00
Richard Patel 0e68fe6661 bazel: add Bazel v5.3.2, rules_go v0.35.0 with Go v1.19.2, Gazelle v0.28.0 2022-10-29 15:56:41 +02:00
Richard Patel ba4e9d4b0d radiance: add car dump command
Adds command to dump the first signature of all txs in a CAR.
2022-10-29 15:00:50 +02:00
Richard Patel fb579ba6fb cargen: add unit test
- Adds unit test running cargen over a fake blockstore.
- Covers the CAR splitting code path.
- Ensures resulting CARs decode and contain expected CIDs.
2022-10-29 14:35:09 +02:00
Richard Patel 56a7065d79 ipldgen: fix shredding serialization
Fixes regression in d4c1dfe1
2022-10-29 14:30:12 +02:00
Richard Patel a6f33dd77d cargen: bug fixes
- Fix last CAR not being flushed
- Fix wrong epoch number in file name
- Print error message in edge case where block exceeds CAR length (splitting fails)
2022-10-29 14:24:57 +02:00
Richard Patel 2f5954278c blockstore: simplify BlockWalkI.Entries()
Shredding is already implied by SlotMeta,
thus no need to return shreds and entries in the iterator.
2022-10-29 13:50:25 +02:00
Richard Patel 776c1e88f1 sort Go imports 2022-10-29 13:30:26 +02:00
Richard Patel b8c9810620 cargen: default to 32GB sectors, make it configurable 2022-10-29 13:29:19 +02:00
Richard Patel aafccecfcc car: add parsing unit test 2022-10-29 13:22:03 +02:00
Richard Patel 1f38306312 car: fix leb128 length estimation 2022-10-29 12:18:26 +02:00
Richard Patel d4c1dfe18c ipld: optimize entry schema
- use tuple CBOR representation
- use efficient entry/shred mapping
- remove RadianceLedger placeholder multicodec type
2022-10-28 20:51:11 +00:00
Richard Patel 2f1f467b3b generate.sh: fix go-ipldtool installation instructions 2022-10-28 20:33:08 +00:00
Richard Patel 77db1c0074 gitignore: add radiance binary 2022-10-28 20:32:38 +00:00
Richard Patel 8e6cbec8d2 radiance: only build car create if RocksDB enabled 2022-10-28 20:24:46 +00:00
Richard Patel 9a647f9aee cargen: use blockstore abstraction
Uses the blockstore.BlockWalkI abstraction introduced in 4644c3f
to decouple cargen from RocksDB.
2022-10-28 20:21:32 +00:00
Richard Patel 4644c3fa8e blockstore: split RocksDB-specific code into _rocks.go files
Splitting pure Go and Cgo code into separate Go files
allows other packages to depend on blockstore mocks.

This is useful for writing test scenarios for external
components that consume data from blockstore.
2022-10-28 18:31:15 +00:00
Richard Patel a739dd5879 poh: add mainnet block 0 and 1 tests 2022-10-20 01:02:47 +00:00
Richard Patel 724d252f6b go mod tidy 2022-10-20 01:02:47 +00:00
Richard Patel dc8246dcfd poh: move package from runtime/poh to poh, add tests 2022-10-20 01:02:47 +00:00
Richard Patel 5d7f351484 replay: mainnet epoch 0 PoH 🏁
- Implements transaction mixin
- Replays fine for a few thousand slots so far
2022-10-19 01:44:30 +00:00
Richard Patel 7b97065d5a go.mod: pin solana-go to fork with vote decoding 2022-10-19 01:41:52 +00:00
Richard Patel 53bbce1304 add merkletree package
Ports the construction algorithm of solana-merkle-tree.
2022-10-18 23:23:09 +00:00
Richard Patel 14594ded1b
replay: mainnet block 0 🏁 2022-10-18 16:48:39 -05:00
Richard Patel 0eeb85b874 add runtime package 2022-10-16 04:10:12 +00:00
Richard Patel 8278a5dc00
cargen: open as O_RDWR
Copying files between CARs needs read/write-file mode.
Embarrassing bug. Adding unit tests next time.
2022-09-28 23:01:56 +02:00
Richard Patel f9acf4f518
cargen: fix cache bug while splitting CARs
When splitHandle moves data from old to new CAR,
it fails to flush the write cache of the old car,
and seeks past the end of the file.

Regression introduced by a319031b
2022-09-26 16:44:19 +02:00
Leo 28a1e298f0 README.md: well, technically 2022-09-26 14:10:36 +02:00
Leo 55c6b113af Make rocksdb optional and document build 2022-09-26 14:07:11 +02:00
Leo 0d5dd26722 radiance: optional build tag for rocksdb commands
Compiling the rocksdb binding isn't as straight-forward
as installing rocksdb headers:

$ go build ./...
# github.com/linxGnu/grocksdb
../../go/pkg/mod/github.com/linx!gnu/grocksdb@v1.7.7/cache.go:84:2: could not determine kind of name for C.rocksdb_lru_cache_options_set_num_shard_bits

Adding an optional build tag to compile radiance with rocksdb
support, otherwise remove the blockstore command. This keeps
everything else working with CGO_ENABLED=0.
2022-09-26 13:29:59 +02:00
Leo a7ce08b6fe Unfork quic-go and update it for Go 1.19 compat
Solana is now properly doing ALPN, so we no longer need the fork.
2022-09-26 13:27:23 +02:00
Leo 7fefc225f6 proto: check in generated code
Add the protos as well to make them "go get"-able
by downstream projects.
2022-09-26 13:10:56 +02:00
Linus Kendall 8a86188bb0
Radiance requires golang 1.9 (#39)
Uses atomic types introduced in 1.9.
2022-09-26 13:05:05 +02:00
Richard Patel df79377116
README: document radiance tool 2022-09-26 00:54:45 +02:00
Richard Patel a319031bd8
cargen: use buffered writer (+120% perf) 2022-09-26 00:41:26 +02:00
Richard Patel 480c66c744
cmd/car: add profiling support 2022-09-26 00:32:04 +02:00
Richard Patel bb94b7e6c0
blockstore: let BlockWalk iterate over rooted slots only 2022-09-25 15:43:57 +02:00