docs: add ADR 055 ORM (#11807)
## Description Adds an ADR for the `orm` module which is already implemented. --- ### Author Checklist *All items are required. Please add a note to the item if the item is not applicable and please add links to any relevant follow up issues.* I have... - [ ] included the correct [type prefix](https://github.com/commitizen/conventional-commit-types/blob/v3.0.0/index.json) in the PR title - [ ] added `!` to the type prefix if API or client breaking change - [ ] targeted the correct branch (see [PR Targeting](https://github.com/cosmos/cosmos-sdk/blob/main/CONTRIBUTING.md#pr-targeting)) - [ ] provided a link to the relevant issue or specification - [ ] followed the guidelines for [building modules](https://github.com/cosmos/cosmos-sdk/blob/main/docs/building-modules) - [ ] included the necessary unit and integration [tests](https://github.com/cosmos/cosmos-sdk/blob/main/CONTRIBUTING.md#testing) - [ ] added a changelog entry to `CHANGELOG.md` - [ ] included comments for [documenting Go code](https://blog.golang.org/godoc) - [ ] updated the relevant documentation or specification - [ ] reviewed "Files changed" and left comments if necessary - [ ] confirmed all CI checks have passed ### Reviewers Checklist *All items are required. Please add a note if the item is not applicable and please add your handle next to the items reviewed if you only reviewed selected items.* I have... - [ ] confirmed the correct [type prefix](https://github.com/commitizen/conventional-commit-types/blob/v3.0.0/index.json) in the PR title - [ ] confirmed `!` in the type prefix if API or client breaking change - [ ] confirmed all author checklist items have been addressed - [ ] reviewed state machine logic - [ ] reviewed API design and naming - [ ] reviewed documentation is accurate - [ ] reviewed tests and test coverage - [ ] manually tested (if applicable)
This commit is contained in:
Aaron Craelius
GitHub
parent
0d846ae2f0
commit
c3edbb22ca
|
|
@ -56,6 +56,7 @@ When writing ADRs, follow the same best practices for writing RFCs. When writing
|
|||
* [ADR 029: Fee Grant Module](./adr-029-fee-grant-module.md)
|
||||
* [ADR 030: Message Authorization Module](./adr-030-authz-module.md)
|
||||
* [ADR 031: Protobuf Msg Services](./adr-031-msg-service.md)
|
||||
* [ADR 055: ORM](./adr-055-orm.md)
|
||||
|
||||
### Proposed
|
||||
|
||||
|
|
|
|||
|
|
@ -0,0 +1,111 @@
|
|||
# ADR 055: ORM
|
||||
|
||||
## Changelog
|
||||
|
||||
* 2022-04-27: First draft
|
||||
|
||||
## Status
|
||||
|
||||
ACCEPTED Implemented
|
||||
|
||||
## Abstract
|
||||
|
||||
In order to make it easier for developers to build Cosmos SDK modules and for clients to query, index and verify proofs
|
||||
against state data, we have implemented an ORM (object-relational mapping) layer for the Cosmos SDK.
|
||||
|
||||
## Context
|
||||
|
||||
Historically modules in the Cosmos SDK have always used the key-value store directly and created various handwritten
|
||||
functions for managing key format as well as constructing secondary indexes. This consumes a significant amount of
|
||||
time when building a module and is error-prone. Because key formats are non-standard, sometimes poorly documented,
|
||||
and subject to change, it is hard for clients to generically index, query and verify merkle proofs against state data.
|
||||
|
||||
The known first instance of an "ORM" in the Cosmos ecosystem was in [weave](https://github.com/iov-one/weave/tree/master/orm).
|
||||
A later version was built for [regen-ledger](https://github.com/regen-network/regen-ledger/tree/157181f955823149e1825263a317ad8e16096da4/orm) for
|
||||
use in the group module and later [ported to the SDK](https://github.com/cosmos/cosmos-sdk/tree/35d3312c3be306591fcba39892223f1244c8d108/x/group/internal/orm)
|
||||
just for that purpose.
|
||||
|
||||
While these earlier designs made it significantly easier to write state machines, they still required a lot of manual
|
||||
configuration, didn't expose state format directly to clients, and were limited in their support of different types
|
||||
of index keys, composite keys, and range queries.
|
||||
|
||||
Discussions about the design continued in https://github.com/cosmos/cosmos-sdk/discussions/9156 and more
|
||||
sophisticated proofs of concept were created in https://github.com/allinbits/cosmos-sdk-poc/tree/master/runtime/orm
|
||||
and https://github.com/cosmos/cosmos-sdk/pull/10454.
|
||||
|
||||
## Decision
|
||||
|
||||
These prior efforts culminated in the creation of the Cosmos SDK `orm` go module which uses protobuf annotations
|
||||
for specifying ORM table definitions. This ORM is based on the new `google.golang.org/protobuf/reflect/protoreflect`
|
||||
API and supports:
|
||||
* sorted indexes for all simple protobuf types (except `bytes`, `enum`, `float`, `double`) as well as `Timestamp` and `Duration`
|
||||
* unsorted `bytes` and `enum` indexes
|
||||
* composite primary and secondary keys
|
||||
* unique indexes
|
||||
* auto-incrementing `uint64` primary keys
|
||||
* complex prefix and range queries
|
||||
* paginated queries
|
||||
* complete logical decoding of KV-store data
|
||||
|
||||
Almost all the information needed to decode state directly is specified in .proto files. Each table definition specifies
|
||||
an ID which is unique per .proto file and each index within a table is unique within that table. Clients then only need
|
||||
to know the name of a module and the prefix ORM data for a specific .proto file within that module in order to decode
|
||||
state data directly. This additional information will be exposed directly through app configs which will be explained
|
||||
in a future ADR related to app wiring.
|
||||
|
||||
The ORM makes optimizations around storage space by not repeating values in the primary key in the key value
|
||||
when storing primary key records. For example, if the object `{"a":0,"b":1}` has the primary key `a`, it will
|
||||
be stored in the key value store as `Key: '0', Value: {"b":1}` (with more efficient protobuf binary encoding).
|
||||
Also, the generated code from https://github.com/cosmos/cosmos-proto does optimizations around the
|
||||
`google.golang.org/protobuf/reflect/protoreflect` API to improve performance.
|
||||
|
||||
A code generator is included with the ORM which creates type safe wrappers around the ORM's dynamic `Table`
|
||||
implementation and is the recommended way for modules to use the ORM.
|
||||
|
||||
The ORM tests provide a simplified bank module demonstration which illustrates:
|
||||
- [ORM proto options](https://github.com/cosmos/cosmos-sdk/blob/0d846ae2f0424b2eb640f6679a703b52d407813d/orm/internal/testpb/bank.proto)
|
||||
- [Generated Code](https://github.com/cosmos/cosmos-sdk/blob/0d846ae2f0424b2eb640f6679a703b52d407813d/orm/internal/testpb/bank.cosmos_orm.go)
|
||||
- [Example Usage in a Module Keeper](https://github.com/cosmos/cosmos-sdk/blob/0d846ae2f0424b2eb640f6679a703b52d407813d/orm/model/ormdb/module_test.go)
|
||||
|
||||
## Consequences
|
||||
|
||||
### Backwards Compatibility
|
||||
|
||||
State machine code that adopts the ORM will need migrations as the state layout is generally backwards incompatible.
|
||||
These state machines will also need to migrate to https://github.com/cosmos/cosmos-proto at least for state data.
|
||||
|
||||
### Positive
|
||||
|
||||
* easier to build modules
|
||||
* easier to add secondary indexes to state
|
||||
* possible to write a generic indexer for ORM state
|
||||
* easier to write clients that do state proofs
|
||||
* possible to automatically write query layers rather than needing to manually implement gRPC queries
|
||||
|
||||
### Negative
|
||||
|
||||
* worse performance than handwritten keys (for now). See [Further Discussions](#further-discussions)
|
||||
for potential improvements
|
||||
|
||||
### Neutral
|
||||
|
||||
## Further Discussions
|
||||
|
||||
Further discussions will happen within the Cosmos SDK Framework Working Group. Current planned and ongoing work includes:
|
||||
* automatically generate client-facing query layer
|
||||
* client-side query libraries that transparently verify light client proofs
|
||||
* index ORM data to SQL databases
|
||||
* improve performance by:
|
||||
* optimizing existing reflection based code to avoid unnecessary gets when doing deletes & updates of simple tables
|
||||
* more sophisticated code generation such as making fast path reflection even faster (avoiding `switch` statements),
|
||||
or even fully generating code that equals handwritten performance
|
||||
|
||||
|
||||
## References
|
||||
|
||||
* https://github.com/iov-one/weave/tree/master/orm).
|
||||
* https://github.com/regen-network/regen-ledger/tree/157181f955823149e1825263a317ad8e16096da4/orm
|
||||
* https://github.com/cosmos/cosmos-sdk/tree/35d3312c3be306591fcba39892223f1244c8d108/x/group/internal/orm
|
||||
* https://github.com/cosmos/cosmos-sdk/discussions/9156
|
||||
* https://github.com/allinbits/cosmos-sdk-poc/tree/master/runtime/orm
|
||||
* https://github.com/cosmos/cosmos-sdk/pull/10454
|
||||
Loading…
Reference in New Issue