Merge PR #5894: ADR 021 - Protocol Buffer Query Support
This commit is contained in:
Aaron Craelius
GitHub
parent
cc2a79ce12
commit
e8a82f78ca
|
|
@ -46,3 +46,4 @@ Please add a entry below in your Pull Request for an ADR.
|
|||
- [ADR 018: Extendable Voting Periods](./adr-018-extendable-voting-period.md)
|
||||
- [ADR 019: Protocol Buffer State Encoding](./adr-019-protobuf-state-encoding.md)
|
||||
- [ADR 020: Protocol Buffer Transaction Encoding](./adr-020-protobuf-transaction-encoding.md)
|
||||
- [ADR 021: Protocol Buffer Query Encoding](./adr-021-protobuf-query-encoding.md)
|
||||
|
|
|
|||
|
|
@ -0,0 +1,258 @@
|
|||
# ADR 021: Protocol Buffer Query Encoding
|
||||
|
||||
## Changelog
|
||||
|
||||
- 2020 March 27: Initial Draft
|
||||
|
||||
## Status
|
||||
|
||||
Proposed
|
||||
|
||||
## Context
|
||||
|
||||
This ADR is a continuation of the motivation, design, and context established in
|
||||
[ADR 019](./adr-019-protobuf-state-encoding.md) and
|
||||
[ARD 020](./adr-019-protobuf-transaction-encoding.md), namely, we aim to design the
|
||||
Protocol Buffer migration path for the client-side of the Cosmos SDK.
|
||||
|
||||
This ADR continues from [ARD 020](./adr-020-protobuf-transaction-encoding.md)
|
||||
to specify the encoding of queries.
|
||||
|
||||
## Decision
|
||||
|
||||
### Custom Query Definition
|
||||
|
||||
Modules define custom queries through a protocol buffers `service` definition.
|
||||
These `service` definitions are generally associated with and used by the
|
||||
GRPC protocol. However, the protocol buffers specification indicates that
|
||||
they can be used more generically by any request/response protocol that uses
|
||||
protocol buffer encoding. Thus, we can use `service` definitions for specifying
|
||||
custom ABCI queries and even reuse a substantial amount of the GRPC infrastructure.
|
||||
|
||||
Each module with custom queries should define a service canonically named `Query`:
|
||||
|
||||
```proto
|
||||
// x/bank/types/types.proto
|
||||
|
||||
service Query {
|
||||
rpc QueryBalance(QueryBalanceParams) returns (cosmos_sdk.v1.Coin) { }
|
||||
rpc QueryAllBalances(QueryAllBalancesParams) returns (QueryAllBalancesResponse) { }
|
||||
}
|
||||
```
|
||||
|
||||
#### Handling of Interface Types
|
||||
|
||||
Modules that use interface types and need true polymorphism generally force a
|
||||
`oneof` up to the app-level that provides the set of concrete implementations of
|
||||
that interface that the app supports. While app's are welcome to do the same for
|
||||
queries and implement an app-level query service, it is recommended that modules
|
||||
provide query methods that expose these interfaces via `google.protobuf.Any`.
|
||||
There is a concern on the transaction level that the overhead of `Any` is too
|
||||
high to justify its usage. However for queries this is not a concern, and
|
||||
providing generic module-level queries that use `Any` does not preclude apps
|
||||
from also providing app-level queries that return use the app-level `oneof`s.
|
||||
|
||||
|
||||
A hypothetical example for the `gov` module would look something like:
|
||||
|
||||
```proto
|
||||
// x/gov/types/types.proto
|
||||
|
||||
import "google/protobuf/any.proto";
|
||||
|
||||
service Query {
|
||||
rpc GetProposal(GetProposalParams) returns (AnyProposal) { }
|
||||
}
|
||||
|
||||
message AnyProposal {
|
||||
ProposalBase base = 1;
|
||||
google.protobuf.Any content = 2;
|
||||
}
|
||||
```
|
||||
|
||||
### Custom Query Implementation
|
||||
|
||||
In order to implement the query service, we can reuse the existing [gogo protobuf](https://github.com/gogo/protobuf)
|
||||
grpc plugin, which for a service named `Query` generates an interface named
|
||||
`QueryServer` as below:
|
||||
|
||||
```go
|
||||
type QueryServer interface {
|
||||
QueryBalance(context.Context, *QueryBalanceParams) (*types.Coin, error)
|
||||
QueryAllBalances(context.Context, *QueryAllBalancesParams) (*QueryAllBalancesResponse, error)
|
||||
}
|
||||
```
|
||||
|
||||
The custom queries for our module are implemented by implementing this interface.
|
||||
|
||||
The first parameter in this generated interface is a generic `context.Context`,
|
||||
whereas querier methods generally need an instance of `sdk.Context` to read
|
||||
from the store. Since arbitrary values can be attached to `context.Context`
|
||||
using the `WithValue` and `Value` methods, the SDK should provide a function
|
||||
`sdk.UnwrapSDKContext` to retrieve the `sdk.Context` from the provided
|
||||
`context.Context`.
|
||||
|
||||
An example implementation of `QueryBalance` for the bank module as above would
|
||||
look something like:
|
||||
|
||||
```go
|
||||
type Querier struct {
|
||||
Keeper
|
||||
}
|
||||
|
||||
func (q Querier) QueryBalance(ctx context.Context, params *types.QueryBalanceParams) (*sdk.Coin, error) {
|
||||
balance := q.GetBalance(sdk.UnwrapSDKContext(ctx), params.Address, params.Denom)
|
||||
return &balance, nil
|
||||
}
|
||||
```
|
||||
|
||||
### Custom Query Registration and Routing
|
||||
|
||||
Query server implementations as above would be registered with `AppModule`s using
|
||||
a new method `RegisterQueryServer(grpc.Server)` which could be implemented simply
|
||||
as below:
|
||||
|
||||
```go
|
||||
// x/bank/module.go
|
||||
func (am AppModule) RegisterQueryServer(server grpc.Server) {
|
||||
types.RegisterQueryServer(server, keeper.Querier{am.keeper})
|
||||
}
|
||||
```
|
||||
|
||||
Underneath the hood, a new method `RegisterService(sd *grpc.ServiceDesc, handler interface{})`
|
||||
will be added to the existing `baseapp.QueryRouter` to add the queries to the custom
|
||||
query routing table (with the routing method being described below).
|
||||
The signature for this method matches the existing
|
||||
`RegisterServer` method on the GRPC `Server` type where `handler` is the custom
|
||||
query server implementation described above.
|
||||
|
||||
|
||||
GRPC-like requests are routed by the service name (ex. `cosmos_sdk.x.bank.v1.Query`)
|
||||
and method name (ex. `QueryBalance`) combined with `/`s to form a full
|
||||
method name (ex. `/cosmos_sdk.x.bank.v1.Query/QueryBalance`). This gets translated
|
||||
into an ABCI query as `custom/cosmos_sdk.x.bank.v1.Query/QueryBalance`. Service handlers
|
||||
registered with `QueryRouter.RegisterService` will be routed this way.
|
||||
|
||||
Beyond the method name, GRPC requests carry a protobuf encoded payload, which maps naturally
|
||||
to `RequestQuery.Data`, and receive a protobuf encoded response or error. Thus
|
||||
there is a quite natural mapping of GRPC-like rpc methods to the existing
|
||||
`sdk.Query` and `QueryRouter` infrastructure.
|
||||
|
||||
This basic specification allows us to reuse protocol buffer `service` definitions
|
||||
for ABCI custom queries substantially reducing the need for manual decoding and
|
||||
encoding in query methods.
|
||||
|
||||
### GRPC Protocol Support
|
||||
|
||||
In addition to providing an ABCI query pathway, we can easily provide a GRPC
|
||||
proxy server that routes requests in the GRPC protocol to ABCI query requests
|
||||
under the hood. In this way, clients could use their host languages' existing
|
||||
GRPC implementations to make direct queries against Cosmos SDK app's using
|
||||
these `service` definitions. In order for this server to work, the `QueryRouter`
|
||||
on `BaseApp` will need to expose the service handlers registered with
|
||||
`QueryRouter.RegisterService` to the proxy server implementation. Nodes could
|
||||
launch the proxy server on a separate port in the same process as the ABCI app
|
||||
with a command-line flag.
|
||||
|
||||
### REST Queries and Swagger Generation
|
||||
|
||||
[grpc-gateway](https://github.com/grpc-ecosystem/grpc-gateway) is a project that
|
||||
translates REST calls into GRPC calls using special annotations on service
|
||||
methods. Modules that want to expose REST queries should add `google.api.http`
|
||||
annotations to their `rpc` methods as in this example below.
|
||||
|
||||
```proto
|
||||
// x/bank/types/types.proto
|
||||
|
||||
service Query {
|
||||
rpc QueryBalance(QueryBalanceParams) returns (cosmos_sdk.v1.Coin) {
|
||||
option (google.api.http) = {
|
||||
get: "/x/bank/v1/balance/{address}/{denom}"
|
||||
};
|
||||
}
|
||||
rpc QueryAllBalances(QueryAllBalancesParams) returns (QueryAllBalancesResponse) {
|
||||
option (google.api.http) = {
|
||||
get: "/x/bank/v1/balances/{address}"
|
||||
};
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
grpc-gateway will work direcly against the GRPC proxy described above which will
|
||||
translate requests to ABCI queries under the hood. grpc-gateway can also
|
||||
generate Swagger definitions automatically.
|
||||
|
||||
In the current implementation of REST queries, each module needs to implement
|
||||
REST queries manually in addition to ABCI querier methods. Using the grpc-gateway
|
||||
approach, there will be no need to generate separate REST query handlers, just
|
||||
query servers as described above as grpc-gateway handles the translation of protobuf
|
||||
to REST as well as Swagger definitions.
|
||||
|
||||
The SDK should provide CLI commands for apps to start GRPC gateway either in
|
||||
a separate process or the same process as the ABCI app, as well as provide a
|
||||
command for generating grpc-gateway proxy `.proto` files and the `swagger.json`
|
||||
file.
|
||||
|
||||
### Client Usage
|
||||
|
||||
The gogo protobuf grpc plugin generates client interfaces in addition to server
|
||||
interfaces. For the `Query` service defined above we would get a `QueryClient`
|
||||
interface like:
|
||||
|
||||
```go
|
||||
type QueryClient interface {
|
||||
QueryBalance(ctx context.Context, in *QueryBalanceParams, opts ...grpc.CallOption) (*types.Coin, error)
|
||||
QueryAllBalances(ctx context.Context, in *QueryAllBalancesParams, opts ...grpc.CallOption) (*QueryAllBalancesResponse, error)
|
||||
}
|
||||
```
|
||||
|
||||
Via a small patch to gogo protobuf ([gogo/protobuf#675](https://github.com/gogo/protobuf/pull/675))
|
||||
we have tweaked the grpc codegen to use an interface rather than concrete type
|
||||
for the generated client struct. This allows us to also reuse the GRPC infrastructure
|
||||
for ABCI client queries.
|
||||
|
||||
`CLIContext` will receive a new method `QueryConn` that returns a `ClientConn`
|
||||
that routes calls to ABCI queries
|
||||
|
||||
Clients (such as CLI methods) will then be able to call query methods like this:
|
||||
|
||||
```go
|
||||
cliCtx := context.NewCLIContext()
|
||||
queryClient := types.NewQueryClient(cliCtx.QueryConn())
|
||||
params := &types.QueryBalanceParams{addr, denom}
|
||||
result, err := queryClient.QueryBalance(gocontext.Background(), params)
|
||||
```
|
||||
|
||||
### Testing
|
||||
|
||||
Tests would be able to create a query client directly from keeper and `sdk.Context`
|
||||
references using a `QueryServerTestHelper` as below:
|
||||
|
||||
```go
|
||||
queryHelper := baseapp.NewQueryServerTestHelper(ctx)
|
||||
types.RegisterQueryServer(queryHelper, keeper.Querier{app.BankKeeper})
|
||||
queryClient := types.NewQueryClient(queryHelper)
|
||||
```
|
||||
|
||||
## Future Improvements
|
||||
|
||||
## Consequences
|
||||
|
||||
### Positive
|
||||
|
||||
* greatly simplified querier implementation (no manual encoding/decoding)
|
||||
* easy query client generation (can use existing grpc and swagger tools)
|
||||
* no need for REST query implementations
|
||||
* type safe query methods (generated via grpc plugin)
|
||||
* going forward, there will be less breakage of query methods because of the
|
||||
backwards compatibility guarantees provided by buf
|
||||
|
||||
### Negative
|
||||
|
||||
* all clients using the existing ABCI/REST queries will need to be refactored
|
||||
for both the new GRPC/REST query paths as well as protobuf/proto-json encoded
|
||||
data, but this is more or less unavoidable in the protobuf refactoring
|
||||
|
||||
### Neutral
|
||||
|
||||
## References
|
||||
Loading…
Reference in New Issue