203 lines
10 KiB
Markdown
203 lines
10 KiB
Markdown
# ADR 031: Protobuf Msg Services
|
||
|
||
## Changelog
|
||
|
||
- 2020-10-05: Initial Draft
|
||
- 2021-04-21: Remove `ServiceMsg`s to follow Protobuf `Any`'s spec, see [#9063](https://github.com/cosmos/cosmos-sdk/issues/9063).
|
||
|
||
## Status
|
||
|
||
Accepted
|
||
|
||
## Abstract
|
||
|
||
We want to leverage protobuf `service` definitions for defining `Msg`s which will give us significant developer UX
|
||
improvements in terms of the code that is generated and the fact that return types will now be well defined.
|
||
|
||
## Context
|
||
|
||
Currently `Msg` handlers in the Cosmos SDK do have return values that are placed in the `data` field of the response.
|
||
These return values, however, are not specified anywhere except in the golang handler code.
|
||
|
||
In early conversations [it was proposed](https://docs.google.com/document/d/1eEgYgvgZqLE45vETjhwIw4VOqK-5hwQtZtjVbiXnIGc/edit)
|
||
that `Msg` return types be captured using a protobuf extension field, ex:
|
||
|
||
```protobuf
|
||
package cosmos.gov;
|
||
|
||
message MsgSubmitProposal
|
||
option (cosmos_proto.msg_return) = “uint64”;
|
||
string delegator_address = 1;
|
||
string validator_address = 2;
|
||
repeated sdk.Coin amount = 3;
|
||
}
|
||
```
|
||
|
||
This was never adopted, however.
|
||
|
||
Having a well-specified return value for `Msg`s would improve client UX. For instance,
|
||
in `x/gov`, `MsgSubmitProposal` returns the proposal ID as a big-endian `uint64`.
|
||
This isn’t really documented anywhere and clients would need to know the internals
|
||
of the SDK to parse that value and return it to users.
|
||
|
||
Also, there may be cases where we want to use these return values programatically.
|
||
For instance, https://github.com/cosmos/cosmos-sdk/issues/7093 proposes a method for
|
||
doing inter-module Ocaps using the `Msg` router. A well-defined return type would
|
||
improve the developer UX for this approach.
|
||
|
||
In addition, handler registration of `Msg` types tends to add a bit of
|
||
boilerplate on top of keepers and is usually done through manual type switches.
|
||
This isn't necessarily bad, but it does add overhead to creating modules.
|
||
|
||
## Decision
|
||
|
||
We decide to use protobuf `service` definitions for defining `Msg`s as well as
|
||
the code generated by them as a replacement for `Msg` handlers.
|
||
|
||
Below we define how this will look for the `SubmitProposal` message from `x/gov` module.
|
||
We start with a `Msg` `service` definition:
|
||
|
||
```proto
|
||
package cosmos.gov;
|
||
|
||
service Msg {
|
||
rpc SubmitProposal(MsgSubmitProposal) returns (MsgSubmitProposalResponse);
|
||
}
|
||
|
||
// Note that for backwards compatibility this uses MsgSubmitProposal as the request
|
||
// type instead of the more canonical MsgSubmitProposalRequest
|
||
message MsgSubmitProposal {
|
||
google.protobuf.Any content = 1;
|
||
string proposer = 2;
|
||
}
|
||
|
||
message MsgSubmitProposalResponse {
|
||
uint64 proposal_id;
|
||
}
|
||
```
|
||
|
||
While this is most commonly used for gRPC, overloading protobuf `service` definitions like this does not violate
|
||
the intent of the [protobuf spec](https://developers.google.com/protocol-buffers/docs/proto3#services) which says:
|
||
> If you don’t want to use gRPC, it’s also possible to use protocol buffers with your own RPC implementation.
|
||
With this approach, we would get an auto-generated `MsgServer` interface:
|
||
|
||
In addition to clearly specifying return types, this has the benefit of generating client and server code. On the server
|
||
side, this is almost like an automatically generated keeper method and could maybe be used intead of keepers eventually
|
||
(see [\#7093](https://github.com/cosmos/cosmos-sdk/issues/7093)):
|
||
|
||
```go
|
||
package gov
|
||
|
||
type MsgServer interface {
|
||
SubmitProposal(context.Context, *MsgSubmitProposal) (*MsgSubmitProposalResponse, error)
|
||
}
|
||
```
|
||
|
||
On the client side, developers could take advantage of this by creating RPC implementations that encapsulate transaction
|
||
logic. Protobuf libraries that use asynchronous callbacks, like [protobuf.js](https://github.com/protobufjs/protobuf.js#using-services)
|
||
could use this to register callbacks for specific messages even for transactions that include multiple `Msg`s.
|
||
|
||
Each `Msg` service method should have exactly one request parameter: its corresponding `Msg` type. For example, the `Msg` service method `/cosmos.gov.v1beta1.Msg/SubmitProposal` above has exactly one request parameter, namely the `Msg` type `/cosmos.gov.v1beta1.MsgSubmitProposal`. It is important the reader understands clearly the nomenclature difference between a `Msg` service (a Protobuf service) and a `Msg` type (a Protobuf message), and the differences in their fully-qualified name.
|
||
|
||
This convention has been decided over the more canonical `Msg...Request` names mainly for backwards compatibility, but also for better readability in `TxBody.messages` (see [Encoding section](#encoding) below): transactions containing `/cosmos.gov.MsgSubmitProposal` read better than those containing `/cosmos.gov.v1beta1.MsgSubmitProposalRequest`.
|
||
|
||
One consequence of this convention is that each `Msg` type can be the request parameter of only one `Msg` service method. However, we consider this limitation a good practice in explicitness.
|
||
|
||
### Encoding
|
||
|
||
Encoding of transactions generated with `Msg` services do not differ from current Protobuf transaction encoding as defined in [ADR-020](./adr-020-protobuf-transaction-encoding.md). We are encoding `Msg` types (which are exactly `Msg` service methods' request parameters) as `Any` in `Tx`s which involves packing the
|
||
binary-encoded `Msg` with its type URL.
|
||
|
||
### Decoding
|
||
|
||
Since `Msg` types are packed into `Any`, decoding transactions messages are done by unpacking `Any`s into `Msg` types. For more information, please refer to [ADR-020](./adr-020-protobuf-transaction-encoding.md#transactions).
|
||
|
||
### Routing
|
||
|
||
We propose to add a `msg_service_router` in BaseApp. This router is a key/value map which maps `Msg` types' `type_url`s to their corresponding `Msg` service method handler. Since there is a 1-to-1 mapping between `Msg` types and `Msg` service method, the `msg_service_router` has exactly one entry per `Msg` service method.
|
||
|
||
When a transaction is processed by BaseApp (in CheckTx or in DeliverTx), its `TxBody.messages` are decoded as `Msg`s. Each `Msg`'s `type_url` is matched against an entry in the `msg_service_router`, and the respective `Msg` service method handler is called.
|
||
|
||
For backward compatability, the old handlers are not removed yet. If BaseApp receives a legacy `Msg` with no correspoding entry in the `msg_service_router`, it will be routed via its legacy `Route()` method into the legacy handler.
|
||
|
||
### Module Configuration
|
||
|
||
In [ADR 021](./adr-021-protobuf-query-encoding.md), we introduced a method `RegisterQueryService`
|
||
to `AppModule` which allows for modules to register gRPC queriers.
|
||
|
||
To register `Msg` services, we attempt a more extensible approach by converting `RegisterQueryService`
|
||
to a more generic `RegisterServices` method:
|
||
|
||
```go
|
||
type AppModule interface {
|
||
RegisterServices(Configurator)
|
||
...
|
||
}
|
||
|
||
type Configurator interface {
|
||
QueryServer() grpc.Server
|
||
MsgServer() grpc.Server
|
||
}
|
||
|
||
// example module:
|
||
func (am AppModule) RegisterServices(cfg Configurator) {
|
||
types.RegisterQueryServer(cfg.QueryServer(), keeper)
|
||
types.RegisterMsgServer(cfg.MsgServer(), keeper)
|
||
}
|
||
```
|
||
|
||
The `RegisterServices` method and the `Configurator` interface are intended to
|
||
evolve to satisfy the use cases discussed in [\#7093](https://github.com/cosmos/cosmos-sdk/issues/7093)
|
||
and [\#7122](https://github.com/cosmos/cosmos-sdk/issues/7421).
|
||
|
||
When `Msg` services are registered, the framework _should_ verify that all `Msg` types
|
||
implement the `sdk.Msg` interface and throw an error during initialization rather
|
||
than later when transactions are processed.
|
||
|
||
### `Msg` Service Implementation
|
||
|
||
Just like query services, `Msg` service methods can retrieve the `sdk.Context`
|
||
from the `context.Context` parameter method using the `sdk.UnwrapSDKContext`
|
||
method:
|
||
|
||
```go
|
||
package gov
|
||
|
||
func (k Keeper) SubmitProposal(goCtx context.Context, params *types.MsgSubmitProposal) (*MsgSubmitProposalResponse, error) {
|
||
ctx := sdk.UnwrapSDKContext(goCtx)
|
||
...
|
||
}
|
||
```
|
||
|
||
The `sdk.Context` should have an `EventManager` already attached by BaseApp's `msg_service_router`.
|
||
|
||
Separate handler definition is no longer needed with this approach.
|
||
|
||
## Consequences
|
||
|
||
This design changes how a module functionality is exposed and accessed. It deprecates the existing `Handler` interface and `AppModule.Route` in favor of [Protocol Buffer Services](https://developers.google.com/protocol-buffers/docs/proto3#services) and Service Routing described above. This dramatically simplifies the code. We don't need to create handlers and keepers any more. Use of Protocol Buffer auto-generated clients clearly separates the communication interfaces between the module and a modules user. The control logic (aka handlers and keepers) is not exposed any more. A module interface can be seen as a black box accessible through a client API. It's worth to note that the client interfaces are also generated by Protocol Buffers.
|
||
|
||
This also allows us to change how we perform functional tests. Instead of mocking AppModules and Router, we will mock a client (server will stay hidden). More specifically: we will never mock `moduleA.MsgServer` in `moduleB`, but rather `moduleA.MsgClient`. One can think about it as working with external services (eg DBs, or online servers...). We assume that the transmission between clients and servers is correctly handled by generated Protocol Buffers.
|
||
|
||
Finally, closing a module to client API opens desirable OCAP patterns discussed in ADR-033. Since server implementation and interface is hidden, nobody can hold "keepers"/servers and will be forced to relay on the client interface, which will drive developers for correct encapsulation and software engineering patterns.
|
||
|
||
### Pros
|
||
|
||
- communicates return type clearly
|
||
- manual handler registration and return type marshaling is no longer needed, just implement the interface and register it
|
||
- communication interface is automatically generated, the developer can now focus only on the state transition methods - this would improve the UX of [\#7093](https://github.com/cosmos/cosmos-sdk/issues/7093) approach (1) if we chose to adopt that
|
||
- generated client code could be useful for clients and tests
|
||
- dramatically reduces and simplifies the code
|
||
|
||
### Cons
|
||
|
||
- using `service` definitions outside the context of gRPC could be confusing (but doesn’t violate the proto3 spec)
|
||
|
||
## References
|
||
|
||
- [Initial Github Issue \#7122](https://github.com/cosmos/cosmos-sdk/issues/7122)
|
||
- [proto 3 Language Guide: Defining Services](https://developers.google.com/protocol-buffers/docs/proto3#services)
|
||
- [Initial pre-`Any` `Msg` designs](https://docs.google.com/document/d/1eEgYgvgZqLE45vETjhwIw4VOqK-5hwQtZtjVbiXnIGc)
|
||
- [ADR 020](./adr-020-protobuf-transaction-encoding.md)
|
||
- [ADR 021](./adr-021-protobuf-query-encoding.md)
|