10 KiB

Raw Blame History

ADR 031: Protobuf Msg Services

Changelog

2020-10-05: Initial Draft

Status

Accepted

Abstract

We want to leverage protobuf service definitions for defining Msgs which will give us significant developer UX improvements in terms of the code that is generated and the fact that return types will now be well defined.

Context

Currently Msg handlers in the Cosmos SDK do have return values that are placed in the data field of the response. These return values, however, are not specified anywhere except in the golang handler code.

In early conversations it was proposed that Msg return types be captured using a protobuf extension field, ex:

package cosmos.gov;

message MsgSubmitProposal
	option (cosmos_proto.msg_return) = “uint64”;
	string delegator_address = 1;
	string validator_address = 2;
	repeated sdk.Coin amount = 3;
}

This was never adopted, however.

Having a well-specified return value for Msgs would improve client UX. For instance, in x/gov, MsgSubmitProposal returns the proposal ID as a big-endian uint64. This isn’t really documented anywhere and clients would need to know the internals of the SDK to parse that value and return it to users.

Also, there may be cases where we want to use these return values programatically. For instance, https://github.com/cosmos/cosmos-sdk/issues/7093 proposes a method for doing inter-module Ocaps using the Msg router. A well-defined return type would improve the developer UX for this approach.

In addition, handler registration of Msg types tends to add a bit of boilerplate on top of keepers and is usually done through manual type switches. This isn't necessarily bad, but it does add overhead to creating modules.

Decision

We decide to use protobuf service definitions for defining Msgs as well as the code generated by them as a replacement for Msg handlers.

Below we define how this will look for the SubmitProposal message from x/gov module. We start with a Msg service definition:

package cosmos.gov;

service Msg {
  rpc SubmitProposal(MsgSubmitProposal) returns (MsgSubmitProposalResponse);
}

// Note that for backwards compatibility this uses MsgSubmitProposal as the request
// type instead of the more canonical MsgSubmitProposalRequest
message MsgSubmitProposal {
  google.protobuf.Any content = 1;
  string proposer = 2;
}

message MsgSubmitProposalResponse {
  uint64 proposal_id;
}

While this is most commonly used for gRPC, overloading protobuf service definitions like this does not violate the intent of the protobuf spec which says:

If you don’t want to use gRPC, it’s also possible to use protocol buffers with your own RPC implementation. With this approach, we would get an auto-generated MsgServer interface:

In addition to clearly specifying return types, this has the benefit of generating client and server code. On the server side, this is almost like an automatically generated keeper method and could maybe be used intead of keepers eventually (see #7093):

package gov

type MsgServer interface {
  SubmitProposal(context.Context, *MsgSubmitProposal) (*MsgSubmitProposalResponse, error)
}

On the client side, developers could take advantage of this by creating RPC implementations that encapsulate transaction logic. Protobuf libraries that use asynchronous callbacks, like protobuf.js could use this to register callbacks for specific messages even for transactions that include multiple Msgs.

For backwards compatibility, existing Msg types should be used as the request parameter for service definitions. Newer Msg types which only support service definitions should use the more canonical Msg...Request names.

Encoding

Currently, we are encoding Msgs as Any in Txs which involves packing the binary-encoded Msg with its type URL.

The type URL for MsgSubmitProposal based on the proto3 spec is /cosmos.gov.MsgSubmitProposal.

The fully-qualified name for the SubmitProposal service method above (also based on the proto3 and gRPC specs) is /cosmos.gov.Msg/SubmitProposal which varies by a single / character. The generated .pb.go files for protobuf services include names of this form and any compliant protobuf/gRPC code generator will generate the same name.

In order to encode service methods in transactions, we encode them as Anys in the same TxBody.messages field as other Msgs. We simply set Any.type_url to the full-qualified method name (ex. /cosmos.gov.Msg/SubmitProposal) and set Any.value to the protobuf encoding of the request message (MsgSubmitProposal in this case).

Decoding

When decoding, TxBody.UnpackInterfaces will need a special case to detect if Any type URLs match the service method format (ex. /cosmos.gov.Msg/SubmitProposal) by checking for two / characters. Messages that are method names plus request parameters instead of a normal Any messages will get unpacked into the ServiceMsg struct:

type ServiceMsg struct {
  // MethodName is the fully-qualified service name
  MethodName string
  // Request is the request payload
  Request MsgRequest
}

Routing

In the future, service definitions may become the primary method for defining Msgs. As a starting point, we need to integrate with the SDK's existing routing and Msg interface.

To do this, ServiceMsg implements the sdk.Msg interface and its handler does the actual method routing, allowing this feature to be added incrementally on top of existing functionality.

`MsgRequest` interface

All request messages will need to implement the MsgRequest interface which is a simplified version of Msg, without Route(), Type() and GetSignBytes() which are no longer needed:

type MsgRequest interface {
  proto.Message
  ValidateBasic() error
  GetSigners() []AccAddress
}

ServiceMsg will forward its ValidateBasic and GetSigners methods to the MsgRequest methods.

Module Configuration

In ADR 021, we introduced a method RegisterQueryService to AppModule which allows for modules to register gRPC queriers.

To register Msg services, we attempt a more extensible approach by converting RegisterQueryService to a more generic RegisterServices method:

type AppModule interface {
  RegisterServices(Configurator)
  ...
}

type Configurator interface {
  QueryServer() grpc.Server
  MsgServer() grpc.Server
}

// example module:
func (am AppModule) RegisterServices(cfg Configurator) {
	types.RegisterQueryServer(cfg.QueryServer(), keeper)
	types.RegisterMsgServer(cfg.MsgServer(), keeper)
}

The RegisterServices method and the Configurator interface are intended to evolve to satisfy the use cases discussed in #7093 and #7122.

When Msg services are registered, the framework should verify that all Msg...Request types implement the MsgRequest interface described above and throw an error during initialization rather than later when transactions are processed.

`Msg` Service Implementation

Just like query services, Msg service methods can retrieve the sdk.Context from the context.Context parameter method using the sdk.UnwrapSDKContext method:

package gov

func (k Keeper) SubmitProposal(goCtx context.Context, params *types.MsgSubmitProposal) (*MsgSubmitProposalResponse, error) {
	ctx := sdk.UnwrapSDKContext(goCtx)
    ...
}

The sdk.Context should have an EventManager already attached by the ServiceMsg router.

Separate handler definition is no longer needed with this approach.

Consequences

This design changes how a module functionality is exposed and accessed. It deprecates the existing Handler interface and AppModule.Route in favor of Protocol Buffer Services and Service Routing described above. This dramatically simplifies the code. We don't need to create handlers and keepers any more. Use of Protocol Buffer auto-generated clients clearly separates the communication interfaces between the module and a modules user. The control logic (aka handlers and keepers) is not exposed any more. A module interface can be seen as a black box accessible through a client API. It's worth to note that the client interfaces are also generated by Protocol Buffers.

This also allows us to change how we perform functional tests. Instead of mocking AppModules and Router, we will mock a client (server will stay hidden). More specifically: we will never mock moduleA.MsgServer in moduleB, but rather moduleA.MsgClient. One can think about it as working with external services (eg DBs, or online servers...). We assume that the transmission between clients and servers is correctly handled by generated Protocol Buffers.

Finally, closing a module to client API opens desirable OCAP patterns discussed in ADR-033. Since server implementation and interface is hidden, nobody can hold "keepers"/servers and will be forced to relay on the client interface, which will drive developers for correct encapsulation and software engineering patterns.

Pros

communicates return type clearly
manual handler registration and return type marshaling is no longer needed, just implement the interface and register it
communication interface is automatically generated, the developer can now focus only on the state transition methods - this would improve the UX of #7093 approach (1) if we chose to adopt that
generated client code could be useful for clients and tests
dramatically reduces and simplifies the code

Cons

supporting both this and the current concrete Msg type approach simultaneously could be confusing (we could choose to deprecate the current approach)
using service definitions outside the context of gRPC could be confusing (but doesn’t violate the proto3 spec)

10 KiB Raw Blame History Unescape Escape