# ADR 031: Protobuf Msg Services

## Changelog

* 2020-10-05: Initial Draft
* 2021-04-21: Remove `ServiceMsg`s to follow Protobuf `Any`'s spec, see [#9063](https://github.com/cosmos/cosmos-sdk/issues/9063).

## Status

Accepted

## Abstract

We want to leverage protobuf `service` definitions for defining `Msg`s which will give us significant developer UX
improvements in terms of the code that is generated and the fact that return types will now be well defined.

## Context

Currently `Msg` handlers in the Cosmos SDK do have return values that are placed in the `data` field of the response.
These return values, however, are not specified anywhere except in the golang handler code.

In early conversations [it was proposed](https://docs.google.com/document/d/1eEgYgvgZqLE45vETjhwIw4VOqK-5hwQtZtjVbiXnIGc/edit)
that `Msg` return types be captured using a protobuf extension field, ex:

```protobuf
package cosmos.gov;

message MsgSubmitProposal
	option (cosmos_proto.msg_return) = “uint64”;
	string delegator_address = 1;
	string validator_address = 2;
	repeated sdk.Coin amount = 3;
}
```

This was never adopted, however.

Having a well-specified return value for `Msg`s would improve client UX. For instance,
in `x/gov`,  `MsgSubmitProposal` returns the proposal ID as a big-endian `uint64`.
This isn’t really documented anywhere and clients would need to know the internals
of the Cosmos SDK to parse that value and return it to users.

Also, there may be cases where we want to use these return values programatically.
For instance, https://github.com/cosmos/cosmos-sdk/issues/7093 proposes a method for
doing inter-module Ocaps using the `Msg` router. A well-defined return type would
improve the developer UX for this approach.

In addition, handler registration of `Msg` types tends to add a bit of
boilerplate on top of keepers and is usually done through manual type switches.
This isn't necessarily bad, but it does add overhead to creating modules.

## Decision

We decide to use protobuf `service` definitions for defining `Msg`s as well as
the code generated by them as a replacement for `Msg` handlers.

Below we define how this will look for the `SubmitProposal` message from `x/gov` module.
We start with a `Msg` `service` definition:

```proto
package cosmos.gov;

service Msg {
  rpc SubmitProposal(MsgSubmitProposal) returns (MsgSubmitProposalResponse);
}

// Note that for backwards compatibility this uses MsgSubmitProposal as the request
// type instead of the more canonical MsgSubmitProposalRequest
message MsgSubmitProposal {
  google.protobuf.Any content = 1;
  string proposer = 2;
}

message MsgSubmitProposalResponse {
  uint64 proposal_id;
}
```

While this is most commonly used for gRPC, overloading protobuf `service` definitions like this does not violate
the intent of the [protobuf spec](https://developers.google.com/protocol-buffers/docs/proto3#services) which says:
> If you don’t want to use gRPC, it’s also possible to use protocol buffers with your own RPC implementation.
With this approach, we would get an auto-generated `MsgServer` interface:

In addition to clearly specifying return types, this has the benefit of generating client and server code. On the server
side, this is almost like an automatically generated keeper method and could maybe be used intead of keepers eventually
(see [\#7093](https://github.com/cosmos/cosmos-sdk/issues/7093)):

```go
package gov

type MsgServer interface {
  SubmitProposal(context.Context, *MsgSubmitProposal) (*MsgSubmitProposalResponse, error)
}
```

On the client side, developers could take advantage of this by creating RPC implementations that encapsulate transaction
logic. Protobuf libraries that use asynchronous callbacks, like [protobuf.js](https://github.com/protobufjs/protobuf.js#using-services)
could use this to register callbacks for specific messages even for transactions that include multiple `Msg`s.

Each `Msg` service method should have exactly one request parameter: its corresponding `Msg` type. For example, the `Msg` service method `/cosmos.gov.v1beta1.Msg/SubmitProposal` above has exactly one request parameter, namely the `Msg` type `/cosmos.gov.v1beta1.MsgSubmitProposal`. It is important the reader understands clearly the nomenclature difference between a `Msg` service (a Protobuf service) and a `Msg` type (a Protobuf message), and the differences in their fully-qualified name.

This convention has been decided over the more canonical `Msg...Request` names mainly for backwards compatibility, but also for better readability in `TxBody.messages` (see [Encoding section](#encoding) below): transactions containing `/cosmos.gov.MsgSubmitProposal` read better than those containing `/cosmos.gov.v1beta1.MsgSubmitProposalRequest`.

One consequence of this convention is that each `Msg` type can be the request parameter of only one `Msg` service method. However, we consider this limitation a good practice in explicitness.

### Encoding

Encoding of transactions generated with `Msg` services do not differ from current Protobuf transaction encoding as defined in [ADR-020](./adr-020-protobuf-transaction-encoding.md). We are encoding `Msg` types (which are exactly `Msg` service methods' request parameters) as `Any` in `Tx`s which involves packing the
binary-encoded `Msg` with its type URL.

### Decoding

Since `Msg` types are packed into `Any`, decoding transactions messages are done by unpacking `Any`s into `Msg` types. For more information, please refer to [ADR-020](./adr-020-protobuf-transaction-encoding.md#transactions).

### Routing

We propose to add a `msg_service_router` in BaseApp. This router is a key/value map which maps `Msg` types' `type_url`s to their corresponding `Msg` service method handler. Since there is a 1-to-1 mapping between `Msg` types and `Msg` service method, the `msg_service_router` has exactly one entry per `Msg` service method.

When a transaction is processed by BaseApp (in CheckTx or in DeliverTx), its `TxBody.messages` are decoded as `Msg`s. Each `Msg`'s `type_url` is matched against an entry in the `msg_service_router`, and the respective `Msg` service method handler is called.

For backward compatability, the old handlers are not removed yet. If BaseApp receives a legacy `Msg` with no correspoding entry in the `msg_service_router`, it will be routed via its legacy `Route()` method into the legacy handler.

### Module Configuration

In [ADR 021](./adr-021-protobuf-query-encoding.md), we introduced a method `RegisterQueryService`
to `AppModule` which allows for modules to register gRPC queriers.

To register `Msg` services, we attempt a more extensible approach by converting `RegisterQueryService`
to a more generic `RegisterServices` method:

```go
type AppModule interface {
  RegisterServices(Configurator)
  ...
}

type Configurator interface {
  QueryServer() grpc.Server
  MsgServer() grpc.Server
}

// example module:
func (am AppModule) RegisterServices(cfg Configurator) {
	types.RegisterQueryServer(cfg.QueryServer(), keeper)
	types.RegisterMsgServer(cfg.MsgServer(), keeper)
}
```

The `RegisterServices` method and the `Configurator` interface are intended to
evolve to satisfy the use cases discussed in [\#7093](https://github.com/cosmos/cosmos-sdk/issues/7093)
and [\#7122](https://github.com/cosmos/cosmos-sdk/issues/7421).

When `Msg` services are registered, the framework _should_ verify that all `Msg` types
implement the `sdk.Msg` interface and throw an error during initialization rather
than later when transactions are processed.

### `Msg` Service Implementation

Just like query services, `Msg` service methods can retrieve the `sdk.Context`
from the `context.Context` parameter method using the `sdk.UnwrapSDKContext`
method:

```go
package gov

func (k Keeper) SubmitProposal(goCtx context.Context, params *types.MsgSubmitProposal) (*MsgSubmitProposalResponse, error) {
	ctx := sdk.UnwrapSDKContext(goCtx)
    ...
}
```

The `sdk.Context` should have an `EventManager` already attached by BaseApp's `msg_service_router`.

Separate handler definition is no longer needed with this approach.

## Consequences

This design changes how a module functionality is exposed and accessed. It deprecates the existing `Handler` interface and `AppModule.Route` in favor of [Protocol Buffer Services](https://developers.google.com/protocol-buffers/docs/proto3#services) and Service Routing described above. This dramatically simplifies the code. We don't need to create handlers and keepers any more. Use of Protocol Buffer auto-generated clients clearly separates the communication interfaces between the module and a modules user. The control logic (aka handlers and keepers) is not exposed any more. A module interface can be seen as a black box accessible through a client API. It's worth to note that the client interfaces are also generated by Protocol Buffers.

This also allows us to change how we perform functional tests. Instead of mocking AppModules and Router, we will mock a client (server will stay hidden). More specifically: we will never mock `moduleA.MsgServer` in `moduleB`, but rather `moduleA.MsgClient`. One can think about it as working with external services (eg DBs, or online servers...). We assume that the transmission between clients and servers is correctly handled by generated Protocol Buffers.

Finally, closing a module to client API opens desirable OCAP patterns discussed in ADR-033. Since server implementation and interface is hidden, nobody can hold "keepers"/servers and will be forced to relay on the client interface, which will drive developers for correct encapsulation and software engineering patterns.

### Pros

* communicates return type clearly
* manual handler registration and return type marshaling is no longer needed, just implement the interface and register it
* communication interface is automatically generated, the developer can now focus only on the state transition methods - this would improve the UX of [\#7093](https://github.com/cosmos/cosmos-sdk/issues/7093) approach (1) if we chose to adopt that
* generated client code could be useful for clients and tests
* dramatically reduces and simplifies the code

### Cons

* using `service` definitions outside the context of gRPC could be confusing (but doesn’t violate the proto3 spec)

## References

* [Initial Github Issue \#7122](https://github.com/cosmos/cosmos-sdk/issues/7122)
* [proto 3 Language Guide: Defining Services](https://developers.google.com/protocol-buffers/docs/proto3#services)
* [Initial pre-`Any` `Msg` designs](https://docs.google.com/document/d/1eEgYgvgZqLE45vETjhwIw4VOqK-5hwQtZtjVbiXnIGc)
* [ADR 020](./adr-020-protobuf-transaction-encoding.md)
* [ADR 021](./adr-021-protobuf-query-encoding.md)