cosmos-sdk/docs/core/encoding.md

103 lines
5.2 KiB
Markdown
Raw Normal View History

Docs: hide frontmatter, bugfixes (#5413) * encoding * working on baseapp doc * baseapp work * reorg * almost there * finish first draft * remove old files * module doc start * finish intro * working * workinnn * add transactions into core * hans comments * add transactions into core * working * gautier comments * clean * working * consolidate intro * querier * workiiiing * refactor for new module interface * karoly review * working on baseapp doc * baseapp work * reorg * almost there * finish first draft * remove old files * finish intro * workinnn * initial commit after rebase * query-lifecycle and started modules-interfaces * query-lifecycle first draft done * module interfaces first draft * rest and intro skeletons * rest and intro done * small edits and links * comments * revisions * cli.md comments * comments * minor edits * better flow for query lifecycle * add transactions into core * hans comments * add transactions into core * checkout master-docs files * deleted some * remove modules readme * cli.md comments * comments * module-interfaces comments * Merge PR #4857: Add Context concept doc * working * working * finish messages and queries * handler * querier * last comments! * punctuation * querier2 * consolidate intro * querier * workiiiing * refactor for new module interface * karoly review * working on baseapp doc * baseapp work * reorg * almost there * finish first draft * remove old files * finish intro * workinnn * initial commit after rebase * query-lifecycle and started modules-interfaces * query-lifecycle first draft done * module interfaces first draft * rest and intro skeletons * rest and intro done * small edits and links * comments * revisions * cli.md comments * comments * minor edits * better flow for query lifecycle * checkout master-docs files * deleted some * remove modules readme * cli.md comments * comments * module-interfaces comments * keeper * genesis * finish * Apply suggestions from code review Co-Authored-By: Hans Schoenburg <hschoenburg@users.noreply.github.com> * hans review * Update docs/core/baseapp.md Co-Authored-By: Hans Schoenburg <hschoenburg@users.noreply.github.com> * working * last comment * workin * Apply suggestions from code review * encoding and node * almost finish store * finish docs * fixes * fede comments + permalinks * hans review * add more permalinks * update docs theme version (#5239) * R4R: Docs Cleanup (#5246) * start * work * work * work * remove table of content * links intro * fix links * remove junk * cleanup * cleanup * work * finish cleanup * addback readmes * remove nft * fix links * remove dup * remove dup * remove dup * remove dup * remove dup * fix links * add subscribe events * refine rest * index page * sidebar * theme version * theme version * testing netlify * theme version * tooltip example * version * testing code embedding * reverting back * theme version * version * version * version * readme and version * cleanup * redo app anatomy * modules readme, theme version * theme version * fix modules list * theme version * new snippets * modules readme * update docs readme * modify synopsis * version * fix yaml * version * version * version * version * version * version * version * version * version * version * add hide banner * version * version * version * small fixes * modules readme, version * remove hotkeys dep, version * version * version * version * version * version * version * version * slight notice * fix links and hide * permalinks * small clean * version * resolve conflicts, add google analytics * fix merge remants * version * changelog 1/2 * Changelog: docs UI * version * remove merge conflicts * Code: Update link for Contributing to the docs to docs_readme * HTML/CSS: Update layout of homepage footer to match new layout in Figma * version * final modifs * modules, version * modules readme * link to module list from homepage * version * building modules link * version * version * fonts * version * version * fix link * fix package.json * links in explore sdk section * core concepts * version * change delimeters for frontmatter * frontmatter in comments * version * temp add tiny-cookie * fixed link issues * fixed styling issues, copy * hide frontmatter * hide frontmatter * layout fixes, padded ascii diagram * fira sans font for code
2019-12-17 03:44:44 -08:00
<!--
order: 6
Docs: hide frontmatter, bugfixes (#5413) * encoding * working on baseapp doc * baseapp work * reorg * almost there * finish first draft * remove old files * module doc start * finish intro * working * workinnn * add transactions into core * hans comments * add transactions into core * working * gautier comments * clean * working * consolidate intro * querier * workiiiing * refactor for new module interface * karoly review * working on baseapp doc * baseapp work * reorg * almost there * finish first draft * remove old files * finish intro * workinnn * initial commit after rebase * query-lifecycle and started modules-interfaces * query-lifecycle first draft done * module interfaces first draft * rest and intro skeletons * rest and intro done * small edits and links * comments * revisions * cli.md comments * comments * minor edits * better flow for query lifecycle * add transactions into core * hans comments * add transactions into core * checkout master-docs files * deleted some * remove modules readme * cli.md comments * comments * module-interfaces comments * Merge PR #4857: Add Context concept doc * working * working * finish messages and queries * handler * querier * last comments! * punctuation * querier2 * consolidate intro * querier * workiiiing * refactor for new module interface * karoly review * working on baseapp doc * baseapp work * reorg * almost there * finish first draft * remove old files * finish intro * workinnn * initial commit after rebase * query-lifecycle and started modules-interfaces * query-lifecycle first draft done * module interfaces first draft * rest and intro skeletons * rest and intro done * small edits and links * comments * revisions * cli.md comments * comments * minor edits * better flow for query lifecycle * checkout master-docs files * deleted some * remove modules readme * cli.md comments * comments * module-interfaces comments * keeper * genesis * finish * Apply suggestions from code review Co-Authored-By: Hans Schoenburg <hschoenburg@users.noreply.github.com> * hans review * Update docs/core/baseapp.md Co-Authored-By: Hans Schoenburg <hschoenburg@users.noreply.github.com> * working * last comment * workin * Apply suggestions from code review * encoding and node * almost finish store * finish docs * fixes * fede comments + permalinks * hans review * add more permalinks * update docs theme version (#5239) * R4R: Docs Cleanup (#5246) * start * work * work * work * remove table of content * links intro * fix links * remove junk * cleanup * cleanup * work * finish cleanup * addback readmes * remove nft * fix links * remove dup * remove dup * remove dup * remove dup * remove dup * fix links * add subscribe events * refine rest * index page * sidebar * theme version * theme version * testing netlify * theme version * tooltip example * version * testing code embedding * reverting back * theme version * version * version * version * readme and version * cleanup * redo app anatomy * modules readme, theme version * theme version * fix modules list * theme version * new snippets * modules readme * update docs readme * modify synopsis * version * fix yaml * version * version * version * version * version * version * version * version * version * version * add hide banner * version * version * version * small fixes * modules readme, version * remove hotkeys dep, version * version * version * version * version * version * version * version * slight notice * fix links and hide * permalinks * small clean * version * resolve conflicts, add google analytics * fix merge remants * version * changelog 1/2 * Changelog: docs UI * version * remove merge conflicts * Code: Update link for Contributing to the docs to docs_readme * HTML/CSS: Update layout of homepage footer to match new layout in Figma * version * final modifs * modules, version * modules readme * link to module list from homepage * version * building modules link * version * version * fonts * version * version * fix link * fix package.json * links in explore sdk section * core concepts * version * change delimeters for frontmatter * frontmatter in comments * version * temp add tiny-cookie * fixed link issues * fixed styling issues, copy * hide frontmatter * hide frontmatter * layout fixes, padded ascii diagram * fira sans font for code
2019-12-17 03:44:44 -08:00
-->
# Encoding
Update docs website (#5827) * query-lifecycle and started modules-interfaces * query-lifecycle first draft done * module interfaces first draft * rest and intro skeletons * rest and intro done * small edits and links * comments * revisions * cli.md comments * comments * minor edits * better flow for query lifecycle * add transactions into core * hans comments * add transactions into core * checkout master-docs files * deleted some * remove modules readme * cli.md comments * comments * module-interfaces comments * Merge PR #4857: Add Context concept doc * working * working * finish messages and queries * handler * querier * last comments! * punctuation * querier2 * consolidate intro * querier * workiiiing * refactor for new module interface * karoly review * working on baseapp doc * baseapp work * reorg * almost there * finish first draft * remove old files * finish intro * workinnn * initial commit after rebase * query-lifecycle and started modules-interfaces * query-lifecycle first draft done * module interfaces first draft * rest and intro skeletons * rest and intro done * small edits and links * comments * revisions * cli.md comments * comments * minor edits * better flow for query lifecycle * checkout master-docs files * deleted some * remove modules readme * cli.md comments * comments * module-interfaces comments * keeper * genesis * finish * Apply suggestions from code review Co-Authored-By: Hans Schoenburg <hschoenburg@users.noreply.github.com> * hans review * Update docs/core/baseapp.md Co-Authored-By: Hans Schoenburg <hschoenburg@users.noreply.github.com> * working * last comment * workin * Apply suggestions from code review * encoding and node * almost finish store * finish docs * fixes * fede comments + permalinks * hans review * add more permalinks * update docs theme version (#5239) * R4R: Docs Cleanup (#5246) * start * work * work * work * remove table of content * links intro * fix links * remove junk * cleanup * cleanup * work * finish cleanup * addback readmes * remove nft * fix links * remove dup * remove dup * remove dup * remove dup * remove dup * fix links * add subscribe events * refine rest * index page * sidebar * theme version * theme version * testing netlify * theme version * tooltip example * version * testing code embedding * reverting back * theme version * version * version * version * readme and version * cleanup * redo app anatomy * modules readme, theme version * theme version * fix modules list * theme version * new snippets * modules readme * update docs readme * modify synopsis * version * fix yaml * version * version * version * version * version * version * version * version * version * version * add hide banner * version * version * version * small fixes * modules readme, version * remove hotkeys dep, version * version * version * version * version * version * version * version * slight notice * fix links and hide * permalinks * small clean * version * resolve conflicts, add google analytics * fix merge remants * version * changelog 1/2 * Changelog: docs UI * version * remove merge conflicts * Code: Update link for Contributing to the docs to docs_readme * HTML/CSS: Update layout of homepage footer to match new layout in Figma * version * final modifs * modules, version * modules readme * link to module list from homepage * version * building modules link * version * version * fonts * version * version * fix link * fix package.json * links in explore sdk section * core concepts * version * change delimeters for frontmatter * frontmatter in comments * version * temp add tiny-cookie * fixed link issues * fixed styling issues, copy * hide frontmatter * hide frontmatter * layout fixes, padded ascii diagram * fira sans font for code * scrollbar color * move synopsis from frontmatter * move synopsis from frontmatter * code styling in synopsis fix * tutorial link * active headers * 404 fix * homepage links fix * fix link in footer * misc fixes * version * prerequisite links on mobile * version * version * Fix footer links * version * Fix permalink popup * Update version * package-lock.json * Update Questions section in the footer * Config for Algolia Docsearch * Update version * Update to the latest version of the theme * Use docs-staging as a branch for staging website * Update version * Add google analytics * Remove {hide} from Pre-Requisite Readings * Replace Riot with Discord Co-Authored-By: billy rennekamp <billy.rennekamp@gmail.com> * Fix yaml syntax error in docs * Fix formatting in keyring.md Co-authored-by: Gloria Zhao <gzhao408@berkeley.edu> Co-authored-by: gamarin <gautier@tendermint.com> Co-authored-by: Jack Zampolin <jack.zampolin@gmail.com> Co-authored-by: Hans Schoenburg <hschoenburg@users.noreply.github.com> Co-authored-by: Federico Kunze <31522760+fedekunze@users.noreply.github.com> Co-authored-by: Alexander Bezobchuk <alexanderbez@users.noreply.github.com> Co-authored-by: billy rennekamp <billy.rennekamp@gmail.com>
2020-03-24 09:20:30 -07:00
The `codec` is used everywhere in the Cosmos SDK to encode and decode structs and interfaces. The specific codec used in the Cosmos SDK is called `go-amino`. {synopsis}
Update docs website (#5827) * query-lifecycle and started modules-interfaces * query-lifecycle first draft done * module interfaces first draft * rest and intro skeletons * rest and intro done * small edits and links * comments * revisions * cli.md comments * comments * minor edits * better flow for query lifecycle * add transactions into core * hans comments * add transactions into core * checkout master-docs files * deleted some * remove modules readme * cli.md comments * comments * module-interfaces comments * Merge PR #4857: Add Context concept doc * working * working * finish messages and queries * handler * querier * last comments! * punctuation * querier2 * consolidate intro * querier * workiiiing * refactor for new module interface * karoly review * working on baseapp doc * baseapp work * reorg * almost there * finish first draft * remove old files * finish intro * workinnn * initial commit after rebase * query-lifecycle and started modules-interfaces * query-lifecycle first draft done * module interfaces first draft * rest and intro skeletons * rest and intro done * small edits and links * comments * revisions * cli.md comments * comments * minor edits * better flow for query lifecycle * checkout master-docs files * deleted some * remove modules readme * cli.md comments * comments * module-interfaces comments * keeper * genesis * finish * Apply suggestions from code review Co-Authored-By: Hans Schoenburg <hschoenburg@users.noreply.github.com> * hans review * Update docs/core/baseapp.md Co-Authored-By: Hans Schoenburg <hschoenburg@users.noreply.github.com> * working * last comment * workin * Apply suggestions from code review * encoding and node * almost finish store * finish docs * fixes * fede comments + permalinks * hans review * add more permalinks * update docs theme version (#5239) * R4R: Docs Cleanup (#5246) * start * work * work * work * remove table of content * links intro * fix links * remove junk * cleanup * cleanup * work * finish cleanup * addback readmes * remove nft * fix links * remove dup * remove dup * remove dup * remove dup * remove dup * fix links * add subscribe events * refine rest * index page * sidebar * theme version * theme version * testing netlify * theme version * tooltip example * version * testing code embedding * reverting back * theme version * version * version * version * readme and version * cleanup * redo app anatomy * modules readme, theme version * theme version * fix modules list * theme version * new snippets * modules readme * update docs readme * modify synopsis * version * fix yaml * version * version * version * version * version * version * version * version * version * version * add hide banner * version * version * version * small fixes * modules readme, version * remove hotkeys dep, version * version * version * version * version * version * version * version * slight notice * fix links and hide * permalinks * small clean * version * resolve conflicts, add google analytics * fix merge remants * version * changelog 1/2 * Changelog: docs UI * version * remove merge conflicts * Code: Update link for Contributing to the docs to docs_readme * HTML/CSS: Update layout of homepage footer to match new layout in Figma * version * final modifs * modules, version * modules readme * link to module list from homepage * version * building modules link * version * version * fonts * version * version * fix link * fix package.json * links in explore sdk section * core concepts * version * change delimeters for frontmatter * frontmatter in comments * version * temp add tiny-cookie * fixed link issues * fixed styling issues, copy * hide frontmatter * hide frontmatter * layout fixes, padded ascii diagram * fira sans font for code * scrollbar color * move synopsis from frontmatter * move synopsis from frontmatter * code styling in synopsis fix * tutorial link * active headers * 404 fix * homepage links fix * fix link in footer * misc fixes * version * prerequisite links on mobile * version * version * Fix footer links * version * Fix permalink popup * Update version * package-lock.json * Update Questions section in the footer * Config for Algolia Docsearch * Update version * Update to the latest version of the theme * Use docs-staging as a branch for staging website * Update version * Add google analytics * Remove {hide} from Pre-Requisite Readings * Replace Riot with Discord Co-Authored-By: billy rennekamp <billy.rennekamp@gmail.com> * Fix yaml syntax error in docs * Fix formatting in keyring.md Co-authored-by: Gloria Zhao <gzhao408@berkeley.edu> Co-authored-by: gamarin <gautier@tendermint.com> Co-authored-by: Jack Zampolin <jack.zampolin@gmail.com> Co-authored-by: Hans Schoenburg <hschoenburg@users.noreply.github.com> Co-authored-by: Federico Kunze <31522760+fedekunze@users.noreply.github.com> Co-authored-by: Alexander Bezobchuk <alexanderbez@users.noreply.github.com> Co-authored-by: billy rennekamp <billy.rennekamp@gmail.com>
2020-03-24 09:20:30 -07:00
## Pre-requisite Readings
- [Anatomy of an SDK application](../basics/app-anatomy.md) {prereq}
## Encoding
The Cosmos SDK utilizes two binary wire encoding protocols, [Amino](https://github.com/tendermint/go-amino/)
and [Protocol Buffers](https://developers.google.com/protocol-buffers), where Amino
is an object encoding specification. It is a subset of Proto3 with an extension for
interface support. See the [Proto3 spec](https://developers.google.com/protocol-buffers/docs/proto3)
for more information on Proto3, which Amino is largely compatible with (but not with Proto2).
Due to Amino having significant performance drawbacks, being reflection-based, and
not having any meaningful cross-language/client support, Protocol Buffers, specifically
[gogoprotobuf](https://github.com/gogo/protobuf/), is being used in place of Amino.
Note, this process of using Protocol Buffers over Amino is still an ongoing process.
Binary wire encoding of types in the Cosmos SDK can be broken down into two main
categories, client encoding and store encoding. Client encoding mainly revolves
around transaction processing and signing, whereas store encoding revolves around
types used in state-machine transitions and what is ultimately stored in the Merkle
tree.
For store encoding, protobuf definitions can exist for any type and will typically
have an Amino-based "intermediary" type. Specifically, the protobuf-based type
definition is used for serialization and persistence, whereas the Amino-based type
is used for business logic in the state-machine where they may converted back-n-forth.
Note, the Amino-based types may slowly be phased-out in the future so developers
should take note to use the protobuf message definitions where possible.
In the `codec` package, there exists two core interfaces, `Marshaler` and `ProtoMarshaler`,
where the former encapsulates the current Amino interface except it operates on
types implementing the latter instead of generic `interface{}` types.
In addition, there exists three implementations of `Marshaler`. The first being
`AminoCodec`, where both binary and JSON serialization is handled via Amino. The
second being `ProtoCodec`, where both binary and JSON serialization is handled
via Protobuf. Finally, `HybridCodec`, a codec that utilizes Protobuf for binary
serialization and Amino for JSON serialization. The `HybridCodec` is typically
the codec that used in majority in situations as it's easier to use for client
and state serialization.
This means that modules may use Amino or Protobuf encoding but the types must
implement `ProtoMarshaler`. If modules wish to avoid implementing this interface
for their types, they may use an Amino codec directly.
### Amino
Every module uses an Amino codec to serialize types and interfaces. This codec typically
has types and interfaces registered in that module's domain only (e.g. messages),
but there are exceptions like `x/gov`. Each module exposes a `RegisterCodec` function
that allows a user to provide a codec and have all the types registered. An application
will call this method for each necessary module.
Where there is no protobuf-based type definition for a module (see below), Amino
is used to encode and decode raw wire bytes to the concrete type or interface:
```go
bz := keeper.cdc.MustMarshalBinaryBare(typeOrInterface)
keeper.cdc.MustUnmarshalBinaryBare(bz, &typeOrInterface)
```
Note, there are length-prefixed variants of the above functionality and this is
typically used for when the data needs to be streamed or grouped together
(e.g. `ResponseDeliverTx.Data`)
Another important use of the Amino is the encoding and decoding of
[transactions](./transactions.md). Transactions are defined by the application or
the SDK, but passed to the underlying consensus engine in order to be relayed to
other peers. Since the underlying consensus engine is agnostic to the application,
it only accepts transactions in the form of raw bytes. The encoding is done by an
object called `TxEncoder` and the decoding by an object called `TxDecoder`.
+++ https://github.com/cosmos/cosmos-sdk/blob/7d7821b9af132b0f6131640195326aa02b6751db/types/tx_msg.go#L45-L49
A standard implementation of both these objects can be found in the [`auth` module](https://github.com/cosmos/cosmos-sdk/blob/master/x/auth):
+++ https://github.com/cosmos/cosmos-sdk/blob/7d7821b9af132b0f6131640195326aa02b6751db/x/auth/types/stdtx.go#L241-L266
### Gogoproto
Modules are encouraged to utilize Protobuf encoding for their respective types.
If modules do not contain any interfaces (e.g. `Account` or `Content`), then they
may simply accept a `Marshaler` as the codec which is implemented via the `HybridCodec`
without any further customization.
However, if modules are to handle type interfaces, they should seek to extend the
`Marshaler` interface contract for these types (e.g. `MarshalAccount`). Note, they
should still use a `HybridCodec` internally. These extended contracts will typically
use concrete types with unique `oneof` messages.
## Next {hide}
Learn about [events](./events.md) {hide}