cosmos-sdk/docs/building-modules/simulator.md

124 lines
5.5 KiB
Markdown
Raw Normal View History

# Module Simulation
## Prerequisites
* [Cosmos Blockchain Simulator](./../using-the-sdk/simulation.md)
## Synopsis
This document details how to define each module simulation functions to be
integrated with the application `SimulationManager`.
* [Simulation package](#simulation-package)
* [Store decoders](#store-decoders)
* [Randomized genesis](#randomized-genesis)
* [Randomized parameters](#randomized-parameters)
* [Random weighted operations](#random-weighted-operations)
* [Random proposal contents](#random-proposal-contents)
* [Registering the module simulation functions](#registering-simulation-functions)
* [App simulator manager](#app-simulator-manager)
* [Simulation tests](#simulation-tests)
## Simulation package
Every module that implements the Cosmos SDK simulator needs to have a `x/<module>/simulation`
package which contains the primary functions required by the fuzz tests: store
decoders, randomized genesis state and parameters, weighted operations and proposal
contents.
### Store decoders
Registering the store decoders is required for the `AppImportExport`. This allows
for the key-value pairs from the stores to be decoded (_i.e_ unmarshalled)
to their corresponding types. In particular, it matches the key to a concrete type
and then unmarshals the value from the `KVPair` to the type provided.
fixes: permalinks for docs (#8838) * fixed broken links, typos * Update docs/ibc/overview.md Co-authored-by: Amaury <1293565+amaurym@users.noreply.github.com> * Update docs/intro/sdk-app-architecture.md Co-authored-by: Marko <markobaricevic3778@gmail.com> * Update docs/building-modules/simulator.md Co-authored-by: Marko <markobaricevic3778@gmail.com> * build(deps): bump JamesIves/github-pages-deploy-action from 4.0.0 to 4.1.0 (#8792) Bumps [JamesIves/github-pages-deploy-action](https://github.com/JamesIves/github-pages-deploy-action) from 4.0.0 to 4.1.0. - [Release notes](https://github.com/JamesIves/github-pages-deploy-action/releases) - [Commits](https://github.com/JamesIves/github-pages-deploy-action/compare/4.0.0...3dbacc7e69578703f91f077118b3475862cb09b8) Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * fix multisig account pubkeys migration (#8794) closes: #8776 * Update mergify (#8784) * Update mergify Prep for the v0.42 release series. * retire v0.41, the hub can upgrade to v0.42 smoothly * perf change (#8796) Co-authored-by: Alessio Treglia <alessio@tendermint.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * Supply by denom Migrations (#8780) * Add back supply proto * Add migration for supply * Fix lint * Update x/bank/spec/01_state.md * Fix test * Proto gen * Update x/bank/spec/01_state.md * Make proto gen Co-authored-by: Jonathan Gimeno <jgimeno@gmail.com> * fix make protoc error (#8799) * reduce gas costs by 10x for transient store operations (#8790) * reduce gas costs by 10x for transient store operations * fix TestTransientGasConfig for ReadCostFlat * added changelog entry * fix changelog * fix changelog Co-authored-by: Alessio Treglia <alessio@tendermint.com> * x/gov: fix NormalizeProposalType() return values (#8808) Closes: #8806 * store/cachekv: use typed types/kv.List instead of container/list.List (#8811) Reduces CPU burn by using a typed List to avoid the expensive type assertions from using an interface. This is the only option for now until Go makes generics generally available. The change brings time spent on the time assertion cummulatively to: 580ms down from 6.88s Explanation: The type assertions were showing up heavily in profiles: * Before this commit ```shell Total: 42.18s ROUTINE ======================== github.com/cosmos/cosmos-sdk/store/cachekv.newMemIterator in /Users/emmanuelodeke/go/src/github.com/cosmos/cosmos-sdk/store/cachekv/memiterator.go 14.01s 18.87s (flat, cum) 44.74% of Total . . 17: items []*kv.Pair . . 18: ascending bool . . 19:} . . 20: . . 21:func newMemIterator(start, end []byte, items *list.List, ascending bool) *memIterator { . 620ms 22: itemsInDomain := make([]*kv.Pair, 0, items.Len()) . . 23: . . 24: var entered bool . . 25: 510ms 870ms 26: for e := items.Front(); e != nil; e = e.Next() { 6.85s 6.88s 27: item := e.Value.(*kv.Pair) 5.71s 8.19s 28: if !dbm.IsKeyInDomain(item.Key, start, end) { 120ms 120ms 29: if entered { . . 30: break . . 31: } . . 32: . . 33: continue . . 34: } . . 35: 820ms 980ms 36: itemsInDomain = append(itemsInDomain, item) . . 37: entered = true . . 38: } . . 39: . 1.21s 40: return &memIterator{ . . 41: start: start, . . 42: end: end, . . 43: items: itemsInDomain, . . 44: ascending: ascending, . . 45: } ``` and given that the list only uses that type, it is only right to lift the code from container/list.List, and only modify Element.Value's type. For emphasis, the code is basically just a retrofit of container/list/list.go but with a typing, and we'll keep it as is until perhaps Go1.17 or Go1.18 or when everyone uses Go1.17+ after generics have landed. * After this commit ```shell Total: 45.25s ROUTINE ======================== github.com/cosmos/cosmos-sdk/store/cachekv.newMemIterator in /Users/emmanuelodeke/go/src/github.com/cosmos/cosmos-sdk/store/cachekv/memiterator.go 4.84s 6.77s (flat, cum) 14.96% of Total . . 16: items []*kv.Pair . . 17: ascending bool . . 18:} . . 19: . . 20:func newMemIterator(start, end []byte, items *kv.List, ascending bool) *memIterator { . 330ms 21: itemsInDomain := make([]*kv.Pair, 0, items.Len()) . . 22: . . 23: var entered bool . . 24: 60ms 160ms 25: for e := items.Front(); e != nil; e = e.Next() { 580ms 580ms 26: item := e.Value 3.68s 4.78s 27: if !dbm.IsKeyInDomain(item.Key, start, end) { 80ms 80ms 28: if entered { . . 29: break . . 30: } . . 31: . . 32: continue . . 33: } . . 34: 440ms 580ms 35: itemsInDomain = append(itemsInDomain, item) . . 36: entered = true . . 37: } . . 38: . 260ms 39: return &memIterator{ . . 40: start: start, . . 41: end: end, . . 42: items: itemsInDomain, . . 43: ascending: ascending, . . 44: } ``` Fixes #8810 * Move all migration scripts to v043 (#8814) * Move all migration scripts to v043 * Fix permaling * Fix test * Fix test again Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * permalinks Co-authored-by: chrly <chrly@chrlys-MacBook-Pro.local> Co-authored-by: Amaury <1293565+amaurym@users.noreply.github.com> Co-authored-by: Jonathan Gimeno <jgimeno@gmail.com> Co-authored-by: Marko <markobaricevic3778@gmail.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Akhil Kumar P <36399231+akhilkumarpilli@users.noreply.github.com> Co-authored-by: Alessio Treglia <alessio@tendermint.com> Co-authored-by: Marko <marbar3778@yahoo.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: Albert Chon <albert@injectiveprotocol.com> Co-authored-by: Emmanuel T Odeke <emmanuel@orijtech.com>
2021-03-10 11:14:12 -08:00
You can use the example [here](https://github.com/cosmos/cosmos-sdk/blob/v0.42.0/x/distribution/simulation/decoder.go) from the distribution module to implement your store decoders.
### Randomized genesis
The simulator tests different scenarios and values for genesis parameters
in order to fully test the edge cases of specific modules. The `simulator` package from each module must expose a `RandomizedGenState` function to generate the initial random `GenesisState` from a given seed.
Once the module genesis parameter are generated randomly (or with the key and
values defined in a `params` file), they are marshaled to JSON format and added
to the app genesis JSON to use it on the simulations.
You can check an example on how to create the randomized genesis [here](https://github.com/cosmos/cosmos-sdk/blob/v0.42.0/x/staking/simulation/genesis.go).
### Randomized parameter changes
The simulator is able to test parameter changes at random. The simulator package from each module must contain a `RandomizedParams` func that will simulate parameter changes of the module throughout the simulations lifespan.
fixes: permalinks for docs (#8838) * fixed broken links, typos * Update docs/ibc/overview.md Co-authored-by: Amaury <1293565+amaurym@users.noreply.github.com> * Update docs/intro/sdk-app-architecture.md Co-authored-by: Marko <markobaricevic3778@gmail.com> * Update docs/building-modules/simulator.md Co-authored-by: Marko <markobaricevic3778@gmail.com> * build(deps): bump JamesIves/github-pages-deploy-action from 4.0.0 to 4.1.0 (#8792) Bumps [JamesIves/github-pages-deploy-action](https://github.com/JamesIves/github-pages-deploy-action) from 4.0.0 to 4.1.0. - [Release notes](https://github.com/JamesIves/github-pages-deploy-action/releases) - [Commits](https://github.com/JamesIves/github-pages-deploy-action/compare/4.0.0...3dbacc7e69578703f91f077118b3475862cb09b8) Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * fix multisig account pubkeys migration (#8794) closes: #8776 * Update mergify (#8784) * Update mergify Prep for the v0.42 release series. * retire v0.41, the hub can upgrade to v0.42 smoothly * perf change (#8796) Co-authored-by: Alessio Treglia <alessio@tendermint.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * Supply by denom Migrations (#8780) * Add back supply proto * Add migration for supply * Fix lint * Update x/bank/spec/01_state.md * Fix test * Proto gen * Update x/bank/spec/01_state.md * Make proto gen Co-authored-by: Jonathan Gimeno <jgimeno@gmail.com> * fix make protoc error (#8799) * reduce gas costs by 10x for transient store operations (#8790) * reduce gas costs by 10x for transient store operations * fix TestTransientGasConfig for ReadCostFlat * added changelog entry * fix changelog * fix changelog Co-authored-by: Alessio Treglia <alessio@tendermint.com> * x/gov: fix NormalizeProposalType() return values (#8808) Closes: #8806 * store/cachekv: use typed types/kv.List instead of container/list.List (#8811) Reduces CPU burn by using a typed List to avoid the expensive type assertions from using an interface. This is the only option for now until Go makes generics generally available. The change brings time spent on the time assertion cummulatively to: 580ms down from 6.88s Explanation: The type assertions were showing up heavily in profiles: * Before this commit ```shell Total: 42.18s ROUTINE ======================== github.com/cosmos/cosmos-sdk/store/cachekv.newMemIterator in /Users/emmanuelodeke/go/src/github.com/cosmos/cosmos-sdk/store/cachekv/memiterator.go 14.01s 18.87s (flat, cum) 44.74% of Total . . 17: items []*kv.Pair . . 18: ascending bool . . 19:} . . 20: . . 21:func newMemIterator(start, end []byte, items *list.List, ascending bool) *memIterator { . 620ms 22: itemsInDomain := make([]*kv.Pair, 0, items.Len()) . . 23: . . 24: var entered bool . . 25: 510ms 870ms 26: for e := items.Front(); e != nil; e = e.Next() { 6.85s 6.88s 27: item := e.Value.(*kv.Pair) 5.71s 8.19s 28: if !dbm.IsKeyInDomain(item.Key, start, end) { 120ms 120ms 29: if entered { . . 30: break . . 31: } . . 32: . . 33: continue . . 34: } . . 35: 820ms 980ms 36: itemsInDomain = append(itemsInDomain, item) . . 37: entered = true . . 38: } . . 39: . 1.21s 40: return &memIterator{ . . 41: start: start, . . 42: end: end, . . 43: items: itemsInDomain, . . 44: ascending: ascending, . . 45: } ``` and given that the list only uses that type, it is only right to lift the code from container/list.List, and only modify Element.Value's type. For emphasis, the code is basically just a retrofit of container/list/list.go but with a typing, and we'll keep it as is until perhaps Go1.17 or Go1.18 or when everyone uses Go1.17+ after generics have landed. * After this commit ```shell Total: 45.25s ROUTINE ======================== github.com/cosmos/cosmos-sdk/store/cachekv.newMemIterator in /Users/emmanuelodeke/go/src/github.com/cosmos/cosmos-sdk/store/cachekv/memiterator.go 4.84s 6.77s (flat, cum) 14.96% of Total . . 16: items []*kv.Pair . . 17: ascending bool . . 18:} . . 19: . . 20:func newMemIterator(start, end []byte, items *kv.List, ascending bool) *memIterator { . 330ms 21: itemsInDomain := make([]*kv.Pair, 0, items.Len()) . . 22: . . 23: var entered bool . . 24: 60ms 160ms 25: for e := items.Front(); e != nil; e = e.Next() { 580ms 580ms 26: item := e.Value 3.68s 4.78s 27: if !dbm.IsKeyInDomain(item.Key, start, end) { 80ms 80ms 28: if entered { . . 29: break . . 30: } . . 31: . . 32: continue . . 33: } . . 34: 440ms 580ms 35: itemsInDomain = append(itemsInDomain, item) . . 36: entered = true . . 37: } . . 38: . 260ms 39: return &memIterator{ . . 40: start: start, . . 41: end: end, . . 42: items: itemsInDomain, . . 43: ascending: ascending, . . 44: } ``` Fixes #8810 * Move all migration scripts to v043 (#8814) * Move all migration scripts to v043 * Fix permaling * Fix test * Fix test again Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * permalinks Co-authored-by: chrly <chrly@chrlys-MacBook-Pro.local> Co-authored-by: Amaury <1293565+amaurym@users.noreply.github.com> Co-authored-by: Jonathan Gimeno <jgimeno@gmail.com> Co-authored-by: Marko <markobaricevic3778@gmail.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Akhil Kumar P <36399231+akhilkumarpilli@users.noreply.github.com> Co-authored-by: Alessio Treglia <alessio@tendermint.com> Co-authored-by: Marko <marbar3778@yahoo.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: Albert Chon <albert@injectiveprotocol.com> Co-authored-by: Emmanuel T Odeke <emmanuel@orijtech.com>
2021-03-10 11:14:12 -08:00
You can see how an example of what is needed to fully test parameter changes [here](https://github.com/cosmos/cosmos-sdk/blob/v0.42.0/x/staking/simulation/params.go)
### Random weighted operations
Operations are one of the crucial parts of the Cosmos SDK simulation. They are the transactions
(`Msg`) that are simulated with random field values. The sender of the operation
is also assigned randomly.
Operations on the simulation are simulated using the full [transaction cycle](../core/transactions.md) of a
`ABCI` application that exposes the `BaseApp`.
Shown below is how weights are set:
+++ https://github.com/cosmos/cosmos-sdk/blob/v0.42.1/x/staking/simulation/operations.go#L18-L68
As you can see, the weights are predefined in this case. Options exist to override this behavior with different weights. One option is to use `*rand.Rand` to define a random weight for the operation, or you can inject your own predefined weights.
Here is how one can override the above package `simappparams`.
+++ https://github.com/cosmos/gaia/blob/master/sims.mk#L9-L22
For the last test a tool called runsim <!-- # TODO: add link to runsim readme when its created --> is used, this is used to parallelize go test instances, provide info to Github and slack integrations to provide information to your team on how the simulations are running.
### Random proposal contents
Randomized governance proposals are also supported on the Cosmos SDK simulator. Each
module must define the governance proposal `Content`s that they expose and register
them to be used on the parameters.
## Registering simulation functions
Now that all the required functions are defined, we need to integrate them into the module pattern within the `module.go`:
fixes: permalinks for docs (#8838) * fixed broken links, typos * Update docs/ibc/overview.md Co-authored-by: Amaury <1293565+amaurym@users.noreply.github.com> * Update docs/intro/sdk-app-architecture.md Co-authored-by: Marko <markobaricevic3778@gmail.com> * Update docs/building-modules/simulator.md Co-authored-by: Marko <markobaricevic3778@gmail.com> * build(deps): bump JamesIves/github-pages-deploy-action from 4.0.0 to 4.1.0 (#8792) Bumps [JamesIves/github-pages-deploy-action](https://github.com/JamesIves/github-pages-deploy-action) from 4.0.0 to 4.1.0. - [Release notes](https://github.com/JamesIves/github-pages-deploy-action/releases) - [Commits](https://github.com/JamesIves/github-pages-deploy-action/compare/4.0.0...3dbacc7e69578703f91f077118b3475862cb09b8) Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * fix multisig account pubkeys migration (#8794) closes: #8776 * Update mergify (#8784) * Update mergify Prep for the v0.42 release series. * retire v0.41, the hub can upgrade to v0.42 smoothly * perf change (#8796) Co-authored-by: Alessio Treglia <alessio@tendermint.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * Supply by denom Migrations (#8780) * Add back supply proto * Add migration for supply * Fix lint * Update x/bank/spec/01_state.md * Fix test * Proto gen * Update x/bank/spec/01_state.md * Make proto gen Co-authored-by: Jonathan Gimeno <jgimeno@gmail.com> * fix make protoc error (#8799) * reduce gas costs by 10x for transient store operations (#8790) * reduce gas costs by 10x for transient store operations * fix TestTransientGasConfig for ReadCostFlat * added changelog entry * fix changelog * fix changelog Co-authored-by: Alessio Treglia <alessio@tendermint.com> * x/gov: fix NormalizeProposalType() return values (#8808) Closes: #8806 * store/cachekv: use typed types/kv.List instead of container/list.List (#8811) Reduces CPU burn by using a typed List to avoid the expensive type assertions from using an interface. This is the only option for now until Go makes generics generally available. The change brings time spent on the time assertion cummulatively to: 580ms down from 6.88s Explanation: The type assertions were showing up heavily in profiles: * Before this commit ```shell Total: 42.18s ROUTINE ======================== github.com/cosmos/cosmos-sdk/store/cachekv.newMemIterator in /Users/emmanuelodeke/go/src/github.com/cosmos/cosmos-sdk/store/cachekv/memiterator.go 14.01s 18.87s (flat, cum) 44.74% of Total . . 17: items []*kv.Pair . . 18: ascending bool . . 19:} . . 20: . . 21:func newMemIterator(start, end []byte, items *list.List, ascending bool) *memIterator { . 620ms 22: itemsInDomain := make([]*kv.Pair, 0, items.Len()) . . 23: . . 24: var entered bool . . 25: 510ms 870ms 26: for e := items.Front(); e != nil; e = e.Next() { 6.85s 6.88s 27: item := e.Value.(*kv.Pair) 5.71s 8.19s 28: if !dbm.IsKeyInDomain(item.Key, start, end) { 120ms 120ms 29: if entered { . . 30: break . . 31: } . . 32: . . 33: continue . . 34: } . . 35: 820ms 980ms 36: itemsInDomain = append(itemsInDomain, item) . . 37: entered = true . . 38: } . . 39: . 1.21s 40: return &memIterator{ . . 41: start: start, . . 42: end: end, . . 43: items: itemsInDomain, . . 44: ascending: ascending, . . 45: } ``` and given that the list only uses that type, it is only right to lift the code from container/list.List, and only modify Element.Value's type. For emphasis, the code is basically just a retrofit of container/list/list.go but with a typing, and we'll keep it as is until perhaps Go1.17 or Go1.18 or when everyone uses Go1.17+ after generics have landed. * After this commit ```shell Total: 45.25s ROUTINE ======================== github.com/cosmos/cosmos-sdk/store/cachekv.newMemIterator in /Users/emmanuelodeke/go/src/github.com/cosmos/cosmos-sdk/store/cachekv/memiterator.go 4.84s 6.77s (flat, cum) 14.96% of Total . . 16: items []*kv.Pair . . 17: ascending bool . . 18:} . . 19: . . 20:func newMemIterator(start, end []byte, items *kv.List, ascending bool) *memIterator { . 330ms 21: itemsInDomain := make([]*kv.Pair, 0, items.Len()) . . 22: . . 23: var entered bool . . 24: 60ms 160ms 25: for e := items.Front(); e != nil; e = e.Next() { 580ms 580ms 26: item := e.Value 3.68s 4.78s 27: if !dbm.IsKeyInDomain(item.Key, start, end) { 80ms 80ms 28: if entered { . . 29: break . . 30: } . . 31: . . 32: continue . . 33: } . . 34: 440ms 580ms 35: itemsInDomain = append(itemsInDomain, item) . . 36: entered = true . . 37: } . . 38: . 260ms 39: return &memIterator{ . . 40: start: start, . . 41: end: end, . . 42: items: itemsInDomain, . . 43: ascending: ascending, . . 44: } ``` Fixes #8810 * Move all migration scripts to v043 (#8814) * Move all migration scripts to v043 * Fix permaling * Fix test * Fix test again Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * permalinks Co-authored-by: chrly <chrly@chrlys-MacBook-Pro.local> Co-authored-by: Amaury <1293565+amaurym@users.noreply.github.com> Co-authored-by: Jonathan Gimeno <jgimeno@gmail.com> Co-authored-by: Marko <markobaricevic3778@gmail.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Akhil Kumar P <36399231+akhilkumarpilli@users.noreply.github.com> Co-authored-by: Alessio Treglia <alessio@tendermint.com> Co-authored-by: Marko <marbar3778@yahoo.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: Albert Chon <albert@injectiveprotocol.com> Co-authored-by: Emmanuel T Odeke <emmanuel@orijtech.com>
2021-03-10 11:14:12 -08:00
+++ https://github.com/cosmos/cosmos-sdk/blob/v0.42.0/x/distribution/module.go
## App Simulator manager
The following step is setting up the `SimulatorManager` at the app level. This
is required for the simulation test files on the next step.
```go
type CustomApp struct {
...
sm *module.SimulationManager
}
```
Then at the instantiation of the application, we create the `SimulationManager`
instance in the same way we create the `ModuleManager` but this time we only pass
the modules that implement the simulation functions from the `AppModuleSimulation`
interface described above.
```go
func NewCustomApp(...) {
// create the simulation manager and define the order of the modules for deterministic simulations
app.sm = module.NewSimulationManager(
auth.NewAppModule(app.accountKeeper),
bank.NewAppModule(app.bankKeeper, app.accountKeeper),
supply.NewAppModule(app.supplyKeeper, app.accountKeeper),
ov.NewAppModule(app.govKeeper, app.accountKeeper, app.supplyKeeper),
mint.NewAppModule(app.mintKeeper),
distr.NewAppModule(app.distrKeeper, app.accountKeeper, app.supplyKeeper, app.stakingKeeper),
staking.NewAppModule(app.stakingKeeper, app.accountKeeper, app.supplyKeeper),
slashing.NewAppModule(app.slashingKeeper, app.accountKeeper, app.stakingKeeper),
)
// register the store decoders for simulation tests
app.sm.RegisterStoreDecoders()
...
}
```