Simulation docs (#5033)

* simulation docs

* update docs with the latest simulation changes

* minor imporvments

* clean up of simulation.md

* expand section on weights

* minor reword

* minor wording fix

Co-authored-by: Marko <marbar3778@yahoo.com>

docs/building-modules/simulator.md

@@ -0,0 +1,123 @@
+# Module Simulation
+
+## Prerequisites
+
+* [Cosmos Blockchain Simulator](./../using-the-sdk/simulation.md)
+
+## Synopsis
+
+This document details how to define each module simulation functions to be
+integrated with the application `SimulationManager`.
+
+* [Simulation package](#simulation-package)
+  * [Store decoders](#store-decoders)
+  * [Randomized genesis](#randomized-genesis)
+  * [Randomized parameters](#randomized-parameters)
+  * [Random weighted operations](#random-weighted-operations)
+  * [Random proposal contents](#random-proposal-contents)
+* [Registering the module simulation functions](#registering-simulation-functions)
+* [App simulator manager](#app-simulator-manager)
+* [Simulation tests](#simulation-tests)
+
+## Simulation package
+
+Every module that implements the SDK simulator needs to have a `x/<module>/simulation`
+package which contains the primary functions required by the fuzz tests: store
+decoders, randomized genesis state and parameters, weighted operations and proposal
+contents.
+
+### Store decoders
+
+Registering the store decoders is required for the `AppImportExport`. This allows
+for the key-value pairs from the stores to be decoded (_i.e_ unmarshalled)
+to their corresponding types. In particular, it matches the key to a concrete type
+and then unmarshals the value from the `KVPair` to the type provided.
+
+You can use the example [here](https://github.com/cosmos/cosmos-sdk/blob/release%2Fv0.38.0/x/distribution/simulation/decoder.go) from the distribution module to implement your store decoders.
+
+### Randomized genesis
+
+The simulator tests different scenarios and values for genesis parameters
+in order to fully test the edge cases of specific modules. The `simulator` package from each module must expose a `RandomizedGenState` function to generate the initial random `GenesisState` from a given seed. In
+
+Once the module genesis parameter are generated randomly (or with the key and
+values defined in a `params` file), they are marshaled to JSON format and added
+to the app genesis JSON to use it on the simulations.
+
+You can check an example on how to create the randomized genesis [here](https://github.com/cosmos/cosmos-sdk/blob/release%2Fv0.38.0/x/staking/simulation/genesis.go).
+
+### Randomized parameter changes
+
+The simulator is able to test parameter changes at random. The simulator package from each module must contain a `RandomizedParams` func that will simulate parameter changes of the module throughout the simulations lifespan.
+
+You can see how an example of what is needed to fully test parameter changes [here](https://github.com/cosmos/cosmos-sdk/blob/release%2Fv0.38.0/x/staking/simulation/params.go)
+
+### Random weighted operations
+
+Operations are one of the crucial parts of the SDK simulation. They are the transactions
+(`Msg`) that are simulated with random field values. The sender of the operation
+is also assigned randomly.
+
+Operations on the simulation are simulated using the full [transaction cycle](../core/transactions.md) of a
+`ABCI` application that exposes the `BaseApp`.
+
+Shown below is how weights are set:
+
++++ https://github.com/cosmos/cosmos-sdk/blob/release%2Fv0.38.0/x/staking/simulation/operations.go#L18-L92
+
+As you can see the weights are predefined in this case but there are options on how to override this behavior with different weights. One is allowing `*rand.Rand` to define a random weight for the operation, or you can inject your own predefined weights.
+
+Here is how one can override the above package `simappparams`.
+
++++ https://github.com/cosmos/gaia/blob/master/sims.mk#L9-L22
+
+For the last test a tool called runsim  <!-- # TODO: add link to runsim readme when its created --> is used, this is used to parallelize go test instances, provide info to Github and slack integrations to provide information to your team on how the simulations are running.  
+
+### Random proposal contents
+
+Randomized governance proposals are also supported on the SDK simulator. Each
+module must define the governance proposal `Content`s that they expose and register
+them to be used on the parameters.
+
+## Registering simulation functions
+
+Now that all the required functions are defined, we need to integrate them into the module pattern within the `module.go`:
+
++++ https://github.com/cosmos/cosmos-sdk/blob/release%2Fv0.38.0/x/distribution/module.go#L156-L185
+
+## App Simulator manager
+
+The following step is setting up the `SimulatorManager` at the app level. This
+is required for the simulation test files on the next step.
+
+```go
+type CustomApp struct {
+  ...
+  sm *module.SimulationManager
+}
+```
+
+Then at the instantiation of the application, we create the `SimulationManager`
+instance in the same way we create the `ModuleManager` but this time we only pass
+the modules that implement the simulation functions from the `AppModuleSimulation`
+interface described above.
+
+```go
+func NewCustomApp(...) {
+  // create the simulation manager and define the order of the modules for deterministic simulations
+  app.sm = module.NewSimulationManager(
+    auth.NewAppModule(app.accountKeeper),
+    bank.NewAppModule(app.bankKeeper, app.accountKeeper),
+    supply.NewAppModule(app.supplyKeeper, app.accountKeeper),
+    ov.NewAppModule(app.govKeeper, app.accountKeeper, app.supplyKeeper),
+    mint.NewAppModule(app.mintKeeper),
+    distr.NewAppModule(app.distrKeeper, app.accountKeeper, app.supplyKeeper, app.stakingKeeper),
+    staking.NewAppModule(app.stakingKeeper, app.accountKeeper, app.supplyKeeper),
+    slashing.NewAppModule(app.slashingKeeper, app.accountKeeper, app.stakingKeeper),
+  )
+
+  // register the store decoders for simulation tests
+  app.sm.RegisterStoreDecoders()
+  ...
+}
+```

docs/building-modules/structure.md

@@ -37,13 +37,18 @@ x/{module}
 │       ├── params.go
 │       ├── ...
 │       └── querier.go
+├── simulation
+│   ├── decoder.go
+│   ├── genesis.go
+│   ├── operations.go
+│   ├── params.go
+│   └── proposals.go
 ├── abci.go
 ├── alias.go
 ├── genesis.go
 ├── handler.go
-├── module.go
 ├── ...
-└── simulation.go
+└── module.go
 ```
 
 - `abci.go`: The module's `BeginBlocker` and `EndBlocker` implementations (if any).
@@ -53,7 +58,7 @@ there is nothing preventing developers from importing other packages from the mo
 (excluding`internal/`) but it is recommended that `alias.go` have everything
 exposed that other modules may need. The majority of the exported values here will
 typically come from `internal/` (see below).
-- `client/`: The module's CLI and REST client functionality implementation and 
+- `client/`: The module's CLI and REST client functionality implementation and
 testing.
 - `exported/`: The module's exported types -- typically type interfaces. If a module
 relies on other module keepers, it is expected to receive them as interface
@@ -79,8 +84,10 @@ allows for greater freedom of development while maintaining API stability.
   implementations such as the querier and invariants.
 - `module.go`: The module's implementation of the `AppModule` and `AppModuleBasic`
 interfaces.
-- `simulation.go`: The module's simulation messages and related types (if any).
+- `simulation/`: The module's simulation package defines all the required functions
+used on the blockchain simulator: randomized genesis state, parameters, weigthed
+operations, proposal contents and types decoders.
 
 ## Next {hide}
 
-Learn about [interfaces](../interfaces/interfaces-intro.md) {hide}
+Learn about [interfaces](../interfaces/interfaces-intro.md) {hide}

docs/package.json

@@ -19,4 +19,4 @@
     "vuepress-plugin-smooth-scroll": "0.0.9",
     "vuepress-theme-cosmos": "^1.0.113"
   }
-}
+}

docs/using-the-sdk/README.md

@@ -7,4 +7,3 @@ parent:
 
 - [Modules](../../x/README.md)
 - [Simulation](./simulation.md)
-

docs/using-the-sdk/simulation.md

@@ -1,49 +1,65 @@
 # Cosmos Blockchain Simulator
 
-The Cosmos SDK offers a full fledged simulation framework to fuzz test every message defined by a module.
+The Cosmos SDK offers a full fledged simulation framework to fuzz test every
+message defined by a module.
 
-This functionality is provided by the[`SimApp`](https://github.com/cosmos/cosmos-sdk/blob/master/simapp/app.go),
+On the SDK, this functionality is provided by the[`SimApp`](https://github.com/cosmos/cosmos-sdk/blob/master/simapp/app.go), which is a
-which is a dummy application that is used for running the [`simulation`](https://github.com/cosmos/cosmos-sdk/tree/master/x/simulation) module.
+`Baseapp` application that is used for running the [`simulation`](https://github.com/cosmos/cosmos-sdk/tree/master/x/simulation) module.
-This module defines all the simulation logic as well as the operations for randomized parameters like accounts, balances etc.
+This module defines all the simulation logic as well as the operations for
+randomized parameters like accounts, balances etc.
 
 ## Goals
 
-The blockchain simulator tests how the blockchain application would behave under real life circumstances by generating and sending randomized messages.
+The blockchain simulator tests how the blockchain application would behave under
-The goal of this is to detect and debug failures that could halt a live chain, by providing logs and statistics about the operations run by the simulator as well as exporting the latest application state when a failure was found.
+real life circumstances by generating and sending randomized messages.
+The goal of this is to detect and debug failures that could halt a live chain,
+by providing logs and statistics about the operations run by the simulator as
+well as exporting the latest application state when a failure was found.
 
-Its main difference with integration testing is that the simulator app allows you to pass parameters to customize the chain that's being simulated.
+Its main difference with integration testing is that the simulator app allows
-This comes in handy when trying to reproduce bugs that were generated in the provided operations (randomized or not).
+you to pass parameters to customize the chain that's being simulated.
+This comes in handy when trying to reproduce bugs that were generated in the
+provided operations (randomized or not).
 
 ## Simulation commands
 
-The simulation app has different commands, each of which tests a different failure type:
+The simulation app has different commands, each of which tests a different
+failure type:
 
-- `AppImportExport`: The simulator exports the initial app state and then it creates a new app with the exported `genesis.json` as an input, checking for inconsistencies between the stores.
+* `AppImportExport`: The simulator exports the initial app state and then it
-- `AppSimulationAfterImport`: Queues two simulations together. The first one provides the app state (_i.e_ genesis) to the second. Useful to test software upgrades or hard-forks from a live chain.
+creates a new app with the exported `genesis.json` as an input, checking for
-- `AppStateDeterminism`: Checks that all the nodes return the same values, in the same order.
+inconsistencies between the stores.
-- `BenchmarkInvariants`: Analyses the performance of running all the modules' invariants (_i.e_ secuentially runs a [benchmark](https://golang.org/pkg/testing/#hdr-Benchmarks) test). An invariant checks for differences between the values that are on the store and the passive tracker. Eg: total coins held by accounts vs total supply tracker.
+* `AppSimulationAfterImport`: Queues two simulations together. The first one provides the app state (_i.e_ genesis) to the second. Useful to test software upgrades or hard-forks from a live chain.
-- `FullAppSimulation`: General simulation mode. Runs the chain and the specified operations for a given number of blocks. Tests that there're no `panics` on the simulation. It does also run invariant checks on every `Period` but they are not benchmarked.
+* `AppStateDeterminism`: Checks that all the nodes return the same values, in the same order.
+* `BenchmarkInvariants`:  Analysis of the performance of running all modules' invariants (_i.e_ sequentially runs a [benchmark](https://golang.org/pkg/testing/#hdr-Benchmarks) test). An invariant checks for
+differences between the values that are on the store and the passive tracker. Eg: total coins held by accounts vs total supply tracker.
+* `FullAppSimulation`: General simulation mode. Runs the chain and the specified operations for a given number of blocks. Tests that there're no `panics` on the simulation. It does also run invariant checks on every `Period` but they are not benchmarked.
 
-Each simulation must receive a set of inputs (_i.e_ flags) such as the number of blocks that the simulation is run, seed, block size, etc.
+Each simulation must receive a set of inputs (_i.e_ flags) such as the number of
+blocks that the simulation is run, seed, block size, etc.
 Check the full list of flags [here](https://github.com/cosmos/cosmos-sdk/blob/adf6ddd4a807c8363e33083a3281f6a5e112ab89/simapp/sim_test.go#L34-L50).
 
 ## Simulator Modes
 
 In addition to the various inputs and commands, the simulator runs in three modes:
 
-1. Completely random where the initial state, module parameters and simulation parameters are **pseudo-randomly generated**.
+1. Completely random where the initial state, module parameters and simulation
+parameters are **pseudo-randomly generated**.
 2. From a `genesis.json` file where the initial state and the module parameters are defined.
 This mode is helpful for running simulations on a known state such as a live network export where a new (mostly likely breaking) version of the application needs to be tested.
 3. From a `params.json` file where the initial state is pseudo-randomly generated but the module and simulation parameters can be provided manually.
-This allows for a more controlled and deterministic simulation setup while allowing the state space to still be pseudo-randomly simulated. The list of available parameters is listed [here](https://github.com/cosmos/cosmos-sdk/blob/adf6ddd4a807c8363e33083a3281f6a5e112ab89/x/simulation/params.go#L170-L178).
+This allows for a more controlled and deterministic simulation setup while allowing the state space to still be pseudo-randomly simulated. 
+The list of available parameters are listed [here](https://github.com/cosmos/cosmos-sdk/blob/adf6ddd4a807c8363e33083a3281f6a5e112ab89/x/simulation/params.go#L170-L178).
 
 ::: tip
-These modes are not mutually exclusive. So you can for example run a randomly generated genesis state (`1`) with manually generated simulation params (`3`).
+These modes are not mutually exclusive. So you can for example run a randomly
+generated genesis state (`1`) with manually generated simulation params (`3`).
 :::
 
 ## Usage
 
-This is a general example of how simulations are run. For more specific examples check the SDK [Makefile](https://github.com/cosmos/cosmos-sdk/blob/adf6ddd4a807c8363e33083a3281f6a5e112ab89/Makefile#L88-L123).
+This is a general example of how simulations are run. For more specific examples
+check the SDK [Makefile](https://github.com/cosmos/cosmos-sdk/blob/adf6ddd4a807c8363e33083a3281f6a5e112ab89/Makefile#L88-L123).
 
 ```bash
  $ go test -mod=readonly github.com/cosmos/cosmos-sdk/simapp \
@@ -56,15 +72,26 @@ This is a general example of how simulations are run. For more specific examples
 
 Here are some suggestions when encountering a simulation failure:
 
-- Export the app state at the height were the failure was found. You can do this by passing the `-ExportStatePath` flag to the simulator.
+* Export the app state at the height were the failure was found. You can do this
-- Use `-Verbose` logs. They could give you a better hint on all the operations involved.
+by passing the `-ExportStatePath` flag to the simulator.
+* Use `-Verbose` logs. They could give you a better hint on all the operations
+involved.
+* Reduce the simulation `-Period`. This will run the invariants checks more
+frequently.
+* Print all the failed invariants at once with `-PrintAllInvariants`.
+* Try using another `-Seed`. If it can reproduce the same error and if it fails
+sooner you will spend less time running the simulations.
+* Reduce the `-NumBlocks` . How's the app state at the height previous to the
+failure?
+* Run invariants on every operation with `-SimulateEveryOperation`. _Note_: this
+will slow down your simulation **a lot**.
+* Try adding logs to operations that are not logged. You will have to define a
+[Logger](https://github.com/cosmos/cosmos-sdk/blob/adf6ddd4a807c8363e33083a3281f6a5e112ab89/x/staking/keeper/keeper.go#L65:17) on your `Keeper`.
 
-- Reduce the simulation `-Period`. This will run the invariants checks more frequently.
+## Use simulation in your SDK-based application
-- Print all the failed invariants at once with `-PrintAllInvariants`.
-- Try using another `-Seed`. If it can reproduce the same error and if it fails sooner you will spend less time running the simulations.
-- Reduce the `-NumBlocks` . How's the app state at the height previous to the failure?
-- Run invariants on every operation with `-SimulateEveryOperation`. _Note_: this will slow down your simulation **a lot**.
-- Try adding logs to operations that are not logged. You will have to define a [Logger](https://github.com/cosmos/cosmos-sdk/blob/adf6ddd4a807c8363e33083a3281f6a5e112ab89/x/staking/keeper/keeper.go#L65:17) on your `Keeper`.
 
-<!-- ## Use simulation in your SDK-based application -->
+Learn how you can integrate the simulation into your SDK-based application:
-<!-- TODO: link to the simulation section on the tutorial for how to add your own simulation messages -->
+
+* Application Simulation Manager
+* [Building modules: Simulator](../building-modules/simulator.md)
+* Simulator tests