docs: adding acceptance tests description (#10534)
## Description We start to adopt acceptance tests when designing new module. We had few internal sessions on how to do and started to practice it. We should better document the process and start requiring it with new documents. --- ### Author Checklist *All items are required. Please add a note to the item if the item is not applicable and please add links to any relevant follow up issues.* I have... - [x] included the correct [type prefix](https://github.com/commitizen/conventional-commit-types/blob/v3.0.0/index.json) in the PR title - [x] added `!` to the type prefix if API or client breaking change - [x] targeted the correct branch (see [PR Targeting](https://github.com/cosmos/cosmos-sdk/blob/master/CONTRIBUTING.md#pr-targeting)) - [x] provided a link to the relevant issue or specification - [x] followed the guidelines for [building modules](https://github.com/cosmos/cosmos-sdk/blob/master/docs/building-modules) - [x] included the necessary unit and integration [tests](https://github.com/cosmos/cosmos-sdk/blob/master/CONTRIBUTING.md#testing) - [x] added a changelog entry to `CHANGELOG.md` - [x] included comments for [documenting Go code](https://blog.golang.org/godoc) - [x] updated the relevant documentation or specification - [x] reviewed "Files changed" and left comments if necessary - [x] confirmed all CI checks have passed ### Reviewers Checklist *All items are required. Please add a note if the item is not applicable and please add your handle next to the items reviewed if you only reviewed selected items.* I have... - [ ] confirmed the correct [type prefix](https://github.com/commitizen/conventional-commit-types/blob/v3.0.0/index.json) in the PR title - [ ] confirmed `!` in the type prefix if API or client breaking change - [ ] confirmed all author checklist items have been addressed - [ ] reviewed state machine logic - [ ] reviewed API design and naming - [ ] reviewed documentation is accurate - [ ] reviewed tests and test coverage - [ ] manually tested (if applicable)
This commit is contained in:
parent
94def69926
commit
f7f1b86506
|
@ -31,7 +31,62 @@ Security:
|
||||||
+ code must be always deterministic
|
+ code must be always deterministic
|
||||||
+ Thread safety. If some functionality is not thread-safe, or uses something that is not thread-safe, then clearly indicate the risk on each level.
|
+ Thread safety. If some functionality is not thread-safe, or uses something that is not thread-safe, then clearly indicate the risk on each level.
|
||||||
|
|
||||||
## Testing
|
## Acceptance tests
|
||||||
|
|
||||||
|
Start the design by defining Acceptance Tests. The purpose of Acceptance Testing is to
|
||||||
|
validate that the product being developed corresponds to the needs of the real users
|
||||||
|
and is ready for launch. Hence we often talk about **User Acceptance Test** (UAT).
|
||||||
|
It also gives a better understanding of the product and helps designing a right interface
|
||||||
|
and API.
|
||||||
|
|
||||||
|
UAT should be revisited at each stage of the product development:
|
||||||
|
|
||||||
|

|
||||||
|
|
||||||
|
### Why Acceptance Testing?
|
||||||
|
|
||||||
|
- Automated acceptance tests catch serious problems that unit or component test suites could never catch.
|
||||||
|
- Automated acceptance tests deliver business value the users are expecting as they test user scenarios.
|
||||||
|
- Automated acceptance tests executed and passed on every build help improve the software delivery process.
|
||||||
|
- Testers, developers, and customers need to work closely to create suitable automated acceptance test suites.
|
||||||
|
|
||||||
|
### How to define Acceptance Test?
|
||||||
|
|
||||||
|
The best way to define AT is by starting from the user stories and think about all positive and negative scenarios a user can perform.
|
||||||
|
|
||||||
|
Product Developers should collaborate with stakeholders to define AT. Functional experts and business users are both needed for defining AT.
|
||||||
|
|
||||||
|
A good pattern for defining AT is listing scenarios with [GIVEN-WHEN-THEN](https://martinfowler.com/bliki/GivenWhenThen.html) format where:
|
||||||
|
|
||||||
|
- **GIVEN**: A set of initial circumstances (e.g. bank balance)
|
||||||
|
- **WHEN**: Some event happens (e.g. customer attempts a transfer)
|
||||||
|
- **THEN**: The expected result as per the defined behavior of the system
|
||||||
|
|
||||||
|
In other words: we define a use case input, current state and the expected outcome. Example:
|
||||||
|
|
||||||
|
> Feature: User trades stocks.
|
||||||
|
> Scenario: User requests a sell before close of trading
|
||||||
|
>
|
||||||
|
> Given I have 100 shares of MSFT stock
|
||||||
|
> And I have 150 shares of APPL stock
|
||||||
|
> And the time is before close of trading
|
||||||
|
>
|
||||||
|
> When I ask to sell 20 shares of MSFT stock
|
||||||
|
>
|
||||||
|
> Then I should have 80 shares of MSFT stock
|
||||||
|
> And I should have 150 shares of APPL stock
|
||||||
|
> And a sell order for 20 shares of MSFT stock should have been executed
|
||||||
|
|
||||||
|
*Reference: [writing acceptance tests](https://openclassrooms.com/en/courses/4544611-write-agile-documentation-user-stories-acceptance-tests/4810081-writing-acceptance-tests)*.
|
||||||
|
|
||||||
|
### How and where to add acceptance tests?
|
||||||
|
|
||||||
|
Acceptance tests are written in the Markdown format, using the scenario template described above, and be part of the specification (`xx_test.md` file in _spec_ directory). Example: (`eco-credits/spec/06.test.md`)[https://github.com/regen-network/regen-ledger/blob/7297783577e6cd102c5093365b573163680f36a1/x/ecocredit/spec/06_tests.md]
|
||||||
|
|
||||||
|
Acceptance tests should be defined during the design phase or at an early stage of development. Moreover, they should be defined before writing a module architecture - it will clarify the purpose and usage of the software.
|
||||||
|
Automated tests should cover all acceptance tests scenarios.
|
||||||
|
|
||||||
|
## Automated Tests
|
||||||
|
|
||||||
Make sure your code is well tested:
|
Make sure your code is well tested:
|
||||||
|
|
||||||
|
@ -87,3 +142,11 @@ Desired outcomes:
|
||||||
- QA reports. Goal is to guide with new tasks and be one of the QA measures.
|
- QA reports. Goal is to guide with new tasks and be one of the QA measures.
|
||||||
|
|
||||||
As a developer, you must help the QA team by providing instructions for User Experience (UX) and functional testing.
|
As a developer, you must help the QA team by providing instructions for User Experience (UX) and functional testing.
|
||||||
|
|
||||||
|
### QA Team to cross check Acceptance Tests
|
||||||
|
|
||||||
|
Once the AT are defined, the QA team will have an overview of the behavior a user can expect and:
|
||||||
|
|
||||||
|
- validate the user experience will be good
|
||||||
|
- validate the implementation conforms the acceptance tests
|
||||||
|
- by having a broader overview of the use cases, QA team should be able to define **test suites** and test data to efficiently automate Acceptance Tests and reuse the work.
|
||||||
|
|
Binary file not shown.
After Width: | Height: | Size: 120 KiB |
|
@ -37,6 +37,7 @@ following list is nonbinding and all files are optional.
|
||||||
- `XX_client.md` - list and describe CLI commands and gRPC and REST endpoints
|
- `XX_client.md` - list and describe CLI commands and gRPC and REST endpoints
|
||||||
- `XX_params.md` - list all module parameters, their types (in JSON) and examples
|
- `XX_params.md` - list all module parameters, their types (in JSON) and examples
|
||||||
- `XX_future_improvements.md` - describe future improvements of this module
|
- `XX_future_improvements.md` - describe future improvements of this module
|
||||||
|
- `XX_tests.md` - acceptance tests
|
||||||
- `XX_appendix.md` - supplementary details referenced elsewhere within the spec
|
- `XX_appendix.md` - supplementary details referenced elsewhere within the spec
|
||||||
|
|
||||||
### Notation for key-value mapping
|
### Notation for key-value mapping
|
||||||
|
|
|
@ -94,7 +94,7 @@ func RandTimestamp(r *rand.Rand) time.Time {
|
||||||
unixTime := r.Int63n(60*60*24*365*200) * 1000 // convert to milliseconds
|
unixTime := r.Int63n(60*60*24*365*200) * 1000 // convert to milliseconds
|
||||||
|
|
||||||
// Get milliseconds for a time between Jan 1, 2062 and Jan 1, 2262
|
// Get milliseconds for a time between Jan 1, 2062 and Jan 1, 2262
|
||||||
rtime := time.UnixMilli(start + unixTime).UnixMilli() / 1000
|
rtime := time.UnixMilli(start+unixTime).UnixMilli() / 1000
|
||||||
return time.Unix(rtime, 0)
|
return time.Unix(rtime, 0)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
|
@ -41,6 +41,7 @@ The primary key parts can be []byte, string, and `uint64` types.
|
||||||
### Key codec
|
### Key codec
|
||||||
|
|
||||||
Key parts, except the last part, follow these rules:
|
Key parts, except the last part, follow these rules:
|
||||||
- []byte is encoded with a single byte length prefix
|
|
||||||
- strings are null-terminated
|
- []byte is encoded with a single byte length prefix
|
||||||
- `uint64` are encoded using 8 byte big endian.
|
- strings are null-terminated
|
||||||
|
- `uint64` are encoded using 8 byte big endian.
|
||||||
|
|
|
@ -13,4 +13,3 @@ The orm package provides a framework for creating relational database tables wit
|
||||||
3. **[Iterator and Pagination](03_iterator_pagination.md)**
|
3. **[Iterator and Pagination](03_iterator_pagination.md)**
|
||||||
- [Iterator](03_iterator_pagination.md#iterator)
|
- [Iterator](03_iterator_pagination.md#iterator)
|
||||||
- [Pagination](03_iterator_pagination.md#pagination)
|
- [Pagination](03_iterator_pagination.md#pagination)
|
||||||
|
|
||||||
|
|
Loading…
Reference in New Issue