* fix: state listener observe writes at wrong time (#13516)
* fix: state listener observe writes at wrong time
Closes: #13457
Currently state listener is notified when the cache store write, which happens in commit event only, which breaks the current design.
The solution (as discussed in the issue) is to listen state writes on rootmulti store only.
It also changes the file streamer to output single data file for the writes in the whole block, since we can't distinguish writes from different stage of abci events.
It adds new config items for file streamer:
- streamers.file.output-metadata
- streamers.file.stop-node-on-error
- streamers.file.fsync
* synchronous abci call, and format doc
* fix comment
* update file streamer readme and fix typos
* typo
* fix: state listener observe writes at wrong time
Closes: #13457
Currently state listener is notified when the cache store write, which happens in commit event only, which breaks the current design.
The solution (as discussed in the issue) is to listen state writes on rootmulti store only.
It also changes the file streamer to output single data file for the writes in the whole block, since we can't distinguish writes from different stage of abci events.
It adds new config items for file streamer:
- streamers.file.output-metadata
- streamers.file.stop-node-on-error
- streamers.file.fsync
synchronous abci call, and format doc
fix comment
update file streamer readme and fix typos
typo
* improve UX of file streamer, make it immediately usable after enabled
- set default value to write_dir.
- make write_dir based on home directory by default.
- auto-create the directory if not exists.
* get homePage from opts
Co-authored-by: Marko <marbar3778@yahoo.com>
(cherry picked from commit 1f91ee2ee941fd9a1dd4bc3acecd753e3cb7e237)
# Conflicts:
# CHANGELOG.md
# baseapp/streaming.go
# store/streaming/constructor.go
# store/streaming/file/service.go
* fix changelog
* fix conflicts
Co-authored-by: yihuang <huang@crypto.com>
Co-authored-by: Julien Robert <julien@rbrt.fr>
<!-- < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < ☺
v ✰ Thanks for creating a PR! ✰
v Before smashing the submit button please review the checkboxes.
v If a checkbox is n/a - please still include it but + a little note why
☺ > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -->
## Description
<!-- Add a description of the changes that this PR introduces and the files that
are the most critical to review.
-->
Hello 👋 this PR introduces the second stage of changes to support [ADR-038](https://github.com/cosmos/cosmos-sdk/pull/8012) state listening. This is rebased on top of the [first segment](https://github.com/cosmos/cosmos-sdk/pull/8551), which introduces the low level changes to the MultiStore and KVStore interfaces and implementations, the new WriteListener types, and the new listen.KVStore type.
In this segment we introduce the StreamingService interface, an implementation that writes out to files, and it's integration and configuration at the BaseApp level.
The idea was to have the first segment reviewed independently first but if people think it is easier/more time efficient to review both at the same time then we could start here.
Thanks!
This work is towards satisfying [ADR-038](https://github.com/cosmos/cosmos-sdk/blob/master/docs/architecture/adr-038-state-listening.md)
---
Before we can merge this PR, please make sure that all the following items have been
checked off. If any of the checklist items are not applicable, please leave them but
write a little note why.
- [x] Targeted PR against correct branch (see [CONTRIBUTING.md](https://github.com/cosmos/cosmos-sdk/blob/master/CONTRIBUTING.md#pr-targeting))
- [x] Linked to Github issue with discussion and accepted design OR link to spec that describes this work.
- [x] Code follows the [module structure standards](https://github.com/cosmos/cosmos-sdk/blob/master/docs/building-modules/structure.md).
- [x] Wrote unit and integration [tests](https://github.com/cosmos/cosmos-sdk/blob/master/CONTRIBUTING.md#testing)
- [x] Updated relevant documentation (`docs/`) or specification (`x/<module>/spec/`)
- [x] Added relevant `godoc` [comments](https://blog.golang.org/godoc-documenting-go-code).
- [x] Added a relevant changelog entry to the `Unreleased` section in `CHANGELOG.md`
- [x] Re-reviewed `Files changed` in the Github PR explorer
- [x] Review `Codecov Report` in the comment section below once CI passes
Julien Robert
mergify[bot]
Ian Norden