After continuously profiling InitGensis with 100K accounts, it showed
pathologically slow code, that was the result of a couple of patterns:
* Unconditional and not always necessary map lookups
* O(n^2) sdk.AccAddressFromBech32 retrievals when the code is expensive,
during a quicksort
The remedy involved 4 parts:
* O(n) sdk.AccAddressFromBech32 invocations, down from O(n^2) in the quicksort
* Only doing map lookups when the domain key check has passed
* Using a black magic compiler technique of the map clearing idiom
* Zero allocation []byte<->string conversion
With 100K accounts, this brings InitGenesis down to ~6min, instead of
20+min, it reduces the sort code from ~7sec down to 50ms.
Also some simple benchmark reflect the change:
```shell
name old time/op new time/op delta
SanitizeBalances500-8 19.3ms ±10% 1.5ms ± 5% -92.46% (p=0.000 n=20+20)
SanitizeBalances1000-8 41.9ms ± 8% 3.0ms ±12% -92.92% (p=0.000 n=20+20)
name old alloc/op new alloc/op delta
SanitizeBalances500-8 9.05MB ± 6% 0.56MB ± 0% -93.76% (p=0.000 n=20+18)
SanitizeBalances1000-8 20.2MB ± 3% 1.1MB ± 0% -94.37% (p=0.000 n=20+19)
name old allocs/op new allocs/op delta
SanitizeBalances500-8 72.4k ± 6% 4.5k ± 0% -93.76% (p=0.000 n=20+20)
SanitizeBalances1000-8 162k ± 3% 9k ± 0% -94.40% (p=0.000 n=20+20)
```
The CPU profiles show the radical change as per
https://github.com/cosmos/cosmos-sdk/issues/7766#issuecomment-786671734
Later on, we shall do more profiling and fixes but for now this brings
down the run-time for InitGenesis.
Fixes#7766
Co-authored-by: Alessio Treglia <alessio@tendermint.com>
* enable the wsl linter
Fix various wsl-related warnings.
x/ibc/04-channel/keeper/handshake.go: fix missing return statement in ChanOpenTry().
* goimports -w files
* remove unknown linter references
* run make format
* Revert "run make format"
This reverts commit f810b62b9e4993f08506663d4e5f2ec2228a9863.
* run make format
* Bump Tendermint version to v0.33.0
* Deprecate old cmn package with new packages
* Update update DB APIs
* More DB updates
* Bump IAVL to v0.13.0
* Handle error returned by iavl.NewMutableTree
* Fix some IAVL stuffs
* Update IAVL
* More updates
* Passing tests
* Fix unit tests
Co-authored-by: Jack Zampolin <jack.zampolin@gmail.com>
Robert Zaremba
Emmanuel T Odeke
Marko
Adam Bozanich
Alexander Bezobchuk
Alessio Treglia
Joon