cosmos-sdk/store/cachekv
Emmanuel T Odeke dbb9923917
store/cachekv, x/bank/types: algorithmically fix pathologically slow code (#8719)
After continuously profiling InitGensis with 100K accounts, it showed
pathologically slow code, that was the result of a couple of patterns:
* Unconditional and not always necessary map lookups
* O(n^2) sdk.AccAddressFromBech32 retrievals when the code is expensive,
during a quicksort

The remedy involved 4 parts:
* O(n) sdk.AccAddressFromBech32 invocations, down from O(n^2) in the quicksort
* Only doing map lookups when the domain key check has passed
* Using a black magic compiler technique of the map clearing idiom
* Zero allocation []byte<->string conversion

With 100K accounts, this brings InitGenesis down to ~6min, instead of
20+min, it reduces the sort code from ~7sec down to 50ms.

Also some simple benchmark reflect the change:
```shell
name                    old time/op    new time/op    delta
SanitizeBalances500-8     19.3ms ±10%     1.5ms ± 5%  -92.46%  (p=0.000 n=20+20)
SanitizeBalances1000-8    41.9ms ± 8%     3.0ms ±12%  -92.92%  (p=0.000 n=20+20)

name                    old alloc/op   new alloc/op   delta
SanitizeBalances500-8     9.05MB ± 6%    0.56MB ± 0%  -93.76%  (p=0.000 n=20+18)
SanitizeBalances1000-8    20.2MB ± 3%     1.1MB ± 0%  -94.37%  (p=0.000 n=20+19)

name                    old allocs/op  new allocs/op  delta
SanitizeBalances500-8      72.4k ± 6%      4.5k ± 0%  -93.76%  (p=0.000 n=20+20)
SanitizeBalances1000-8      162k ± 3%        9k ± 0%  -94.40%  (p=0.000 n=20+20)
```

The CPU profiles show the radical change as per
https://github.com/cosmos/cosmos-sdk/issues/7766#issuecomment-786671734

Later on, we shall do more profiling and fixes but for now this brings
down the run-time for InitGenesis.

Fixes #7766

Co-authored-by: Alessio Treglia <alessio@tendermint.com>
2021-02-27 07:26:22 -08:00
..
memiterator.go tendermint: update to rc3 (#6892) 2020-08-14 13:58:53 -04:00
mergeiterator.go tendermint: update to rc3 (#6892) 2020-08-14 13:58:53 -04:00
store.go store/cachekv, x/bank/types: algorithmically fix pathologically slow code (#8719) 2021-02-27 07:26:22 -08:00
store_bench_test.go all: ensure b.ReportAllocs() in all the benchmarks (#8460) 2021-01-27 23:52:08 -08:00
store_test.go all: ensure b.ReportAllocs() in all the benchmarks (#8460) 2021-01-27 23:52:08 -08:00