Commit Graph

913 Commits

Author SHA1 Message Date
Robert Zaremba 3e4d81c3d3
Fix proto format (#9279)
* makefile: fix proto-format

* reformat all proto files

* regenerating proto

* remove dangling semicolon
2021-05-07 21:33:17 +00:00
Amaury 56ab4e4c93
Cleanup remains of ServiceMsg (#9236)
* Remove ServiceMsgClientConn from CLI

* Cleanup Operations

* Remove some legacytx.LegacyMsg

* Update CL

* Remove mentions of ServiceMsg

* Simply reflection MsgDescriptor

* Simplify more

* Add code comment

Co-authored-by: technicallyty <48813565+technicallyty@users.noreply.github.com>
2021-05-07 09:31:20 +00:00
Anil Kumar Kammari 2eeb464939
Update `memo` usage docs (#9265)
* Update `memo` usage docs

* update doc

Co-authored-by: Aaron Craelius <aaron@regen.network>

* update doc

Co-authored-by: Aaron Craelius <aaron@regen.network>

Co-authored-by: Aaron Craelius <aaron@regen.network>
2021-05-06 08:47:02 +00:00
Robert Zaremba 68d461052b
add sorted check for the coins sub/add fun parameter (#9240)
* add sorted check for the coins sub/add fun parameter

* adding internal tests

* fix tests

* docs update

* add self sorted check

* add unit test for self sorted check

* adding a comment

* review updates

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2021-05-05 15:55:28 +00:00
Cuong Manh Le 3e6c4a6c79
types: make chopPrecisionAndTruncate always non-mutative (#9250)
Currently, chopPrecisionAndTruncate/chopPrecisionAndTruncateNonMutative
are doing the same thing, except that the former do it mutatively.

Looking at the usages of chopPrecisionAndTruncate, the callers must
alloc a new big.Int before calling it anyway, so any mutation will be
done with this new allocated big.Int, instead of input argument.

So, by making chopPrecisionAndTruncate always non-mutative, we simplify
the code, and make the code faster, less allocation in some benchmarks.

Benchmark for github.com/cosmos/cosmos-sdk/types:

name                                           old time/op    new time/op    delta
CoinsAdditionIntersect/sizes:_A_1,_B_1-8         35.4ns ± 0%    35.1ns ± 0%  -0.92%  (p=0.008 n=5+5)
CoinsAdditionIntersect/sizes:_A_5,_B_5-8          366ns ± 1%     363ns ± 1%    ~     (p=0.056 n=5+5)
CoinsAdditionIntersect/sizes:_A_5,_B_20-8         557ns ± 0%     553ns ± 0%  -0.64%  (p=0.008 n=5+5)
CoinsAdditionIntersect/sizes:_A_1,_B_1000-8      6.17µs ± 0%    6.18µs ± 0%    ~     (p=0.151 n=5+5)
CoinsAdditionIntersect/sizes:_A_2,_B_1000-8      6.30µs ± 0%    6.32µs ± 0%  +0.23%  (p=0.008 n=5+5)
CoinsAdditionNoIntersect/sizes:_A_1,_B_1-8       30.2ns ± 0%    29.9ns ± 1%  -0.98%  (p=0.008 n=5+5)
CoinsAdditionNoIntersect/sizes:_A_5,_B_5-8        226ns ± 0%     224ns ± 0%  -0.54%  (p=0.008 n=5+5)
CoinsAdditionNoIntersect/sizes:_A_5,_B_20-8       373ns ± 0%     372ns ± 0%  -0.26%  (p=0.040 n=5+5)
CoinsAdditionNoIntersect/sizes:_A_1,_B_1000-8    6.21µs ± 0%    6.22µs ± 0%    ~     (p=0.421 n=5+5)
CoinsAdditionNoIntersect/sizes:_A_2,_B_1000-8    6.22µs ± 0%    6.23µs ± 0%    ~     (p=0.286 n=5+5)
CoinsAdditionNoIntersect/sizes:_A_1000,_B_2-8    6.22µs ± 0%    6.23µs ± 0%    ~     (p=0.143 n=5+5)
ParseCoin-8                                      5.46µs ± 0%    5.39µs ± 0%  -1.26%  (p=0.008 n=5+5)
UintMarshal-8                                    1.01µs ± 0%    1.00µs ± 0%  -1.10%  (p=0.008 n=5+5)
IntMarshal-8                                      764ns ± 0%     758ns ± 1%  -0.79%  (p=0.024 n=5+5)
MarshalTo-8                                       192ns ± 0%     190ns ± 0%  -1.03%  (p=0.008 n=5+5)

name                                           old alloc/op   new alloc/op   delta
CoinsAdditionIntersect/sizes:_A_1,_B_1-8          32.0B ± 0%     32.0B ± 0%    ~     (all equal)
CoinsAdditionIntersect/sizes:_A_5,_B_5-8           520B ± 0%      520B ± 0%    ~     (all equal)
CoinsAdditionIntersect/sizes:_A_5,_B_20-8        1.38kB ± 0%    1.38kB ± 0%    ~     (all equal)
CoinsAdditionIntersect/sizes:_A_1,_B_1000-8      49.2kB ± 0%    49.2kB ± 0%    ~     (all equal)
CoinsAdditionIntersect/sizes:_A_2,_B_1000-8      49.3kB ± 0%    49.3kB ± 0%    ~     (all equal)
CoinsAdditionNoIntersect/sizes:_A_1,_B_1-8        24.0B ± 0%     24.0B ± 0%    ~     (all equal)
CoinsAdditionNoIntersect/sizes:_A_5,_B_5-8         488B ± 0%      488B ± 0%    ~     (all equal)
CoinsAdditionNoIntersect/sizes:_A_5,_B_20-8      1.35kB ± 0%    1.35kB ± 0%    ~     (all equal)
CoinsAdditionNoIntersect/sizes:_A_1,_B_1000-8    49.2kB ± 0%    49.2kB ± 0%    ~     (all equal)
CoinsAdditionNoIntersect/sizes:_A_2,_B_1000-8    49.2kB ± 0%    49.2kB ± 0%    ~     (all equal)
CoinsAdditionNoIntersect/sizes:_A_1000,_B_2-8    49.2kB ± 0%    49.2kB ± 0%    ~     (all equal)
ParseCoin-8                                      2.21kB ± 0%    2.21kB ± 0%    ~     (all equal)
UintMarshal-8                                      392B ± 0%      392B ± 0%    ~     (all equal)
IntMarshal-8                                       168B ± 0%      168B ± 0%    ~     (all equal)
MarshalTo-8                                       80.0B ± 0%     80.0B ± 0%    ~     (all equal)

name                                           old allocs/op  new allocs/op  delta
CoinsAdditionIntersect/sizes:_A_1,_B_1-8           1.00 ± 0%      1.00 ± 0%    ~     (all equal)
CoinsAdditionIntersect/sizes:_A_5,_B_5-8           12.0 ± 0%      12.0 ± 0%    ~     (all equal)
CoinsAdditionIntersect/sizes:_A_5,_B_20-8          14.0 ± 0%      14.0 ± 0%    ~     (all equal)
CoinsAdditionIntersect/sizes:_A_1,_B_1000-8        3.00 ± 0%      3.00 ± 0%    ~     (all equal)
CoinsAdditionIntersect/sizes:_A_2,_B_1000-8        6.00 ± 0%      6.00 ± 0%    ~     (all equal)
CoinsAdditionNoIntersect/sizes:_A_1,_B_1-8         1.00 ± 0%      1.00 ± 0%    ~     (all equal)
CoinsAdditionNoIntersect/sizes:_A_5,_B_5-8         5.00 ± 0%      5.00 ± 0%    ~     (all equal)
CoinsAdditionNoIntersect/sizes:_A_5,_B_20-8        6.00 ± 0%      6.00 ± 0%    ~     (all equal)
CoinsAdditionNoIntersect/sizes:_A_1,_B_1000-8      3.00 ± 0%      3.00 ± 0%    ~     (all equal)
CoinsAdditionNoIntersect/sizes:_A_2,_B_1000-8      4.00 ± 0%      4.00 ± 0%    ~     (all equal)
CoinsAdditionNoIntersect/sizes:_A_1000,_B_2-8      3.00 ± 0%      3.00 ± 0%    ~     (all equal)
ParseCoin-8                                        71.0 ± 0%      71.0 ± 0%    ~     (all equal)
UintMarshal-8                                      25.0 ± 0%      25.0 ± 0%    ~     (all equal)
IntMarshal-8                                       18.0 ± 0%      18.0 ± 0%    ~     (all equal)
MarshalTo-8                                        2.00 ± 0%      2.00 ± 0%    ~     (all equal)

name                                           old speed      new speed      delta
UintMarshal-8                                  6.91MB/s ± 0%  6.99MB/s ± 0%  +1.04%  (p=0.008 n=5+5)
IntMarshal-8                                   9.16MB/s ± 0%  9.23MB/s ± 1%  +0.83%  (p=0.032 n=5+5)

Benchmark for github.com/cosmos/cosmos-sdk/x/mint/types:

name              old time/op    new time/op    delta
BlockProvision-8     263ns ± 0%     257ns ± 0%   -2.23%  (p=0.008 n=5+5)

name              old alloc/op   new alloc/op   delta
BlockProvision-8      112B ± 0%      104B ± 0%   -7.14%  (p=0.008 n=5+5)

name              old allocs/op  new allocs/op  delta
BlockProvision-8      6.00 ± 0%      5.00 ± 0%  -16.67%  (p=0.008 n=5+5)

Fixes #9249

Co-authored-by: Emmanuel T Odeke <emmanuel@orijtech.com>
2021-05-04 00:32:15 +01:00
Amaury dfe3e7a8d7
Remove ServiceMsgs from ADR-031 (#9139)
* wip

* wip

* wip

* wip on refactoring adr 031 type URLs

* Fix msg_service_router

* Fix gov client queries

* Fix some modules tests

* Remove all instances of "*.Msg/*"

* Uncomment code

* Remove commented code

* // simulation.NewWeightedOperation(

* Fix CopyTx

* Fix more tests

* Fix x/gov test

* proto.MessageName->sdk.MsgName

* Fix authz tests

* Use MsgRoute in feegrant and staking/authz

* Fix more tests

* Fix sims?

* Add norace tag

* Add CL

* rebuild rosetta api test data

* Update ADR

* Rename MsgRoute -> MsgTypeURL

* Fix codec registration

* Remove sdk.GetLegacySignBytes

* Update types/tx_msg.go

* Update x/authz/simulation/operations.go

* Move LegacyMsg to legacytx

* Update CHANGELOG.md

Co-authored-by: Aaron Craelius <aaron@regen.network>

* Remove NewAnyWithCustomTypeURL

* Keep support for ServiceMsgs

* Fix TxBody UnpackInterfaces

* Fix test

* Address review

* Remove support for ServiceMsg typeURLs

* Fix lint

* Update changelog

* Fix tests

* Use sdk.MsgTypeURL everywhere

* Fix tests

* Fix rosetta, run make rosetta-data

* Fix rosetta thanks to froydi

* Address reviews

* Fix test

* Remove stray log

* Update CL

Co-authored-by: Aaron Craelius <aaronc@users.noreply.github.com>
Co-authored-by: Alessio Treglia <alessio@tendermint.com>
Co-authored-by: Aaron Craelius <aaron@regen.network>
2021-04-30 11:00:47 +00:00
Robert Zaremba be4a965599
codec: Rename codec and marshaler interfaces (#9226)
* codec: Rename codec and marshaler interfaces, ref: 8413

* codec: remove BinaryBare

* changelog update

* Update comments and documentation

* adding doc string comments

* Update CHANGELOG.md

Co-authored-by: Federico Kunze <31522760+fedekunze@users.noreply.github.com>

* Update codec/codec.go

Co-authored-by: Marko <marbar3778@yahoo.com>

Co-authored-by: Federico Kunze <31522760+fedekunze@users.noreply.github.com>
Co-authored-by: Amaury <1293565+amaurym@users.noreply.github.com>
Co-authored-by: Marko <marbar3778@yahoo.com>
2021-04-29 10:46:22 +00:00
Cuong Manh Le e28271b8e6
types: make NewDecFromStr returns error for too large decimal (#9157)
Otherwise, NewDecFromStr may accept very large input, causing Dec
methods panic, e.g Dec.TruncateInt

Found by oss-fuzz: https://oss-fuzz.com/testcase-detail/6454129938530304

Fixes #9160
2021-04-22 11:21:27 +01:00
Dev Ojha 82dc0081fd
Minor docs update to sdk.Int (#9126)
Co-authored-by: Federico Kunze <31522760+fedekunze@users.noreply.github.com>
2021-04-16 08:43:49 +00:00
Robert Zaremba e43edc4749
ADR-28 derive address functions (#9088)
* update Derive functions

* update adr-028

* changelog update

* Apply suggestions from code review

Co-authored-by: Marie Gauthier <marie.gauthier63@gmail.com>

* review updates

* remove DeriveMulti and rollback some changes in CHANGELOG

* add noop error check

Co-authored-by: Marie Gauthier <marie.gauthier63@gmail.com>
2021-04-15 21:32:45 +00:00
technicallyty 3ebf06ce0d
module/types code hygiene - utilize map access 2nd return value (#9098)
* update docs/code to utilize map access return values

* remove mock module in fromVM to simulate truly non-existent module

* update test docs to refelect change

Co-authored-by: technicallyty <48813565+tytech3@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2021-04-13 19:00:04 +00:00
Hanjun Kim cf7b03efcd
add AddAmount/SubAmount methods to sdk.Coin (#9091)
Co-authored-by: Marko <marbar3778@yahoo.com>
Co-authored-by: Amaury <1293565+amaurym@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2021-04-13 15:01:21 +00:00
Sunny Aggarwal 10b2b08ece Merge branch 'master' into powerreduction_param 2021-04-10 16:26:50 -04:00
MD Aleem 8a376a41c9
x/authz charge gas for expensive authorizations (#8995)
* WIP

* Add consume gas

* update authz.go

* add build flag

* Update x/staking/types/authz.go

* Update x/staking/types/authz.go

* add tests docs

* review changes

* fix operations

* Update x/authz/types/msgs.go

Co-authored-by: Amaury <1293565+amaurym@users.noreply.github.com>

* review changes

* update gas cost

* review changes

Co-authored-by: SaReN <sahithnarahari@gmail.com>
Co-authored-by: Amaury <1293565+amaurym@users.noreply.github.com>
2021-04-09 12:33:27 +00:00
Alessio Treglia 6e26ee9805
bump tendermint core (#9081) 2021-04-08 20:22:00 +01:00
Sunny Aggarwal ca34808fdc merged master 2021-04-06 20:18:20 -04:00
Frojdi Dymylja 29ff333007
cosmos-reflection: extend to support writing by reflection clients (#8965)
Co-authored-by: SaReN <sahithnarahari@gmail.com>
Co-authored-by: Alessio Treglia <alessio@tendermint.com>
2021-04-07 00:33:54 +01:00
Sunny Aggarwal 05e2e45010 merged in master 2021-04-06 19:19:59 -04:00
SaReN 0effd3a9d3
Paginate supply queries (#8798)
* paginate grpc query

* fix lint

* update tests

* remove GetTotalSupply

* fix test

* remove GetTotalSupply

* add changelog

* update changelog

* update rosetta data

* update limit

* update genesis

* add max limit to query

* fix lint

* go imports

* Update types/query/pagination.go

* update supply

Co-authored-by: Marko <marbar3778@yahoo.com>
2021-04-06 14:43:08 +00:00
Amaury 5d13b1fc63
InitGenesis in migrations when fromVersion==0 (#9007)
* InitGenesis in migrations when fromVersion==0

* Add test

* Fix test

* Fix comment

* cdc=>codec

* Don't break Configurator

* Remove method

* Add comments

* Add more comments

* Update types/module/module.go

Co-authored-by: technicallyty <48813565+technicallyty@users.noreply.github.com>

* Update types/module/module.go

Co-authored-by: technicallyty <48813565+technicallyty@users.noreply.github.com>

* Update types/module/module.go

Co-authored-by: technicallyty <48813565+technicallyty@users.noreply.github.com>

Co-authored-by: technicallyty <48813565+technicallyty@users.noreply.github.com>
2021-04-02 15:41:35 +00:00
Aaron Craelius 697d8d52f9
Make module.Manager.RunMigrations handle new modules correctly (#8988)
* Make module.Manager.RunMigrations handle module additions correctly

* fix if

Co-authored-by: Alessio Treglia <alessio@tendermint.com>
2021-03-25 19:51:38 +00:00
Robert Zaremba 7568b6680a
Remove bech32 PubKey support (#7477)
* Move PubKey bech32 to legacy package and migrate the usage where possible

* update /server

* wip

* move proto json encoding helper functions to internal

* update internal/marshal

* wip

* update sections which needs legacybech32

* update validators output

* fix conflicts

* slashing update

* add more tests and helper function for ANY JSON serialization

* update slashing

* Update function documentation

* Rename code any-marshal helper functions

* Update pubkey unpacking test

* Update test comments

* solve TestDecodeStore

* solve legacytx issues

* all code compiles

* keyring tests

* keyring cleanup

* remove AssertMsg

* fix some tests

* fix add_ledger_test.go

* update cli tests

* debug cli test

* rename clashed bech32 names

* linter fixes

* update tmservice tests

* linter: update legacy deprecated checks

* fix names

* linting

* legacybech32 pubkey type rename

* fix staking client

* fix test compilation

* fix TestGetCmdQuerySigningInfo

* rename NewIfcJSONAnyMarshaler

* keyring: remove duplicated information from multinfo structure

* todo cleanups

* Update Changelog

* remove some legacybech32 from tests

* remove todos

* remove printlnJSON from /server CLI and amino encoding

* remove protocdc.MarshalJSON

* client/show remove duplicated function

* remove protocdc package

* comment update

* remove legacybech32.MustMarshalPubKey from a test

* add todo

* fix TestPublicKeyUnsafe test

* review update

* fix bech32 UnmarshalPubKey

* Use codec.MarshalIfcJSON

* fix linter issues

* merging conflict: fix codec.Unmarshal calls

* cleanups

* Update CHANGELOG.md

Co-authored-by: Aaron Craelius <aaron@regen.network>

* Reword changelog updates

* use pubkey.String for comparison in Test_runAddCmdLedgerWithCustomCoinType

* Update GetCmdQuerySigningInfo example

* cli: update keys add docs

* Add errors AsOf and errors.ErrIO type

* restore multisigPubKeyInfo structure bring it back to multiInfo struct

* Update codec/proto_codec.go

Co-authored-by: Marie Gauthier <marie.gauthier63@gmail.com>

* Update crypto/keys/ed25519/ed25519_test.go

Co-authored-by: Marie Gauthier <marie.gauthier63@gmail.com>

* Update codec/proto_codec.go

Co-authored-by: Marie Gauthier <marie.gauthier63@gmail.com>

* move pubkey any marshaling tests

* Apply suggestions from code review

Co-authored-by: Marie Gauthier <marie.gauthier63@gmail.com>

* review updates

* adding missing return

* errors: use IsOf instead of AsOf

* keyring: add a correct check for key not found in keyring.Get

* add checkKeyNotFound

* fix linter issues

* fix: keyring key not found check

* fix keyring tests

* fix linting issues

* cli tests

* fix: 'simd keys show <key> -p'

* fix: TestVerifyMultisignature

* rename keyring Bech32... functions to Mk...

* fix RunAddCmd

* Update pubkey display

* wip

* add more tests

* udate keyring output tests

* remove todo from ledger tests

* rename MkKeyOutput

* Changelog update

* solve liner issues

* add link to github issue

Co-authored-by: Aaron Craelius <aaron@regen.network>
Co-authored-by: Marie Gauthier <marie.gauthier63@gmail.com>
2021-03-25 14:53:22 +00:00
Robert Zaremba 0c824f11e9
fix race condition on address.String (#8997) 2021-03-25 13:51:34 +00:00
Amaury 1fa2c22d8a
Prefer sending tx_bytes to Simulate gRPC endpoint (#8926)
* First run

* Remove dead code

* Make test pass

* Proto gen

* Fix lint

* Add changelog

* Fix tests

* Fix test

* Update x/auth/tx/service.go

Co-authored-by: Marie Gauthier <marie.gauthier63@gmail.com>

* Remove protoTxProvider

* Add grpc-gateway test

* Add comment

* move to api breaking

* lesser diff

* remove conflict

* empty commit to rerun CI

* empty commit to rerun CI

Co-authored-by: Marie Gauthier <marie.gauthier63@gmail.com>
Co-authored-by: Alessio Treglia <alessio@tendermint.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2021-03-25 10:52:54 +00:00
Hanjun Kim 8ee9da775e
types: add Abs() method to sdk.Int (#8963)
Fixes #8962

Co-authored-by: Alessio Treglia <alessio@tendermint.com>
2021-03-24 16:33:55 +00:00
MD Aleem a78f777ea8
add reverse iteration to pagination (#8875)
* WIP

* add tests

* add tests

* fix lint

* fix pagination

* add proto message doc

* fix filtered_pagination

* fix test

* cleanup

* add reverse flag to pagination

* changelog

* Update client/flags/flags.go

* Update CHANGELOG.md

Co-authored-by: Alessio Treglia <alessio@tendermint.com>
Co-authored-by: Federico Kunze <31522760+fedekunze@users.noreply.github.com>
2021-03-23 09:28:20 +00:00
Sunny Aggarwal b54e2763db simply default power reduction definition 2021-03-21 20:29:39 -04:00
Sunny Aggarwal 1f99aa3fb2 merged in master 2021-03-21 16:37:06 -04:00
technicallyty 5bd93bfe7b
x/upgrade: added consensus version tracking (part of ADR-041) (#8743)
* -added consensus version tracking to x/upgrade

* -added interface to module manager -added e2e test for migrations using consensus version store in x/upgrade -cleaned up x/upgrade Keeper -handler in apply upgrade now handles errors and setting consensus versions -cleaned up migration map keys -removed init chainer method -simapp now implements GetConsensusVersions to assist with testing

* Changed MigrationMap identifier to VersionMap
removed module_test

* updated docs

* forgot this

* added line to changelog for this PR

* Change set consensus version function to match adr 041 spec

* add documentation

* remove newline from changelog

unnecessary newline removed

* updated example in simapp for RunMigrations, SetCurrentConsensusVersions now returns an error

* switch TestMigrations to use Require instead of t.Fatal

* Update CHANGELOG.md

Co-authored-by: Aaron Craelius <aaron@regen.network>

* docs for SetVersionManager

* -init genesis method added -removed panics/fails from setting consensus versions

* update identifiers to be more go-like

* update docs and UpgradeHandler fnc sig

* Upgrade Keeper now takes a VersionMap instead of a VersionManager interface

* upgrade keeper transition to Version Map

* cleanup, added versionmap return to RunMigrations

* quick fix

* Update docs/architecture/adr-041-in-place-store-migrations.md

Co-authored-by: Amaury <1293565+amaurym@users.noreply.github.com>

* remove support for versionmap field on upgrade keeper

* cleanup

* rename get/set version map keeper functions

* update adr doc to match name changes

* remove redudant line

Co-authored-by: technicallyty <48813565+tytech3@users.noreply.github.com>
Co-authored-by: Aaron Craelius <aaron@regen.network>
Co-authored-by: Amaury <1293565+amaurym@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2021-03-19 22:01:29 +00:00
Robert Zaremba 05d4e43d1c
account.Address: use RWLocker (#8774)
Co-authored-by: Alessio Treglia <alessio@tendermint.com>
2021-03-16 17:48:54 +00:00
Robert Zaremba 4bd695de4e
errors: Add Wrap handy method (#8845)
Co-authored-by: Alessio Treglia <alessio@tendermint.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2021-03-16 01:09:31 +00:00
MD Aleem adf5f625c1
add orderBy parameter to TxsByEvents (#8815)
Co-authored-by: Alessio Treglia <alessio@tendermint.com>
Co-authored-by: Jonathan Gimeno <jgimeno@gmail.com>
2021-03-12 18:47:10 +00:00
Emmanuel T Odeke b9f3db1be8
all: skip noisy/faulty benchmarks + add b.ReportAllocs for every benchmark (#8856)
* Skips very noisy benchmarks that end up running only for b.N=1 because
their entire time is spent in setup, and varying parameters doesn't change
much given that the number of stores is what dominates the expense. To
ensure we can provide reliable benchmarks, progressively for the project,
skip these until there is a proper re-work of what the benchmarks need to do

* Previously sub-benchmarks: b.Run(...) did not b.ReportAllocs() due to a faulty
assumption that invoking b.ReportAllocs() at the top would be inherited by
all sub-benchmarks. This change fixes that

Fixes #8779
Fixes #8855

Co-authored-by: Alessio Treglia <alessio@tendermint.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2021-03-11 15:59:13 +00:00
Frojdi Dymylja 288f8dda4b
[rosetta] implement balance tracking and redo tx construction (#8729)
Co-authored-by: Alessio Treglia <alessio@tendermint.com>
Co-authored-by: Robert Zaremba <robert@zaremba.ch>
2021-03-11 15:01:29 +00:00
Emmanuel T Odeke c2d5b24f58
store/cachekv: use typed types/kv.List instead of container/list.List (#8811)
Reduces CPU burn by using a typed List to avoid the expensive type
assertions from using an interface. This is the only option for now
until Go makes generics generally available.

The change brings time spent on the time assertion cummulatively to:
    580ms down from 6.88s

Explanation:
The type assertions were showing up heavily in profiles:
* Before this commit
```shell
Total: 42.18s
ROUTINE ======================== github.com/cosmos/cosmos-sdk/store/cachekv.newMemIterator
in /Users/emmanuelodeke/go/src/github.com/cosmos/cosmos-sdk/store/cachekv/memiterator.go
    14.01s     18.87s (flat, cum) 44.74% of Total
         .          .     17:	items      []*kv.Pair
         .          .     18:	ascending  bool
         .          .     19:}
         .          .     20:
         .          .     21:func newMemIterator(start, end []byte, items *list.List, ascending bool) *memIterator {
         .      620ms     22:	itemsInDomain := make([]*kv.Pair, 0, items.Len())
         .          .     23:
         .          .     24:	var entered bool
         .          .     25:
     510ms      870ms     26:	for e := items.Front(); e != nil; e = e.Next() {
     6.85s      6.88s     27:		item := e.Value.(*kv.Pair)
     5.71s      8.19s     28:		if !dbm.IsKeyInDomain(item.Key, start, end) {
     120ms      120ms     29:			if entered {
         .          .     30:				break
         .          .     31:			}
         .          .     32:
         .          .     33:			continue
         .          .     34:		}
         .          .     35:
     820ms      980ms     36:		itemsInDomain = append(itemsInDomain, item)
         .          .     37:		entered = true
         .          .     38:	}
         .          .     39:
         .      1.21s     40:	return &memIterator{
         .          .     41:		start:     start,
         .          .     42:		end:       end,
         .          .     43:		items:     itemsInDomain,
         .          .     44:		ascending: ascending,
         .          .     45:	}
```

and given that the list only uses that type, it is only right to lift the
code from container/list.List, and only modify Element.Value's type.

For emphasis, the code is basically just a retrofit of
container/list/list.go but with a typing, and we'll keep it as is
until perhaps Go1.17 or Go1.18 or when everyone uses Go1.17+ after
generics have landed.

* After this commit
```shell
Total: 45.25s
ROUTINE ======================== github.com/cosmos/cosmos-sdk/store/cachekv.newMemIterator
in /Users/emmanuelodeke/go/src/github.com/cosmos/cosmos-sdk/store/cachekv/memiterator.go
     4.84s      6.77s (flat, cum) 14.96% of Total
         .          .     16:	items      []*kv.Pair
         .          .     17:	ascending  bool
         .          .     18:}
         .          .     19:
         .          .     20:func newMemIterator(start, end []byte, items *kv.List, ascending bool) *memIterator {
         .      330ms     21:	itemsInDomain := make([]*kv.Pair, 0, items.Len())
         .          .     22:
         .          .     23:	var entered bool
         .          .     24:
      60ms      160ms     25:	for e := items.Front(); e != nil; e = e.Next() {
     580ms      580ms     26:		item := e.Value
     3.68s      4.78s     27:		if !dbm.IsKeyInDomain(item.Key, start, end) {
      80ms       80ms     28:			if entered {
         .          .     29:				break
         .          .     30:			}
         .          .     31:
         .          .     32:			continue
         .          .     33:		}
         .          .     34:
     440ms      580ms     35:		itemsInDomain = append(itemsInDomain, item)
         .          .     36:		entered = true
         .          .     37:	}
         .          .     38:
         .      260ms     39:	return &memIterator{
         .          .     40:		start:     start,
         .          .     41:		end:       end,
         .          .     42:		items:     itemsInDomain,
         .          .     43:		ascending: ascending,
         .          .     44:	}
```

Fixes #8810
2021-03-08 09:16:23 -08:00
Robert Zaremba 0b6833c333
Lock mutex on getting element from cache (#8767) 2021-03-03 15:52:38 +00:00
Cuong Manh Le 91affb5167
simapp, types: fix benchmarks panics by honoring skip if set (#8763)
- BenchmarkAccAddressString must initialize the index less than ed25519.PubKeySize to avoid panics
 - BenchmarkFullAppSimulation should be skipped as it has the same logic as tests

Fixes #8762
2021-03-03 03:54:14 -08:00
Robert Zaremba 2864eb69a3
Robert/addr memory leak (#8717)
* address: adding cache address.String() cache benchmark

* address: use LRU cache for .String()

* optimize address.Empty

* move cache initialization to init function

* Use UnsafeBytesToStr convertion with Addr cache

* add cache for other address String() methods

* fix linter issue

* add a non trivial address for Address.String benchmark

* add comment about cache size and update cashe size to 60k

* fix syntax

Co-authored-by: Alessio Treglia <alessio@tendermint.com>
2021-03-03 03:53:28 +00:00
Robert Zaremba 5f2b90c3c7
internal: create package for unsafe bytes convertion (#8733)
Co-authored-by: Alessio Treglia <alessio@tendermint.com>
2021-03-01 15:10:22 +00:00
Emmanuel T Odeke 929b62cd46
types: intern AccAddress.String() to gut wasted expensive recomputations (#8694)
Given that AccAddress is a []byte type, and its .String() method is
quite expensive, continuously invoking the method doesn't easily provide
a way for the result to be memoized. In memory profiles from
benchmarking OneBankSendTxPerBlock and run for a while, it showed >2GiB burnt
and SendCoins consuming a bunch of memory: >2GiB.

This change introduces a simple global cache with a map to intern
AccAddress values, so that we quickly look up the expensively computed
values. With it, the prior memory profiles are erased, and benchmarks
show improvements:

```shell
$ benchstat before.txt after.txt
name                     old time/op    new time/op    delta
OneBankSendTxPerBlock-8    1.94ms ± 9%    1.92ms ±11%   -1.34%  (p=0.011 n=58+57)

name                     old alloc/op   new alloc/op   delta
OneBankSendTxPerBlock-8     428kB ± 0%     365kB ± 0%  -14.67%  (p=0.000 n=58+54)

name                     old allocs/op  new allocs/op  delta
OneBankSendTxPerBlock-8     6.34k ± 0%     5.90k ± 0%   -7.06%  (p=0.000 n=58+57)
```

Fixes #8693

Co-authored-by: Alessio Treglia <alessio@tendermint.com>
2021-02-25 03:01:03 -08:00
Amaury ba74a7c737
x{stake,slash,gov,distrib} In-place Store Migrations (#8504)
* Add 1st version of migrate

* Put migration logic into Configurator

* add test to bank store migration

* add test for configurator

* Error if no migration found

* Remove RunMigrations from Configurator interface

* Update spec

* Rename folders

* copy-paste from keys.go

* Fix nil map

* rename function

* Update simapp/app.go

Co-authored-by: Robert Zaremba <robert@zaremba.ch>

* Update simapp/app_test.go

Co-authored-by: Robert Zaremba <robert@zaremba.ch>

* Adderss reviews

* Fix tests

* Update testutil/context.go

Co-authored-by: Robert Zaremba <robert@zaremba.ch>

* Update docs for ConsensusVersion

* Rename to forVersion

* Fix tests

* Check error early

* Return 1 for intiial version

* Use MigrationKeeper

* Fix test

* Revert adding marshaler to Configurator

* Godoc updates

* Update docs

* Add distrib legacy folder

* Add tests for distrib migration

* Add gov migrations

* Copy paste whole keys file

* Add gov migrations

* Add staking

* Fix staking tests

* Update spec and module.go

* Update to latest changes

* Update migration scripts

* capability to 1

* Fix tests

* Add package godoc

* Remove whitespace

* Remove global

* Use Migrator

* Remove 042 keys files

* Fix build

* Unlambda

* Rename to Migrate1to2

Co-authored-by: Robert Zaremba <robert@zaremba.ch>
2021-02-25 10:43:31 +00:00
Robert Zaremba 7969135abc
rework unsafe string -> bytes conversion #8674 (#8684)
Changing the unsafe string -> bytes conversion to fix
a security concern.

Closes: #8670

Co-authored-by: Alessio Treglia <alessio@tendermint.com>
2021-02-24 12:47:32 +00:00
Robert Zaremba 8db9bbba04
linter: ignore false positives in types/address (#8674) 2021-02-23 13:38:37 +00:00
Alessio Treglia f2ee972e31
various linter fixes (#8666) 2021-02-23 08:46:01 +00:00
Riccardo Montagnin f49e1554e6
Deprecate SetFullFundraiserPath in favor of SetPurpose and SetCoinType (#8629)
Co-authored-by: Alessio Treglia <alessio@tendermint.com>
2021-02-22 15:14:09 +00:00
MD Aleem 81cc5c1194
gRPC gateway issue with colons in the account url path (#8649)
* add allow_colon_final_segments flag

* add test

* Update x/auth/client/rest/rest_test.go

Co-authored-by: Amaury <1293565+amaurym@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2021-02-22 11:30:45 +00:00
Alessio Treglia 9b7a12f7db
run make format (#8642)
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2021-02-19 15:02:53 +00:00
Amaury 73e38e4009
Use service Msgs in CLI tx commands (#8512)
* Use service Msgs in CLI tx commands

* Update comment

* Gracefully support amino signing

* CLI to use svc msg

* Fix lint

* Use fq method name in events

* Update tests

* Fix quering events

* Add docs

* Fix test build

* Fix events

* Fix search for events

* Handle old andd new event quering

* Use merge events

* Better encCfg

* Add page in search

* Fix tests

* Update test and comments

* Update x/auth/legacy/legacytx/stdsign.go

Co-authored-by: atheeshp <59333759+atheeshp@users.noreply.github.com>

* Fix conflict for weighted vote

* Make CopyTx actually run

* Fix CopyTx

* Fix test

* Add changelog entry

* Remove useless code

* Add msgs tests

* Remove testing proto Msg via CLI

* Fix lint

* Fix test

* Implement GetSignBytes on ServiceMsg

* Fix %T

Co-authored-by: atheeshp <59333759+atheeshp@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2021-02-18 18:00:19 +00:00
Jonathan Gimeno abb3dfefa0
[Bank] Remove the unsafe balance changing API (#8473)
* temp commit

* setbalance now is internal

* remove set balances in genesis

* feedback test commit

* update tests

* fix: genesis panic message

* fix not bonded pool

* fix(staking): genesis test

* fix(simapp): rollback state fix change

* fix(staking): genesis large val set test

* [Bank Refactor] Frojdi jonathan/remove setsupply (#8491)

* init supply in a different way

* remove external usage of set supply

* change(staking): replace SetSupply with MintCoins in tests

* change(evidence): replace SetSupply with MintCoins in tests

* change(crisis): remove SetSupply in tests

* change(bank): remove set supply from genesis tests

* change(bank): remove set supply from keeper tests

* change(bank): remove remaining set supply usage from keeper tests

* change(bank): remove set supply usage from grpc query and querier tests

* change(bank): remove SetSupply from keeper interface

Co-authored-by: Frojdi Dymylja <frojdi.dymylja@gmail.com>

* remove setbalances from genesis in gov

* remove keyring

* add init genesis state

* change(staking): make genesis checks coherent and add tests

* remove setbalances on distribution

* fix(staking): genesis tests

* [Bank Refactor]: Remove SetBalances usage from the code and tests (#8509)

* change(distribution): remove SetBalances usage from keeper tests

* add(simapp): FundAccount utility function

* chore(staking): use FundAccount in keeper tests

* change(staking): remove usage of SetBalance in allocation tests

* change(staking): remove usage of SetBalance in delegation tests

* change(staking): remove usage of SetBalance in proposal handler tests

* change(staking): remove usage of SetBalances in grpc query tests

* change(staking): remove usage of SetBalances in operations tests

* change(distribution): remove usage of SetBalances in genesis

* change(authz): remove usage of SetBalances keeper and operations test

* fix(authz): TestKeeperFees failing test

* change(slashing): remove SetBalances from expected BankKeeper

* change(slashing): remove usage of SetBalances in tests

* change(distribution): remove SetBalances from expected BankKeeper

* change(genutil): remove usage of SetBalances from tests

* change(gov): remove SetBalances from expected BankKeeper

* change(gov): remove usage of SetBalances from tests

* change(staking): remove usage of SetBalances from slash tests

* change(staking): remove SetBalances from expected BankKeeper

* change(staking): remove usage of SetBalances from delegation tests

* change(staking): remove usage of SetBalances from operations tests

* change(staking): remove usage of SetBalances from validator tests

* change(bank): remove usage of SetBalances from app tests

* change(bank): remove usage of SetBalances from bench tests

* change(bank): remove usage of SetBalances from querier tests

* change(bank): remove usage of SetBalances from grpc query tests

* change(bank): remove usage of SetBalances from operations tests

* change(bank): partially remove usage of SetBalances from keeper tests

* change(bank): finalize removal of usage of SetBalances from keeper tests

* change(auth): remove usage of SetBalances from verify tests

* change(auth): partially remove usage of SetBalances from tests

* [Bank refactor]: finalize removal of setbalances from auth (#8527)

* add tests with is check tx

* temp commit

* fix test

* fix other test and remove setbalances

* change(auth): remove usage of SetBalances is vesting tests

Co-authored-by: Jonathan Gimeno <jgimeno@gmail.com>

* change(types): remove usage of SetBalances in queries

* fix(types): pagination tests

* [Bank refactor] fix pagination tests (#8550)

* fix tests

* lint

* change(bank): remove SetBalances from keeper public API

Co-authored-by: Jonathan Gimeno <jgimeno@gmail.com>
Co-authored-by: SaReN <sahithnarahari@gmail.com>

* change(bank): remove SubtractCoins from keeper public API

* change(ibc/transfer): remove AddCoins from relay tests

* change(bank): remove AddCoins from public keeper API

* fix imports

* remove set balances

* fix fee test

* remove set balances

* fix(staking): remove dependency on minter authorization for staking pools

* chore: update CHANGELOG.md

* update: x/distribution/keeper/keeper_test.go

Co-authored-by: Robert Zaremba <robert@zaremba.ch>

* Update simapp/test_helpers.go

Co-authored-by: Robert Zaremba <robert@zaremba.ch>

* Update x/staking/genesis_test.go

Co-authored-by: Robert Zaremba <robert@zaremba.ch>

* fix(simapp): FundAccount amount variable name

* fix some PR issues

Co-authored-by: Frojdi Dymylja <frojdi.dymylja@gmail.com>
Co-authored-by: Frojdi Dymylja <33157909+fdymylja@users.noreply.github.com>
Co-authored-by: SaReN <sahithnarahari@gmail.com>
Co-authored-by: Alessio Treglia <alessio@tendermint.com>
Co-authored-by: Robert Zaremba <robert@zaremba.ch>
2021-02-17 18:20:33 +00:00
Robert Zaremba adbf5a71e6
adr-028 address generation (#8415)
* Optimize secp256k1 hashing

* Add ADR-028 related functions

* Update ed25519

* fix errors/handle

* fix build

* fix build

* Add tests and update function names

* wip

* Use LengthPrefix for composed addresses

* add tests for NewComposed

* add module hash function

* fix append

* rollback ed25519 ADR-28 update

* rollback ed25519 ADR-28 test

* Adding Module tests and convert tests to test suite

* convert store_key_test.go to test suite

* rollback test check comment

* Rename assert.Panic and add comment

* add note to ed25519 about SDK support with regards to ADR-28

* Update ed25519 TestAddress

* Adding Deprecated notes for ed25519.PrivKey

* Update crypto/keys/ed25519/ed25519.go

Co-authored-by: Marie Gauthier <marie.gauthier63@gmail.com>

* Update types/address/hash_test.go

Co-authored-by: Marie Gauthier <marie.gauthier63@gmail.com>

* solve linter issues

* linter: remove gocritic

Co-authored-by: Marie Gauthier <marie.gauthier63@gmail.com>
2021-02-15 15:32:51 +00:00