Avoids a bleed out attack in which a node can be made to allocate
memory slowly or very fast in small strides, by sending bad data
to code that invokes NewAnyWithCustomTypeURL, in which we
unconditionally returned a new Any object. On a 64-bit machine,
this would waste 96 bytes per invocation even on error.
Added a test to ensure zero allocations with a fixed error returned.
Also added a benchmark which shows reduction in wasted allocations and
wasted CPU time:
```shell
$ benchstat before.txt after.txt
name old time/op new time/op delta
NewAnyWithCustomTypeURLWithErrorReturned-8 142ns ± 6% 55ns ±12% -61.65% (p=0.000 n=9+10)
name old alloc/op new alloc/op delta
NewAnyWithCustomTypeURLWithErrorReturned-8 96.0B ± 0% 0.0B -100.00% (p=0.000 n=10+10)
name old allocs/op new allocs/op delta
NewAnyWithCustomTypeURLWithErrorReturned-8 1.00 ± 0% 0.00 -100.00% (p=0.000 n=10+10)
```
Fixes#8537
* remove any.ClearCachedValue method
This method is in a public interface but used only for tests.
It shouldn't be used - `Any` values shouldn't be manipulated. This could lead to bugs.
* rename any_test.go
Co-authored-by: Alessio Treglia <alessio@tendermint.com>
* Clean Any interface
+ removed Any.Pack method. This method is not needed, because we have specialized
functions for createing new Any values. It could be used inappropriately with existing
Any objects.
* removed deprecated PackAny function
* fix linter issue
* update nil handling
* NewAnyWithValue returns error on nil instead of panic
* NewMsgCreateValidator: handle nil pubkey
* Remove AsAny and tx builder workarounds
* fix linter issue
* Add AsAny methods to TxBuilder and StdTxConfig, but keep intoAny interface private
* remove tx.PubKeyToAny
* cosmetic updates
* fix method interface
* move ProtoTxProvider to x/auth/tx
* add test case for decoding & re-encoding a full transaction using amino
* drop equal assertion for tx and txRequest
* Fix BroadcastReq UnpackInterfaces
* Remove unrelated files
* order imports
* sort imports again
Co-authored-by: Cory Levinson <cjlevinson@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* Changelog update
* Rename codec.MarshalAny
* move codec.MarshalInterface to codec.Marshaler
* fix tests
* Update amino_codec for compliance with MarshalerInterface
* update tests and comments
* add tests
* change order of args in UnmarshalInterface to a canonical one
* uplift MarshalInterface to take ProtoMessage as an argument
* wip
* add nil check
* make tests working
* tests cleanup
* add support for *JSON methods
* Update changelog
* linter fixes
* fix test types
* update evidence genesis_test
* adding test
* review updates
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* Update tooling around docker
* Run commands
* Remove swagger gen for now
* Update proto-gen-any
* Fix indentn
* Use CURDIR
Co-authored-by: Alessio Treglia <alessio@tendermint.com>
* unknownproto: check result from protowire.ConsumeFieldValue and return error
Given that protowire.ConsumeFieldValue returns -1 when it encounters an
error, perform a check for n < 0 and return the respectively obtained
error with context about the details.
Fixes an issue identified from a go-fuzz session, thanks to Ethan
Buchman and the IBC auditors from Informal Systems et al.
Fixes#7739.
* Address AlexanderBez's suggestions
* Use require in tests
* Add issue #7846 to TODO
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* testing: refactore Validation and Delegation handling of x/staking
This Changeset introduces set of improvements for writing tests.
The idea is to create a testing subpackage which will provide functions
to make tests more dev-friendly and wrap higher level use-cases.
Here is a show-up of of creating a service for staking module
for tests.
This PR also changes the `x/staking/types.MsgCreateValidator.Pubkey` from string
to types.Any. This change motivated the other change to show the pattern I'm describing here.
* add validator checks
* type change fixes
* use deprecated
* adding test slashing
* new network comment update
* working on tests
* Fix TestMsgPkDecode test
* Add UnpackInterfaces to MsgCreateValidator
* Fix tests
* Convert bech32 pubkey to proto
* Fix test
* fix v039/migrate_test/TestMigrate
* fix tests
* testslashing: rename Service to Helper
* file rename
* update TestMsgDecode
Co-authored-by: blushi <marie.gauthier63@gmail.com>
Co-authored-by: Amaury Martiny <amaury.martiny@protonmail.com>
Co-authored-by: Cory Levinson <cjlevinson@gmail.com>
* WIP on protobuf keys
* Use Type() and Bytes() in sr25519 pub key Equals
* Add tests
* Add few more tests
* Update other pub/priv key types Equals
* Fix PrivKey's Sign method
* Rename variables in tests
* Fix infinite recursive calls
* Use tm ed25519 keys
* Add Sign and VerifySignature tests
* Remove ed25519 and sr25519 references
* proto linting
* Add proto crypto file
* Implement some of the new multisig proto type methods
* Add tests for MultisigThresholdPubKey
* Add tests for pubkey pb/amino conversion functions
* Move crypto types.go and register new proto pubkeys
* Add missing pointer ref
* Address review comments
* panic in MultisigThresholdPubKey VerifySignature
* Use internal crypto.PubKey in multisig
* Add tests for MultisigThresholdPubKey VerifyMultisignature
* Only keep LegacyAminoMultisigThresholdPubKey and move to proto keys to v1
* Remove conversion functions and introduce internal PubKey type
* Remove old secp256k1 PubKey and PrivKey
* Uncomment test case
* Fix linting issues
* More linting
* Revert tests keys values
* Add Amino overrides to proto keys
* Add pubkey test
* Fix tests
* Use threshold isntead of K
* Standardize Type
* Revert standardize types commit
* Add comment
* Simplify proto names
* Add comment about multisig codec
Co-authored-by: Aaron Craelius <aaronc@users.noreply.github.com>
Co-authored-by: Amaury Martiny <amaury.martiny@protonmail.com>
Co-authored-by: Alexander Bezobchuk <alexanderbez@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* Make JSONMarshaler require proto.Message
* Use &msg with MarshalJSON
* Use *LegacyAmino in queriers instead of JSONMarshaler
* Revert ABCIMessageLogs String() and coins tests
* Use LegacyAmino in client/debug and fix subspace tests
* Use LegacyAmino in all legacy queriers and adapt simulation
* Make AminoCodec implement Marshaler and some godoc fixes
* Test fixes
* Remove unrelevant comment
* Use TxConfig.TxJSONEncoder
* Use encoding/json in genutil cli migrate/validate genesis cmds
* Address simulation related comments
* Use JSONMarshaler in cli tests
* Use proto.Message as respType in cli tests
* Use tmjson for tm GenesisDoc
* Update types/module/simulation.go
Co-authored-by: Federico Kunze <31522760+fedekunze@users.noreply.github.com>
* Update types/module/module_test.go
Co-authored-by: Federico Kunze <31522760+fedekunze@users.noreply.github.com>
* Add godoc comments
* Remove unused InsertKeyJSON
* Fix tests
Co-authored-by: Aaron Craelius <aaronc@users.noreply.github.com>
Co-authored-by: Federico Kunze <31522760+fedekunze@users.noreply.github.com>
* create base proto package, move ValAddresses to staking
* move tx, crypto, query proto packages into base; mark module level packages as beta
* new cosmos/base/abci.proto file, refactor some message names
* add comments to TxResponse
* refactor RegisterInterface calls to new proto package names
* fix bug in merge commit
* move missing module proto packages to beta, move cosmos.kv to cosmos.base.kv
* add tx.proto files, holding proto messages for module Msgs
* rm old generated pb.go files
* cosmos/base/base.proto -> cosmos/base/coin.proto
* mark genutil proto package as beta
* Fix conflicts
* Restructure proto files
* Re put signing
* Fix test
* Update slashing genesis
* Ignore confio proto files
* Fix interface register
* Make proto-gen
* Fix lint
* Put tx service into tx.v1beta1
* Use v1beta1 in interface registry
* Remove to cosmos.base.store
* Remove extra confio in buf
Co-authored-by: Amaury Martiny <amaury.martiny@protonmail.com>
Co-authored-by: Alexander Bezobchuk <alexanderbez@users.noreply.github.com>
Co-authored-by: Federico Kunze <31522760+fedekunze@users.noreply.github.com>
* codec: remove unnecessary allocations in ProtoCodec.MarshalBinaryLengthPrefixed
Improve ProtoCodec.MarshalBinaryLengthPrefixed by removing the need
to use a *bytes.Buffer and instead use an array and invoke
binary.PutUvarint, and directly create the respective length prefixed
concatentation.
With this change we get the following improvement in all dimenions
of throughput, bytes allocated, number of allocations per operation.
```shell
$ benchstat before.txt after.txt
name old time/op new time/op delta
ProtoCodecMarshalBinaryLengthPrefixed-8 295ns ± 2% 177ns ± 3% -39.92% (p=0.000 n=20+20)
name old speed new speed delta
ProtoCodecMarshalBinaryLengthPrefixed-8 505MB/s ± 2% 841MB/s ± 3% +66.44% (p=0.000 n=20+20)
name old alloc/op new alloc/op delta
ProtoCodecMarshalBinaryLengthPrefixed-8 576B ± 0% 336B ± 0% -41.67% (p=0.000 n=20+20)
name old allocs/op new allocs/op delta
ProtoCodecMarshalBinaryLengthPrefixed-8 5.00 ± 0% 3.00 ± 0% -40.00% (p=0.000 n=20+20)
```
Fixes#6875
* Address @marbar3778's feedback
Use binary.MaxVarintLen64 instead of 10
Co-authored-by: Marko <marbar3778@yahoo.com>
Co-authored-by: Marko <marbar3778@yahoo.com>
Co-authored-by: Aaron Craelius <aaron@regen.network>
* WIP: using encoding config
* Make it compile, test fails
* test should be okay
* Make tests pass
* Add comments
* Convert more tests
* Make TestAnteHandlerSigErrors work
* Make first 2 tests pass
* TestAnteHandlerAccountNumbers
* Use table tests
* Remove print
* Use test table
* TestAnteHandlerSigErrors
* TestAnteHandlerAccountNumbers
* TestAnteHandlerAccountNumbers
* Refactor TestAccount
* Refactor getSignBytes
* TestAnteHandlerAccountNumbersAtBlockHeightZero
* TestAnteHandlerSequences
* TestAnteHandlerFees
* TestAnteHandlerMultiSigner
* TestAnteHandlerBadSignBytes
* TestAnteHandlerSetPubKey
* TestAnteHandlerSigLimitExceeded
* TestCustomSignatureVerificationGasConsumer
* TestAnteHandlerReCheck
* Make all tests pass
* Refactor a little bit more
* Fee test
* SetupTest
* All tests pass
* Refactor to RunTestCase
* Don't use StdFee
* Revert some little stuff
* Finish up last couple of test cases
* Less verbose
* s/TxGenerator/TxConfig
* Add comments
* Indent
* Move KeyTestPubAddr to testdata
* Move testdata to /testutil
* Revert to use signature: nil step in signing
* Add comments
Co-authored-by: Alexander Bezobchuk <alexanderbez@users.noreply.github.com>
* refactor auths broadcast cmd in alignment with #6216
* add TxResponse proto definition, and related refactoring
* re-run make proto-gen, updating staking.pb.go
* cleanup TxResponse tests to handle nil return values
* properly handle nil Tx value in TxResponse's implementation of UnpackInterfaces
Co-authored-by: Federico Kunze <31522760+fedekunze@users.noreply.github.com>
Cory
Amaury
Robert Zaremba
Emmanuel T Odeke
Amaury
vincent
Jack Zampolin
Marie Gauthier
SaReN
Anil Kumar Kammari
Aaron Craelius
Marko
dauTT