Commit Graph

113 Commits

Author SHA1 Message Date
Zarko Milosevic 7b88172f41 Implement BFT time (#2203)
* Implement BFT time

* set LastValidators when creating state in state helper

for heights >= 2
2018-08-31 19:33:51 -04:00
Ethan Buchman e3f54ece2f abci: VoteInfo, ValidatorUpdate. See ADR-018 2018-08-16 13:19:13 -04:00
Dev Ojha 2756be5a59 libs: Remove usage of custom Fmt, in favor of fmt.Sprintf (#2199)
* libs: Remove usage of custom Fmt, in favor of fmt.Sprintf

Closes #2193

* Fix bug that was masked by custom Fmt!
2018-08-10 09:25:57 +04:00
ValarDragon a83eed104c libs/cmn: Remove Tempfile, Tempdir, switch to ioutil variants (#2114)
Our Tempfile was just a wrapper on ioutil that panicked instead of error.

Our Tempdir was a less safe variant of ioutil's Tempdir.
2018-07-31 19:43:36 +02:00
Ethan Buchman fe5b0d7074 consensus: fix test for blocks with evidence 2018-07-27 14:00:05 -04:00
Ethan Buchman d55243f0e6 fix import paths 2018-07-01 22:36:49 -04:00
Alexander Simmerl a605b66c5a
Move abci imports 2018-06-22 07:28:07 +02:00
Ethan Buchman 3d33226e80 move types/services.go to state pkg. pass State to evpool.Update 2018-06-04 21:20:23 -07:00
Alexander Simmerl bf370d36c2
Extract priv_validator into first class package
This is a maintenance change to move the private validator package out
of the types and to a top-level location. There is no good reason to
keep it under the types and it will more clearly coommunicate where
additions related to the privval belong. It leaves the interface and the
mock in types for now as it would introduce circular dependency between
privval and types, this should be resolved eventually.

* mv priv_validator to privval pkg
* use consistent `privval` as import

Follow-up to #1255
2018-06-03 13:51:58 +02:00
Anton Kaliaev 21f5f3faa7
use channels to send votes, ... from consensus state to reactor
Refs #847
2018-05-21 10:50:55 +04:00
Vladislav Dmitriyev 7c22e47629 Replaced NodeInfo's pubkey to ID (#1443)
* Replaced NodeInfo PubKey to NodeID

* Fixed tests and replaced NodeID with ID

* Removed unnecessary method ID()

* Fixed codec_test.go

* Fixed codec_test.go

* Removed unnecessary bracket

* Fixed all tests

* Fixed peer_set_test.go

* Fixed peer_test.go

* Fixed common_test.go

* Fixed common_test.go

* Renamed node_id to id

* Removed peer.ID() from RPC net.go

* Replaced NodeInfo pubKey to ID

* Fixed codec_test.go

* Fixed peer_set_test.go

* Fix pex_reactor_test.go

* Refactored code for privateKey initiali

* Fixed peer_set_test.go

* Fixed test.proto and removed orphan string in codec_test.go

* Fixed pointer to a string

* generate node_key when running tendermint init

* [docs] prefix IPs with node IDs

Refs #1429

* gen_node_key cmd

* [docs/specification/secure-p2p] add a note about config

* fix data race

Closes #1442

```
WARNING: DATA RACE
Write at 0x00c4209de7c8 by goroutine 23:
  github.com/tendermint/tendermint/types.(*Block).fillHeader()
      /home/vagrant/go/src/github.com/tendermint/tendermint/types/block.go:88 +0x157
  github.com/tendermint/tendermint/types.(*Block).Hash()
      /home/vagrant/go/src/github.com/tendermint/tendermint/types/block.go:104 +0x121
  github.com/tendermint/tendermint/types.(*Block).HashesTo()
      /home/vagrant/go/src/github.com/tendermint/tendermint/types/block.go:135 +0x4f
  github.com/tendermint/tendermint/consensus.(*ConsensusState).enterPrecommit()
      /home/vagrant/go/src/github.com/tendermint/tendermint/consensus/state.go:1037 +0x182d
  github.com/tendermint/tendermint/consensus.(*ConsensusState).addVote()
      /home/vagrant/go/src/github.com/tendermint/tendermint/consensus/state.go:1425 +0x1a6c
  github.com/tendermint/tendermint/consensus.(*ConsensusState).tryAddVote()
      /home/vagrant/go/src/github.com/tendermint/tendermint/consensus/state.go:1318 +0x77
  github.com/tendermint/tendermint/consensus.(*ConsensusState).handleMsg()
      /home/vagrant/go/src/github.com/tendermint/tendermint/consensus/state.go:581 +0x7a9
  github.com/tendermint/tendermint/consensus.(*ConsensusState).receiveRoutine()
      /home/vagrant/go/src/github.com/tendermint/tendermint/consensus/state.go:539 +0x6c3

Previous read at 0x00c4209de7c8 by goroutine 47:
  github.com/tendermint/tendermint/vendor/github.com/tendermint/tmlibs/common.(*HexBytes).MarshalJSON()
      <autogenerated>:1 +0x52
  github.com/tendermint/tendermint/vendor/github.com/tendermint/go-amino.invokeMarshalJSON()
      /home/vagrant/go/src/github.com/tendermint/tendermint/vendor/github.com/tendermint/go-amino/json-encode.go:433 +0x88
  github.com/tendermint/tendermint/vendor/github.com/tendermint/go-amino.(*Codec)._encodeReflectJSON()
      /home/vagrant/go/src/github.com/tendermint/tendermint/vendor/github.com/tendermint/go-amino/json-encode.go:82 +0x8d2
  github.com/tendermint/tendermint/vendor/github.com/tendermint/go-amino.(*Codec).encodeReflectJSON()
      /home/vagrant/go/src/github.com/tendermint/tendermint/vendor/github.com/tendermint/go-amino/json-encode.go:50 +0x10e
  github.com/tendermint/tendermint/vendor/github.com/tendermint/go-amino.(*Codec).encodeReflectJSONStruct()
      /home/vagrant/go/src/github.com/tendermint/tendermint/vendor/github.com/tendermint/go-amino/json-encode.go:348 +0x539
  github.com/tendermint/tendermint/vendor/github.com/tendermint/go-amino.(*Codec)._encodeReflectJSON()
      /home/vagrant/go/src/github.com/tendermint/tendermint/vendor/github.com/tendermint/go-amino/json-encode.go:119 +0x83f
  github.com/tendermint/tendermint/vendor/github.com/tendermint/go-amino.(*Codec).encodeReflectJSON()
      /home/vagrant/go/src/github.com/tendermint/tendermint/vendor/github.com/tendermint/go-amino/json-encode.go:50 +0x10e
  github.com/tendermint/tendermint/vendor/github.com/tendermint/go-amino.(*Codec).encodeReflectJSONStruct()
      /home/vagrant/go/src/github.com/tendermint/tendermint/vendor/github.com/tendermint/go-amino/json-encode.go:348 +0x539
  github.com/tendermint/tendermint/vendor/github.com/tendermint/go-amino.(*Codec)._encodeReflectJSON()
      /home/vagrant/go/src/github.com/tendermint/tendermint/vendor/github.com/tendermint/go-amino/json-encode.go:119 +0x83f
  github.com/tendermint/tendermint/vendor/github.com/tendermint/go-amino.(*Codec).encodeReflectJSON()
      /home/vagrant/go/src/github.com/tendermint/tendermint/vendor/github.com/tendermint/go-amino/json-encode.go:50 +0x10e
  github.com/tendermint/tendermint/vendor/github.com/tendermint/go-amino.(*Codec).encodeReflectJSONStruct()
      /home/vagrant/go/src/github.com/tendermint/tendermint/vendor/github.com/tendermint/go-amino/json-encode.go:348 +0x539
  github.com/tendermint/tendermint/vendor/github.com/tendermint/go-amino.(*Codec)._encodeReflectJSON()
      /home/vagrant/go/src/github.com/tendermint/tendermint/vendor/github.com/tendermint/go-amino/json-encode.go:119 +0x83f
  github.com/tendermint/tendermint/vendor/github.com/tendermint/go-amino.(*Codec).encodeReflectJSON()
      /home/vagrant/go/src/github.com/tendermint/tendermint/vendor/github.com/tendermint/go-amino/json-encode.go:50 +0x10e
  github.com/tendermint/tendermint/vendor/github.com/tendermint/go-amino.(*Codec).encodeReflectJSONStruct()
      /home/vagrant/go/src/github.com/tendermint/tendermint/vendor/github.com/tendermint/go-amino/json-encode.go:348 +0x539
  github.com/tendermint/tendermint/vendor/github.com/tendermint/go-amino.(*Codec)._encodeReflectJSON()
      /home/vagrant/go/src/github.com/tendermint/tendermint/vendor/github.com/tendermint/go-amino/json-encode.go:119 +0x83f
  github.com/tendermint/tendermint/vendor/github.com/tendermint/go-amino.(*Codec).encodeReflectJSON()
      /home/vagrant/go/src/github.com/tendermint/tendermint/vendor/github.com/tendermint/go-amino/json-encode.go:50 +0x10e
  github.com/tendermint/tendermint/vendor/github.com/tendermint/go-amino.(*Codec).MarshalJSON()
      /home/vagrant/go/src/github.com/tendermint/tendermint/vendor/github.com/tendermint/go-amino/amino.go:296 +0x182
  github.com/tendermint/tendermint/rpc/lib/types.NewRPCSuccessResponse()
      /home/vagrant/go/src/github.com/tendermint/tendermint/rpc/lib/types/types.go:100 +0x12c
  github.com/tendermint/tendermint/rpc/lib/server.makeJSONRPCHandler.func1()
      /home/vagrant/go/src/github.com/tendermint/tendermint/rpc/lib/server/handlers.go:152 +0xab7
  net/http.HandlerFunc.ServeHTTP()
      /usr/lib/go-1.9/src/net/http/server.go:1918 +0x51
  net/http.(*ServeMux).ServeHTTP()
      /usr/lib/go-1.9/src/net/http/server.go:2254 +0xa2
  github.com/tendermint/tendermint/rpc/lib/server.RecoverAndLogHandler.func1()
      /home/vagrant/go/src/github.com/tendermint/tendermint/rpc/lib/server/http_server.go:138 +0x4fa
  net/http.HandlerFunc.ServeHTTP()
      /usr/lib/go-1.9/src/net/http/server.go:1918 +0x51
  net/http.serverHandler.ServeHTTP()
      /usr/lib/go-1.9/src/net/http/server.go:2619 +0xbc
  net/http.(*conn).serve()
      /usr/lib/go-1.9/src/net/http/server.go:1801 +0x83b

Goroutine 23 (running) created at:
  github.com/tendermint/tendermint/consensus.(*ConsensusState).OnStart()
      /home/vagrant/go/src/github.com/tendermint/tendermint/consensus/state.go:250 +0x35b
  github.com/tendermint/tendermint/vendor/github.com/tendermint/tmlibs/common.(*BaseService).Start()
      /home/vagrant/go/src/github.com/tendermint/tendermint/vendor/github.com/tendermint/tmlibs/common/service.go:130 +0x5fc
  github.com/tendermint/tendermint/consensus.(*ConsensusReactor).OnStart()
      /home/vagrant/go/src/github.com/tendermint/tendermint/consensus/reactor.go:69 +0x1b4
  github.com/tendermint/tendermint/vendor/github.com/tendermint/tmlibs/common.(*BaseService).Start()
      /home/vagrant/go/src/github.com/tendermint/tendermint/vendor/github.com/tendermint/tmlibs/common/service.go:130 +0x5fc
  github.com/tendermint/tendermint/consensus.(*ConsensusReactor).Start()
      <autogenerated>:1 +0x43
  github.com/tendermint/tendermint/p2p.(*Switch).OnStart()
      /home/vagrant/go/src/github.com/tendermint/tendermint/p2p/switch.go:177 +0x124
  github.com/tendermint/tendermint/vendor/github.com/tendermint/tmlibs/common.(*BaseService).Start()
      /home/vagrant/go/src/github.com/tendermint/tendermint/vendor/github.com/tendermint/tmlibs/common/service.go:130 +0x5fc
  github.com/tendermint/tendermint/node.(*Node).OnStart()
      /home/vagrant/go/src/github.com/tendermint/tendermint/node/node.go:416 +0xa1b
  github.com/tendermint/tendermint/vendor/github.com/tendermint/tmlibs/common.(*BaseService).Start()
      /home/vagrant/go/src/github.com/tendermint/tendermint/vendor/github.com/tendermint/tmlibs/common/service.go:130 +0x5fc
  github.com/tendermint/tendermint/rpc/test.StartTendermint()
      /home/vagrant/go/src/github.com/tendermint/tendermint/rpc/test/helpers.go:100 +0x5b
  github.com/tendermint/tendermint/rpc/client_test.TestMain()
      /home/vagrant/go/src/github.com/tendermint/tendermint/rpc/client/main_test.go:17 +0x4c
  main.main()
      github.com/tendermint/tendermint/rpc/client/_test/_testmain.go:76 +0x1cd

Goroutine 47 (running) created at:
  net/http.(*Server).Serve()
      /usr/lib/go-1.9/src/net/http/server.go:2720 +0x37c
  net/http.Serve()
      /usr/lib/go-1.9/src/net/http/server.go:2323 +0xe2
  github.com/tendermint/tendermint/rpc/lib/server.StartHTTPServer.func1()
      /home/vagrant/go/src/github.com/tendermint/tendermint/rpc/lib/server/http_server.go:35 +0xb3
```

* removed excessive comment

Refs https://github.com/tendermint/tendermint/pull/1446#discussion_r180353446

* use the tag interface for pubsub. (#1438)

* use the tag interface for pubsub.

* update tmlibs.

* Fix unresolved conflict.

* improve `show_node_id` (#1433)

* fix show_node_id

* make LoadNodeKey public

* make LoadNodeKey public

* remove if

* remove if
2018-04-11 10:11:11 +02:00
Ethan Buchman c170800fbd Merge branch 'develop' into jae/aminoify 2018-04-07 16:16:53 +03:00
Jae Kwon fb64314d1c Review from Anton 2018-04-06 13:46:40 -07:00
Jae Kwon e4492afbad Merge 2018-04-05 08:17:10 -07:00
Jae Kwon 45ec5fd170 WIP consensus 2018-04-05 07:05:45 -07:00
Jae Kwon 196f8410ba WIP commit; Fix types/results_test 2018-04-03 07:03:08 -07:00
Thomas Corbière 2644a529f0 Fix lint errors (#1390)
* use increment and decrement operators.

* remove unnecessary else branches.

* fix package comment with leading space.

* fix receiver names.

* fix error strings.

* remove omittable code.

* remove redundant return statement.

* Revert changes (code is generated.)

* use cfg as receiver name for all config-related types.

* use lsi as the receiver name for the LastSignedInfo type.
2018-04-02 10:21:17 +02:00
Zach 2cc63069c6 rename dummy to kvstore (#1223)
* remove accidental binary

* docs: s/Dummy&dummy/KVStore&kvstore/g

* glide update to abci

* update abci import paths

* dummy begone, hello kvstore

* RequestInitChain needs genesisBytes

* glide update
2018-02-27 18:01:10 +04:00
Ethan Buchman 3ae26bd6e6 consensus: fix SetLogger in tests 2018-01-24 23:34:57 -05:00
Ethan Buchman 8171628ee5 make tests run faster 2018-01-19 00:59:09 -05:00
Zach e0e600df05
Merge branch 'develop' into config 2018-01-06 05:30:38 +00:00
Ethan Buchman ae68fcb78a move fireEvents to ApplyBlock 2017-12-28 23:15:54 -05:00
Ethan Buchman 1d6f00859d fixes from review 2017-12-28 23:15:54 -05:00
Ethan Buchman 537b0dfa1a use NopEventBus 2017-12-28 23:15:54 -05:00
Ethan Buchman 0acca7fe69 final updates for state 2017-12-28 23:15:54 -05:00
Ethan Buchman a8e625e99d config: unexpose chainID 2017-12-28 20:49:02 +00:00
Ethan Buchman 7a18fa887d evidence linked with consensus/node. compiles 2017-12-26 20:26:21 -05:00
Anton Kaliaev 0093f9877a
change voting power change, not number of vals 2017-12-25 17:49:36 -06:00
Ethan Frey a29c781295 Add default timestamp to all instances of *types.Vote 2017-12-12 12:59:51 +01:00
Ethan Buchman 9af8da7aad update for new abci int types 2017-12-02 01:47:55 -05:00
Anton Kaliaev 922af7c405
int64 height
uint64 is considered dangerous. the details will follow in a blog post.
2017-12-01 19:04:53 -06:00
Anton Kaliaev b3492356e6 uint64 height (Refs #911) 2017-12-01 17:17:22 -05:00
Anton Kaliaev 69b5da766c
service#Start, service#Stop signatures were changed
See https://github.com/tendermint/tmlibs/issues/45
2017-11-29 10:38:58 -06:00
Zach Ramsay 331857c9e6 linting: apply errcheck part2 2017-11-27 22:39:11 +00:00
Anton Kaliaev f6539737de
new pubsub package
comment out failing consensus tests for now

rewrite rpc httpclient to use new pubsub package

import pubsub as tmpubsub, query as tmquery

make event IDs constants
EventKey -> EventTypeKey

rename EventsPubsub to PubSub

mempool does not use pubsub

rename eventsSub to pubsub

new subscribe API

fix channel size issues and consensus tests bugs

refactor rpc client

add missing discardFromChan method

add mutex

rename pubsub to eventBus

remove IsRunning from WSRPCConnection interface (not needed)

add a comment in broadcastNewRoundStepsAndVotes

rename registerEventCallbacks to broadcastNewRoundStepsAndVotes

See https://dave.cheney.net/2014/03/19/channel-axioms

stop eventBuses after reactor tests

remove unnecessary Unsubscribe

return subscribe helper function

move discardFromChan to where it is used

subscribe now returns an err

this gives us ability to refuse to subscribe if pubsub is at its max
capacity.

use context for control overflow

cache queries

handle err when subscribing in replay_test

rename testClientID to testSubscriber

extract var

set channel buffer capacity to 1 in replay_file

fix byzantine_test

unsubscribe from single event, not all events

refactor httpclient to return events to appropriate channels

return failing testReplayCrashBeforeWriteVote test

fix TestValidatorSetChanges

refactor code a bit

fix testReplayCrashBeforeWriteVote

add comment

fix TestValidatorSetChanges

fixes from Bucky's review

update comment [ci skip]

test TxEventBuffer

update changelog

fix TestValidatorSetChanges (2nd attempt)

only do wg.Done when no errors

benchmark event bus

create pubsub server inside NewEventBus

only expose config params (later if needed)

set buffer capacity to 0 so we are not testing cache

new tx event format: key = "Tx" plus a tag {"tx.hash": XYZ}

This should allow to subscribe to all transactions! or a specific one
using a query: "tm.events.type = Tx and tx.hash = '013ABF99434...'"

use TimeoutCommit instead of afterPublishEventNewBlockTimeout

TimeoutCommit is the time a node waits after committing a block, before
it goes into the next height. So it will finish everything from the last
block, but then wait a bit. The idea is this gives it time to hear more
votes from other validators, to strengthen the commit it includes in the
next block. But it also gives it time to hear about new transactions.

waitForBlockWithUpdatedVals

rewrite WAL crash tests

Task:
test that we can recover from any WAL crash.

Solution:
the old tests were relying on event hub being run in the same thread (we
were injecting the private validator's last signature).

when considering a rewrite, we considered two possible solutions: write
a "fuzzy" testing system where WAL is crashing upon receiving a new
message, or inject failures and trigger them in tests using something
like https://github.com/coreos/gofail.

remove sleep

no cs.Lock around wal.Save

test different cases (empty block, non-empty block, ...)

comments

add comments

test 4 cases: empty block, non-empty block, non-empty block with smaller part size, many blocks

fixes as per Bucky's last review

reset subscriptions on UnsubscribeAll

use a simple counter to track message for which we panicked

also, set a smaller part size for all test cases
2017-10-30 00:32:22 -05:00
Ethan Buchman fd60621a8e update cswal test 2017-10-23 10:03:54 -04:00
Anton Kaliaev dc0e8de9b0
extract some of the consensus types into ./types
so they can be used in rpc/core/types/responses.go.

```
So, it seems like we could use the actual structs here, but we don't want to have to import consensus to get them, as then clients are importing too much crap. So probably we should move some types from consensus into consensus/types so we can import.

Will these raw messages be identical to:

type ResultDumpConsensusState struct {
  RoundState cstypes.RoundState
  PeerRoundStates map[string]cstypes.PeerRoundState
}
```
https://github.com/tendermint/tendermint/pull/724#discussion_r143598193
2017-10-10 12:39:21 +04:00
Zach Ramsay d56b44f3a5 all: no more anonymous imports 2017-10-04 16:40:45 -04:00
Ethan Buchman 75b97a5a65 PrivValidatorFS is like old PrivValidator, for now 2017-09-21 16:46:31 -04:00
Ethan Buchman 779c2a22d0 node: NewNode takes DBProvider and GenDocProvider 2017-09-21 15:54:33 -04:00
Ethan Buchman 4382c8d28b fix tests 2017-09-21 15:52:25 -04:00
Ethan Buchman 944ebccfe9 more PrivValidator interface 2017-09-21 15:51:20 -04:00
Adrian Brink 7dd3c007c7 Refactor priv_validator
Users can now just pass an object that implements the Signer interface.
2017-09-21 15:50:43 -04:00
Ethan Buchman 3089bbf2b8 Amount -> Power. Closes #166 2017-09-21 14:59:27 -04:00
Ethan Buchman aea8629272 peer interface 2017-09-15 18:40:59 -04:00
Ethan Buchman c5a657f540 consensus: test proposal heartbeat 2017-08-10 01:24:23 -04:00
Ethan Buchman d2f3e9faf4 test CreateEmptyBlocksInterval 2017-08-08 16:35:25 -04:00
Ethan Buchman 37f1390473 CreateEmptyBlocks and CreateEmptyBlocksInterval 2017-08-08 16:22:37 -04:00
Ethan Buchman 3444bee47f fixes from review; use mempool.TxsAvailable() directly 2017-07-28 23:42:43 -04:00
Ethan Buchman 678a9a2e42 TxsAvailable tests 2017-07-28 22:11:45 -04:00