Commit Graph

93 Commits

Author SHA1 Message Date
Zarko Milosevic f99e4010f2 Add stats related channel between consensus state and reactor (#2388) 2018-09-21 14:36:48 -04:00
Anton Kaliaev 0e1cd88863 Remove ConsensusParams.TxSize and ConsensusParams.BlockGossip (#2364)
* remove ConsensusParams.TxSize and ConsensusParams.BlockGossip

Refs #2347

* block part size is now fixed

Refs #2347

* use max data size, not max bytes for tx limit

Refs #2347
2018-09-12 15:44:43 -04:00
Dev Ojha 2756be5a59 libs: Remove usage of custom Fmt, in favor of fmt.Sprintf (#2199)
* libs: Remove usage of custom Fmt, in favor of fmt.Sprintf

Closes #2193

* Fix bug that was masked by custom Fmt!
2018-08-10 09:25:57 +04:00
Ethan Buchman e1062a657f fixes for ProposerAddress
- state.MakeBlock takes a proposerAddr
- validateBlock only checks that the ProposerAddress is in the validator
  set
- fix raceyness from bad proposer test:
  - use privValidator to get the proposer address (instead of racy
    state)
  - note we had to remove the test that checked the correct proposer was
    included for higher rounds because we don't have a good way to test
    this with multiple consensus states and not using the
    privValidator.Address while calling createProposalBlock was a hack!
2018-08-05 15:19:21 -04:00
Ethan Buchman 4d998b7c03 consensus: failing test for ProposerAddress 2018-08-05 15:17:43 -04:00
Ethan Buchman d55243f0e6 fix import paths 2018-07-01 22:36:49 -04:00
ia b5775b56c6 all: gofmt (#1743)
* all: gofmt

Run 'gofmt -w .' from project root.

* Update changelog to say that I ran gofmt

* Revert "Update changelog to say that I ran gofmt"

This reverts commit 956f133ff0354fd7338e7df7c823e6f98b655da6.
2018-06-15 02:03:50 +02:00
Anton Kaliaev bb9aa85d22
copy events and pubsub packages from tmlibs
Refs #847
2018-05-21 10:51:47 +04:00
Jae Kwon e4492afbad Merge 2018-04-05 08:17:10 -07:00
Ethan Buchman 799beebd36 fix consensus tests 2018-04-05 17:54:26 +03:00
Jae Kwon 45ec5fd170 WIP consensus 2018-04-05 07:05:45 -07:00
Ethan Buchman 1cb76625d3 consensus: rename test funcs 2018-01-19 00:59:09 -05:00
Ethan Buchman a8e625e99d config: unexpose chainID 2017-12-28 20:49:02 +00:00
Ethan Buchman 3d00c477fc separate block vs state based validation 2017-12-21 16:49:47 -05:00
Zach Ramsay 331857c9e6 linting: apply errcheck part2 2017-11-27 22:39:11 +00:00
Anton Kaliaev fa60d8120e
fix TestFullRound1 race (Refs #846)
```
==================
WARNING: DATA RACE
Write at 0x00c42d7605f0 by goroutine 844:
  github.com/tendermint/tendermint/consensus.(*ConsensusState).updateToState()
      /home/vagrant/go/src/github.com/tendermint/tendermint/consensus/state.go:465 +0x59e
I[11-14|22:37:28.781] Added to prevote                             vote="Vote{0:646753DCE124 1/02/1(Prevote) E9B19636DCDB {/CAD5FA805E8C.../}}" prevotes="VoteSet{H:1 R:2 T:1 +2/3:<nil> BA{2:X_} map[]}"
  github.com/tendermint/tendermint/consensus.(*ConsensusState).finalizeCommit()
      /home/vagrant/go/src/github.com/tendermint/tendermint/consensus/state.go:1229 +0x16a9
  github.com/tendermint/tendermint/consensus.(*ConsensusState).tryFinalizeCommit()
      /home/vagrant/go/src/github.com/tendermint/tendermint/consensus/state.go:1135 +0x721
  github.com/tendermint/tendermint/consensus.(*ConsensusState).enterCommit.func1()
      /home/vagrant/go/src/github.com/tendermint/tendermint/consensus/state.go:1087 +0x153
  github.com/tendermint/tendermint/consensus.(*ConsensusState).enterCommit()
      /home/vagrant/go/src/github.com/tendermint/tendermint/consensus/state.go:1114 +0xa34
  github.com/tendermint/tendermint/consensus.(*ConsensusState).addVote()
      /home/vagrant/go/src/github.com/tendermint/tendermint/consensus/state.go:1423 +0xdd6
  github.com/tendermint/tendermint/consensus.(*ConsensusState).tryAddVote()
      /home/vagrant/go/src/github.com/tendermint/tendermint/consensus/state.go:1317 +0x77
  github.com/tendermint/tendermint/consensus.(*ConsensusState).handleMsg()
      /home/vagrant/go/src/github.com/tendermint/tendermint/consensus/state.go:565 +0x7a9
  github.com/tendermint/tendermint/consensus.(*ConsensusState).receiveRoutine()
      /home/vagrant/go/src/github.com/tendermint/tendermint/consensus/state.go:523 +0x6d2

Previous read at 0x00c42d7605f0 by goroutine 654:
  github.com/tendermint/tendermint/consensus.validatePrevote()
      /home/vagrant/go/src/github.com/tendermint/tendermint/consensus/common_test.go:149 +0x57
  github.com/tendermint/tendermint/consensus.TestFullRound1()
      /home/vagrant/go/src/github.com/tendermint/tendermint/consensus/state_test.go:256 +0x3c5
  testing.tRunner()
      /usr/local/go/src/testing/testing.go:746 +0x16c

Goroutine 844 (running) created at:
  github.com/tendermint/tendermint/consensus.(*ConsensusState).startRoutines()
      /home/vagrant/go/src/github.com/tendermint/tendermint/consensus/state.go:258 +0x8c
  github.com/tendermint/tendermint/consensus.startTestRound()
      /home/vagrant/go/src/github.com/tendermint/tendermint/consensus/common_test.go:118 +0x63
  github.com/tendermint/tendermint/consensus.TestFullRound1()
      /home/vagrant/go/src/github.com/tendermint/tendermint/consensus/state_test.go:247 +0x1fb
  testing.tRunner()
      /usr/local/go/src/testing/testing.go:746 +0x16c

Goroutine 654 (running) created at:
  testing.(*T).Run()
      /usr/local/go/src/testing/testing.go:789 +0x568
  testing.runTests.func1()
      /usr/local/go/src/testing/testing.go:1004 +0xa7
  testing.tRunner()
      /usr/local/go/src/testing/testing.go:746 +0x16c
  testing.runTests()
      /usr/local/go/src/testing/testing.go:1002 +0x521
  testing.(*M).Run()
      /usr/local/go/src/testing/testing.go:921 +0x206
  main.main()
      github.com/tendermint/tendermint/consensus/_test/_testmain.go:106 +0x1d3
==================
```
2017-11-14 17:41:30 -06:00
Anton Kaliaev f6539737de
new pubsub package
comment out failing consensus tests for now

rewrite rpc httpclient to use new pubsub package

import pubsub as tmpubsub, query as tmquery

make event IDs constants
EventKey -> EventTypeKey

rename EventsPubsub to PubSub

mempool does not use pubsub

rename eventsSub to pubsub

new subscribe API

fix channel size issues and consensus tests bugs

refactor rpc client

add missing discardFromChan method

add mutex

rename pubsub to eventBus

remove IsRunning from WSRPCConnection interface (not needed)

add a comment in broadcastNewRoundStepsAndVotes

rename registerEventCallbacks to broadcastNewRoundStepsAndVotes

See https://dave.cheney.net/2014/03/19/channel-axioms

stop eventBuses after reactor tests

remove unnecessary Unsubscribe

return subscribe helper function

move discardFromChan to where it is used

subscribe now returns an err

this gives us ability to refuse to subscribe if pubsub is at its max
capacity.

use context for control overflow

cache queries

handle err when subscribing in replay_test

rename testClientID to testSubscriber

extract var

set channel buffer capacity to 1 in replay_file

fix byzantine_test

unsubscribe from single event, not all events

refactor httpclient to return events to appropriate channels

return failing testReplayCrashBeforeWriteVote test

fix TestValidatorSetChanges

refactor code a bit

fix testReplayCrashBeforeWriteVote

add comment

fix TestValidatorSetChanges

fixes from Bucky's review

update comment [ci skip]

test TxEventBuffer

update changelog

fix TestValidatorSetChanges (2nd attempt)

only do wg.Done when no errors

benchmark event bus

create pubsub server inside NewEventBus

only expose config params (later if needed)

set buffer capacity to 0 so we are not testing cache

new tx event format: key = "Tx" plus a tag {"tx.hash": XYZ}

This should allow to subscribe to all transactions! or a specific one
using a query: "tm.events.type = Tx and tx.hash = '013ABF99434...'"

use TimeoutCommit instead of afterPublishEventNewBlockTimeout

TimeoutCommit is the time a node waits after committing a block, before
it goes into the next height. So it will finish everything from the last
block, but then wait a bit. The idea is this gives it time to hear more
votes from other validators, to strengthen the commit it includes in the
next block. But it also gives it time to hear about new transactions.

waitForBlockWithUpdatedVals

rewrite WAL crash tests

Task:
test that we can recover from any WAL crash.

Solution:
the old tests were relying on event hub being run in the same thread (we
were injecting the private validator's last signature).

when considering a rewrite, we considered two possible solutions: write
a "fuzzy" testing system where WAL is crashing upon receiving a new
message, or inject failures and trigger them in tests using something
like https://github.com/coreos/gofail.

remove sleep

no cs.Lock around wal.Save

test different cases (empty block, non-empty block, ...)

comments

add comments

test 4 cases: empty block, non-empty block, non-empty block with smaller part size, many blocks

fixes as per Bucky's last review

reset subscriptions on UnsubscribeAll

use a simple counter to track message for which we panicked

also, set a smaller part size for all test cases
2017-10-30 00:32:22 -05:00
Anton Kaliaev 716364182d
[state] expose ChainID and Params
```
jaekwon
Yeah we should definitely expose ChainID.
ConsensusParams is small enough, we can just write it.
```
https://github.com/tendermint/tendermint/pull/676#discussion_r144123203
2017-10-16 10:34:02 +04:00
Anton Kaliaev dc0e8de9b0
extract some of the consensus types into ./types
so they can be used in rpc/core/types/responses.go.

```
So, it seems like we could use the actual structs here, but we don't want to have to import consensus to get them, as then clients are importing too much crap. So probably we should move some types from consensus into consensus/types so we can import.

Will these raw messages be identical to:

type ResultDumpConsensusState struct {
  RoundState cstypes.RoundState
  PeerRoundStates map[string]cstypes.PeerRoundState
}
```
https://github.com/tendermint/tendermint/pull/724#discussion_r143598193
2017-10-10 12:39:21 +04:00
Zach Ramsay d56b44f3a5 all: no more anonymous imports 2017-10-04 16:40:45 -04:00
Ethan Buchman 75b97a5a65 PrivValidatorFS is like old PrivValidator, for now 2017-09-21 16:46:31 -04:00
Ethan Buchman 4382c8d28b fix tests 2017-09-21 15:52:25 -04:00
Ethan Buchman 2b6db268cf genesis json tests and mv ConsensusParams to types 2017-09-21 14:51:29 -04:00
Ethan Buchman 1f3e4d2d9a move PartSetSize out of the config, into ConsensusParams 2017-09-21 14:51:29 -04:00
Ethan Buchman 0d1fa8e884 fixes from review 2017-06-28 11:12:45 -04:00
zramsay cf31f8d06f core: apply megacheck vet tool (unused, gosimple, staticcheck) 2017-05-29 23:11:40 -04:00
spring1843 cf686d4f83 Fix commonly misspelled words 2017-05-20 21:43:00 -07:00
Anton Kaliaev f803544195
new logging 2017-05-13 10:24:58 +02:00
Ethan Buchman 46151720f8 fix tests 2017-05-04 22:46:41 -04:00
Ethan Buchman 92bafa7ecd consensus: fix tests 2017-05-04 22:46:13 -04:00
Ethan Buchman 07e59e63f9 TMEventDataInner 2017-04-28 17:57:06 -04:00
Ethan Buchman d1926bcad1 use tmlibs 2017-04-21 18:12:54 -04:00
Ethan Buchman b1cd677711 types: valSet LastProposer->Proposer and Proposer()->GetProposer() 2017-03-05 23:28:42 -05:00
Jae Kwon d83fc02597 MakePartSet takes partSize from config. fix replay test 2016-11-16 01:23:16 -05:00
Ethan Buchman 57da2e4af5 make byzantine logic testable 2016-11-15 18:45:36 -05:00
Ethan Buchman f837252ff1 consensus: test reactor 2016-11-15 18:37:33 -05:00
Jae Kwon 655d829314 Fix proposal sign bytes. Start tracking blockID in POL 2016-11-15 18:35:17 -05:00
Jae Kwon 3c5a2f55c2 Add validator index and address to Vote. 2016-11-15 18:33:16 -05:00
Ethan Buchman 35d4cca8bb type safe events 2016-10-10 03:10:29 -04:00
Ethan Buchman 987dac9ee0 consensus: no sign err in replay; fix a race 2016-09-08 19:00:59 -04:00
Ethan Buchman 54357bcf8f crank consensus timeout parameters for tests (see #236) 2016-07-12 14:25:27 -04:00
Ethan Buchman 47acada2cb consensus: t.Fatal -> panic 2016-07-11 22:37:39 -04:00
Ethan Buchman e4c795f15c consensus: fix more races in tests 2016-07-11 20:40:48 -04:00
Ethan Buchman bc0b52293e update circle and glide 2016-07-05 15:40:53 -04:00
Jae Kwon ad17090a0f No global config 2016-05-08 15:00:58 -07:00
Ethan Buchman 523a170c3e EventDataTypeNewBlockHeader 2016-04-19 20:59:52 -04:00
Ethan Buchman 922f720cf6 cswal_light logs own votes; fix tests 2016-04-19 20:45:33 -04:00
Jae Kwon 03115cbf93 Fix tests; Tests run in their own directory 2016-03-06 15:05:50 -08:00
Ethan Buchman c9ec9cf00e config: block size, consensus timeouts, recheck tx 2016-02-29 17:04:33 -05:00
Jae Kwon 0f1cf243fd Merge remote-tracking branch 'origin/replay' 2016-01-28 19:53:22 -08:00