Commit Graph

599 Commits

Author SHA1 Message Date
Anton Kaliaev 1e19860585
fixes from my own review 2017-11-29 14:24:18 -06:00
Anton Kaliaev f65e357d2b
adapt Tendermint to new abci.Client interface
which was introduced in https://github.com/tendermint/abci/pull/130
2017-11-29 14:23:44 -06:00
Anton Kaliaev a52cdbfe43
extract tags from DeliverTx/Result
and send them along with predefined
2017-11-29 14:23:43 -06:00
Anton Kaliaev 69b5da766c
service#Start, service#Stop signatures were changed
See https://github.com/tendermint/tmlibs/issues/45
2017-11-29 10:38:58 -06:00
Ethan Buchman 2009c3d4f1
Merge pull request #884 from tendermint/wal-benchmark-decode
consensus/WAL: benchmark WALDecode across data sizes
2017-11-28 03:39:55 +00:00
Ethan Buchman 9529f12c28 more linting 2017-11-27 22:39:12 +00:00
Ethan Buchman 55b81cc1a1 address linting FIXMEs 2017-11-27 22:39:12 +00:00
Zach Ramsay c4caad7720 lint madness 2017-11-27 22:39:12 +00:00
Zach Ramsay 2563b4fc92 lint fixes 2017-11-27 22:39:12 +00:00
Zach Ramsay 6f3c05545d fix new linting errors 2017-11-27 22:39:12 +00:00
Zach Ramsay 7ad8a8ab55 Tests almost passing 2017-11-27 22:39:12 +00:00
Zach Ramsay a15c7f221d linting: moar fixes 2017-11-27 22:39:11 +00:00
Zach Ramsay d7cb291fb2 errcheck; sort some stuff out 2017-11-27 22:39:11 +00:00
Zach Ramsay 563faa98de address comments, pr #643 2017-11-27 22:39:11 +00:00
Zach Ramsay bc2aa79f9a linter: sort through each kind and address small fixes 2017-11-27 22:39:11 +00:00
Zach Ramsay 48aca642e3 linter: address deadcode, implement incremental lint testing 2017-11-27 22:39:11 +00:00
Zach Ramsay 68e7983c70 linting errors: afew more 2017-11-27 22:39:11 +00:00
Zach Ramsay 8f0237610e linting errors: clean it all up 2017-11-27 22:39:11 +00:00
Zach Ramsay b75d4f73e7 errcheck: PR comment fixes 2017-11-27 22:39:11 +00:00
Zach Ramsay 331857c9e6 linting: apply errcheck part2 2017-11-27 22:39:11 +00:00
Zach Ramsay 57ea4987f7 linting: apply errcheck part1 2017-11-27 22:39:11 +00:00
Zach Ramsay 46ccbcbff6 linting: apply 'gofmt -s -w' throughout 2017-11-27 22:39:11 +00:00
Emmanuel Odeke 42da8cd297
consensus/WAL: benchmark WALDecode across data sizes 2017-11-23 12:43:11 -07:00
Anton Kaliaev fa60d8120e
fix TestFullRound1 race (Refs #846)
```
==================
WARNING: DATA RACE
Write at 0x00c42d7605f0 by goroutine 844:
  github.com/tendermint/tendermint/consensus.(*ConsensusState).updateToState()
      /home/vagrant/go/src/github.com/tendermint/tendermint/consensus/state.go:465 +0x59e
I[11-14|22:37:28.781] Added to prevote                             vote="Vote{0:646753DCE124 1/02/1(Prevote) E9B19636DCDB {/CAD5FA805E8C.../}}" prevotes="VoteSet{H:1 R:2 T:1 +2/3:<nil> BA{2:X_} map[]}"
  github.com/tendermint/tendermint/consensus.(*ConsensusState).finalizeCommit()
      /home/vagrant/go/src/github.com/tendermint/tendermint/consensus/state.go:1229 +0x16a9
  github.com/tendermint/tendermint/consensus.(*ConsensusState).tryFinalizeCommit()
      /home/vagrant/go/src/github.com/tendermint/tendermint/consensus/state.go:1135 +0x721
  github.com/tendermint/tendermint/consensus.(*ConsensusState).enterCommit.func1()
      /home/vagrant/go/src/github.com/tendermint/tendermint/consensus/state.go:1087 +0x153
  github.com/tendermint/tendermint/consensus.(*ConsensusState).enterCommit()
      /home/vagrant/go/src/github.com/tendermint/tendermint/consensus/state.go:1114 +0xa34
  github.com/tendermint/tendermint/consensus.(*ConsensusState).addVote()
      /home/vagrant/go/src/github.com/tendermint/tendermint/consensus/state.go:1423 +0xdd6
  github.com/tendermint/tendermint/consensus.(*ConsensusState).tryAddVote()
      /home/vagrant/go/src/github.com/tendermint/tendermint/consensus/state.go:1317 +0x77
  github.com/tendermint/tendermint/consensus.(*ConsensusState).handleMsg()
      /home/vagrant/go/src/github.com/tendermint/tendermint/consensus/state.go:565 +0x7a9
  github.com/tendermint/tendermint/consensus.(*ConsensusState).receiveRoutine()
      /home/vagrant/go/src/github.com/tendermint/tendermint/consensus/state.go:523 +0x6d2

Previous read at 0x00c42d7605f0 by goroutine 654:
  github.com/tendermint/tendermint/consensus.validatePrevote()
      /home/vagrant/go/src/github.com/tendermint/tendermint/consensus/common_test.go:149 +0x57
  github.com/tendermint/tendermint/consensus.TestFullRound1()
      /home/vagrant/go/src/github.com/tendermint/tendermint/consensus/state_test.go:256 +0x3c5
  testing.tRunner()
      /usr/local/go/src/testing/testing.go:746 +0x16c

Goroutine 844 (running) created at:
  github.com/tendermint/tendermint/consensus.(*ConsensusState).startRoutines()
      /home/vagrant/go/src/github.com/tendermint/tendermint/consensus/state.go:258 +0x8c
  github.com/tendermint/tendermint/consensus.startTestRound()
      /home/vagrant/go/src/github.com/tendermint/tendermint/consensus/common_test.go:118 +0x63
  github.com/tendermint/tendermint/consensus.TestFullRound1()
      /home/vagrant/go/src/github.com/tendermint/tendermint/consensus/state_test.go:247 +0x1fb
  testing.tRunner()
      /usr/local/go/src/testing/testing.go:746 +0x16c

Goroutine 654 (running) created at:
  testing.(*T).Run()
      /usr/local/go/src/testing/testing.go:789 +0x568
  testing.runTests.func1()
      /usr/local/go/src/testing/testing.go:1004 +0xa7
  testing.tRunner()
      /usr/local/go/src/testing/testing.go:746 +0x16c
  testing.runTests()
      /usr/local/go/src/testing/testing.go:1002 +0x521
  testing.(*M).Run()
      /usr/local/go/src/testing/testing.go:921 +0x206
  main.main()
      github.com/tendermint/tendermint/consensus/_test/_testmain.go:106 +0x1d3
==================
```
2017-11-14 17:41:30 -06:00
Ethan Buchman 30f675aafa
Merge pull request #839 from tendermint/bugfix/pubsub-failures
Fix nondeterministic tests failures related to pubsub
2017-11-14 18:13:47 +00:00
Ethan Buchman 3db44dacae
Merge pull request #840 from tendermint/fix/tests
Fix/tests
2017-11-14 15:48:17 +00:00
Ethan Buchman 238e2b72ee
Merge pull request #834 from tendermint/829-enable-logs-by-default
Enable logs by default
2017-11-12 07:06:14 +00:00
Ethan Buchman aba8a8f4fc consensus: crank timeout in timeoutWaitGroup 2017-11-12 06:41:15 +00:00
Ethan Buchman 0448c2b437 consensus: fix LastCommit log 2017-11-12 06:40:27 +00:00
Anton Kaliaev 7fa12662c4
check whatever we can read from the channel
```
panic: interface conversion: interface {} is nil, not types.TMEventData

goroutine 7690 [running]:
github.com/tendermint/tendermint/consensus.waitForAndValidateBlock.func1(0xc427727620, 0x3)
        /go/src/github.com/tendermint/tendermint/consensus/reactor_test.go:292 +0x62b
created by github.com/tendermint/tendermint/consensus.timeoutWaitGroup
        /go/src/github.com/tendermint/tendermint/consensus/reactor_test.go:349 +0xa4
exit status 2
FAIL    github.com/tendermint/tendermint/consensus      38.614s

```
2017-11-10 18:16:31 -05:00
Ethan Buchman 2cda777900 consensus: make mempool_test deterministic 2017-11-09 23:54:02 +00:00
Anton Kaliaev a1cdc2b68a
set logger for peer's MConnection 2017-11-09 14:57:40 -05:00
Anton Kaliaev ad03491ee6
remove duplicated key 2017-11-09 13:37:29 -05:00
Ethan Buchman 4b9dfc8990 consensus: fix for initializing block parts during catchup 2017-11-09 18:14:41 +00:00
Ethan Buchman c0e2649ed6
Merge pull request #788 from tendermint/feature/548-indexing-tags
new pubsub package
2017-11-08 01:02:48 +00:00
Ethan Buchman dd47884661
Merge pull request #820 from tendermint/790-use-tickers-instead-of-time-Sleep
prefer tickers to time.Sleep
2017-11-08 00:53:42 +00:00
Ethan Buchman 47f5e37205 copy RoundState for event 2017-11-07 23:57:23 +00:00
Anton Kaliaev e0daca5693
fixes from Bucky's review 2017-11-07 18:20:24 -05:00
Anton Kaliaev 2d4ad02356
prefer tickers to time.Sleep (Refs #790) 2017-11-07 15:38:25 -05:00
Ethan Buchman 3b81d3fea4 consensus: ensure prs.ProposalBlockParts is initialized. fixes #810 2017-11-07 17:14:40 +00:00
Anton Kaliaev b1eec3a5d3
remove test_data/empty_block and test_data/small_blockN 2017-11-02 13:20:14 -05:00
Ethan Buchman ec87c740a7
Merge pull request #794 from tendermint/243-restart-app-via-os
Kill Tendermint when App dies
2017-10-31 16:27:23 -04:00
Ethan Buchman d71aed309f some minor changes 2017-10-30 22:52:03 -04:00
Anton Kaliaev 61d76a273f
fixes from Bucky's and Emmanuel's reviews 2017-10-30 11:12:01 -05:00
Anton Kaliaev 6d18e2f447
do not send whole round state via eventHub
Fixes

```
WARNING: DATA RACE
Write at 0x00c4200715b8 by goroutine 24:
  github.com/tendermint/tendermint/consensus.(*ConsensusState).enterPrevote.func1()
      /go/src/github.com/tendermint/tendermint/consensus/state.go:359 +0x3f
  github.com/tendermint/tendermint/consensus.(*ConsensusState).enterPrevote()
      /go/src/github.com/tendermint/tendermint/consensus/state.go:897 +0x8de
  github.com/tendermint/tendermint/consensus.(*ConsensusState).addProposalBlockPart()
      /go/src/github.com/tendermint/tendermint/consensus/state.go:1303 +0x701
  github.com/tendermint/tendermint/consensus.(*ConsensusState).handleMsg()
      /go/src/github.com/tendermint/tendermint/consensus/state.go:560 +0x88c
  github.com/tendermint/tendermint/consensus.(*ConsensusState).receiveRoutine()
      /go/src/github.com/tendermint/tendermint/consensus/state.go:525 +0x6d2

Previous read at 0x00c4200715b8 by goroutine 19:
  github.com/tendermint/tendermint/consensus.makeRoundStepMessages()
      /go/src/github.com/tendermint/tendermint/consensus/reactor.go:415 +0x192
  github.com/tendermint/tendermint/consensus.(*ConsensusReactor).broadcastNewRoundStep()
      /go/src/github.com/tendermint/tendermint/consensus/reactor.go:377 +0x3c
  github.com/tendermint/tendermint/consensus.(*ConsensusReactor).broadcastNewRoundStepsAndVotes.func1()
      /go/src/github.com/tendermint/tendermint/consensus/reactor.go:350 +0x275
```
2017-10-30 00:32:23 -05:00
Anton Kaliaev 1c1c68df8d
fixes from my own review 2017-10-30 00:32:23 -05:00
Anton Kaliaev f6539737de
new pubsub package
comment out failing consensus tests for now

rewrite rpc httpclient to use new pubsub package

import pubsub as tmpubsub, query as tmquery

make event IDs constants
EventKey -> EventTypeKey

rename EventsPubsub to PubSub

mempool does not use pubsub

rename eventsSub to pubsub

new subscribe API

fix channel size issues and consensus tests bugs

refactor rpc client

add missing discardFromChan method

add mutex

rename pubsub to eventBus

remove IsRunning from WSRPCConnection interface (not needed)

add a comment in broadcastNewRoundStepsAndVotes

rename registerEventCallbacks to broadcastNewRoundStepsAndVotes

See https://dave.cheney.net/2014/03/19/channel-axioms

stop eventBuses after reactor tests

remove unnecessary Unsubscribe

return subscribe helper function

move discardFromChan to where it is used

subscribe now returns an err

this gives us ability to refuse to subscribe if pubsub is at its max
capacity.

use context for control overflow

cache queries

handle err when subscribing in replay_test

rename testClientID to testSubscriber

extract var

set channel buffer capacity to 1 in replay_file

fix byzantine_test

unsubscribe from single event, not all events

refactor httpclient to return events to appropriate channels

return failing testReplayCrashBeforeWriteVote test

fix TestValidatorSetChanges

refactor code a bit

fix testReplayCrashBeforeWriteVote

add comment

fix TestValidatorSetChanges

fixes from Bucky's review

update comment [ci skip]

test TxEventBuffer

update changelog

fix TestValidatorSetChanges (2nd attempt)

only do wg.Done when no errors

benchmark event bus

create pubsub server inside NewEventBus

only expose config params (later if needed)

set buffer capacity to 0 so we are not testing cache

new tx event format: key = "Tx" plus a tag {"tx.hash": XYZ}

This should allow to subscribe to all transactions! or a specific one
using a query: "tm.events.type = Tx and tx.hash = '013ABF99434...'"

use TimeoutCommit instead of afterPublishEventNewBlockTimeout

TimeoutCommit is the time a node waits after committing a block, before
it goes into the next height. So it will finish everything from the last
block, but then wait a bit. The idea is this gives it time to hear more
votes from other validators, to strengthen the commit it includes in the
next block. But it also gives it time to hear about new transactions.

waitForBlockWithUpdatedVals

rewrite WAL crash tests

Task:
test that we can recover from any WAL crash.

Solution:
the old tests were relying on event hub being run in the same thread (we
were injecting the private validator's last signature).

when considering a rewrite, we considered two possible solutions: write
a "fuzzy" testing system where WAL is crashing upon receiving a new
message, or inject failures and trigger them in tests using something
like https://github.com/coreos/gofail.

remove sleep

no cs.Lock around wal.Save

test different cases (empty block, non-empty block, ...)

comments

add comments

test 4 cases: empty block, non-empty block, non-empty block with smaller part size, many blocks

fixes as per Bucky's last review

reset subscriptions on UnsubscribeAll

use a simple counter to track message for which we panicked

also, set a smaller part size for all test cases
2017-10-30 00:32:22 -05:00
Ethan Buchman fe1c60b5cf consensus: kill process on app error 2017-10-27 10:55:20 -04:00
Ethan Buchman 591dd9e662 dont catchupReplay on wal if we fast synced 2017-10-27 10:46:19 -04:00
Ethan Buchman 12c703c1c3 Merge branch 'develop' into 573-wal-issues 2017-10-25 23:56:08 -04:00