Commit Graph

340 Commits

Author SHA1 Message Date
Anton Kaliaev 992c54253f
fixes from gometalinter review 2017-07-18 11:53:41 +03:00
Anton Kaliaev 17d6091ef4
updates as per Bucky's comments 2017-07-15 13:33:47 +03:00
Anton Kaliaev e664f9c688
use context to provide timeouts! 2017-07-14 14:49:25 +03:00
Anton Kaliaev 0006bfc359
return ErrorOverflow on Subscribe if server is overflowed
> why we need it?

most of our subscribers will be RPC WS subscribers, so if there are too
many, nothing wrong with rejecting to subscribe.

however, consensus reactor must be the first to subscribe, since its
work depends on the pubsub package.
2017-07-14 13:02:32 +03:00
Anton Kaliaev 13207a5927
remove overflow options 2017-07-14 12:32:01 +03:00
Anton Kaliaev 4aa024d843
add more info to error messages 2017-07-12 23:10:36 +03:00
Anton Kaliaev e4f3f9d9bf
remove comment about LRU cache (see comments below)
I've tried https://github.com/hashicorp/golang-lru/tree/master/simplelru today and here are
the results:

with LRU cache:

```
Benchmark10Clients-2                               50000             29021 ns/op         3976 B/op         105 allocs/op
Benchmark100Clients-2                               3000            363432 ns/op        36382 B/op        1005 allocs/op
Benchmark1000Clients-2                               500           2473752 ns/op       360500 B/op       10009 allocs/op
Benchmark10ClientsUsingTheSameQuery-2             300000              4059 ns/op          773 B/op          15 allocs/op
Benchmark100ClientsUsingTheSameQuery-2            500000              4360 ns/op          773 B/op          15 allocs/op
Benchmark1000ClientsUsingTheSameQuery-2           300000              4204 ns/op          773 B/op          15 allocs/op
```

without LRU cache:

```
Benchmark10Clients-2                      200000              5267 ns/op             616 B/op       25 allocs/op
Benchmark100Clients-2                      30000             42134 ns/op            2776 B/op      205 allocs/op
Benchmark1000Clients-2                      3000            552648 ns/op           24376 B/op     2005 allocs/op
Benchmark10ClientsOneQuery-2             1000000              2127 ns/op             462 B/op        9 allocs/op
Benchmark100ClientsOneQuery-2             500000              2353 ns/op             462 B/op        9 allocs/op
Benchmark1000ClientsOneQuery-2            500000              2339 ns/op             462 B/op        9 allocs/op
```

> How were you using the lru cache exactly?

I was adding a KV pair each time there is a match plus checking if
`lru.Contains(key)` before running the actual check (`q.Matches(tags)`).

```
key = fmt.Sprintf("%s/%v", query + tags)
```
2017-07-12 22:52:13 +03:00
Anton Kaliaev 8062ade787
remove all clients (including closing all channels) on shutdown 2017-07-12 13:10:36 +03:00
Anton Kaliaev a99b8a6210
new events package
query parser

use parser compiler to generate query parser

I used https://github.com/pointlander/peg which has a nice API and seems
to be the most popular Golang compiler parser using PEG on Github.

More about PEG:

- https://en.wikipedia.org/wiki/Parsing_expression_grammar
- https://github.com/PhilippeSigaud/Pegged/wiki/PEG-Basics
- https://github.com/PhilippeSigaud/Pegged/wiki/Grammar-Examples

rename

implement query match function

match function

uncomment test lines

add more test cases for query#Matches

fix int case

rename events to pubsub

add comment about cache

assertReceive helper to not block on receive in tests

fix bug with multiple conditions

uncomment benchmark

first results:

```
Benchmark10Clients-2                1000           1305493 ns/op         3957519 B/op     355 allocs/op
Benchmark100Clients-2                100          12278304 ns/op        39571751 B/op    3505 allocs/op
Benchmark1000Clients-2                10         124120909 ns/op        395714004 B/op   35005 allocs/op
```

124ms to publish message to 1000 clients. A lot.

use AST from query.peg.go

separate pubsub and query packages by using Query interface in pubsub

wrote docs and refactor code

updates from Frey's review

refactor type assertion to use type switch

cleanup during shutdown

subscriber should create output channel, not the server

overflow strategies, server buffer capacity

context as the first argument for Publish

log error

introduce Option type

update NewServer comment

move helpers into pubsub_test

increase assertReceive timeout

add query.MustParse

add more false tests for parser

add more false tests for query.Matches

parse numbers as int64 / float64

try our best to convert from other types

add number to panic output

add more comments

save commit

introduce client argument as first argument to Subscribe

> Why we do not specify buffer size on the output channel in Subscribe?

The choice of buffer size of N here depends on knowing the number of
messages server will receive and the number of messages downstream
subscribers will consume. This is fragile: if we publish an additional
message, or if one of the downstream subscribers reads any fewer
messages, we will again have blocked goroutines.

save commit

remove reference counting

fix test

test client resubscribe

test UnsubscribeAll

client options

[pubsub/query] fuzzy testing

do not print msg as it creates data race!
2017-07-12 09:48:01 +03:00
Ethan Buchman efb56aaea7 Merge pull request #25 from tendermint/stderr
CLI Execute Error prints to stderr
2017-06-23 22:22:07 -04:00
Ethan Buchman a28e35fd98 Merge branch 'develop' into stderr 2017-06-23 22:21:36 -04:00
Ethan Buchman bf55624f75 Merge pull request #24 from tendermint/dateparse
common date parsing
2017-06-23 22:20:32 -04:00
Ethan Buchman 7fbe6adf24 Merge branch 'develop' into dateparse 2017-06-23 22:16:37 -04:00
rigel rozanski cc364b14e2 changelog and PR changes 2017-06-20 17:18:55 -04:00
rigel rozanski f3eaf9b870 quickfix 2017-06-20 16:52:22 -04:00
rigel rozanski 34bcb30f1c changelog 2017-06-20 16:40:32 -04:00
rigel rozanski 0a3a08a3bc stderr PR revisions 2017-06-17 18:35:05 -04:00
Ethan Buchman 7ce4da1eee Merge pull request #26 from tendermint/develop
v0.2.2
2017-06-16 11:42:58 -04:00
Ethan Buchman bd9d0d1637 changelog and version 2017-06-16 11:40:14 -04:00
Ethan Frey 3400cee845 Handle --two-words as TMTWO_WORDS env var 2017-06-15 20:16:22 +02:00
Ethan Frey 59a77e7bef Remove Printf 2017-06-14 17:01:15 +02:00
Ethan Frey 0ecb38c6da Return exit code on error, disable in tests 2017-06-14 16:51:33 +02:00
Ethan Buchman c5644aad31 Merge pull request #21 from tendermint/feature/tracing-logger
[log] tracing logger
2017-06-07 13:26:27 -04:00
Anton Kaliaev c332a21fb2 Merge pull request #20 from tendermint/bugfix/16-flowrate
[flowrate] refactor clock functions (Refs #16)
2017-06-07 13:14:47 +03:00
rigel rozanski 94c0172618 doc update 2017-06-07 05:05:11 -04:00
rigel rozanski 33d0dd0bfc add stderr to tests 2017-06-06 18:36:28 -04:00
rigel rozanski c76dca0456 cli stderr output for Execute 2017-06-06 16:13:44 -04:00
rigel rozanski f913ed8134 date simplify 2017-06-06 16:08:57 -04:00
rigel rozanski 304be4ec2f date parse functionality 2017-06-06 04:00:36 -04:00
Ethan Buchman 462243e31a Merge pull request #23 from tendermint/hexfuncs
IsHex and StripHex
2017-06-05 17:32:49 -04:00
rigel rozanski 925f2b3350 golint corrections 2017-06-05 16:22:01 -04:00
rigel rozanski 295f6c2cc6 IsHex and StripHex 2017-06-05 15:50:11 -04:00
Anton Kaliaev a24a0ff003
Merge tag 'v0.2.1' into develop
v0.2.1
2017-06-02 11:57:04 +03:00
Anton Kaliaev 6b619742ac
Merge branch 'release/0.2.1' 2017-06-02 11:56:17 +03:00
Anton Kaliaev 4ef77c008c
update changelog 2017-06-02 11:55:43 +03:00
Anton Kaliaev 8c0959604c Merge pull request #19 from tendermint/feature/log-level-parsing-from-tendermint
copy log level parsing from tendemint
2017-06-02 10:42:23 +03:00
Anton Kaliaev 5f20b3323e
don't do DeepEqual, compare ranges for durations and rates (Refs #16) 2017-05-25 13:06:42 +02:00
Anton Kaliaev b36203bb02
[cli] add a test case to TestParseLogLevel where there is no module key 2017-05-23 23:31:29 +02:00
Anton Kaliaev ddaa4d9b4c
[log] tracing logger 2017-05-23 23:06:20 +02:00
Ethan Frey de02488778 Enhance the tests to make it clearer how these levels work together 2017-05-23 23:04:33 +02:00
Anton Kaliaev b5c57967b7
[flowrate] improve error formatting (Refs #16) 2017-05-23 15:24:00 +02:00
Anton Kaliaev 6b10432463
[flowrate] refactor clock functions (Refs #16)
this commit does not fix the original bug
2017-05-23 15:20:27 +02:00
Anton Kaliaev 5032b224bc
copy log level parsing from tendemint
API change due to me not wanting `flags` package to depend on
tendermint's config package.

Refs https://github.com/tendermint/tendermint/issues/504
2017-05-23 00:43:12 +02:00
Ethan Buchman 306795ae1d Merge pull request #15 from tendermint/develop
v0.2.0
2017-05-18 11:28:34 +02:00
Ethan Buchman 2733f5a738 CHANGELOG: update release date 2017-05-18 11:27:26 +02:00
Ethan Frey 8af1c70a8b Renamed --debug to --trace, used for light-client and basecoin 2017-05-17 12:03:26 +02:00
Ethan Buchman 67f558cff0 Merge pull request #17 from tendermint/feature/log-levels-per-key-in-filter
Allow custom levels for the keyvals in filter
2017-05-16 23:29:02 +02:00
Anton Kaliaev d0cae7b6ed
[log] change helper func 2017-05-16 12:49:23 +02:00
Anton Kaliaev 761b1553aa
[log] allow filtering with fields 2017-05-16 12:48:01 +02:00
Ethan Buchman 812d9f9b84 add changelog 2017-05-15 09:09:42 -04:00