Commit Graph

217 Commits

Author SHA1 Message Date
Anton Kaliaev 4aa024d843
add more info to error messages 2017-07-12 23:10:36 +03:00
Anton Kaliaev e4f3f9d9bf
remove comment about LRU cache (see comments below)
I've tried https://github.com/hashicorp/golang-lru/tree/master/simplelru today and here are
the results:

with LRU cache:

```
Benchmark10Clients-2                               50000             29021 ns/op         3976 B/op         105 allocs/op
Benchmark100Clients-2                               3000            363432 ns/op        36382 B/op        1005 allocs/op
Benchmark1000Clients-2                               500           2473752 ns/op       360500 B/op       10009 allocs/op
Benchmark10ClientsUsingTheSameQuery-2             300000              4059 ns/op          773 B/op          15 allocs/op
Benchmark100ClientsUsingTheSameQuery-2            500000              4360 ns/op          773 B/op          15 allocs/op
Benchmark1000ClientsUsingTheSameQuery-2           300000              4204 ns/op          773 B/op          15 allocs/op
```

without LRU cache:

```
Benchmark10Clients-2                      200000              5267 ns/op             616 B/op       25 allocs/op
Benchmark100Clients-2                      30000             42134 ns/op            2776 B/op      205 allocs/op
Benchmark1000Clients-2                      3000            552648 ns/op           24376 B/op     2005 allocs/op
Benchmark10ClientsOneQuery-2             1000000              2127 ns/op             462 B/op        9 allocs/op
Benchmark100ClientsOneQuery-2             500000              2353 ns/op             462 B/op        9 allocs/op
Benchmark1000ClientsOneQuery-2            500000              2339 ns/op             462 B/op        9 allocs/op
```

> How were you using the lru cache exactly?

I was adding a KV pair each time there is a match plus checking if
`lru.Contains(key)` before running the actual check (`q.Matches(tags)`).

```
key = fmt.Sprintf("%s/%v", query + tags)
```
2017-07-12 22:52:13 +03:00
Anton Kaliaev 8062ade787
remove all clients (including closing all channels) on shutdown 2017-07-12 13:10:36 +03:00
Anton Kaliaev a99b8a6210
new events package
query parser

use parser compiler to generate query parser

I used https://github.com/pointlander/peg which has a nice API and seems
to be the most popular Golang compiler parser using PEG on Github.

More about PEG:

- https://en.wikipedia.org/wiki/Parsing_expression_grammar
- https://github.com/PhilippeSigaud/Pegged/wiki/PEG-Basics
- https://github.com/PhilippeSigaud/Pegged/wiki/Grammar-Examples

rename

implement query match function

match function

uncomment test lines

add more test cases for query#Matches

fix int case

rename events to pubsub

add comment about cache

assertReceive helper to not block on receive in tests

fix bug with multiple conditions

uncomment benchmark

first results:

```
Benchmark10Clients-2                1000           1305493 ns/op         3957519 B/op     355 allocs/op
Benchmark100Clients-2                100          12278304 ns/op        39571751 B/op    3505 allocs/op
Benchmark1000Clients-2                10         124120909 ns/op        395714004 B/op   35005 allocs/op
```

124ms to publish message to 1000 clients. A lot.

use AST from query.peg.go

separate pubsub and query packages by using Query interface in pubsub

wrote docs and refactor code

updates from Frey's review

refactor type assertion to use type switch

cleanup during shutdown

subscriber should create output channel, not the server

overflow strategies, server buffer capacity

context as the first argument for Publish

log error

introduce Option type

update NewServer comment

move helpers into pubsub_test

increase assertReceive timeout

add query.MustParse

add more false tests for parser

add more false tests for query.Matches

parse numbers as int64 / float64

try our best to convert from other types

add number to panic output

add more comments

save commit

introduce client argument as first argument to Subscribe

> Why we do not specify buffer size on the output channel in Subscribe?

The choice of buffer size of N here depends on knowing the number of
messages server will receive and the number of messages downstream
subscribers will consume. This is fragile: if we publish an additional
message, or if one of the downstream subscribers reads any fewer
messages, we will again have blocked goroutines.

save commit

remove reference counting

fix test

test client resubscribe

test UnsubscribeAll

client options

[pubsub/query] fuzzy testing

do not print msg as it creates data race!
2017-07-12 09:48:01 +03:00
Ethan Buchman c5644aad31 Merge pull request #21 from tendermint/feature/tracing-logger
[log] tracing logger
2017-06-07 13:26:27 -04:00
Anton Kaliaev c332a21fb2 Merge pull request #20 from tendermint/bugfix/16-flowrate
[flowrate] refactor clock functions (Refs #16)
2017-06-07 13:14:47 +03:00
Ethan Buchman 462243e31a Merge pull request #23 from tendermint/hexfuncs
IsHex and StripHex
2017-06-05 17:32:49 -04:00
rigel rozanski 925f2b3350 golint corrections 2017-06-05 16:22:01 -04:00
rigel rozanski 295f6c2cc6 IsHex and StripHex 2017-06-05 15:50:11 -04:00
Anton Kaliaev a24a0ff003
Merge tag 'v0.2.1' into develop
v0.2.1
2017-06-02 11:57:04 +03:00
Anton Kaliaev 6b619742ac
Merge branch 'release/0.2.1' 2017-06-02 11:56:17 +03:00
Anton Kaliaev 4ef77c008c
update changelog 2017-06-02 11:55:43 +03:00
Anton Kaliaev 8c0959604c Merge pull request #19 from tendermint/feature/log-level-parsing-from-tendermint
copy log level parsing from tendemint
2017-06-02 10:42:23 +03:00
Anton Kaliaev 5f20b3323e
don't do DeepEqual, compare ranges for durations and rates (Refs #16) 2017-05-25 13:06:42 +02:00
Anton Kaliaev b36203bb02
[cli] add a test case to TestParseLogLevel where there is no module key 2017-05-23 23:31:29 +02:00
Anton Kaliaev ddaa4d9b4c
[log] tracing logger 2017-05-23 23:06:20 +02:00
Ethan Frey de02488778 Enhance the tests to make it clearer how these levels work together 2017-05-23 23:04:33 +02:00
Anton Kaliaev b5c57967b7
[flowrate] improve error formatting (Refs #16) 2017-05-23 15:24:00 +02:00
Anton Kaliaev 6b10432463
[flowrate] refactor clock functions (Refs #16)
this commit does not fix the original bug
2017-05-23 15:20:27 +02:00
Anton Kaliaev 5032b224bc
copy log level parsing from tendemint
API change due to me not wanting `flags` package to depend on
tendermint's config package.

Refs https://github.com/tendermint/tendermint/issues/504
2017-05-23 00:43:12 +02:00
Ethan Buchman 306795ae1d Merge pull request #15 from tendermint/develop
v0.2.0
2017-05-18 11:28:34 +02:00
Ethan Buchman 2733f5a738 CHANGELOG: update release date 2017-05-18 11:27:26 +02:00
Ethan Frey 8af1c70a8b Renamed --debug to --trace, used for light-client and basecoin 2017-05-17 12:03:26 +02:00
Ethan Buchman 67f558cff0 Merge pull request #17 from tendermint/feature/log-levels-per-key-in-filter
Allow custom levels for the keyvals in filter
2017-05-16 23:29:02 +02:00
Anton Kaliaev d0cae7b6ed
[log] change helper func 2017-05-16 12:49:23 +02:00
Anton Kaliaev 761b1553aa
[log] allow filtering with fields 2017-05-16 12:48:01 +02:00
Ethan Buchman 812d9f9b84 add changelog 2017-05-15 09:09:42 -04:00
Ethan Buchman da68014a50 Merge pull request #10 from tendermint/bugfix/rotating-fails-on-windows
Close file before renaming it
2017-05-15 14:47:57 +02:00
Anton Kaliaev 74a7f8c92b
[autofile] close file before renaming it
this might fix our windows bug https://github.com/tendermint/tendermint/issues/444

0980f8e197
2017-05-15 10:47:16 +02:00
Ethan Buchman 4fdeaa70af Merge pull request #14 from tendermint/feature/allow-for-custom-color-functions
[log] allow for custom color funcs
2017-05-14 23:24:03 +02:00
Anton Kaliaev a5fcc94a3b
[log] allow for custom color funcs 2017-05-14 12:03:45 +02:00
Anton Kaliaev 8f5a175ff4 Merge pull request #13 from tendermint/feature/filter-returns-error
[log] NewFilterByLevel returns an error
2017-05-13 16:01:02 +02:00
Anton Kaliaev dd3e433d32
[log] NewFilterByLevel returns an error 2017-05-13 15:54:11 +02:00
Anton Kaliaev 9e1852c08d Merge pull request #12 from tendermint/feature/helper-log-func
[log] add NewFilterByLevel helper func
2017-05-12 23:12:36 +02:00
Anton Kaliaev 3007668274
[log] add NewFilterByLevel helper func 2017-05-12 22:59:31 +02:00
Ethan Buchman dd592a21b9 Merge pull request #11 from tendermint/bugfix/multiple-module-keys
Squash module keyvals if multiple keyvals were provided
2017-05-12 22:08:36 +02:00
Anton Kaliaev f202d02d0d
[log] add separator 2017-05-12 17:39:00 +02:00
Anton Kaliaev b2a116863c
squash module key if multiple keyvals were provided
last keyvalue wins
2017-05-08 17:30:21 +04:00
Anton Kaliaev 25a5bc2697 Merge pull request #9 from tendermint/bugfix/filter-with-func
[log] proper impl of With for filter
2017-05-08 14:42:21 +04:00
Anton Kaliaev bc6baf6774
[log] proper impl of With for filter 2017-05-06 11:21:07 +04:00
Ethan Buchman d72136da0d Merge pull request #7 from tendermint/log
Log
2017-05-05 14:02:38 -04:00
Anton Kaliaev 240215f2aa
return back all in Makefile 2017-05-05 21:52:01 +04:00
Anton Kaliaev f4be75cb1e
remove warn mentions 2017-05-05 21:48:03 +04:00
Anton Kaliaev 2bf6ebf379
filter 2017-05-05 21:48:03 +04:00
Anton Kaliaev ea01d003d1
changes per @ethanfrey comments 2017-05-05 21:48:03 +04:00
Anton Kaliaev b2bd039923
correct impl of WithLevel 2017-05-05 21:48:03 +04:00
Anton Kaliaev 6ff7b4395d
update glide.yaml and glide.lock files 2017-05-05 21:48:02 +04:00
Ethan Frey 3420b389e9 Add testify version for consistency 2017-05-05 19:45:02 +02:00
Anton Kaliaev 520561e94a
add testing logger 2017-05-05 21:43:07 +04:00
Ethan Buchman 66c9401c07
log: Tm -> TM 2017-05-05 21:43:07 +04:00