Lesson articulated by @jaekwon on why we need 80 bits
of entropy at least before we can think of cryptographic
safety. math/rand's seed is a max of 64 bits so can never
be cryptographically secure.
Also added some benchmarks for RandBytes
Fixes https://github.com/tendermint/tmlibs/issues/99
Updates https://github.com/tendermint/tendermint/issues/973
Removed usages of math/rand.DefaultSource in favour of our
own source that's seeded with a completely random source
and is safe for use in concurrent in multiple goroutines.
Also extend some functionality that the stdlib exposes such as
* RandPerm
* RandIntn
* RandInt31
* RandInt63
Also added an integration test whose purpose is to be run as
a consistency check to ensure that our results never repeat
hence that our internal PRNG is uniquely seeded each time.
This integration test can be triggered by setting environment variable:
`TENDERMINT_INTEGRATION_TESTS=true`
for example
```shell
TENDERMINT_INTEGRATION_TESTS=true go test
```
```
@melekes
yeah, bool is superfluous
@ethanfrey
If I remember correctly when I was writing test code, if I call Start() on a Service that is already running, it returns (false, nil). Only if I try to legitimately start it, but it fails in startup do I get an error.
The distinction is quite important to make it safe for reentrant calls. The other approach would be to have a special error type like ErrAlreadyStarted, then check for that in your code explicitly. Kind of like if I make a db call in gorm, and get an error, I check if it is a RecordNotFound error, or whether there was a real error with the db query.
@melekes
Ah, I see. Thanks. I must say I like ErrAlreadyStarted approach more (not just in Golang)
```
Fix rename /root/.tendermint_test/consensus_replay_test/priv_validator.json.new /root/.tendermint_test/consensus_replay_test/priv_validator.json: no such file or directory
Currently IsDirEmpty returns true, err if it encounters
any error after trying to os.Open the directory.
I noticed this while studying the code and recalled a bug
from an earlier project in which doing the exact same thing
on code without permissions would trip out and falsely report
that the directory was empty.
Given demo.go in https://play.golang.org/p/vhTPU2RiCJ
* Demo:
```shell
$ mkdir -p sample-demo/1 && touch sample-demo/2
$ echo "1st round" && go run demo.go sample-demo
$ sudo chown root sample-demo && sudo chmod 0700 sample-demo
$ echo "2nd round" && go run demo.go sample-demo
```
That then prints out
```shell
1st round
original:: empty: false err: <nil>
updated:: empty: false err: <nil>
2nd round
original:: empty: true err: open data/: permission denied
updated:: empty: false err: open data/: permission denied
```
where in "2nd round", the original code falsely reports that
the directory is empty but that's a permission error.
I could write a code test for it, but that test requires me to change
users and switch to root as a Go user so no point in complicating our
tests, but otherwise it is a 1-to-1 translation between shell and Go.
Found common http utils that were being multiply duplicated across
many libraries and since am moving things in basecoin/unstable to
add for more functionality, it's better to put them in one
place.
Utilities and tests added:
- [X] FparseJSON
- [X] FparseAndValidateJSON
- [X] ParseRequestJSON
- [X] ParseAndValidateRequestJSON
- [X] WriteCode
- [X] WriteError
- [X] WriteSuccess
- [X] ErrorResponse
During review from @ethanfrey, made updates:
* Removed tt.want since it was a distraction/artifact that made
the reviewer think the tests weren't testing for both failed
and passed results.
* Added ErrorWithCode as WithCode is a common options pattern
in Go that could cause confusion:
ErrorWithCode(error, int) ErrorResponse
* Using json.NewDecoder(io.Reader) error instead of
ioutil.ReadAll(io.Reader) to slurp all the bytes.
* Added more test scenarios to achieve 100% coverage of http.go
We use WriteFileAtomic in two places:
```
p2p/addrbook.go
338: err = cmn.WriteFileAtomic(filePath, jsonBytes, 0644)
types/priv_validator.go
162: err = WriteFileAtomic(privVal.filePath, jsonBytes, 0600)
```
and we don't need .bak in any of the above. We save priv_validator every
10ms and addrbook every 2 min.
Ethan Buchman
Anton Kaliaev
Emmanuel Odeke
Ethan Frey
Wolf
Zach Ramsay