Commit Graph

71 Commits

Author SHA1 Message Date
Felix Lange 40cdcf1183 trie, core/state: improve memory usage and performance (#3135)
* trie: store nodes as pointers

This avoids memory copies when unwrapping node interface values.

name      old time/op  new time/op  delta
Get        388ns ± 8%   215ns ± 2%  -44.56%  (p=0.000 n=15+15)
GetDB      363ns ± 3%   202ns ± 2%  -44.21%  (p=0.000 n=15+15)
UpdateBE  1.57µs ± 2%  1.29µs ± 3%  -17.80%  (p=0.000 n=13+15)
UpdateLE  1.92µs ± 2%  1.61µs ± 2%  -16.25%  (p=0.000 n=14+14)
HashBE    2.16µs ± 6%  2.18µs ± 6%     ~     (p=0.436 n=15+15)
HashLE    7.43µs ± 3%  7.21µs ± 3%   -2.96%  (p=0.000 n=15+13)

* trie: close temporary databases in GetDB benchmark

* trie: don't keep []byte from DB load around

Nodes decoded from a DB load kept hashes and values as sub-slices of
the DB value. This can be a problem because loading from leveldb often
returns []byte with a cap that's larger than necessary, increasing
memory usage.

* trie: unload old cached nodes

* trie, core/state: use cache unloading for account trie

* trie: use explicit private flags (fixes Go 1.5 reflection issue).

* trie: fixup cachegen overflow at request of nick

* core/state: rename journal size constant
2016-10-14 19:04:33 +03:00
Felix Lange c3a77d6268 trie: fix delete bug for values contained in fullNode
Delete crashed if a fullNode contained a valueNode directly. This bug is
very unlikely to occur with SecureTrie, but can happen with regular
tries. This commit also introduces a randomised test which triggers all
trie operations, which should prevent such bugs in the future.

Credit for finding this bug goes to Github user @rjl493456442.
2016-09-29 22:53:59 +02:00
Felix Lange ba8c4c6b1a trie: remove dependency on package crypto
Package crypto needs cgo, which is inconvenient for some build
configurations.
2016-09-29 22:23:24 +02:00
Péter Szilágyi 710435b51b core, eth, trie: reuse trie journals in all our code 2016-09-28 11:27:31 +03:00
Felix Lange cd791bd855 core, trie: replace state caches with trie journal 2016-09-28 11:27:28 +03:00
Péter Szilágyi 1d5d21726a Merge pull request #2627 from karalabe/concurrent-head-sync
eth/downloader, trie: pull head state concurrently with chain
2016-05-31 11:52:16 +03:00
Péter Szilágyi 4f1d92b332 eth/downloader, trie: pull head state concurrently with chain 2016-05-27 14:26:00 +03:00
Péter Szilágyi 748d1c171d core, core/state, trie: enterprise hand-tuned multi-level caching 2016-05-26 16:33:09 +03:00
Felix Lange 6fdd0893c3 all: fix go vet warnings 2016-04-15 11:17:27 +02:00
Felix Lange d04a2e7557 all: update license information 2016-04-15 09:48:05 +02:00
Péter Szilágyi e90958cd29 cmd, eth, ethdb, node: prioritise chaindata for resources, bump cache 2016-03-09 10:33:39 +02:00
Jeffrey Wilcke 483feb0d3f Merge pull request #2242 from jimenezrick/upstream-crypto
Closes #2241: Use Keccak-256 from golang.org/x/crypto/sha3 and mention explicitly
2016-02-24 12:57:57 +01:00
Ricardo Catalinas Jiménez 436fc8d76a all: Rename crypto.Sha3{,Hash}() to crypto.Keccak256{,Hash}()
As we aren't really using the standarized SHA-3
2016-02-21 22:34:34 +00:00
Péter Szilágyi b8d59d9c98 core/state, trie: switch iterator panics to error fields 2016-02-16 12:37:00 +02:00
Péter Szilágyi 151c7bef41 core/state, trie: node iterator reports parent hashes too 2016-02-16 12:21:08 +02:00
Péter Szilágyi 5a057a8ded core/state, trie: surface iterator entry hashes 2016-02-16 12:21:08 +02:00
Péter Szilágyi 7e29b0b5b4 core/state, trie: add node iterator, test state/trie sync consistency 2016-02-16 12:21:08 +02:00
Péter Szilágyi f3d4ce0d16 core/state, ethdb, trie: test intermediate secure key leak, fix memdb bug 2016-01-20 16:06:28 +02:00
Péter Szilágyi 886478b18b core/state, trie: don't leak database writes before commit 2016-01-20 12:09:24 +02:00
zsfelfoldi ef422ee1e1 light: implemented odr-capable trie and state structures 2015-12-17 16:07:54 +01:00
zsfelfoldi 52904ae32f trie: added error handling
Created alternate versions of Trie and SecureTrie functions that can return a MissingNodeError (used by ODR services)
2015-12-01 01:38:32 +01:00
Péter Szilágyi 5b0ee8ec30 core, eth, trie: fix data races and merge/review issues 2015-10-21 16:49:55 +03:00
Péter Szilágyi b97e34a8e4 eth/downloader: concurrent receipt and state processing 2015-10-19 10:03:10 +03:00
Péter Szilágyi ab27bee25a core, eth, trie: direct state trie synchronization 2015-10-19 10:03:09 +03:00
Felix Lange c1a352c108 trie: add merkle proof functions 2015-09-22 22:57:37 +02:00
Felix Lange 565d9f2306 core, trie: new trie 2015-09-22 22:53:49 +02:00
Felix Lange 6b91a4abe5 trie: improve benchmarks 2015-09-22 22:49:27 +02:00
Jeffrey Wilcke b8ca0a830e eth, trie: removed key prefixing from state entries & merge db fix
Fixed database merge strategy to use the correct database. Due to a copy
paste fail when doing type evaluation the same database was being
iterated (chain), all others were ignored.

Removed state prefixing because {H(code): code} is stored in the same
database as the rest of the state.
2015-08-13 20:44:03 +02:00
Jeffrey Wilcke a23478c0be core, eth, trie, xeth: merged state, chain, extra databases in one 2015-08-07 22:29:02 +02:00
Ethan Buchman 46c9594081 trie: run codec tests, add benchmarks, faster 2015-08-06 14:04:16 -04:00
Ethan Buchman c1d516546d faster hex-prefix codec and string -> []byte 2015-08-06 03:17:59 -04:00
Jeffrey Wilcke 1fad8798ec Merge pull request #1515 from fjl/license-fixes
all: fix license headers one more time
2015-07-28 04:29:42 -07:00
Felix Lange bfbcfbe4a9 all: fix license headers one more time
I forgot to update one instance of "go-ethereum" in commit 3f047be5a.
2015-07-23 18:35:11 +02:00
Jeffrey Wilcke 8f56eea77d ethdb, trie: removed RLE compression 2015-07-23 12:57:33 +02:00
Felix Lange 3f047be5aa all: update license headers to distiguish GPL/LGPL
All code outside of cmd/ is licensed as LGPL. The headers
now reflect this by calling the whole work "the go-ethereum library".
2015-07-22 18:51:45 +02:00
Felix Lange bdae4fd573 all: add some godoc synopsis comments 2015-07-07 14:12:45 +02:00
Felix Lange ea54283b30 all: update license information 2015-07-07 14:12:44 +02:00
Jeffrey Wilcke 47460b3b4a trie: removed shallow copies (thanks to @fjl) 2015-07-04 02:51:37 +02:00
Jeffrey Wilcke 0a1ff68c11 trie: dirty tracking 2015-07-04 02:51:36 +02:00
obscuren c850c41ec1 trie: Implemented a batch write approach for flushing 2015-06-29 18:51:48 +02:00
obscuren a40a91d60f trie: fixed tests 2015-06-21 17:08:47 +02:00
obscuren c590b505ed core, ethdb, trie: validate database errors 2015-06-21 16:59:15 +02:00
obscuren bbe795455a Secure trie shakey / key matching 2015-03-24 15:14:03 +01:00
obscuren cf45b939a0 fixed tests 2015-03-19 14:31:14 +01:00
obscuren d7ab716eea Fixed mkdnode & added some tests 2015-03-19 10:57:02 +01:00
obscuren b523441361 Moved ethutil => common 2015-03-16 11:27:38 +01:00
obscuren d44fe4ec62 Merge branch 'develop' into poc-9 2015-03-05 17:46:40 +01:00
obscuren 357d17ae58 Start at 0 2015-03-05 17:46:00 +01:00
= a6fe92194f Fixed iterator bug which skips the 0x000...000 key 2015-03-05 01:47:49 +00:00
obscuren b909a5ec18 Trie => SecureTrie 2015-03-03 12:25:44 +01:00