Commit Graph

941 Commits

Author SHA1 Message Date
Michael Vines 3d44cffcda
Beautify metrics datapoint logging (#5998) 2019-09-20 12:00:43 -07:00
carllin c9be9acd14
log snapshot time (#5996) 2019-09-20 08:03:00 -07:00
dependabot-preview[bot] 8eab673b1c Bump serde from 1.0.100 to 1.0.101 (#5994)
Bumps [serde](https://github.com/serde-rs/serde) from 1.0.100 to 1.0.101.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.100...v1.0.101)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-09-20 07:05:16 -06:00
dependabot-preview[bot] e5806d07a6 Bump jsonrpc-pubsub from 13.1.0 to 13.2.0 (#5995)
Bumps [jsonrpc-pubsub](https://github.com/paritytech/jsonrpc) from 13.1.0 to 13.2.0.
- [Release notes](https://github.com/paritytech/jsonrpc/releases)
- [Commits](https://github.com/paritytech/jsonrpc/compare/v13.1.0...v13.2.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-09-20 07:04:14 -06:00
Parth 11e6197a83
require stake, vote and executable accounts to be rent exempt (#5928)
* require vote account to be exempt

* make stake account rent exempt

* add rent exempted system instruction

* use rent exemption instruction in vote and stake api

* use rent exempted account while creating executable account

* updating chacha golden hash as instruction data has changed

* rent will be initialized for genesis bank too
2019-09-20 16:52:17 +05:30
Michael Vines 1d0be265d9
Add explicit validator-cuda crate (#5985) 2019-09-19 20:50:34 -07:00
Sagar Dhawan d379786c90
Fix bind errors (#5986)
* Add ability to bind to a common tcp/udp port

* Extend port range for local-net sanity and fix validator executable
2019-09-19 17:16:22 -07:00
Pankaj Garg ca9d4e34df
Broadcast stage tuning (#5989) 2019-09-19 16:29:52 -07:00
dependabot-preview[bot] 5157bdd8ce Bump jsonrpc-http-server from 13.1.0 to 13.2.0 (#5975)
Bumps [jsonrpc-http-server](https://github.com/paritytech/jsonrpc) from 13.1.0 to 13.2.0.
- [Release notes](https://github.com/paritytech/jsonrpc/releases)
- [Commits](https://github.com/paritytech/jsonrpc/compare/v13.1.0...v13.2.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-09-19 13:10:54 -06:00
Rob Walker 5cbd1190b2
transaction batch (#5962)
* transaction batch

* fixup
2019-09-19 10:06:08 -07:00
dependabot-preview[bot] 1a71804ef2 Bump bs58 from 0.2.5 to 0.3.0 (#5974)
Bumps [bs58](https://github.com/mycorrhiza/bs58-rs) from 0.2.5 to 0.3.0.
- [Release notes](https://github.com/mycorrhiza/bs58-rs/releases)
- [Commits](https://github.com/mycorrhiza/bs58-rs/compare/0.2.5...0.3.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-09-19 10:39:37 -06:00
dependabot-preview[bot] e83f6332bf Bump serde_derive from 1.0.100 to 1.0.101 (#5945)
Bumps [serde_derive](https://github.com/serde-rs/serde) from 1.0.100 to 1.0.101.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.100...v1.0.101)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-09-18 21:29:40 -07:00
Pankaj Garg 0dbf7995b5
Remove unnecessary serialize of shred data (#5967)
* Remove unnecessary serialize of shred data

* remove obsolete code

* fix golden hash
2019-09-18 20:08:27 -07:00
Sagar Dhawan e0858cfe06
Add parallel shred signing to shredder (#5964) 2019-09-18 18:00:07 -07:00
Pankaj Garg 783e8672e7
Removed Shred enum (#5963)
* Remove shred enum and it's references

* rename ShredInfo to Shred

* clippy
2019-09-18 16:24:30 -07:00
Pankaj Garg 6c4e656795
Remove obsoleted code from shred (#5954)
* Remove obsoleted code from shred

* fix broken test
2019-09-18 13:56:44 -07:00
Sagar Dhawan b757294864
Add minor performance bump to shredding (#5956) 2019-09-18 12:35:52 -07:00
Rob Walker 0d4a2c5eb0
simplify poh recorder => broadcast channel (#5940)
* simplify poh recorder broadcast channel

* fixup

* fixup
2019-09-18 12:16:22 -07:00
dependabot-preview[bot] 31a276b628 Bump jsonrpc-core from 13.1.0 to 13.2.0 (#5894)
Bumps [jsonrpc-core](https://github.com/paritytech/jsonrpc) from 13.1.0 to 13.2.0.
- [Release notes](https://github.com/paritytech/jsonrpc/releases)
- [Commits](https://github.com/paritytech/jsonrpc/compare/v13.1.0...v13.2.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-09-18 11:44:47 -06:00
dependabot-preview[bot] 75c0a268e0 Bump jsonrpc-derive from 13.1.0 to 13.2.0 (#5893)
Bumps [jsonrpc-derive](https://github.com/paritytech/jsonrpc) from 13.1.0 to 13.2.0.
- [Release notes](https://github.com/paritytech/jsonrpc/releases)
- [Commits](https://github.com/paritytech/jsonrpc/compare/v13.1.0...v13.2.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-09-18 10:48:37 -06:00
carllin 2f50d0e145
Refactor confidence from replay stage (#5938) 2019-09-17 19:43:40 -07:00
Pankaj Garg ff608992ee
Replace Shred usage with ShredInfo (#5939)
* Replace Shred usage with ShredInfo

* Fix tests

* fix clippy
2019-09-17 18:22:46 -07:00
Rob Walker a2595b44c6
test randomize with error (#5916)
* test randomize with error

* update magic numbers

* fixup

* fixup

* fixup

* no more blobs

* fixup
2019-09-17 15:11:29 -07:00
Sagar Dhawan 14cb6353c0
Change erasure ratio to 0.25 and increase data shreds to 16 (#5931)
* Change erasure ratio to 0.25 and increase data shreds to 16

* Fix case where no coding shreds are requested
2019-09-17 11:59:14 -07:00
Michael Vines e26f68fe62
Get transactions from LockedAccountsResults when possible (#5923) 2019-09-17 08:41:56 -07:00
Michael Vines f10438d530
Respect randomized transaction order when unlocking accounts (#5918) 2019-09-16 21:45:16 -07:00
Pankaj Garg 7459eb15c3
A new data-structure in shreds for partial deserialization (#5915)
* A new datastructure in shreds for partial deserialization

* fix chacha golden hash

* fix clippy and address review comments
2019-09-16 20:28:54 -07:00
Sagar Dhawan c44e7ce184
Leaders should not broadcast to replicators (#5917) 2019-09-16 17:56:34 -07:00
Sagar Dhawan 82615c703b
Switch erasure to solana-reed-solomon-erasure (#5913)
* Switch to solana-reed-solomon-erasure

* Disable Rayon for solana-reed-solomon-erasure
2019-09-16 16:14:55 -07:00
dependabot-preview[bot] bc2141fbe0 Bump ureq from 0.11.0 to 0.11.1 (#5905)
Bumps [ureq](https://github.com/algesten/ureq) from 0.11.0 to 0.11.1.
- [Release notes](https://github.com/algesten/ureq/releases)
- [Commits](https://github.com/algesten/ureq/commits)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-09-16 16:51:45 -06:00
carllin d5ba90d375
Don't verify blobs that are less than root in window service (#5901) 2019-09-16 13:13:53 -07:00
Pankaj Garg 5f54573613
More shred related cleanup (#5909)
* More shred related cleanup

* fix uncle
2019-09-16 10:28:28 -07:00
Pankaj Garg c1880e3f3e
Reduce number of shreds per FEC block (#5908) 2019-09-15 10:37:12 -07:00
Pankaj Garg ee791e2e3e
Optimizations to shred writing and signing (#5890)
* Optimizations to shred writing and signing

* fix broken tests

* fixes
2019-09-14 21:05:54 -07:00
sakridge ceacc42126
Call gpu init earlier to force compilation. (#5902) 2019-09-14 12:32:57 -07:00
Michael Vines 84c8a5bbec
Add replay-stage-mark_dead_slot datapoint (#5907) 2019-09-14 08:50:53 -07:00
Pankaj Garg 8135279335
Reduce serialize/deserialize in shred recovery (#5887) 2019-09-12 21:52:13 -07:00
Sagar Dhawan c1d788880d
Limit Rayon threadpool threads (#5871) 2019-09-12 11:39:39 -07:00
Pankaj Garg 385086359c
Reduce serializations/deserializations of shreds (#5879) 2019-09-12 10:10:25 -07:00
dependabot-preview[bot] a85604b2ba Bump sys-info from 0.5.7 to 0.5.8 (#5877)
Bumps [sys-info](https://github.com/FillZpp/sys-info-rs) from 0.5.7 to 0.5.8.
- [Release notes](https://github.com/FillZpp/sys-info-rs/releases)
- [Commits](https://github.com/FillZpp/sys-info-rs/commits)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-09-11 10:53:23 -07:00
Jack May 1853771930
Add support for SDK sysvar types (#5876) 2019-09-10 18:53:02 -07:00
carllin ee4266bc59
Remove banks in locktower not in bank_forks (#5837)
* Remove unnecessary calculations from collect_vote_lockouts

* Add test for locktower startup from snapshot
2019-09-10 13:58:27 -07:00
dependabot-preview[bot] 294d531e0b Bump serde_derive from 1.0.99 to 1.0.100 (#5864)
automerge
2019-09-10 13:31:11 -07:00
dependabot-preview[bot] 0d7efe5176 Bump serde from 1.0.99 to 1.0.100 (#5862)
Bumps [serde](https://github.com/serde-rs/serde) from 1.0.99 to 1.0.100.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.99...v1.0.100)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-09-10 12:27:41 -07:00
Pankaj Garg b426dfb2c0
Change tx batching in banking process and record (#5832)
* Change tx batching in banking process and record

* Change batching to reduce impact on replay stage
2019-09-10 11:04:03 -07:00
Pankaj Garg fd33b27af1
Fix coding shred generator (#5865) 2019-09-10 09:35:07 -07:00
Pankaj Garg 7682db4826
Generate coding shreds on the fly based on erasure limits (#5852)
* Generate coding shreds on the fly based on erasure limits

* fix uncle
2019-09-09 17:26:51 -07:00
carllin 7607800d47
Refactor restart function in local cluster to support separate exit and restart functions (#5845) 2019-09-08 17:53:34 -07:00
Michael Vines df205f8752
Use ureq instead of influx_db_client (#5839) 2019-09-07 12:48:45 -07:00
Jack May e8d88f3237
Split SDK's timing.rs (#5823) 2019-09-06 14:30:56 -07:00
dependabot-preview[bot] f28782cb84 Bump chrono from 0.4.8 to 0.4.9 (#5775)
Bumps [chrono](https://github.com/chronotope/chrono) from 0.4.8 to 0.4.9.
- [Release notes](https://github.com/chronotope/chrono/releases)
- [Changelog](https://github.com/chronotope/chrono/blob/master/CHANGELOG.md)
- [Commits](https://github.com/chronotope/chrono/commits/v0.4.9)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-09-06 13:55:36 -06:00
Michael Vines 1f9fde5f7b
ThinClient internal name grooming (#5800) 2019-09-06 09:07:40 -07:00
Sagar Dhawan a452249bf3
Use retain on Packets instead of creating new ones (#5804)
* Use remove on Packets instead of creating a new one

* Fix compile after rebase
2019-09-05 19:16:18 -07:00
Pankaj Garg 3d3b03a123
Verify signature of recovered shred before adding them to blocktree (#5811)
* Verify signature of recovered shred before adding them to blocktree

* fix failing tests, and review comments
2019-09-05 18:20:30 -07:00
Pankaj Garg 05d2eec45c
Remove unnecessary erasure config references (#5809) 2019-09-05 14:46:41 -07:00
Pankaj Garg 3237e897d7
Adjust packet batching post-decoupling from blobs (#5783) 2019-09-05 11:22:39 -07:00
carllin bd74e63702
Offload remaining confidence cache computation to separate thread (#5792)
* Move remaining confidence cache computation to separate thread

* Move confidence cache out of bank forks
2019-09-04 23:10:25 -07:00
Pankaj Garg f78b865cba
Cleanup shreds to remove FirstShred data structure (#5789)
* Cleanup shreds to remove FirstShred data structure

* Also reduce size used by parent slot information in shred header

* clippy

* fixes

* fix chacha test
2019-09-04 21:06:47 -07:00
carllin 7062fe4b47
Refactor Blocktree for clarity and correctness (#5700)
* Refactor shreds to prevent insertion of any metadata on bad shreds

* Refactor fetching Index in blocktree

* Refactor get_slot_meta_entry

* Re-enable local cluster test

* cleanup

* Add tests for success/fail insertion of coding/data shreds

* Remove assert

* Fix and add tests for should_insert coding and data blobs
2019-09-04 17:14:42 -07:00
Pankaj Garg 02ee2a601c
Further cleanup of blocktree after Blob deprecation (#5780) 2019-09-04 12:47:09 -07:00
Tyera Eulberg b19d9a50d3
Transition to ureq http client (#5777)
* Transition to ureq http client

* Remove unwrap
2019-09-04 12:11:44 -07:00
carllin dfa6238342
Remove unnecessary construction of descendants (#5742) 2019-09-04 01:49:42 -07:00
Pankaj Garg 3b0d48e3b8
Remove blocktree blob references (#5691)
* Remove blocktree blob references

* fixes and cleanup

* replace uninitialized() call with MaybeUninit

* fix bench
2019-09-03 21:32:51 -07:00
Sagar Dhawan 62f6a78ccd
Make data plane shred filter parallel again (#5740) 2019-09-03 21:50:57 +00:00
dependabot-preview[bot] f7e039e7ac Bump chrono from 0.4.7 to 0.4.8 (#5761)
Bumps [chrono](https://github.com/chronotope/chrono) from 0.4.7 to 0.4.8.
- [Release notes](https://github.com/chronotope/chrono/releases)
- [Changelog](https://github.com/chronotope/chrono/blob/master/CHANGELOG.md)
- [Commits](https://github.com/chronotope/chrono/commits)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-09-03 14:49:48 -07:00
dependabot-preview[bot] 61bd14c40a Bump rayon from 1.1.0 to 1.2.0 (#5758)
Bumps [rayon](https://github.com/rayon-rs/rayon) from 1.1.0 to 1.2.0.
- [Release notes](https://github.com/rayon-rs/rayon/releases)
- [Changelog](https://github.com/rayon-rs/rayon/blob/master/RELEASES.md)
- [Commits](https://github.com/rayon-rs/rayon/compare/rayon-core-v1.1.0...v1.2.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-09-03 13:39:58 -06:00
Tyera Eulberg 85e5fbeb35
Add absoluteSlot to getEpochInfo (#5765) 2019-09-02 12:21:06 -07:00
anatoly yakovenko 475f6fe666
votes only need slots and the last bank hash (#5499)
churn

cleanup

reverse test slot hashes

test check_slots_are_valid

updates

only send the minimum bank vote difference

fixup! only send the minimum bank vote difference

some banks may not have a voting account setup

fixup! votes only need slots and the last bank hash

fixup! fixup! votes only need slots and the last bank hash

fmt

fixed compare

fixed vote

fixup! fixed vote

poke ci

filter the local votes via the last bank vote
2019-09-02 12:01:09 -07:00
sakridge 0c2a49391a
Disable pinned gpu memory (#5753) 2019-08-31 16:44:07 -07:00
Michael Vines e3a6c9234a Entrypoint RPC service discovery now blocks until the entrypoint is actually found (#5756)
automerge
2019-08-30 16:12:58 -07:00
Michael Vines 643d0b0868 Make the world flat again; remove utils/ subdirectory (#5752)
automerge
2019-08-30 11:57:39 -07:00
dependabot-preview[bot] f840eefcbf Bump bs58 from 0.2.4 to 0.2.5 (#5747)
Bumps [bs58](https://github.com/mycorrhiza/bs58-rs) from 0.2.4 to 0.2.5.
- [Release notes](https://github.com/mycorrhiza/bs58-rs/releases)
- [Commits](https://github.com/mycorrhiza/bs58-rs/compare/0.2.4...0.2.5)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-08-30 11:41:41 -06:00
dependabot-preview[bot] e1f3e33bfb Bump jsonrpc-pubsub from 13.0.0 to 13.1.0 (#5708)
Bumps [jsonrpc-pubsub](https://github.com/paritytech/jsonrpc) from 13.0.0 to 13.1.0.
- [Release notes](https://github.com/paritytech/jsonrpc/releases)
- [Commits](https://github.com/paritytech/jsonrpc/compare/v13.0.0...v13.1.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-08-30 11:40:58 -06:00
Michael Vines 4786143524 Add a more helpful error on genesis block mismatch (#5744)
automerge
2019-08-30 09:10:22 -07:00
dependabot-preview[bot] c3f07eb85a Bump jsonrpc-ws-server from 13.0.0 to 13.1.0 (#5721)
Bumps [jsonrpc-ws-server](https://github.com/paritytech/jsonrpc) from 13.0.0 to 13.1.0.
- [Release notes](https://github.com/paritytech/jsonrpc/releases)
- [Commits](https://github.com/paritytech/jsonrpc/compare/v13.0.0...v13.1.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-08-29 09:12:02 -06:00
Pankaj Garg 8adac30c05
Integrate shreds to the replicators (#5711)
* Integrate shreds to the replicators

* fix cuda stuff

* fix cuda tests
2019-08-28 22:34:47 -07:00
dependabot-preview[bot] 156399e8aa Bump jsonrpc-http-server from 13.0.0 to 13.1.0 (#5707)
Bumps [jsonrpc-http-server](https://github.com/paritytech/jsonrpc) from 13.0.0 to 13.1.0.
- [Release notes](https://github.com/paritytech/jsonrpc/releases)
- [Commits](https://github.com/paritytech/jsonrpc/compare/v13.0.0...v13.1.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-08-28 14:21:17 -06:00
dependabot-preview[bot] 5745a54d4c Bump indexmap from 1.0.2 to 1.1.0 (#5706)
Bumps [indexmap](https://github.com/bluss/indexmap) from 1.0.2 to 1.1.0.
- [Release notes](https://github.com/bluss/indexmap/releases)
- [Commits](https://github.com/bluss/indexmap/compare/1.0.2...v1.1.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-08-28 14:20:45 -06:00
Parth 7dfb735db9
randomize tx ordering (#4978)
Summary of Changes:
This change adds functionality to randomize tx execution for every entry. It does this by implementing OrderedIterator that iterates tx slice as per the order specified. The order is generated randomly for every entry.
2019-08-28 21:08:32 +05:30
Pankaj Garg 2510f3d352
Remove extra call to serialize in shred verify (#5698) 2019-08-27 19:28:00 -07:00
Pankaj Garg 47535b9ff1 Use serialize_into to fill in shreds instead of writing one byte at a time (#5695)
automerge
2019-08-27 17:11:24 -07:00
Sunny Gleason 34ab25a88b
feat: getInflation() endpoint (#5681) 2019-08-27 18:17:03 -04:00
carllin 8b9c3a2561
Blocktree last_root to enforce a slot floor (#5593)
* Add last_root to blocktree

* Don't repair earlier than last_root

* Add integration test to make sure blocktree floor is enforced
2019-08-27 15:09:41 -07:00
Michael Vines 0c62cf8980
Add metrics for snapshot generation (#5677) 2019-08-27 13:04:20 -07:00
Pankaj Garg f1d58f980b Ignore retransmit channel error (#5680)
automerge
2019-08-27 12:41:04 -07:00
Pankaj Garg 12ad95eb5e
Erasure statistics for shreds (#5676) 2019-08-27 11:22:06 -07:00
dependabot-preview[bot] 85f03b590d Bump jsonrpc-derive from 13.0.0 to 13.1.0 (#5668)
Bumps [jsonrpc-derive](https://github.com/paritytech/jsonrpc) from 13.0.0 to 13.1.0.
- [Release notes](https://github.com/paritytech/jsonrpc/releases)
- [Commits](https://github.com/paritytech/jsonrpc/compare/v13.0.0...v13.1.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-08-27 08:48:34 -07:00
Michael Vines a29f0484dc
Add newline before cluster info log (#5671) 2019-08-27 08:33:48 -07:00
dependabot-preview[bot] 8e6e72babd Bump jsonrpc-core from 13.0.0 to 13.1.0 (#5669)
Bumps [jsonrpc-core](https://github.com/paritytech/jsonrpc) from 13.0.0 to 13.1.0.
- [Release notes](https://github.com/paritytech/jsonrpc/releases)
- [Commits](https://github.com/paritytech/jsonrpc/compare/v13.0.0...v13.1.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-08-27 07:49:16 -07:00
dependabot-preview[bot] 7257d2845d Bump hex-literal from 0.2.0 to 0.2.1 (#5638)
Bumps [hex-literal](https://github.com/RustCrypto/utils) from 0.2.0 to 0.2.1.
- [Release notes](https://github.com/RustCrypto/utils/releases)
- [Commits](https://github.com/RustCrypto/utils/compare/hex-literal-v0.2.0...hex-literal-v0.2.1)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-08-26 21:32:42 -07:00
Michael Vines db768b4c3a
Log contact info every 10 seconds (#5663) 2019-08-26 18:31:14 -07:00
Pankaj Garg 4ac1213c9c
Integrate coding shreds and recovery (#5625)
* Integrate coding shreds and recovery

* More tests for shreds and some fixes

* address review comments

* fixes to code shred generation

* unignore tests

* fixes to recovery
2019-08-26 18:27:45 -07:00
Michael Vines a0f3208828 Ignore flaky test_banking_stage_entryfication (#5659)
automerge
2019-08-26 16:49:34 -07:00
carllin 28f2c75137 Add bigger buffers for shred column families in rocks (#5653)
automerge
2019-08-26 15:58:26 -07:00
Rob Walker 615da845cd
remove replicode in run_purge_batch() (#5630)
* remove replicode

* bigger timeout

* backout 90 min timeout
2019-08-26 09:47:48 -07:00
Michael Vines ba59741b60
Bump to 0.19.0-pre0 2019-08-25 21:47:29 -07:00
Michael Vines 52da207f83 test_snapshots_restart_validity now passes (#5644)
automerge
2019-08-25 21:33:41 -07:00
carllin 6e82978931 Fix race with LedgerCleanupService (#5622) 2019-08-23 23:40:20 -07:00
Michael Vines 4e827af392
Remove unnecessary trailing semicolons (#5636) 2019-08-23 22:47:54 -07:00
Rob Walker 0ffe7a9c8f
plumb some rent (#5610)
* plumb some rent

* nits

* fixups

* fixups

* fixups
2019-08-23 14:04:53 -07:00