Commit Graph

131 Commits

Author SHA1 Message Date
carllin fd3c6eb320 Remove print in test (#7758)
automerge
2020-01-10 15:37:22 -08:00
carllin 27d2c0aaf3
Handle errors on replaying ledger properly (#7741) 2020-01-10 12:16:44 -08:00
Ryo Onodera 865c42465a
Cap file size for snapshot data files (#7182)
* save limit deserialize

* save

* Save

* Clean up

* rustfmt

* rustfmt

* Just comment out to please CI

* Fix ci...

* Move code

* Rustfmt

* Crean up control flow

* Add another comment

* Introduce predetermined constant limit on snapshot data files (deserialize side)

* Introduce predetermined constant limit on snapshot data files (serialize side)

* rustfmt

* Tweak message

* Revert dynamic memory limit

* Limit size of snapshot data file (de)serialization

* Fix test breakage

* Clean up

* Fix uses formatting

* Rename: deserialize_{for,from}_snapshot

* Simplify comment

* Use Slot

* Provide slot for status cache

* Align variable name with snapshot_status_cache_file_path

* Define serialize_snapshot_data_file_with_metrics

* Fix build.......

* De-marco serialize_snapshot_data_file_with_metrics

* Revert u64 => Slot
2020-01-10 09:49:36 +09:00
carllin 85c3d64f29
Fix rooted slot iterator (#7695)
* Enable jumping gaps caused by snapshots in rooted slot iterator
2020-01-07 22:51:28 -05:00
Tyera Eulberg e75a64a8a2
getBlockTime: Fix RootedSlotIterator lowest root (#7681)
* Determine lowest_nonzero_root for purged blocktrees, and clean up slot offset math

* Filter duplicate timestamp votes

* Refactor deduping code
2020-01-05 23:38:27 -07:00
Tyera Eulberg 84a37a2c0c Make validator timestamping more coincident, and increase timestamp sample range (#7673)
automerge
2020-01-03 22:38:00 -08:00
Michael Vines 7002ccb866 Log root slots while processing ledger 2020-01-03 13:25:37 -07:00
Michael Vines 4fe0b116ae Measure heap usage while processing the ledger 2020-01-03 13:25:37 -07:00
Rob Walker 6b7d9942a7
Add authorized_voter history (#7643)
* Add authorized_voter history

* fixups

* coverage

* bigger vote state
2019-12-30 19:57:53 -08:00
Pankaj Garg 87b2525e03
Limit maximum number of shreds in a slot to 32K (#7584)
* Limit maximum number of shreds in a slot to 32K

* mark dead slot replay as fatal error
2019-12-30 07:42:09 -08:00
Parth 727be309b2 fix entryverification state (#7169)
automerge
2019-12-23 23:26:27 -08:00
Michael Vines 9f65d22909
Groom log messages (#7610) 2019-12-23 10:43:07 -07:00
Sagar Dhawan 6a9005645a
Update "limit-ledger-size" to use DeleteRange for much faster deletes (#7515)
* Update "limit-ledger-size" to use DeleteRange for much faster deletes

* Update core/src/ledger_cleanup_service.rs

Co-Authored-By: Michael Vines <mvines@gmail.com>

* Rewrite more idiomatically

* Move max_ledger_slots to a fn for clippy

* Remove unused import

* Detect when all columns have been purged and fix a bug in deletion

* Check that more than 1 column is actually deleted

* Add helper to test that ledger meets minimum slot bounds

* Remove manual batching of deletes

* Refactor to keep some N slots older than the highest root

* Define MAX_LEDGER_SLOTS that ledger_cleanup_service will try to keep around

* Refactor compact range
2019-12-18 11:50:09 -08:00
Tyera Eulberg 6aaf742dfe Extend getConfirmedBlock rpc to return account pre- and post-balances (#7543)
automerge
2019-12-18 09:56:29 -08:00
Sunny Gleason 323673c3c0
Add `compact_cf` calls to reclaim storage during ledger slot purge (#7264) 2019-12-18 10:29:46 -05:00
Sunny Gleason 84b07c81fd
Fix/remove short circuit logic in ledger_purge_batch (&& -> &), found/fixed by @sdhawan (#7526) 2019-12-17 15:08:40 -05:00
anatoly yakovenko 97589f77f8 Pipeline broadcast socket transmit and blocktree record (#7481)
automerge
2019-12-16 17:11:18 -08:00
Tyera Eulberg 6b88da2b82
Hook up getBlockTime rpc to real data (#7476)
* Add blocktree timestamp helper functions and tests

* Flesh out blocktree::get_block_time

* Move stakes up into rpc to make testing easier; expand tests

* Review comments
2019-12-14 12:23:02 -07:00
Rob Walker f05860672c
Add create-address-with-seed to solana-cli (#7472)
* Add create account with seed to cli

* clippy

* fixup
2019-12-14 04:38:24 -08:00
Pankaj Garg 7adc721d96
Generate MAX_DATA_SHREDS_PER_FEC_BLOCK coding shreds for each FEC block (#7474)
* Generate MAX_DATA_SHREDS_PER_FEC_BLOCK coding shreds for each FEC block

* fix tests, and detangle ErasureMeta::status
2019-12-13 14:50:22 -08:00
carllin b7d6ff6770
Fix Blocktree Config (#7399) 2019-12-12 23:27:24 -08:00
Pankaj Garg 75d1aa5135 Perform erasure recovery when repaired data shreds are received (#7463)
automerge
2019-12-12 17:50:28 -08:00
Pankaj Garg d94041e98d
Allow coding shred index to be different than data shred index (#7438)
* Allow coding shred index to be different than data shred index

* move fec_set_index to shred's common header

* fix bench
2019-12-12 16:50:29 -08:00
Sunny Gleason 115bf2613d
feat: add analyze-storage command to ledger-tool (#7165) 2019-12-12 18:54:50 -05:00
anatoly yakovenko e082418e4a Cleanup the sign shreds interface (#7456)
automerge
2019-12-12 13:27:33 -08:00
Sunny Gleason 83218c479a
code cleanup, storage_size() was Option<u64>, now Result<u64> (#7424) 2019-12-12 14:55:30 -05:00
sakridge dd54fff978
Use pinned memory for entry verify (#7440) 2019-12-12 10:36:27 -08:00
Tyera Eulberg 1d0ba0d1f2
Add special handling for snapshot root slot in get_confirmed_block (#7430)
* Add special handling for snapshot root slot

* Improve test
2019-12-11 15:06:54 -07:00
anatoly yakovenko bec5835289 Automatically call `.recycle` in Drop (#7429)
automerge
2019-12-11 11:58:40 -08:00
sakridge f526c424c5
Move slow shred multi_fec test to integration tests folder (#7426) 2019-12-11 11:10:21 -08:00
Sunny Gleason 601d7a52e9
add documentation for storage_size() method (#7416) 2019-12-11 09:47:12 -05:00
Sunny Gleason 06415de8ee
change blocktree*::storage_size() to return Option<u64> to handle live fs changes (#7401) 2019-12-10 19:12:49 -05:00
Michael Vines 781ce30e27 Continue processing the ledger on InvalidTickCount errors 2019-12-09 16:34:37 -07:00
carllin bee3829960
Remove redundant check (#7369) 2019-12-09 01:12:14 -08:00
carllin e0600e5a91
Properly set parallelism (#7370) 2019-12-09 00:14:09 -08:00
carllin b55b646f12
Fix Erasure Index (#7319)
Fix Erasure Index Check to set the erasure presence
2019-12-09 00:13:36 -08:00
Trent Nelson 1ffd6b4b4d
Add program and runtime support for Durable Transaction Nonces (#6845)
* Rework transaction processing result forwarding

Durable nonce prereq

* Add Durable Nonce program API

* Add runtime changes for Durable Nonce program

* Register Durable Nonce program

* Concise comments and bad math

* Fix c/p error

* Add rent sysvar to withdraw ix

* Remove rent exempt required balance from Meta struct

* Use the helper
2019-12-07 12:54:10 -07:00
Sunny Gleason c00216e3be
feat: ledger size and cleanup metrics (#7335) 2019-12-06 22:32:45 -05:00
Justin Starry b7d4330dd4
Fail fast if account paths cannot be canonicalized (#7300)
* Canonicalize account paths to avoid symlink issues

* fixes
2019-12-05 21:41:29 -05:00
sakridge cfc21e1225
Only serialize rooted append vecs (#7281) 2019-12-05 14:27:46 -08:00
Sagar Dhawan a95d37ea25
Fix repair slowness when most peers are unable to serve requests (#7287)
* Fix repair when most peers are incapable of serving requests

* Add a test for getting the lowest slot in blocktree

* Replace some more u64s with Slot
2019-12-05 11:25:13 -08:00
carllin f0a40862d6
Support local cluster edge case testing (#7135)
* Refactor local cluster to support killing a partition

* Rework run_network_partition

* Introduce fixed leader schedule

* Plumb fixed schedule into test
2019-12-03 16:31:59 -08:00
Pankaj Garg a0eafa12e3
Update to hashes_per_tick computation, and tick duration datapoint (#7127) 2019-12-02 18:02:11 -08:00
Greg Fitzgerald 6796b08909
Migrate to thiserror (#7177)
* Migrate to thiserror

* Discourage the use of other modules' Result alias

`io::Result` set a bad precedent. Don't import other `Result`
aliases.
2019-12-02 15:42:05 -07:00
carllin c706f9b2cd Change from using fixed number of ticks in delay calculation (#7152)
automerge
2019-11-26 16:21:02 -08:00
Tyera Eulberg 58c144ee55
Add getBlockTime rpc api (#7130)
* Add getBlockTime rpc api

* Add getBlockTime to docs

* Fix duration rounding for common tick/slot durations; add slot duration calculation

* Expose slots_per_year

* Use genesis values instead of clock defaults to calculate block offset

* Add get-block-time cli subcommand

* Fix test_rent: decrease magic number usage
2019-11-26 00:40:36 -07:00
Michael Vines 280315a314
mut 2019-11-25 22:49:15 -07:00
Michael Vines acd1505050
Stop open measurement before logging it 2019-11-25 22:20:34 -07:00
Rob Walker acbe89a159
shrink stakes (#7122) 2019-11-25 13:14:32 -08:00
Tyera Eulberg c965a110f2
Use unbounded channel (#7081) 2019-11-21 14:23:40 -07:00