84a37a2c0c
Make validator timestamping more coincident, and increase timestamp sample range ( #7673 )
...
automerge
2020-01-03 22:38:00 -08:00
7002ccb866
Log root slots while processing ledger
2020-01-03 13:25:37 -07:00
4fe0b116ae
Measure heap usage while processing the ledger
2020-01-03 13:25:37 -07:00
6b7d9942a7
Add authorized_voter history ( #7643 )
...
* Add authorized_voter history
* fixups
* coverage
* bigger vote state
2019-12-30 19:57:53 -08:00
87b2525e03
Limit maximum number of shreds in a slot to 32K ( #7584 )
...
* Limit maximum number of shreds in a slot to 32K
* mark dead slot replay as fatal error
2019-12-30 07:42:09 -08:00
727be309b2
fix entryverification state ( #7169 )
...
automerge
2019-12-23 23:26:27 -08:00
9f65d22909
Groom log messages ( #7610 )
2019-12-23 10:43:07 -07:00
6a9005645a
Update "limit-ledger-size" to use DeleteRange for much faster deletes ( #7515 )
...
* Update "limit-ledger-size" to use DeleteRange for much faster deletes
* Update core/src/ledger_cleanup_service.rs
Co-Authored-By: Michael Vines <mvines@gmail.com>
* Rewrite more idiomatically
* Move max_ledger_slots to a fn for clippy
* Remove unused import
* Detect when all columns have been purged and fix a bug in deletion
* Check that more than 1 column is actually deleted
* Add helper to test that ledger meets minimum slot bounds
* Remove manual batching of deletes
* Refactor to keep some N slots older than the highest root
* Define MAX_LEDGER_SLOTS that ledger_cleanup_service will try to keep around
* Refactor compact range
2019-12-18 11:50:09 -08:00
6aaf742dfe
Extend getConfirmedBlock rpc to return account pre- and post-balances ( #7543 )
...
automerge
2019-12-18 09:56:29 -08:00
323673c3c0
Add `compact_cf` calls to reclaim storage during ledger slot purge ( #7264 )
2019-12-18 10:29:46 -05:00
84b07c81fd
Fix/remove short circuit logic in ledger_purge_batch (&& -> &), found/fixed by @sdhawan ( #7526 )
2019-12-17 15:08:40 -05:00
97589f77f8
Pipeline broadcast socket transmit and blocktree record ( #7481 )
...
automerge
2019-12-16 17:11:18 -08:00
6b88da2b82
Hook up getBlockTime rpc to real data ( #7476 )
...
* Add blocktree timestamp helper functions and tests
* Flesh out blocktree::get_block_time
* Move stakes up into rpc to make testing easier; expand tests
* Review comments
2019-12-14 12:23:02 -07:00
f05860672c
Add create-address-with-seed to solana-cli ( #7472 )
...
* Add create account with seed to cli
* clippy
* fixup
2019-12-14 04:38:24 -08:00
7adc721d96
Generate MAX_DATA_SHREDS_PER_FEC_BLOCK coding shreds for each FEC block ( #7474 )
...
* Generate MAX_DATA_SHREDS_PER_FEC_BLOCK coding shreds for each FEC block
* fix tests, and detangle ErasureMeta::status
2019-12-13 14:50:22 -08:00
b7d6ff6770
Fix Blocktree Config ( #7399 )
2019-12-12 23:27:24 -08:00
75d1aa5135
Perform erasure recovery when repaired data shreds are received ( #7463 )
...
automerge
2019-12-12 17:50:28 -08:00
d94041e98d
Allow coding shred index to be different than data shred index ( #7438 )
...
* Allow coding shred index to be different than data shred index
* move fec_set_index to shred's common header
* fix bench
2019-12-12 16:50:29 -08:00
115bf2613d
feat: add analyze-storage command to ledger-tool ( #7165 )
2019-12-12 18:54:50 -05:00
e082418e4a
Cleanup the sign shreds interface ( #7456 )
...
automerge
2019-12-12 13:27:33 -08:00
83218c479a
code cleanup, storage_size() was Option<u64>, now Result<u64> ( #7424 )
2019-12-12 14:55:30 -05:00
dd54fff978
Use pinned memory for entry verify ( #7440 )
2019-12-12 10:36:27 -08:00
1d0ba0d1f2
Add special handling for snapshot root slot in get_confirmed_block ( #7430 )
...
* Add special handling for snapshot root slot
* Improve test
2019-12-11 15:06:54 -07:00
bec5835289
Automatically call `.recycle` in Drop ( #7429 )
...
automerge
2019-12-11 11:58:40 -08:00
f526c424c5
Move slow shred multi_fec test to integration tests folder ( #7426 )
2019-12-11 11:10:21 -08:00
601d7a52e9
add documentation for storage_size() method ( #7416 )
2019-12-11 09:47:12 -05:00
06415de8ee
change blocktree*::storage_size() to return Option<u64> to handle live fs changes ( #7401 )
2019-12-10 19:12:49 -05:00
781ce30e27
Continue processing the ledger on InvalidTickCount errors
2019-12-09 16:34:37 -07:00
bee3829960
Remove redundant check ( #7369 )
2019-12-09 01:12:14 -08:00
e0600e5a91
Properly set parallelism ( #7370 )
2019-12-09 00:14:09 -08:00
b55b646f12
Fix Erasure Index ( #7319 )
...
Fix Erasure Index Check to set the erasure presence
2019-12-09 00:13:36 -08:00
1ffd6b4b4d
Add program and runtime support for Durable Transaction Nonces ( #6845 )
...
* Rework transaction processing result forwarding
Durable nonce prereq
* Add Durable Nonce program API
* Add runtime changes for Durable Nonce program
* Register Durable Nonce program
* Concise comments and bad math
* Fix c/p error
* Add rent sysvar to withdraw ix
* Remove rent exempt required balance from Meta struct
* Use the helper
2019-12-07 12:54:10 -07:00
c00216e3be
feat: ledger size and cleanup metrics ( #7335 )
2019-12-06 22:32:45 -05:00
b7d4330dd4
Fail fast if account paths cannot be canonicalized ( #7300 )
...
* Canonicalize account paths to avoid symlink issues
* fixes
2019-12-05 21:41:29 -05:00
cfc21e1225
Only serialize rooted append vecs ( #7281 )
2019-12-05 14:27:46 -08:00
a95d37ea25
Fix repair slowness when most peers are unable to serve requests ( #7287 )
...
* Fix repair when most peers are incapable of serving requests
* Add a test for getting the lowest slot in blocktree
* Replace some more u64s with Slot
2019-12-05 11:25:13 -08:00
f0a40862d6
Support local cluster edge case testing ( #7135 )
...
* Refactor local cluster to support killing a partition
* Rework run_network_partition
* Introduce fixed leader schedule
* Plumb fixed schedule into test
2019-12-03 16:31:59 -08:00
a0eafa12e3
Update to hashes_per_tick computation, and tick duration datapoint ( #7127 )
2019-12-02 18:02:11 -08:00
6796b08909
Migrate to thiserror ( #7177 )
...
* Migrate to thiserror
* Discourage the use of other modules' Result alias
`io::Result` set a bad precedent. Don't import other `Result`
aliases.
2019-12-02 15:42:05 -07:00
c706f9b2cd
Change from using fixed number of ticks in delay calculation ( #7152 )
...
automerge
2019-11-26 16:21:02 -08:00
58c144ee55
Add getBlockTime rpc api ( #7130 )
...
* Add getBlockTime rpc api
* Add getBlockTime to docs
* Fix duration rounding for common tick/slot durations; add slot duration calculation
* Expose slots_per_year
* Use genesis values instead of clock defaults to calculate block offset
* Add get-block-time cli subcommand
* Fix test_rent: decrease magic number usage
2019-11-26 00:40:36 -07:00
280315a314
mut
2019-11-25 22:49:15 -07:00
acd1505050
Stop open measurement before logging it
2019-11-25 22:20:34 -07:00
acbe89a159
shrink stakes ( #7122 )
2019-11-25 13:14:32 -08:00
c965a110f2
Use unbounded channel ( #7081 )
2019-11-21 14:23:40 -07:00
8cbc450192
Create genesis.tar.bz2 in solana-genesis ( #7039 )
...
* Use clap_utils
* Create genesis.tar.bz2 in solana-genesis
* Remove shell-based genesis.tar.bz2 generation
* Make Option=>Result conv more rusty
* stop using solana_logger
* Simplify by just using vec!
* clean up abit
2019-11-21 10:57:27 -07:00
3415db9739
Merge api/program into single units ( #7061 )
2019-11-20 16:32:19 -08:00
97ca6858b7
Write transaction status and fee into persistent store ( #7030 )
...
* Pass blocktree into execute_batch, if persist_transaction_status
* Add validator arg to enable persistent transaction status store
* Pass blocktree into banking_stage, if persist_transaction_status
* Add validator params to bash scripts
* Expose actual transaction statuses outside Bank; add tests
* Fix benches
* Offload transaction status writes to a separate thread
* Enable persistent transaction status along with rpc service
* nudge
* Review comments
2019-11-20 16:43:10 -07:00
d184d3a732
Merge native programs parts into one unit ( #7047 )
2019-11-20 10:12:43 -08:00
d2ed921bc6
Cleanup nightly warnings ( #7055 )
2019-11-19 20:15:37 -08:00
ea656b1a3f
Add parent slot to getConfirmedBlock ( #7038 )
...
* Add parent slot to getConfirmedBlock
* Fix bad text-replace
* Use camelCase in getConfirmedBlock
2019-11-19 09:39:55 -07:00
6bfe0fca1f
Add a version field to shreds ( #7023 )
...
* Add a version field to shreds
* Clippy
* Fix Chacha Golden
* Fix shredder bench compile
* Fix blocktree bench compile
2019-11-18 18:05:02 -08:00
e0a2bb9d86
Legitimately map transactions to statuses in blocktree ( #7011 )
...
* Refactor rocksdb TransactionStatus to store/return struct; hook up map_transactions_to_statuses
* Cleanup use statements
2019-11-18 09:12:42 -07:00
3bc8d78801
Add ConfirmedBlock struct, and rework Blocktree apis to include block… ( #7004 )
...
* Add RpcConfirmedBlock struct, and rework Blocktree apis to include blockhash info and dummy tx statuses
* Remove unused lifetime
2019-11-17 20:17:15 -07:00
086e5da8d0
feat: add TransactionStatus column family and test ( #6958 )
2019-11-17 11:26:01 -05:00
c1b06817a2
Add non-dev value for slots_per_epoch and use that as default ( #6984 )
...
When --dev flag is not passed.
2019-11-16 20:53:54 -08:00
7c32640a9b
Set index and set data should write into shred data ( #6995 )
2019-11-16 02:41:59 -08:00
0ca943f49b
RecyclerCache for shred_sigverify ( #6986 )
...
automerge
2019-11-15 12:16:56 -08:00
852a2146ab
Add Blocktree api to get transactions by slot ( #6966 )
...
* Add blocktree method to get confirmed-block txs
* Clean up use statements
* Add test, and fmt
* Plumb new blocktree method into getConfirmedBlock
2019-11-14 16:34:39 -07:00
79d7090867
Remove obsolete references to Blob ( #6957 )
...
* Remove the name "blob" from archivers
* Remove the name "blob" from broadcast
* Remove the name "blob" from Cluset Info
* Remove the name "blob" from Repair
* Remove the name "blob" from a bunch more places
* Remove the name "blob" from tests and book
2019-11-14 11:49:31 -08:00
e7f63cd336
Upgrade to rust 1.39.0 ( #6939 )
...
* Upgrade to rust 1.39.0
* 1.39.0 clippy
2019-11-14 12:27:01 -07:00
d6cbb02c92
Bump rocksdb from 0.12.4 to 0.13.0 ( #6952 )
...
automerge
2019-11-14 09:59:54 -08:00
43e2301e2c
Fix roots overrunning broadcast ( #6884 )
...
* Add trusted pathway for insert_shreds to avoid checks
2019-11-14 00:32:07 -08:00
6009801c5f
More granular timings in shred generation ( #6900 )
2019-11-13 16:30:12 -08:00
ee5cc733a1
Log blocktree and snapshot open times ( #6930 )
...
automerge
2019-11-13 11:20:39 -08:00
9026339d35
Restore is_frozen() asserts ( #6925 )
2019-11-13 10:40:51 -07:00
fcc2874591
Remove/address some TODOs ( #6923 )
2019-11-13 09:43:15 -07:00
30a08f4282
Cleanup ledger macros ( #6916 )
...
automerge
2019-11-13 07:14:09 -08:00
6d76c34291
Handle dead slots when loading ledger ( #6887 )
2019-11-12 13:38:26 -07:00
5670cafda4
Fix caching data shreds as coding shreds ( #6877 )
2019-11-12 10:29:58 -08:00
e8e13fdeeb
Insert coding shreds to blocktree only if needed in future ( #6836 )
...
* Insert coding shreds to blocktree only if needed in future
* fixes
2019-11-11 13:12:55 -08:00
9807f47d4e
Rename genesis block to genesis config ( #6816 )
2019-11-08 23:56:57 -05:00
02058ea699
Reject blocks with invalid last ticks in replay stage ( #6833 )
...
* Reject blocks with invalid last ticks in replay stage
* slot_full
2019-11-08 20:21:54 -05:00
efdfc5c327
Remove TODOs ( #6843 )
2019-11-08 16:43:18 -07:00
66a0f54097
Replay should respect order of register_ticks with respect to blockhashes ( #6805 )
2019-11-08 12:29:41 -07:00
5bd05fba09
require `to` account signature ( #6658 )
...
* require to signature
* fixing invocation to create_account
* fix create_account references
* address review comment
* whacking bugs in tests
* fixing stake program tests
2019-11-08 15:57:35 +05:30
8e81bc1b49
Fix pinning ( #6604 )
...
Remove Deref implementations and add more pass-throughs to the PinnedVec
wrapper.
Warm recyclers
set_pinnable
2019-11-07 19:48:33 -08:00
b64b54f48f
unfork dalek ed25519 ( #6776 )
2019-11-07 17:08:10 -08:00
ccac35fc01
Increase FEC ratio to 32:32 ( #6800 )
...
automerge
2019-11-07 16:38:06 -08:00
67d1e2903c
Upgrade Repair be more intelligent and agressive ( #6789 )
...
* Upgrade Repair be more intelligent and agressive
* Fix u64 casts
* Fix missing bracket
* Add 1 second delay to test to allow repair to kick in
2019-11-07 11:08:09 -08:00
0ace79939b
Add reference tick to data shreds ( #6772 )
...
* Add reference tick to data shreds
* fix tests
2019-11-06 13:27:58 -08:00
5e8668799c
Fewer recyclers. ( #6770 )
...
automerge
2019-11-06 12:35:51 -08:00
67f636545a
Refactor sigverify to stage for signing shreds on the GPU ( #6635 )
...
automerge
2019-11-06 10:52:30 -08:00
7ff2a44a63
Make last shred for an interrupted slot signed + typed ( #6760 )
2019-11-06 08:25:17 -07:00
9614d17024
Limit deserialization of data coming off the wire ( #6751 )
...
* Limit deserialization of data coming off the wire
* Feedback and cleanup
2019-11-06 00:07:57 -08:00
09e648f957
ledger-tool/: Include full validator voting history in fork-graph ( #6756 )
2019-11-05 19:40:00 -07:00
85c9a231c1
Include the affected slot in blocktree error metrics ( #6734 )
2019-11-05 13:25:21 -07:00
7203036e3e
Adjust nofiles within Blocktree::open() for all ledger/ users ( #6737 )
...
automerge
2019-11-05 11:18:49 -08:00
d207a34736
remove duplicate signal handling ( #6702 )
2019-11-05 11:36:51 +05:30
fba1af6ea9
ledger-tool can now load a ledger snapshot ( #6729 )
2019-11-04 22:14:55 -07:00
b825d04597
Pull perf into a separate module. ( #6718 )
...
automerge
2019-11-04 20:13:43 -08:00
4d52f47f87
Move get_bank_forks() into ledger/ so its available for use by ledger-tool/ ( #6720 )
2019-11-04 19:10:06 -07:00
efe260f12e
sysvar trait ( #6667 )
...
* sysvar trait
* get the new guy in on it
2019-11-04 12:31:24 -08:00
9ea398416e
Sign shreds on the GPU ( #6595 )
...
* sign gpu shreds
* wip
* checks
* tests build
* test
* tests
* test
* nits
* sign cpu test
* write out the sigs in parallel
* clippy
* cpu test
* prepare secret for gpu
* woot!
* update
* bump perf libs
2019-11-02 06:23:14 -07:00
50a17fc00b
Use Slot and Epoch type aliases instead of raw u64 ( #6693 )
...
automerge
2019-11-02 00:38:30 -07:00
a4d48df30a
Add assertion when filling blocktree slot with ticks ( #6664 )
...
automerge
2019-10-31 14:15:07 -07:00
c52830980a
Rework get_slot_meta ( #6642 )
...
* Assert slotmeta is not orphan
* Clean up get_slot_meta functionality
* Add test
2019-10-31 14:03:41 -07:00
e8e5ddc55d
Verify number of hashes for each block of entries ( #6262 )
...
* Verify number of hashes for each block of entries
* Fix blocktree processor tick check
* Rebase once more
2019-10-31 16:38:50 -04:00
111942a47d
document clock ( #6662 )
2019-10-31 13:26:55 -07:00
9d65e6f183
Fix check in should_insert_data_shred ( #6649 )
2019-10-30 23:37:25 -07:00
Tyera Eulberg
Michael Vines
Rob Walker
Pankaj Garg
Parth
Sagar Dhawan
Sunny Gleason
anatoly yakovenko
carllin
sakridge
Trent Nelson
Justin Starry
Greg Fitzgerald
Ryo Onodera
Jack May
dependabot-preview[bot]