aa6832964c
ledger_cleanup_service: compact at a slower rate than purging ( #10414 )
2020-06-04 21:06:06 -07:00
caa7f7a0c9
Support opening an in-use rocksdb as secondary ( #10209 )
...
automerge
2020-06-02 21:32:44 -07:00
eee9a08376
Purge TransactionStatus and AddressSignatures exactly from ledger-tool ( #10358 )
...
* Add failing test
* Add execution path to purge primary-index columns exactly
* Fail gracefully if older TransactionStatus rocksdb keys are present
* Remove columns_empty check for special columns
* Move blockstore purge methods to submodule
* Remove unused column empty check
2020-06-02 19:49:31 -06:00
19d11800bf
Remove WithSigner ( #10325 )
...
automerge
2020-05-29 23:17:44 -07:00
bc37dfc4c1
log leader ( #10316 )
...
automerge
2020-05-29 04:35:20 -07:00
fb4d8e1f62
cleanup clippy tests ( #10172 )
...
automerge
2020-05-29 00:26:06 -07:00
5ac2ae1178
Purge next slots to avoid a blockstore_processor panic on restart ( #10281 )
2020-05-27 19:55:40 -07:00
e3b444834f
Add more logging while unpacking snapshots ( #10266 )
2020-05-27 10:41:05 -07:00
7abdfc23f9
succceed is not a word
2020-05-26 23:17:50 -07:00
156387aba4
LedgerCleanupService no longer causes an OOM and actually purges ( #10199 )
...
* cleanup_ledger() now services new_root_receiver while purging
* purge_slots() now fully deletes before compacting
* Add ledger pruning grafana graph
2020-05-24 21:41:54 -07:00
b7a32f01c0
Multi-version snapshot support ( #9980 )
...
* Multi-version snapshot support
* rustfmt
* Remove CLI options and runtime support for selection output snapshot version.
Address some clippy complaints.
* Muzzle clippy type complexity warning.
Despite clippy's suggestion, it is not currently possible to create type aliases
for traits and so everything within the 'Box<...>' cannot be type aliased.
This then leaves creating full blown traits, and either implementing
said traits by closure (somehow) or moving the closures into new structs
implementing said traits which seems a bit of a palaver.
Alternatively it is possible to define and use the type alias 'type ResultBox<T> = Result<Box<T>>'
which does seems rather pointless and not a great reduction in complexity but is enough to keep clippy happy.
In the end I simply went with squelching the clippy warning.
* Remove now unused Serialize/Deserialize trait implementations for AccountStorageEntry and AppendVec
* refactor versioned de/serialisers
* rename serde_utils to serde_snapshot
* move call to accounts_db.generate_index() back down to context_accountsdb_from_stream()
* update version 1.1.1 to 1.2.0
remove nested use of serialize_bytes
* cleanups
* Add back measurement of account storage entry serialization.
Remove construction of Vec and HashMap temporaries during serialization.
* consolidate serialisation test cases into serde_snapshot.
clean up leakage of implementation details in serde_snapshot.
* move short term / legacy snapshot code into child module
* add serialize_iter_as_tuple
* preliminary integration of following commit
commit 6d58b73c47294bfb93465d5a83cd2175660b6e6d
Author: Ryo Onodera <ryoqun@gmail.com>
Date: Wed May 20 14:02:02 2020 +0900
Confine snapshot 1.1 relic to versioned codepath
* refactored serde_snapshot, rustfmt
legacy accounts_db format now "owns" both leading u64s, legacy bank_rc format has none
* reduce type complexity (clippy)
2020-05-23 02:54:24 +09:00
439fd30840
Fix erasure ( #10095 )
...
* Fix bad FEC blocks
* Add test
Co-authored-by: Carl <carl@solana.com>
2020-05-19 16:13:12 -07:00
97f2bcff69
master: Add nonce to shreds repairs, add shred data size to header ( #10109 )
...
* Add nonce to shreds/repairs
* Add data shred size to header
Co-authored-by: Carl <carl@solana.com>
2020-05-19 12:38:18 -07:00
f562ed4cc8
Distinguish between shred type in shred fetch stage duplicate filter ( #10068 )
...
* Shred type check
* Test
2020-05-15 13:23:56 -07:00
5e89bd8868
Panic if no fee-payer found via Message::new() ( #10050 )
...
automerge
2020-05-15 12:23:09 -07:00
7080fb9b37
Abort if the open fd limit cannot be increased ( #10064 )
...
automerge
2020-05-15 12:14:21 -07:00
58ef02f02b
9951 clippy errors in the test suite ( #10030 )
...
automerge
2020-05-15 09:35:43 -07:00
9575afc8fa
Refactor blockstore recovery code ( #10008 )
2020-05-13 10:09:38 -07:00
1eb40c3fe0
Introduce eager rent collection ( #9527 )
...
* Switch AccountsIndex.account_maps from HashMap to BTreeMap
* Introduce eager rent collection
* Start to add tests
* Avoid too short eager rent collection cycles
* Add more tests
* Add more tests...
* Refacotr!!!!!!
* Refactoring follow up
* More tiny cleanups
* Don't rewrite 0-lamport accounts to be deterministic
* Refactor a bit
* Do hard fork, restore tests, and perf. mitigation
* Fix build...
* Refactor and add switch over for testnet (TdS)
* Use to_be_bytes
* cleanup
* More tiny cleanup
* Rebase cleanup
* Set Bank::genesis_hash when resuming from snapshot
* Reorder fns and clean ups
* Better naming and commenting
* Yet more naming clarifications
* Make prefix width strictly uniform for 2-base partition_count
* Fix typo...
* Revert cluster-dependent gate
* kick ci?
* kick ci?
* kick ci?
2020-05-13 16:22:14 +09:00
903a8a3196
Add retransmit packets_by_slot metrics ( #9975 )
2020-05-11 13:49:10 -07:00
ad3c8fb812
More logging around failure ( #9967 )
...
automerge
2020-05-10 16:01:20 -07:00
f12c6c1ed1
BSD compat for 9493de4 ( #9922 )
...
automerge
2020-05-07 16:38:33 -07:00
09ae61651a
Eliminate BankForksInfo ( #9887 )
2020-05-06 08:24:59 -07:00
72312ad615
Avoid holding the entire rooted path while loading bank forks ( #9885 )
2020-05-05 19:45:41 -07:00
3442f36f8a
Repair alternate versions of dead slots ( #9805 )
...
Co-authored-by: Carl <carl@solana.com>
2020-05-05 14:07:21 -07:00
a91236012d
Pass around --max-genesis-archive-unpacked-size ( #9161 )
...
automerge
2020-04-29 18:53:34 -07:00
eb24f3df84
Update dalek version
2020-04-28 21:02:47 -06:00
a12428a5b8
Use Blockstore lowest_slot to start root iterator ( #9738 )
2020-04-28 10:22:10 -06:00
efad193180
Make default programs static ( #9717 )
2020-04-27 21:05:12 -07:00
c11abf88b7
Clean up `use` to keep rust 1.43.0 from complaining ( #9740 )
2020-04-27 16:54:11 -07:00
a7f33b5014
Cache banks in BankForks until optional largest_confirmed_root ( #9678 )
...
automerge
2020-04-24 15:49:57 -07:00
fa20963b93
Revert shred fs ( #9712 )
...
* Revert "Untar is called for shred archives that do not exist. (#9565 )"
This reverts commit 729cb5eec6#9366 )"
This reverts commit 5ed39de8c5
2020-04-24 15:04:23 -07:00
76b1c2baf0
One less alloc per transaction ( #9705 )
...
* One less alloc per transaction
* Fix benches
* Fix compiler warnings in bench build
* Fix move build
* Fix bench
2020-04-24 13:03:46 -06:00
504160b11f
Update to rocksdb 0.14 and set max wal size ( #9668 )
2020-04-23 08:38:09 -07:00
ecb343c23b
reduce errors ( #9669 )
...
Co-authored-by: Anatoly Yakovenko <anatoly@solana.com>
2020-04-22 19:46:06 -07:00
5d2f488004
Add getLowestNonpurgedBlock rpc; use blockstore api in getConfirmedBlocks ( #9656 )
...
automerge
2020-04-22 13:33:06 -07:00
a5419fe79e
Error for invalid shred. ( #9588 )
2020-04-19 21:15:09 -07:00
729cb5eec6
Untar is called for shred archives that do not exist. ( #9565 )
...
automerge
2020-04-18 08:34:55 -07:00
5ed39de8c5
Dont insert shred payload into rocksdb ( #9366 )
...
automerge
2020-04-16 18:20:55 -07:00
66abe45ea1
Decouple accounts hash calculation from snapshot hash ( #9507 )
2020-04-16 15:12:20 -07:00
30b3862770
Don't unwrap on session new
2020-04-16 08:25:45 -07:00
530c542002
Rpc: Speed up getBlockTime ( #9510 )
...
* Add get-block-time metrics
* Add datapoints to blockstore rpc apis
* Tune timestamp_slot_range
* Refactor get_timestamp_slots
* Cargo.lock
2020-04-15 18:09:14 -06:00
c77ed82caa
Use same max_age regardless of leader/not-leader ( #9423 )
...
automerge
2020-04-14 00:34:41 -07:00
91159ea8e3
Rpc: Add getConfirmedSignaturesForAddress ( #9407 )
...
automerge
2020-04-09 20:21:31 -07:00
d5a9ee97f2
Add --allow-dead-slots argument to `slot`/`print`/`json` commands ( #9408 )
...
automerge
2020-04-09 20:10:51 -07:00
f3afe5c99c
Remove dead code ( #9404 )
...
automerge
2020-04-09 13:09:59 -07:00
e1aa247548
Rpc: Add getConfirmedTransaction ( #9381 )
...
* Add blockstore method to return a complete transaction by signature
* Plumb getConfirmedTransaction rpc
* Add doc
2020-04-09 00:57:30 -06:00
f655b3f0fd
Fix lowest_cleanup_slot check in Blockstore ( #9382 )
...
automerge
2020-04-08 18:47:16 -07:00
4522e85ac4
Add Metrics/Dashboards tracking block production ( #9342 )
...
* Add metric tracking blocks/dropped blocks
Co-authored-by: Carl <carl@solana.com>
2020-04-08 14:35:24 -07:00
36e73cada4
Add blockstore address-to-signature index ( #9367 )
...
automerge
2020-04-08 12:50:39 -07:00
acf64f8476
Update getSignatureStatuses to return historical statuses ( #9314 )
...
automerge
2020-04-06 03:04:54 -07:00
49e2cc6593
Rework TransactionStatus index in blockstore ( #9281 )
...
automerge
2020-04-04 20:24:06 -07:00
ad0997e15f
RPC: add `err` field to TransactionStatus, alongside the now deprecated `status` field ( #9296 )
...
automerge
2020-04-04 16:13:26 -07:00
b557b3170e
Add log before opening database
2020-04-03 14:55:06 -07:00
9493de4443
Add snapshot compression option ( #9276 )
2020-04-03 13:13:49 -07:00
73e99cc513
Ledger cleanup fixes ( #9131 )
...
* Fix purging happening every slot when cleanup service is not started at slot 0
* Purge by shred count instead of slots since slots can have variable
number of shreds
2020-03-30 19:02:12 -07:00
2863f8ec65
Use 1gb as genesis limit to fix bench-tps ledger from not starting ( #9133 )
...
automerge
2020-03-27 16:50:19 -07:00
b130c298df
Remove chatty 'setting snapshot root:' info log ( #9122 )
2020-03-27 10:24:59 -07:00
b7b4aa5d4d
move rpc types from client to client-types crate ( #9039 )
...
* Separate client types into own crate, so ledger does not need it
Removes about 50 crates of dependency from ledger
* Drop Rpc name from transaction-status types
2020-03-26 13:29:30 -07:00
ed036b978d
Accumulate blockstore metrics and submit every 2s ( #9075 )
2020-03-26 12:51:41 -07:00
b8fda9d730
Log how much data the ledger holds before processing it ( #9079 )
2020-03-25 21:41:50 -07:00
9dc69d9843
Store and compute node/stake state in EpochStakes struct ( #8958 )
...
* Store and compute needed bank state in EpochStakes struct
2020-03-25 12:19:15 -07:00
f987c18a7e
Strictly validate the contents of snapshot/genesis ( #8959 )
...
automerge
2020-03-25 02:46:41 -07:00
70c167182a
ledger tool now outputs transaction status information if available ( #9024 )
...
automerge
2020-03-23 12:49:21 -07:00
d75a470ffa
Ledger processing speed tracking ( #9005 )
2020-03-23 12:19:11 -07:00
1b8f9e75dd
Update getSignatureStatus: support multiple signatures, include slot in each response item ( #9022 )
...
* Rename enable-rpc-get-confirmed-block
* Rename RpcTransactionStatus -> RpcTransactionStatusMeta
* Return simplified RpcTransactionStatus; Add support for multiple transactions
* Update docs
* typo
2020-03-23 11:25:39 -06:00
4d2b83d01f
Add option to disable rocks compaction ( #9011 )
2020-03-23 08:42:32 -07:00
88ba8439fc
Add frozen account support ( #8989 )
...
automerge
2020-03-22 11:10:04 -07:00
4dd0367136
Rwlock storage opt ( #9006 )
...
* Remove unecessary account paths rwlock
* Remove path rwlock in accounts_db and optimize storage critical section
2020-03-22 10:04:03 -07:00
dc1db33ec9
Add Capabilities to Signal BroadcastStage to Retransmit ( #8899 )
2020-03-19 23:35:01 -07:00
f78a90bce2
Vote InitializeAccount and UpdateNode instructions now need a signature from the validator identity ( #8947 )
...
automerge
2020-03-19 01:58:52 -07:00
bf60345b7a
Remove all snapshots not matching the desired hash
2020-03-17 22:58:01 -07:00
7079559c2d
Fix windows build by removing sys-info ( #8860 )
...
Doesn't build for windows.
2020-03-16 12:53:13 -07:00
009c124fac
Remove generic
2020-03-16 07:57:07 -07:00
7029c88305
use matches macro
2020-03-16 07:57:07 -07:00
9411fc00b8
Lower error level
2020-03-16 07:57:07 -07:00
b7b36bb0a4
Upgrade to Rust 1.42 ( #8836 )
...
* Upgrade to Rust 1.42
* deref
* parens
Co-authored-by: Trent Nelson <trent@solana.com>
2020-03-13 14:15:22 -06:00
6eb4973780
Don't use move semantics if not needed ( #8793 )
2020-03-11 14:37:23 -07:00
9872430bd2
Add VoteTracker for tracking cluster's votes in gossip ( #8327 )
...
Track votes by slot in cluster_vote_listener
2020-03-09 22:03:09 -07:00
f47a789b15
Add find_incomplete_slots ( #8654 )
...
* Add find_incomplete_slots
* Add live slots iterator
2020-03-05 10:58:00 -08:00
01607b9860
Add NextSlotsIterator ( #8652 )
...
automerge
2020-03-04 20:46:58 -08:00
23d8c7ff0e
Generate a snapshot at synchronized points ( #8532 )
...
Co-authored-by: anatoly yakovenko <anatoly@solana.com>
2020-03-04 19:23:40 -08:00
dec3da8f9d
Add orphan iterator ( #8636 )
2020-03-04 18:10:30 -08:00
b68b74ac32
Check transaction signatures in entry verify ( #8596 )
2020-03-03 20:49:51 -08:00
b084c1d437
Remove accounts hack and correctly restore accounts store counts ( #8569 )
...
* Remove accounts hack and correctly restore append-vec counts
* Add test
2020-03-03 20:48:55 -08:00
d86103383a
Do periodic inbound cleaning for rooted slots ( #8436 )
...
* Do periodic inbound compaction for rooted slots
* Add comment
* nits
* Consider not_compacted_roots in cleanup_dead_slot
* Renames in AccountsIndex
* Rename to reflect expansion of removed accounts
* Fix a comment
* rename
* Parallelize clean over AccountsIndex
* Some niceties
* Reduce locks and real chunked parallelism
* Measure each step for sampling opportunities
* Just noticed par iter is maybe lazy
* Replace storage scan with optimized index scan
* Various clean-ups
* Clear uncleared_roots even if no updates
2020-03-03 14:57:25 +09:00
5d9130a3c4
Hack to skip cleanup_dead_slots upon snapshot load
2020-03-02 10:24:12 -07:00
3acf956f6f
Fix test_concurrent_snapshot_packaging
2020-02-26 23:32:53 -07:00
87b13bef8e
Remove bank_slot_from_archive
2020-02-26 23:32:53 -07:00
0d4cb252c4
Adapt local-cluster/
2020-02-26 23:32:53 -07:00
fcabc6f799
Rename snapshot.tar.bz2 to snapshot-<slot>-<hash>.tar.bz2
2020-02-26 23:32:53 -07:00
848c43a9ab
Peg snapshot version to 1.0.0
2020-02-26 22:44:39 -07:00
7a2bf7e7eb
Limit leader schedule search space ( #8468 )
...
* Limit leader schedule search space
* Fix and add test
* Rename
2020-02-26 13:35:50 -08:00
fbf78b83c4
Add retry mechanism when downloading genesis and snapshots
2020-02-26 14:21:37 -07:00
73063544bd
Move shred_version module to sdk/
2020-02-24 14:46:12 -07:00
90240bf11d
r
2020-02-24 14:45:32 -07:00
b106d3ba60
Fix local cluster test, check for accounts hash ( #8411 )
2020-02-24 10:23:47 -08:00
947a339714
Add snapshot hash of full accounts state ( #8295 )
...
* Add snapshot hash of full accounts state
* Use normal hashing for the accounts delta state
* Add merkle
2020-02-22 13:46:40 -08:00
b7386f9d84
Add --trusted-validator support for snapshot hash validation ( #8390 )
2020-02-21 18:42:24 -08:00
d238371b0c
Correct missing entry handling to avoid bad warns ( #8339 )
...
* Correct missing entry handling to avoid bad warns
* Pass storage entries to AccountStorageSerialize
* Fix CI.....
* Add tests and reorder condition for cheapest first
* Remove unneeded reference
2020-02-21 15:27:55 +09:00
18fd52367e
If the node was loaded from a snapshot, advertise it in gossip ( #8364 )
...
automerge
2020-02-20 18:53:26 -08:00
5ef06a9d36
Add non-bz2 snapshot for faster creation for dev. ( #8350 )
...
* Add non-bz2 snapshot for faster creation for dev.
* Fix tests..
* Revert and always just use snapshot.tar.bz2
2020-02-21 10:19:45 +09:00
ab361a8073
Rename KeypairUtil to Signer ( #8360 )
...
automerge
2020-02-20 13:28:55 -08:00
e8124324ff
Support transaction signing by heterogenous lists of keypairs ( #8342 )
...
automerge
2020-02-20 12:13:23 -08:00
1720fe6a46
Snapshot hash gossip changes ( #8358 )
2020-02-20 11:46:13 -08:00
83a8e82626
Remove dead code
2020-02-18 21:08:43 -07:00
17fb8258e5
Datapoints overwhelm the metrics queue and blow up ram usage. ( #8272 )
...
automerge
2020-02-14 11:11:55 -08:00
ab475e4849
get_confirmed_block: expect() less
2020-02-13 10:56:34 -07:00
127553ce4b
Wrap ed25519_dalek::Keypair ( #8247 )
2020-02-12 14:15:12 -07:00
72b11081a4
Report validator rewards in getConfirmedBlock JSON RPC
2020-02-11 17:25:45 -07:00
46b6cedff4
Fix nightly clippy warnings ( #8199 )
...
automerge
2020-02-10 22:48:50 -08:00
84b3e12e1f
Minor logging improvements
2020-02-10 10:42:42 -07:00
485806c488
Just define BnakSlotDelta type alias ( #8186 )
...
automerge
2020-02-10 03:11:37 -08:00
8b5598fabd
Surface shred version more in tools ( #8163 )
...
automerge
2020-02-07 08:57:54 -08:00
5b070ad014
CLI: Support offline fee payers ( #8009 )
...
* CLI: Support offline fee-payer
* Add some knobs to test genesis/validator helpers
* Add tests
2020-02-07 09:14:26 -07:00
f016c9a669
Maintenance : simplify a few patterns, remove unneeded dependencies ( #8137 )
...
* Simplify a few pattern matches
* Removing unneeded dependencies, upgrading internal version #s
net-shaper: Removing log, semver, serde_derive
bench-tps: Removing serde, serde_derive
banking-bench: Removing solana
ledger-tool: Removing bincode, serde, serde_derive
librapay: Removing solana, language_e2e_tests
log-analyzer: Removing log, semver, serde_derive
exchange: Removing solana
core: Removing crc, memmap, symlink, untrusted
perf: Removing serde_derive
genesis: Removing hex, serde_derive
sdk-c: Removing sha2
sys-tuner: Removing semver
bench-exchange: Removing bincode, bs58, env_logger, serde, serde_derive, untrusted, ws
btc_spv_bin: Removing serde_json
btc_spv: Removing chrono
bpf_loader: Removing serde
ledger: Removing dlopen, dlopen_derive, serde_derive
move_loader: Removing byteorder, libc, language_e2e_tests
ownable: Removing serde, serde_derive
client: Removing rand
archiver-utils: Removing rand_chacha
validator: Removing serde_json, tempfile
param_passing_dep: Removing solana
failure: Removing log
vest: Removing log
vote-signer: Removing bs58, serde
local-cluster: Removing symlink
keygen: Removing rpassword
install: Removing bs58, log
upload-perf: Removing log
runtime: Removing serde_json
stake: Removing rand
* Add modified Cargo.lock
* fixup! Simplify a few pattern matches
* fixup! Simplify a few pattern matches
2020-02-06 10:02:38 -07:00
8436457e75
Rename program_utils.rs ( #8127 )
2020-02-05 12:48:30 -08:00
ed5101b031
Generate max coding shreds when necessary ( #8099 )
...
* Generate max coding shreds when necessary
* test
2020-02-04 15:45:01 -08:00
3bc9789e8d
Remove support for 0.22.3 snapshots
2020-01-30 23:34:15 -07:00
4ffd7693d6
Add lock to make sure slot-based locktree calls are safe ( #7993 )
2020-01-28 13:45:41 -08:00
fd7d5cbe0d
Fix compute_shred_version() ( #7989 )
...
automerge
2020-01-27 17:05:31 -08:00
912aafcefd
Reduce epoch duration from 2 weeks to 2 days
2020-01-27 10:34:55 -07:00
989355e885
Add ability to hard fork at any slot ( #7801 )
...
automerge
2020-01-24 17:27:04 -08:00
a2f2c46f87
Ensure shred version is never 0
2020-01-24 17:41:20 -07:00
d62ed4f6b3
Add BlockstoreProcessorResult
2020-01-23 16:52:47 -07:00
b045f9a50d
codemod --extensions rs get_snapshot_tar_path get_snapshot_archive_path
2020-01-23 13:37:13 -07:00
ce231602dc
Move snapshot archive generation out of the SnapshotPackagerService
2020-01-23 13:37:13 -07:00
6f5e0cd161
Type grooming
2020-01-23 13:37:13 -07:00
8b8033c72b
Set BankRc slot correctly when restoring a bank snapshot
2020-01-23 13:37:13 -07:00
7ca0109732
--halt-at-slot 1 now halts at slot 1
2020-01-23 13:37:13 -07:00
6b5172d002
add_snapshot now returns SlotSnapshotPaths
2020-01-23 13:37:13 -07:00
934c32cbc6
Add mechanism to load v0.22.3 snapshots on newer Solana versions
2020-01-22 15:40:32 -07:00
3aabeb2b81
Rename bootstrap leader ( #7906 )
...
* Rename bootstrap leader to bootstrap validator
It's a normal validator as soon as other validators enter the
leader schedule.
* cargo fmt
* Fix build
Thanks @CriesofCarrots!
2020-01-22 09:22:09 -07:00
37d1daf58e
Revert "Generate MAX_DATA_SHREDS_PER_FEC_BLOCK coding shreds for each FEC block ( #7474 )" ( #7898 )
...
automerge
2020-01-21 11:48:09 -08:00
2dd8ab197d
Remove redundant threadpools in sigverify ( #7888 )
...
* Limit the number of thread pools sigverify creates
* Name local threadpools
2020-01-20 20:08:19 -08:00
21d5fe6272
Fix timestamp overflow ( #7886 )
...
* Split timestamp calculation into separate fn for math unit testing
* Add failing test
* Fix failing test; also bump stakes to near expected cluster max supply
* Don't error on timestamp of slot 0
2020-01-20 17:54:44 -07:00
0526d4ff21
Add logging surrounding failure in `get_slot_entries_with_shred_info()` ( #7846 )
...
* Add logging surrounding failure
2020-01-16 16:03:24 -08:00
76e20015a4
Add separate thread to check for and store duplicate slot proofs ( #7834 )
2020-01-16 15:27:54 -08:00
6d3b8b6d7d
Remove tuples from JSON RPC responses ( #7806 )
...
* Remove RpcConfirmedBlock tuple
* Remove getRecentBlockhash tuple
* Remove getProgramAccounts tuple
* Remove tuple from get_signature_confirmation_status
* Collect Rpc response types
* Camel-case epoch schedule for rpc response
* Remove getBlockCommitment tuple
* Remove getStorageTurn tuple
* Update json-rpc docs
2020-01-15 00:25:45 -07:00
ff1ca1e0d3
Consolidate entry tick verification into one function ( #7740 )
...
* Consolidate entry tick verification into one function
* Mark bad slots as dead in blocktree processor
* more feedback
* Add bank.is_complete
* feedback
2020-01-15 09:15:26 +08:00
721c4378c1
Plumb ability to handle duplicate shreds into shred insertion functions ( #7784 )
2020-01-14 15:37:53 -08:00
f91ffbbfdf
Add support in BlockStore for tracking duplicate slots ( #7761 )
...
* Add test
* Add new column family to track duplicate slots
* Fix clippy errors
* Introduce new SlotColumn for common implementation of Column trait
2020-01-13 17:21:39 -08:00
156292e408
Reduce grace ticks, and ignore grace ticks for missing leaders ( #7764 )
...
* Reduce grace ticks, and ignore grace ticks for missing leaders
* address review comments
* blockstore related renames
2020-01-14 05:25:41 +05:30
b5dba77056
Rename blocktree to blockstore ( #7757 )
...
automerge
2020-01-13 13:13:52 -08:00
8f7ded33e0
coalesce data and coding index ( #7765 )
2020-01-13 12:03:19 -08:00
a17d5795fb
getConfirmedBlock: add encoding optional parameter ( #7756 )
...
automerge
2020-01-12 21:34:30 -08:00
fd3c6eb320
Remove print in test ( #7758 )
...
automerge
2020-01-10 15:37:22 -08:00
27d2c0aaf3
Handle errors on replaying ledger properly ( #7741 )
2020-01-10 12:16:44 -08:00
865c42465a
Cap file size for snapshot data files ( #7182 )
...
* save limit deserialize
* save
* Save
* Clean up
* rustfmt
* rustfmt
* Just comment out to please CI
* Fix ci...
* Move code
* Rustfmt
* Crean up control flow
* Add another comment
* Introduce predetermined constant limit on snapshot data files (deserialize side)
* Introduce predetermined constant limit on snapshot data files (serialize side)
* rustfmt
* Tweak message
* Revert dynamic memory limit
* Limit size of snapshot data file (de)serialization
* Fix test breakage
* Clean up
* Fix uses formatting
* Rename: deserialize_{for,from}_snapshot
* Simplify comment
* Use Slot
* Provide slot for status cache
* Align variable name with snapshot_status_cache_file_path
* Define serialize_snapshot_data_file_with_metrics
* Fix build.......
* De-marco serialize_snapshot_data_file_with_metrics
* Revert u64 => Slot
2020-01-10 09:49:36 +09:00
85c3d64f29
Fix rooted slot iterator ( #7695 )
...
* Enable jumping gaps caused by snapshots in rooted slot iterator
2020-01-07 22:51:28 -05:00
e75a64a8a2
getBlockTime: Fix RootedSlotIterator lowest root ( #7681 )
...
* Determine lowest_nonzero_root for purged blocktrees, and clean up slot offset math
* Filter duplicate timestamp votes
* Refactor deduping code
2020-01-05 23:38:27 -07:00
84a37a2c0c
Make validator timestamping more coincident, and increase timestamp sample range ( #7673 )
...
automerge
2020-01-03 22:38:00 -08:00
7002ccb866
Log root slots while processing ledger
2020-01-03 13:25:37 -07:00
4fe0b116ae
Measure heap usage while processing the ledger
2020-01-03 13:25:37 -07:00
6b7d9942a7
Add authorized_voter history ( #7643 )
...
* Add authorized_voter history
* fixups
* coverage
* bigger vote state
2019-12-30 19:57:53 -08:00
87b2525e03
Limit maximum number of shreds in a slot to 32K ( #7584 )
...
* Limit maximum number of shreds in a slot to 32K
* mark dead slot replay as fatal error
2019-12-30 07:42:09 -08:00
727be309b2
fix entryverification state ( #7169 )
...
automerge
2019-12-23 23:26:27 -08:00
9f65d22909
Groom log messages ( #7610 )
2019-12-23 10:43:07 -07:00
6a9005645a
Update "limit-ledger-size" to use DeleteRange for much faster deletes ( #7515 )
...
* Update "limit-ledger-size" to use DeleteRange for much faster deletes
* Update core/src/ledger_cleanup_service.rs
Co-Authored-By: Michael Vines <mvines@gmail.com>
* Rewrite more idiomatically
* Move max_ledger_slots to a fn for clippy
* Remove unused import
* Detect when all columns have been purged and fix a bug in deletion
* Check that more than 1 column is actually deleted
* Add helper to test that ledger meets minimum slot bounds
* Remove manual batching of deletes
* Refactor to keep some N slots older than the highest root
* Define MAX_LEDGER_SLOTS that ledger_cleanup_service will try to keep around
* Refactor compact range
2019-12-18 11:50:09 -08:00
6aaf742dfe
Extend getConfirmedBlock rpc to return account pre- and post-balances ( #7543 )
...
automerge
2019-12-18 09:56:29 -08:00
323673c3c0
Add `compact_cf` calls to reclaim storage during ledger slot purge ( #7264 )
2019-12-18 10:29:46 -05:00
84b07c81fd
Fix/remove short circuit logic in ledger_purge_batch (&& -> &), found/fixed by @sdhawan ( #7526 )
2019-12-17 15:08:40 -05:00
97589f77f8
Pipeline broadcast socket transmit and blocktree record ( #7481 )
...
automerge
2019-12-16 17:11:18 -08:00
6b88da2b82
Hook up getBlockTime rpc to real data ( #7476 )
...
* Add blocktree timestamp helper functions and tests
* Flesh out blocktree::get_block_time
* Move stakes up into rpc to make testing easier; expand tests
* Review comments
2019-12-14 12:23:02 -07:00
f05860672c
Add create-address-with-seed to solana-cli ( #7472 )
...
* Add create account with seed to cli
* clippy
* fixup
2019-12-14 04:38:24 -08:00
7adc721d96
Generate MAX_DATA_SHREDS_PER_FEC_BLOCK coding shreds for each FEC block ( #7474 )
...
* Generate MAX_DATA_SHREDS_PER_FEC_BLOCK coding shreds for each FEC block
* fix tests, and detangle ErasureMeta::status
2019-12-13 14:50:22 -08:00
b7d6ff6770
Fix Blocktree Config ( #7399 )
2019-12-12 23:27:24 -08:00
75d1aa5135
Perform erasure recovery when repaired data shreds are received ( #7463 )
...
automerge
2019-12-12 17:50:28 -08:00
d94041e98d
Allow coding shred index to be different than data shred index ( #7438 )
...
* Allow coding shred index to be different than data shred index
* move fec_set_index to shred's common header
* fix bench
2019-12-12 16:50:29 -08:00
115bf2613d
feat: add analyze-storage command to ledger-tool ( #7165 )
2019-12-12 18:54:50 -05:00
e082418e4a
Cleanup the sign shreds interface ( #7456 )
...
automerge
2019-12-12 13:27:33 -08:00
83218c479a
code cleanup, storage_size() was Option<u64>, now Result<u64> ( #7424 )
2019-12-12 14:55:30 -05:00
dd54fff978
Use pinned memory for entry verify ( #7440 )
2019-12-12 10:36:27 -08:00
1d0ba0d1f2
Add special handling for snapshot root slot in get_confirmed_block ( #7430 )
...
* Add special handling for snapshot root slot
* Improve test
2019-12-11 15:06:54 -07:00
bec5835289
Automatically call `.recycle` in Drop ( #7429 )
...
automerge
2019-12-11 11:58:40 -08:00
f526c424c5
Move slow shred multi_fec test to integration tests folder ( #7426 )
2019-12-11 11:10:21 -08:00
601d7a52e9
add documentation for storage_size() method ( #7416 )
2019-12-11 09:47:12 -05:00
06415de8ee
change blocktree*::storage_size() to return Option<u64> to handle live fs changes ( #7401 )
2019-12-10 19:12:49 -05:00
781ce30e27
Continue processing the ledger on InvalidTickCount errors
2019-12-09 16:34:37 -07:00
bee3829960
Remove redundant check ( #7369 )
2019-12-09 01:12:14 -08:00
e0600e5a91
Properly set parallelism ( #7370 )
2019-12-09 00:14:09 -08:00
b55b646f12
Fix Erasure Index ( #7319 )
...
Fix Erasure Index Check to set the erasure presence
2019-12-09 00:13:36 -08:00
1ffd6b4b4d
Add program and runtime support for Durable Transaction Nonces ( #6845 )
...
* Rework transaction processing result forwarding
Durable nonce prereq
* Add Durable Nonce program API
* Add runtime changes for Durable Nonce program
* Register Durable Nonce program
* Concise comments and bad math
* Fix c/p error
* Add rent sysvar to withdraw ix
* Remove rent exempt required balance from Meta struct
* Use the helper
2019-12-07 12:54:10 -07:00
c00216e3be
feat: ledger size and cleanup metrics ( #7335 )
2019-12-06 22:32:45 -05:00
b7d4330dd4
Fail fast if account paths cannot be canonicalized ( #7300 )
...
* Canonicalize account paths to avoid symlink issues
* fixes
2019-12-05 21:41:29 -05:00
cfc21e1225
Only serialize rooted append vecs ( #7281 )
2019-12-05 14:27:46 -08:00
a95d37ea25
Fix repair slowness when most peers are unable to serve requests ( #7287 )
...
* Fix repair when most peers are incapable of serving requests
* Add a test for getting the lowest slot in blocktree
* Replace some more u64s with Slot
2019-12-05 11:25:13 -08:00
f0a40862d6
Support local cluster edge case testing ( #7135 )
...
* Refactor local cluster to support killing a partition
* Rework run_network_partition
* Introduce fixed leader schedule
* Plumb fixed schedule into test
2019-12-03 16:31:59 -08:00
a0eafa12e3
Update to hashes_per_tick computation, and tick duration datapoint ( #7127 )
2019-12-02 18:02:11 -08:00
6796b08909
Migrate to thiserror ( #7177 )
...
* Migrate to thiserror
* Discourage the use of other modules' Result alias
`io::Result` set a bad precedent. Don't import other `Result`
aliases.
2019-12-02 15:42:05 -07:00
c706f9b2cd
Change from using fixed number of ticks in delay calculation ( #7152 )
...
automerge
2019-11-26 16:21:02 -08:00
58c144ee55
Add getBlockTime rpc api ( #7130 )
...
* Add getBlockTime rpc api
* Add getBlockTime to docs
* Fix duration rounding for common tick/slot durations; add slot duration calculation
* Expose slots_per_year
* Use genesis values instead of clock defaults to calculate block offset
* Add get-block-time cli subcommand
* Fix test_rent: decrease magic number usage
2019-11-26 00:40:36 -07:00
280315a314
mut
2019-11-25 22:49:15 -07:00
acd1505050
Stop open measurement before logging it
2019-11-25 22:20:34 -07:00
acbe89a159
shrink stakes ( #7122 )
2019-11-25 13:14:32 -08:00
c965a110f2
Use unbounded channel ( #7081 )
2019-11-21 14:23:40 -07:00
8cbc450192
Create genesis.tar.bz2 in solana-genesis ( #7039 )
...
* Use clap_utils
* Create genesis.tar.bz2 in solana-genesis
* Remove shell-based genesis.tar.bz2 generation
* Make Option=>Result conv more rusty
* stop using solana_logger
* Simplify by just using vec!
* clean up abit
2019-11-21 10:57:27 -07:00
3415db9739
Merge api/program into single units ( #7061 )
2019-11-20 16:32:19 -08:00
97ca6858b7
Write transaction status and fee into persistent store ( #7030 )
...
* Pass blocktree into execute_batch, if persist_transaction_status
* Add validator arg to enable persistent transaction status store
* Pass blocktree into banking_stage, if persist_transaction_status
* Add validator params to bash scripts
* Expose actual transaction statuses outside Bank; add tests
* Fix benches
* Offload transaction status writes to a separate thread
* Enable persistent transaction status along with rpc service
* nudge
* Review comments
2019-11-20 16:43:10 -07:00
d184d3a732
Merge native programs parts into one unit ( #7047 )
2019-11-20 10:12:43 -08:00
Michael Vines
Ryo Onodera
Tyera Eulberg
Greg Fitzgerald
carllin
Kristofer Peterson
sakridge
Ryo Onodera
Trent Nelson
Jack May
anatoly yakovenko
Carl
Grimes
sakridge
François Garillot
Pankaj Garg
Sagar Dhawan
Tyera Eulberg
Justin Starry
Rob Walker
Parth
Sunny Gleason