Commit Graph

637 Commits

Author SHA1 Message Date
HaoranYi e4a1799334
fix typos in ledge code comments (#23068) 2022-02-10 20:43:44 -07:00
Yueh-Hsuan Chiang 9213fcb11b
Introduce experimental FIFO-compaction option for shreds in blockstore (#22140) 2022-02-10 11:34:03 -08:00
Justin Starry d5dec989b9
Enforce tx metadata upload with static types (#23028) 2022-02-10 13:28:18 +08:00
Pankaj Garg a146f2d853 suppress clippy 2022-02-07 08:02:58 -08:00
Pankaj Garg c5d8560cdb updates to address review feedback 2022-02-07 08:02:58 -08:00
Pankaj Garg dfef68f985 fix build 2022-02-07 08:02:58 -08:00
Pankaj Garg 4de14e530b Optimize batching of transactions during replay for parallel processing 2022-02-07 08:02:58 -08:00
Yueh-Hsuan Chiang 37afdd1a65
(Ledger Store) Improve comments for blockstore_purge (#22808) 2022-02-06 21:56:23 -08:00
steviez c7ca2f41f5
Add some comments for RocksDB option configuration (#22655) 2022-02-06 23:28:24 -06:00
Justin Starry ba215e94f6
Refactor: Add AccountKeys struct for static and dynamic message keys (#22960) 2022-02-05 20:00:31 +08:00
Jeff Washington (jwash) fb95fa68a2
cleanup api (#22845) 2022-01-31 11:39:23 -06:00
Yueh-Hsuan Chiang aaae5b3ba6
(Ledger Store) Add comment blocks for six pub functions in blockstore.rs (#22476) 2022-01-28 22:25:07 -08:00
Yueh-Hsuan Chiang 40a49081b9
(Ledger Store) Add comments for find slot functions. (#22666) 2022-01-28 12:07:09 -08:00
Michael Vines 6d5bbca630 Pacify clippy 2022-01-21 19:12:57 -08:00
Yueh-Hsuan Chiang fe7543c31a
(Ledger Store) APIs for obtaining physical size of all data and coding shreds (#22443) 2022-01-19 19:31:19 -08:00
Tao Zhu a724fa2347
Add hidden cli option to allow validator reports replayed transaction cost metrics (#22369)
* add hidden cli option to allow validator reports replayed transaction cost detail metrics

* Update validator/src/main.rs

Co-authored-by: Michael Vines <mvines@gmail.com>

* - rebase master, using unbounded instead of channel; dowgrade to datapoint_trace

* removed cli arg, prefer log at trace

Co-authored-by: Michael Vines <mvines@gmail.com>
2022-01-15 00:31:21 +00:00
Justin Starry f804ccdece
Store address table lookups in blockstore and bigtable (#22402) 2022-01-14 15:24:41 +08:00
carllin eca8d21249
log internals (#22493) 2022-01-13 19:50:46 -05:00
Jeff Biseda 8b66625c95
convert std::sync::mpsc to crossbeam_channel (#22264) 2022-01-11 02:44:46 -08:00
carllin 813006b33b
remove per program timings from blockstore processor ledger replay (#22370) 2022-01-08 01:52:08 -05:00
Yueh-Hsuan Chiang 5771c36d3f
Rename open_with_access_type() to open_with_options() (#22123) 2022-01-07 12:11:43 -08:00
steviez 207825d30b
Minimize boilerplate code around Rocks column families (#22345) 2022-01-06 23:39:09 -06:00
Justin Starry 52d12cc802
Add runtime support for address table lookups (#22223)
* Add support for address table lookups in runtime

* feedback

* feedback
2022-01-07 11:59:09 +08:00
Trent Nelson 7d32909e17 move `ExecuteTimings` from `runtime::bank` to `program_runtime::timings` 2022-01-06 03:56:46 -07:00
Justin Starry 45458e7139
Refactor: Improve type safety and readability of transaction execution (#22215)
* Refactor Bank::load_and_execute_transactions

* Refactor: improve type safety of TransactionExecutionResult

* Add enum for extra type safety in execution results

* feedback
2022-01-05 10:15:15 +08:00
behzad nouri 01a096adc8 adds bitflags to Packet.Meta
Instead of a separate bool type for each flag, all the flags can be
encoded in a type-safe bitflags encoded in a single u8:
https://github.com/solana-labs/solana/blob/d6ec103be/sdk/src/packet.rs#L19-L31
2022-01-04 13:53:40 +00:00
Jeff Biseda ca8fef5855
retransmit consecutive leader blocks (#22157) 2022-01-04 00:24:16 -08:00
Yueh-Hsuan Chiang e8b7f96a89
Add struct BlockstoreOptions (#22121) 2022-01-03 18:30:45 -10:00
Justin Starry 2a00382d71
Refactor: cleanup solana_transaction_status crate (#22230) 2022-01-03 15:45:18 +00:00
carllin 33d0b5e011
Revert "Count compute units even when transaction errors (#22059)" (#22174)
This reverts commit eaa8c67bde.
2021-12-30 02:42:32 -05:00
Omar Kilani bac6821e19
get_signatures_for_address does not correctly account for result sets that span local and Bigtable sources (#22115)
* get_signatures_for_address does not correctly account for result sets that span Blockstore and Bigtable.

This causes Bigtable to return `RowNotFound` until the new tx is uploaded.

Check that `before` exists in Bigtable, and if not, set it to `None` to return the full data set.

References #21442
Closes #22110

* Differentiate between before sig not found and no newer signatures

* Dedupe bigtable results to account for potential upload race

Co-authored-by: Tyera Eulberg <tyera@solana.com>
2021-12-29 10:25:10 -07:00
carllin eaa8c67bde
Count compute units even when transaction errors (#22059) 2021-12-28 17:05:11 -05:00
Yueh-Hsuan Chiang b89cd8cd1a
Avoid cloning Vec<Entry> when calling entries_to_test_shreds() (#22093) 2021-12-24 12:32:43 -08:00
Yueh-Hsuan Chiang bf8fbf8383
Add code comment for handle_chaining in blockstore.rs (#21876) 2021-12-21 22:36:24 -08:00
Yueh-Hsuan Chiang 3155a04189
Add comment block for insert_shreds_handle_duplicate in blockstore.rs (#21877) 2021-12-21 22:36:13 -08:00
behzad nouri 65d59f4ef0
tracks erasure coding shreds' indices explicitly (#21822)
The indices for erasure coding shreds are tied to data shreds:
https://github.com/solana-labs/solana/blob/90f41fd9b/ledger/src/shred.rs#L921

However with the upcoming changes to erasure schema, there will be more
erasure coding shreds than data shreds and we can no longer infer coding
shreds indices from data shreds.

The commit adds constructs to track coding shreds indices explicitly.
2021-12-19 22:37:55 +00:00
behzad nouri 89d66c3210
removes next_shred_index from return value of entries to shreds api (#21961)
next-shred-index is already readily available from returned data shreds.
The commit simplifies the api for upcoming changes to erasure coding
schema which will require explicit tracking of indices for coding shreds
as well as data shreds.
2021-12-17 15:01:55 +00:00
behzad nouri efd64a3862
simplifies ShredIndex api (#21932) 2021-12-16 19:17:32 +00:00
behzad nouri 8183f28636
adds ErasureSetId identifying erasure coding sets of shreds (#21928) 2021-12-16 14:18:55 +00:00
Yueh-Hsuan Chiang 882f886450
Add comment block for commit_slot_meta_working_set in blockstore.rs (#21852) 2021-12-15 13:12:50 -08:00
Yueh-Hsuan Chiang e374fb1d60
Add code comment for get_slot_meta_entry in blockstore.rs (#21844) 2021-12-15 13:12:38 -08:00
Yueh-Hsuan Chiang 71b12b1f56
Add comment for clear_unconfirmed_slot() in blockstore.rs (#21837) 2021-12-15 00:37:09 -08:00
Yueh-Hsuan Chiang e476e17abf
Add code comment for check_insert_data_shred in blockstore.rs (#21845) 2021-12-15 00:36:11 -08:00
behzad nouri 8d980f07ba
uses Option<Slot> for SlotMeta.parent_slot (#21808)
SlotMeta.parent_slot for the head of a detached chain of slots is
unknown and that is indicated by u64::MAX which lacks type-safety:
https://github.com/solana-labs/solana/blob/6c108c8fc/ledger/src/blockstore_meta.rs#L203-L205

The commit changes the type to Option<Slot>. Backward compatibility is
maintained by customizing serde serialize/deserialize implementations.
2021-12-14 18:57:11 +00:00
behzad nouri 4ceb2689f5
adds ShredId uniquely identifying each shred (#21820) 2021-12-14 17:34:02 +00:00
Ikko Ashimine e19c7923c3
chore: fix typo in shred.rs (#21890)
begining -> beginning
2021-12-14 11:19:27 -05:00
steviez 4a9d7318d1
Rework test parameters to be shreds instead of entries (#21780)
The number of shreds that result from a given number of entries is
variable and in our test case, somewhat unintuitive to think about when
trying to determine how much data we're pushing into the blockstore. So,
this change converts the unit of test parameters from entries to shreds.

This change also cleans up some variable naming for clarity and prints.
2021-12-13 23:34:43 -06:00
behzad nouri e08139f949
uses Option<u64> for SlotMeta.last_index (#21775)
SlotMeta.last_index may be unknown and current code is using u64::MAX to
indicate that:
https://github.com/solana-labs/solana/blob/6c108c8fc/ledger/src/blockstore_meta.rs#L169-L174

This lacks type-safety and can introduce bugs if not always checked for
Several instances of slot_meta.last_index + 1 are also subject to
overflow.

This commit updates the type to Option<u64>. Backward compatibility is
maintained by customizing serde serialize/deserialize implementations.
2021-12-11 14:47:20 +00:00
Justin Starry 254ef3e7b6
Rename Packets to PacketBatch (#21794) 2021-12-11 09:44:15 -05:00
behzad nouri 49ba09b333
adds back ErasureMeta::first_coding_index field (#21623)
https://github.com/solana-labs/solana/pull/16646
removed first_coding_index since the field is currently redundant and
always equal to fec_set_index.
However, with upcoming changes to erasure coding schema, this will no
longer be the same as fec_set_index and so requires a separate field to
represent.
2021-12-10 20:08:04 +00:00