Commit Graph

23990 Commits

Author SHA1 Message Date
Wen 295d610f43
We need to publish solana-wen-restart so we can publish 1.18.0 later. (#33662) 2023-10-11 11:27:09 -07:00
steviez 6009d49cc3
Remove dummy entries in Blockstore special columns (part 2) (#33649)
* Always call initialize_transaction_status_index() at startup,
  doing so will ensure dummy entries are actually cleaned
* Rename initialize_transaction_status_index()
* Stop initializing TransactionStatusIndex column entries, these
  are no longer needed and old software will initialize if needed
2023-10-11 13:13:10 -05:00
Jeff Biseda 0f82662a7f
allow empty string for SOLANA_METRICS_CONFIG sanity checking (#33515) 2023-10-11 09:58:39 -07:00
Brennan b36d051b51
Update hashes per tick with feature gates (#33600)
* Update hashes per tick with feature gates
2023-10-11 09:58:06 -07:00
Yihau Chen 15debcd6a1
chore: remove unused deps (#33652) 2023-10-11 16:54:13 +00:00
Will Hickey cd743dc496
Update the changelog for v1.18 (#33636)
* Update the changelog for v1.18

* Add sub-headings under v1.18.0
2023-10-11 10:06:38 -05:00
Brooks 99542d9b59
docs: Removes accounts-on-ramdisk section (#33655) 2023-10-11 10:39:09 -04:00
dependabot[bot] fa21a3d78e
build(deps): bump libc from 0.2.148 to 0.2.149 (#33653)
* build(deps): bump libc from 0.2.148 to 0.2.149

Bumps [libc](https://github.com/rust-lang/libc) from 0.2.148 to 0.2.149.
- [Release notes](https://github.com/rust-lang/libc/releases)
- [Commits](https://github.com/rust-lang/libc/compare/0.2.148...0.2.149)

---
updated-dependencies:
- dependency-name: libc
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2023-10-11 13:36:51 +00:00
Ryo Onodera 2f090a5882
Define PohRecorder set_bank related test helper methods (#33626)
Define PohRecorder set_bank related test methods
2023-10-11 10:34:39 +09:00
Lijun Wang 7006a6f94f
Reduce ConnectionPool size used by send-transaction-service (#33548)
Reduce pool size for ConnectionCache used by send-transaction-service to 2 from 4. No significant slow down of performance from bench-tps testing using rpc-client which is used by send-transaction-service. This will reduce active connections maintained both on the server and client. This will enable us to cache connections for more nodes.
2023-10-10 16:34:03 -07:00
Pankaj Garg c929775106
Enable running remote cargo registry server (#33629)
* Enable running remote cargo registry server

* use server URL to configure index
2023-10-10 22:54:34 +00:00
Wen 7c80fa1f67
Hide wen_restart and do not take default value for now. (#33637) 2023-10-10 15:05:55 -07:00
steviez 33e1dd71f3
Remove dated Blockstore PurgeType::PrimaryIndex code (#33631)
* Update instances of PurgeType::PrimaryIndex to PurgeType::Exact
* Remove now unused functions
* Remove unused active_transaction_status_index field
2023-10-10 16:35:42 -05:00
steviez 73a9a14731
Fold noisy metric into struct of metrics that is reported every 10s (#33635) 2023-10-10 16:35:16 -05:00
Kirill Fomichev a22678312e
Allow to create HTTP Sender with custom Client (#33580)
* Allow to create HTTP Sender with custom Client

* Update rpc-client/src/http_sender.rs

Co-authored-by: Tyera <teulberg@gmail.com>

---------

Co-authored-by: Tyera <teulberg@gmail.com>
2023-10-10 14:40:35 -06:00
steviez 982d29cf60
Revert "stop padding new append vecs to page size (#33607)" (#33634)
This reverts commit b7962a3610.
2023-10-10 14:43:52 -05:00
Jeff Biseda 8c27d8bbb3
indicate that test-checks.sh requires cargo-hack (#33519) 2023-10-10 12:08:14 -07:00
Pankaj Garg ad949b21b7
Fix failure in updating GIT index of cargo registry server (#33628) 2023-10-10 11:48:52 -07:00
Tyera 509d6acd2b
Remove primary index from Blockstore special-column keys (#33419)
* Add helper trait for column key deprecation

* Add WriteBatch::delete_raw

* Add ProtobufColumn::get_raw_protobuf_or_bincode

* Add ColumnIndexDeprecation iterator methods

* Impl ColumnIndexDeprecation for TransactionStatus (doesn't build)

* Update TransactionStatus put

* Update TransactionStatus purge_exact

* Fix read_transaction_status

* Fix get_transaction_status_with_counter

* Fix test_all_empty_or_min (builds except tests)

* Fix test_get_rooted_block

* Fix test_persist_transaction_status

* Fix test_get_transaction_status

* Fix test_get_rooted_transaction

* Fix test_get_complete_transaction

* Fix test_lowest_cleanup_slot_and_special_cfs

* Fix test_map_transactions_to_statuses

* Fix test_transaction_status_protobuf_backward_compatability

* Fix test_special_columns_empty

* Delete test_transaction_status_index

* Delete test_purge_transaction_status

* Ignore some tests until both special columns are dealt with (all build)

* Impl ColumnIndexDeprecation for AddressSignatures (doesn't build)

* Add BlockstoreError variant

* Update AddressSignatures put

* Remove unneeded active_transaction_status_index column lock

* Update AddressSignatures purge_exact

* Fix find_address_signatures_for_slot methods

* Fix get_block_signatures methods

* Fix get_confirmed_signatures_for_address2

* Remove unused method

* Fix test_all_empty_or_min moar (builds except tests)

* Fix tests (all build)

* Fix test_get_confirmed_signatures_for_address

* Fix test_lowest_cleanup_slot_and_special_cfs moar

* Unignore tests (builds except tests)

* Fix test_purge_transaction_status_exact

* Fix test_purge_front_of_ledger

* Fix test_purge_special_columns_compaction_filter (all build)

* Move some test-harness stuff around

* Add test cases for purge_special_columns_with_old_data

* Add test_read_transaction_status_with_old_data

* Add test_get_transaction_status_with_old_data

* Review comments

* Move rev of block-signatures into helper

* Improve deprecated_key impls

* iter_filtered -> iter_current_index_filtered

* Add comment to explain why use the smallest (index, Signature) to seed the iterator

* Impl ColumnIndexDeprecation for TransactionMemos (doesn't build)

* Update TransactionMemos put

* Add LedgerColumn::get_raw

* Fix read_transaction_memos

* Add TransactionMemos to purge_special_columns_exact

* Add TransactionMemos to compaction filter

* Take find_address_signatures out of service

* Remove faulty delete_new_column_key logic

* Simplify comments
2023-10-10 10:40:36 -06:00
Alexander Meißner cb695c7b32
Cleanup - Feature Gate of `enable_early_verification_of_account_modifications` (#33579)
Cleans up the feature gate of enable_early_verification_of_account_modifications:
- Removes PreAccount
- Removes InvokeContext::pre_accounts and InvokeContext::rent
- Removes InvokeContext::verify() and InvokeContext::verify_and_update()
- Removes TransactionContext::is_early_verification_of_account_modifications_enabled()
- Removes TransactionAccounts::is_early_verification_of_account_modifications_enabled
- No longer optional: TransactionContext::rent
2023-10-10 17:26:17 +02:00
dependabot[bot] dc522012c8
build(deps): bump proptest from 1.2.0 to 1.3.1 (#33621)
Bumps [proptest](https://github.com/proptest-rs/proptest) from 1.2.0 to 1.3.1.
- [Release notes](https://github.com/proptest-rs/proptest/releases)
- [Changelog](https://github.com/proptest-rs/proptest/blob/master/CHANGELOG.md)
- [Commits](https://github.com/proptest-rs/proptest/compare/v1.2.0...v1.3.1)

---
updated-dependencies:
- dependency-name: proptest
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-10-10 14:42:04 +00:00
Alexander Meißner 1262ff7589
Fix - CPI interface `bool` masking (#33623)
Adds masking of booleans in CPI interface to disable_cpi_setting_executable_and_rent_epoch.
2023-10-10 16:09:12 +02:00
Jeff Washington (jwash) b7962a3610
stop padding new append vecs to page size (#33607)
* stop padding new append vecs to page size

* for creating test accounts, allocate larger like we used to
2023-10-10 07:03:37 -07:00
Jeff Washington (jwash) 6c7d3c8aeb
remove redundant page_align (#33608) 2023-10-10 07:02:47 -07:00
dependabot[bot] 42b63a021f
build(deps): bump semver from 1.0.19 to 1.0.20 (#33622)
* build(deps): bump semver from 1.0.19 to 1.0.20

Bumps [semver](https://github.com/dtolnay/semver) from 1.0.19 to 1.0.20.
- [Release notes](https://github.com/dtolnay/semver/releases)
- [Commits](https://github.com/dtolnay/semver/compare/1.0.19...1.0.20)

---
updated-dependencies:
- dependency-name: semver
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2023-10-10 13:25:11 +00:00
dependabot[bot] 288e0b7536
build(deps): bump regex from 1.9.6 to 1.10.0 (#33620)
* build(deps): bump regex from 1.9.6 to 1.10.0

Bumps [regex](https://github.com/rust-lang/regex) from 1.9.6 to 1.10.0.
- [Release notes](https://github.com/rust-lang/regex/releases)
- [Changelog](https://github.com/rust-lang/regex/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/regex/compare/1.9.6...1.10.0)

---
updated-dependencies:
- dependency-name: regex
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2023-10-10 13:24:42 +00:00
Ryo Onodera 1704789247
Define tick related helper test methods (#33537)
* Define tick related helper methods

* dcou VoteSimulator

* blacklist ledger-tool for dcou

* fix dcou ci...

* github
2023-10-10 09:23:18 +09:00
Wen 0a3810854f
Add RestartLastVotedForkSlots for wen_restart. (#33239)
* Add RestartLastVotedForkSlots and RestartHeaviestFork for wen_restart.

* Fix linter errors.

* Revert RestartHeaviestFork, it will be added in another PR.

* Update frozen abi message.

* Fix wrong number in test generation, change to pub(crate) to limit scope.

* Separate push_epoch_slots and push_restart_last_voted_fork_slots.

* Add RestartLastVotedForkSlots data structure.

* Remove unused parts to make PR smaller.

* Remove unused clone.

* Use CompressedSlotsVec to share code between EpochSlots and RestartLastVotedForkSlots.

* Add total_messages to show how many messages are there.

* Reduce RestartLastVotedForkSlots to one packet (16k slots).

* Replace last_vote_slot with shred_version, revert CompressedSlotsVec.
2023-10-09 15:01:50 -07:00
Trent Nelson 55f3f203c6
bank: remove ambiguating "helper" method (#33606) 2023-10-09 21:47:21 +00:00
Pankaj Garg 4c664a8d31
Cargo registry service for deploying programs (#33570) 2023-10-09 14:32:35 -07:00
Pankaj Garg 2d84c1dbba
Fix deployment of program-v4 in freshly started test validator (#33583) 2023-10-09 13:41:35 -07:00
Brooks fc73813db2
Adds AccountHash newtype (#33597) 2023-10-09 16:00:52 -04:00
Jeff Washington (jwash) 052677595c
in hash calc, delete old cache files that will not be used earlier (#33432)
* in hash calc, delete old cache files that will not be used earlier

* only delete if supposed to

* fmt
2023-10-09 11:47:39 -07:00
Brooks c924719040
Removes hash param from AccountsCache::store() (#33598) 2023-10-09 18:16:15 +00:00
Wen 2d5496a564
Fix wen_restart proto compilation: (#33596)
* Fix wen_restart proto compilation:
- should recompile when proto changes
- no need for customization

* There is only one proto file, no need for loop.
2023-10-09 10:51:44 -07:00
behzad nouri 1d91b60a57
removes unused legacy-snapshot-hashes api in gossip (#33593)
https://github.com/solana-labs/solana/pull/33576
stops broadcasting legacy snapshot hashes over gossip, and this commit
removes unused legacy snapshot hashed code in gossip.
2023-10-09 15:22:34 +00:00
HaoranYi 72574dac02
Assert acount hash mmap file capacity > 0 (#33575)
assert mmap capacity > 0

Co-authored-by: HaoranYi <haoran.yi@solana.com>
2023-10-09 09:38:00 -05:00
dependabot[bot] 46cf79a9c4
build(deps): bump num-traits from 0.2.16 to 0.2.17 (#33589)
* build(deps): bump num-traits from 0.2.16 to 0.2.17

Bumps [num-traits](https://github.com/rust-num/num-traits) from 0.2.16 to 0.2.17.
- [Changelog](https://github.com/rust-num/num-traits/blob/master/RELEASES.md)
- [Commits](https://github.com/rust-num/num-traits/compare/num-traits-0.2.16...num-traits-0.2.17)

---
updated-dependencies:
- dependency-name: num-traits
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2023-10-09 14:21:09 +00:00
dependabot[bot] d74de6780e
build(deps): bump proc-macro2 from 1.0.67 to 1.0.69 (#33591)
* build(deps): bump proc-macro2 from 1.0.67 to 1.0.69

Bumps [proc-macro2](https://github.com/dtolnay/proc-macro2) from 1.0.67 to 1.0.69.
- [Release notes](https://github.com/dtolnay/proc-macro2/releases)
- [Commits](https://github.com/dtolnay/proc-macro2/compare/1.0.67...1.0.69)

---
updated-dependencies:
- dependency-name: proc-macro2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2023-10-09 13:42:44 +00:00
dependabot[bot] 1a2c0943db
build(deps): bump byteorder from 1.4.3 to 1.5.0 (#33590)
* build(deps): bump byteorder from 1.4.3 to 1.5.0

Bumps [byteorder](https://github.com/BurntSushi/byteorder) from 1.4.3 to 1.5.0.
- [Changelog](https://github.com/BurntSushi/byteorder/blob/master/CHANGELOG.md)
- [Commits](https://github.com/BurntSushi/byteorder/compare/1.4.3...1.5.0)

---
updated-dependencies:
- dependency-name: byteorder
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2023-10-09 13:40:45 +00:00
DimAn 7afb11f1e6
validator: skip health check (#33568)
* validator: skip health check

* keep `healthy` as a boolean
2023-10-08 16:33:53 -07:00
Ikko Eltociear Ashimine bd8cfc9923
fix typo in latest_validator_votes_for_frozen_banks.rs (#33585)
upate -> update
2023-10-08 18:12:52 +00:00
Brooks c588f25eca
Stops pushing legacy snapshot hashes to crds (#33576) 2023-10-07 14:29:42 -04:00
Ryo Onodera 95810d876a
Enable frozen_abi on banking trace file (#33501)
* Enable frozen_abi on banking trace file

* Fix ci with really correct bugfix...

* Remove tracker_callers

* Fix typo...

* Fix AbiExample for Arc/Rc's Weaks

* Added comment for AbiExample impl of SystemTime

* Simplify and document EvenAsOpaque with new usage

* Minor clean-ups

* Simplify SystemTime::example() with UNIX_EPOCH...

* Add comment for AbiExample subtleties
2023-10-07 13:15:38 +09:00
Wen 630feeddf2
Add wen_restart module (#33344)
* Add wen_restart module:
- Implement reading LastVotedForkSlots from blockstore.
- Add proto file to record the intermediate results.
- Also link wen_restart into validator.
- Move recreation of tower outside replay_stage so we can get last_vote.

* Update lock file.

* Fix linter errors.

* Fix depencies order.

* Update wen_restart explanation and small fixes.

* Generate tower outside tvu.

* Update validator/src/cli.rs

Co-authored-by: Tyera <teulberg@gmail.com>

* Update wen-restart/protos/wen_restart.proto

Co-authored-by: Tyera <teulberg@gmail.com>

* Update wen-restart/build.rs

Co-authored-by: Tyera <teulberg@gmail.com>

* Update wen-restart/src/wen_restart.rs

Co-authored-by: Tyera <teulberg@gmail.com>

* Rename proto directory.

* Rename InitRecord to MyLastVotedForkSlots, add imports.

* Update wen-restart/Cargo.toml

Co-authored-by: Tyera <teulberg@gmail.com>

* Update wen-restart/src/wen_restart.rs

Co-authored-by: Tyera <teulberg@gmail.com>

* Move prost-build dependency to project toml.

* No need to continue if the distance between slot and last_vote is
already larger than MAX_SLOTS_ON_VOTED_FORKS.

* Use 16k slots instead of 81k slots, a few more wording changes.

* Use AncestorIterator which does the same thing.

* Update Cargo.lock

* Update Cargo.lock

---------

Co-authored-by: Tyera <teulberg@gmail.com>
2023-10-06 15:04:37 -07:00
Tyera f075867ceb
Blockstore::get_sigs_for_addr2: ensure lowest_slot >= first_available_block (#33556)
* Set empty lowest_slot to first_available_block and remove check in loop

* Ensure get_transaction_status on_with_counter returns slots >= first_available_block

* Actually cleanup ledger
2023-10-06 15:12:08 -06:00
Sammy Harris 41ed9ab6d9
Fix: handle `-- --target-dir` arg in `cargo build-sbf` (#33555)
* fix: handle target dir override in build-sbf cargo args

* fix: refactor to canonicalize target arg for workspace absolute paths

* fix: nightly linting
2023-10-06 13:48:12 -07:00
Nick Frostbutter 77632daca5
[docs] removed >600 unused icon files (#33569)
refactor: removed icons since none are used
2023-10-06 15:24:28 -05:00
Brooks bb27bd88d4
Removes write version from tiered storage (#33566) 2023-10-06 16:19:35 -04:00
Yueh-Hsuan Chiang ecb1f8a9d7
[TieredStorage] Include Hot Account in StoredAccountMeta and ReadableAccount (#33544)
#### Problem
All account storage formats are required to implement both StoredAccountMeta
and ReadableAccount, but the implementation for the hot account format is missing.

#### Summary of Changes
This PR includes hot account format into StoredAccountMeta and ReadableAccount
enum.  This will allow the TieredStorageReader in the future PRs to return hot account
format in its `get_account` implementation.
2023-10-06 11:54:02 -07:00