Commit Graph

164 Commits

Author SHA1 Message Date
Jeff Washington (jwash) 288e8a682a
conditionally erase folders on drop of `BucketMap` (#33309) 2023-09-19 13:02:22 -07:00
Jeff Washington (jwash) 056e7cc240
bucket storage refactoring (#33308) 2023-09-19 13:01:49 -07:00
Alexander Meißner 9e703f85de
Upgrades Rust to 1.72.0 & nightly-2023-08-25 (#32961)
* allow pedantic invalid cast lint

* allow lint with false-positive triggered by `test-case` crate

* nightly `fmt` correction

* adapt to rust layout changes

* remove dubious test

* Use transmute instead of pointer cast and de/ref when check_aligned is false.

* Renames clippy::integer_arithmetic to clippy::arithmetic_side_effects.

* bump rust nightly to 2023-08-25

* Upgrades Rust to 1.72.0

---------

Co-authored-by: Trent Nelson <trent@solana.com>
2023-09-01 07:26:13 +00:00
Jeff Washington (jwash) 9a7d503fe4
rework sorting during generate_index (#33076)
* drag data len around during generate_index

* fix tests

* update comments
2023-08-31 12:51:56 -07:00
Brooks f8789c79a2
Refactor bucket storage bad header size test (#33091) 2023-08-31 13:01:20 -04:00
Jeff Washington (jwash) 551317dd31
return file name as u128 for disk buckets (#33044) 2023-08-29 08:16:54 -07:00
Jeff Washington (jwash) 9b4feddb55
add PartialEq and Debug to bucket map (#33043) 2023-08-29 07:13:21 -07:00
Alexander Meißner 29bbda0c11
General cleanup (Part 2) (#32998)
* bucket-map: remove unused `mut`

* remove unnecessary explicit `into_iter()` calls

* don't default-construct unit-structs

* prefer `or_default()` to `or_insert_with(T::default())`

* replace "slow zero-filled vec initialization"

---------

Co-authored-by: Trent Nelson <trent@solana.com>
2023-08-25 19:07:38 +02:00
Alexander Meißner a8be70fa7a
General cleanup (#32980)
* Fixes `cargo::` in build.rs

* Fixes overshadowing reexports.

* Removes unnecessary `as *const u8`.

* Removes unnecessary `.into_iter()`.

* cargo clippy
2023-08-24 21:44:19 +02:00
Jeff Washington (jwash) 3b5e35e819
disk bucket: factor out allocate_bucket (#32920) 2023-08-22 14:21:07 -07:00
Jon Cinque 0fe902ced7
Bump rand to 0.8, rand_chacha to 0.3, getrandom to 0.2 (#32871)
* sdk: Add concurrent support for rand 0.7 and 0.8

* Update rand, rand_chacha, and getrandom versions

* Run command to replace `gen_range`

Run `git grep -l gen_range | xargs sed -i'' -e 's/gen_range(\(\S*\), /gen_range(\1../'

* sdk: Fix users of older `gen_range`

* Replace `hash::new_rand` with `hash::new_with_thread_rng`

Run:
```
git grep -l hash::new_rand | xargs sed -i'' -e 's/hash::new_rand([^)]*/hash::new_with_thread_rng(/'
```

* perf: Use `Keypair::new()` instead of `generate`

* Use older rand version in zk-token-sdk

* program-runtime: Inline random key generation

* bloom: Fix clippy warnings in tests

* streamer: Scope rng usage correctly

* perf: Fix clippy warning

* accounts-db: Map to char to generate a random string

* Remove `from_secret_key_bytes`, it's just `keypair_from_seed`

* ledger: Generate keypairs by hand

* ed25519-tests: Use new rand

* runtime: Use new rand in all tests

* gossip: Clean up clippy and inline keypair generators

* core: Inline keypair generation for tests

* Push sbf lockfile change

* sdk: Sort dependencies correctly

* Remove `hash::new_with_thread_rng`, use `Hash::new_unique()`

* Use Keypair::new where chacha isn't used

* sdk: Fix build by marking rand 0.7 optional

* Hardcode secret key length, add static assertion

* Unify `getrandom` crate usage to fix linking errors

* bloom: Fix tests that require a random hash

* Remove some dependencies, try to unify others

* Remove unnecessary uses of rand and rand_core

* Update lockfiles

* Add back some dependencies to reduce rebuilds

* Increase max rebuilds from 14 to 15

* frozen-abi: Remove `getrandom`

* Bump rebuilds to 17

* Remove getrandom from zk-token-proof
2023-08-21 19:11:21 +02:00
behzad nouri d54b6204be
removes instances of clippy::manual_let_else (#32417) 2023-07-09 21:41:36 +00:00
Jeff Washington (jwash) 4f677ff4f8
remove compiler warning (#32081) 2023-06-12 13:48:17 -07:00
Jeff Washington (jwash) bace5679a1
avoid unnecessary disk bucket data file allocation (#31659) 2023-05-16 07:10:46 -07:00
Brooks 9a4618a049
bucket_map: Length must be 1 if including header in get_slice() (#31445) 2023-05-03 10:44:58 -04:00
Brooks 26809522b7
bucket_map: get_from_bytes should check alignment (#31444) 2023-05-02 15:17:36 -04:00
Brooks 1a6c75a115
bucket_map: Moves get_from_bytes() to index_entry.rs (#31443) 2023-05-02 16:58:16 +00:00
Brooks bac1b2a584
bucket_map: Renames fns to get_slice and get_slice_mut (#31442) 2023-05-02 10:49:25 -04:00
dependabot[bot] 2b2eccfff9
Bump num_enum from 0.5.9 to 0.6.1 (#31122)
* Bump num_enum from 0.5.9 to 0.6.0

Bumps [num_enum](https://github.com/illicitonion/num_enum) from 0.5.9 to 0.6.0.
- [Release notes](https://github.com/illicitonion/num_enum/releases)
- [Commits](https://github.com/illicitonion/num_enum/compare/0.5.9...0.6.0)

---
updated-dependencies:
- dependency-name: num_enum
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* Use workspace dep for bucket_map

* Bump num_enum to v0.6.1

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
2023-04-17 23:49:43 -06:00
Jeff Washington (jwash) 0121be5a9f
disk index: add more tests (#31135) 2023-04-11 07:30:11 -07:00
Jeff Washington (jwash) 4f0adcde89
disk index: correctly keep track of count during batch add (#31133) 2023-04-10 14:01:25 -07:00
Jeff Washington (jwash) d63359a3ff
disk index: batch insert (#31094) 2023-04-10 11:08:54 -07:00
Jeff Washington (jwash) 03abaf76d0
disk index: bucket_index_ix doesn't % by capacity (#31096) 2023-04-07 06:59:35 -07:00
Jeff Washington (jwash) 3422823932
disk index: keep same random during resize (#31095) 2023-04-07 06:54:50 -07:00
Jeff Washington (jwash) 06461fb348
disk index: index files grow by bytes instead of doubling (#31013) 2023-04-05 06:30:37 -07:00
Jeff Washington (jwash) 9fb22bc0be
disk index: store ref_count in data file (#30974) 2023-04-04 19:17:48 -07:00
Jeff Washington (jwash) fd28fd1d34
disk index: move capacity to contents (#31040) 2023-04-04 13:08:13 -07:00
Jeff Washington (jwash) aa3e0b9c20
disk index: add stat: disk_index_failed_resizes (#31039) 2023-04-04 09:39:06 -07:00
Jeff Washington (jwash) 514816a31f
disk index: move key enum to bitvec (#31026) 2023-04-04 08:21:29 -07:00
Jeff Washington (jwash) b0540ff5ad
disk index: add `Capacity` (#31031)
* disk index: add `Capacity`

* pr feedback
2023-04-04 06:57:09 -07:00
Jeff Washington (jwash) bc343a431c
disk index: set_anticipated_count to optimally grow disk buckets at startup (#31033)
* disk index: set_anticipated_count to optimally grow disk buckets at startup

* remove atomic
2023-04-03 20:28:50 -07:00
Jeff Washington (jwash) 05f2a01a80
disk index: refactor OccupiedEnumTag (#31028) 2023-04-03 13:12:06 -07:00
Jeff Washington (jwash) f289a0726b
disk index: refactoring get/get_mut fns (#31025) 2023-04-03 11:40:49 -07:00
Jeff Washington (jwash) 7edef94088
disk index: remove Option on read_value (#31023) 2023-04-03 11:36:58 -07:00
Jeff Washington (jwash) ef6b6793dd
disk bucket: add trait fn copying_entry (#31024) 2023-04-03 11:35:15 -07:00
Jeff Washington (jwash) 9219a16c42
disk bucket: new_map takes bytes (#31020) 2023-04-03 06:57:06 -07:00
Jeff Washington (jwash) 8051aea88e
disk index: store single slot list in index entry (#31002) 2023-04-03 06:51:59 -07:00
Jeff Washington (jwash) fc2bcdffe2
fix up refcounts on disk bucket test (#31001) 2023-03-31 12:37:21 -07:00
Jeff Washington (jwash) a2797ebfa9
disk index: use bits in ref count to store occupied (#31004) 2023-03-31 10:50:55 -07:00
Jeff Washington (jwash) db5cdbedae
disk index remove dead code (#30999) 2023-03-31 09:08:16 -07:00
Jeff Washington (jwash) 3f6c33d7e2
disk index: break ref_count u64 into reserved bits (#30997) 2023-03-31 06:51:00 -07:00
Jeff Washington (jwash) 32780353c7
disk index: move fn data_loc to MultipleSlots (#30993)
disk index: move fn `data_loc` to `MultipleSlots`
2023-03-30 16:23:57 -07:00
Brooks 399b7ba722
Removes unnecessary Sized bounds in bucket map (#30990) 2023-03-30 16:34:59 -04:00
Jeff Washington (jwash) 89f7d4ab9a
Move `data_bucket_ix()` method to `MultipleSlots` (#30992)
move data_bucket_ix to MultipleSlots
2023-03-30 13:20:10 -07:00
Jeff Washington (jwash) b5afcbb72f
disk index: move num_slots() to MultipleSlots (#30991) 2023-03-30 12:39:15 -07:00
Jeff Washington (jwash) 2672637395
disk index: move fns to MultipleSlots (#30989) 2023-03-30 12:00:43 -07:00
Jeff Washington (jwash) 2cafd641fe
disk index: introduce MultipleSlots (#30988)
* disk index: introduce MultipleSlots

* pr feedback
2023-03-30 11:27:45 -07:00
Brooks eb47e44c6b
disk bucket: refactor storage get/get_mut fns (#30985) 2023-03-30 12:38:35 -04:00
Jeff Washington (jwash) 5ae3288a0b
disk index IndexEntry gets <T> (#30984)
* disk index IndexEntry gets <T>

* remove unused phantom
2023-03-30 09:23:51 -07:00
Jeff Washington (jwash) e6ca734ac4
disk index: include T on a few structs (#30983) 2023-03-30 07:44:50 -07:00