Commit Graph

58 Commits

Author SHA1 Message Date
Jeff Washington (jwash) 823d55ea9f
disk index find_entry -> find_index_entry (#30941) 2023-03-29 06:54:03 -07:00
Jeff Washington (jwash) 3ebfda5aef
add stat for disk buckets flushed (#30850) 2023-03-22 15:04:17 -07:00
Jeff Washington (jwash) df2c95119c
add buckets stats for # files and total file size (#30780)
* add buckets stats for # files and total file size

* change metrics to bytes

* get index file size correct
2023-03-20 08:22:31 -07:00
Jeff Washington (jwash) b7d05dd755
consolidate held in mem stats (#30574) 2023-03-03 10:05:05 -08:00
Jeff Washington (jwash) 590ee58667
wip: throttle correctly (#30547)
throttle correctly
2023-03-01 17:08:46 -08:00
Jeff Washington (jwash) 7c086ca42d
add DiskIndexValue trait for disk index (#30503) 2023-02-24 11:02:06 -08:00
Jeff Washington (jwash) e39626ab14
add second type to accounts index for disk (#30446) 2023-02-23 13:05:06 -08:00
Jeff Washington (jwash) db8764f98d
hold items with ref count != 1 in memory (#30414)
* hold items with ref count != 1 in memory

* fix tests
2023-02-22 06:57:59 -08:00
HaoranYi d3daea65b9
use atomicu64 for bucket maxsize instead of lock (#29569) 2023-01-10 09:15:27 -06:00
behzad nouri 5c9beef498
fixes errors from clippy::useless_conversion (#29534)
https://rust-lang.github.io/rust-clippy/master/index.html#useless_conversion
2023-01-05 18:05:32 +00:00
Brooks Prumo d1ba42180d
clippy for rust 1.65.0 (#28765) 2022-11-09 19:39:38 +00:00
Jeff Washington (jwash) 6f5995c841
add stats for disk_index_find_entry_mut_us (#26753) 2022-07-25 14:47:17 -05:00
Jeff Washington (jwash) 8e2bae7bf9
batch insert updates stat count correctly (#26107) 2022-06-22 08:45:42 -05:00
Jeff Washington (jwash) 7a3dc3a57b
remove unused param (#26096) 2022-06-21 13:10:41 -05:00
HaoranYi 28142f8a6d
Refactor: separate delete and insert account index stats (#25800)
* fix double count of account index del stats

* refactor to seperate delete and insert account index stats

* fix test

* increase test timing tolerance for bucket age

* Revert "increase test timing tolerance for bucket age"

This reverts commit 2b91cf902208e4585d25dff1d4298d412c9da516.
2022-06-08 09:05:37 -05:00
Jeff Washington (jwash) e8df90c34b
rename count_buckets_flushed (#25733) 2022-06-02 12:16:01 -05:00
Jeff Washington (jwash) 89ba3ff139
log fail to evict (#23815) 2022-03-22 09:19:38 -05:00
Jeff Washington (jwash) dd69f3baf5
throttle index adding to allow disk flushing to keep up and reduce startup ram usage (#23773) 2022-03-20 19:56:20 -05:00
Jeff Washington (jwash) df29276eb0
AcctIdx: remove -> evict (#23775) 2022-03-18 17:13:21 -05:00
Jeff Washington (jwash) 857576d76f
AcctIdx: move write to disk outside in mem write lock (#23731) 2022-03-17 23:09:41 -05:00
Jeff Washington (jwash) 3a46f45650
AcctIdx: add stats for flushing and estimated memory (#23709) 2022-03-17 08:46:00 -05:00
Jeff Washington (jwash) 729698e815
AcctIdx: items() uses held ranges (#21954) 2021-12-17 09:59:29 -06:00
Jeff Washington (jwash) 6374995522
AcctIdx: share bucket map size for perf (#21935) 2021-12-16 21:25:54 -06:00
Jeff Washington (jwash) 1e0d3f13e6
AcctIdx: fix metrics bug (#21934) 2021-12-15 17:05:38 -06:00
Michael Vines b8837c04ec Reformat imports to a consistent style for imports
rustfmt.toml configuration:
  imports_granularity = "One"
  group_imports = "One"
2021-12-03 09:19:13 -08:00
Jeff Washington (jwash) 7ec88226ee
AcctIdx: stats are usize (#21493)
* AcctIdx: stats are usize

* rename test function
2021-11-30 11:35:21 -06:00
Jeff Washington (jwash) a762066310
AcctIdx: don't log disk stats when disk isn't enabled (#21371) 2021-11-22 15:26:25 -06:00
Jeff Washington (jwash) a272e19f8d
AcctIdx: report on partial progress during flushing (#21306) 2021-11-18 11:01:19 -06:00
Jeff Washington (jwash) 83de2f7376
fix initial massive metrics spike (#21304) 2021-11-17 17:16:48 -06:00
Jeff Washington (jwash) f8dcb2f38b
report mem stats (#21258) 2021-11-13 00:59:41 +00:00
Jeff Washington (jwash) 6a1ff60c61
percent stats 0-1 -> 0-100 (#21028) 2021-10-27 18:08:27 -05:00
Jeff Washington (jwash) 4d6190a1ba
change bg waiting metrics to percent (#20955)
* Revert part of "AcctIdx: cleanup bg threads (#20731)"

This reverts part of commit 47de4f31b6.

* change bg waiting metrics to percent
2021-10-26 15:09:17 -05:00
Jeff Washington (jwash) 5a6d732d48
AcctIdx: separate account index by startup and !startup (#20711) 2021-10-15 13:04:32 -05:00
Jeff Washington (jwash) 7631011d8c
AcctIdx: add flush remove stat (#20625) 2021-10-12 16:13:32 -05:00
Jeff Washington (jwash) 0da677e213
AcctIdx: write disk only when removing from mem (#20578) 2021-10-11 14:47:06 -05:00
Jeff Washington (jwash) 33d8c07364
AcctIdx: combine scan and update loops (#20546) 2021-10-10 13:47:52 -05:00
Jeff Washington (jwash) 4f6a0b2650
AcctIdx: stats for buckets on disk, add median (#20528) 2021-10-08 13:58:38 -05:00
Jeff Washington (jwash) 2c3d52b4cc
AcctIdx: avoid overcounting inserts (#20544) 2021-10-08 13:57:47 -05:00
Jeff Washington (jwash) 0d98a91511
AcctIdx: metric for bucket grow time (#20445) 2021-10-05 20:22:01 +00:00
Jeff Washington (jwash) 59112adac1
AcctIdx: us stats were showing ms (#20324) 2021-09-29 22:18:07 +00:00
Jeff Washington (jwash) bd9b2f6f39
AcctIdx: throttling bg flushing to spread out work over interval (#20176)
* AcctIdx: throttling bg flushing to spread out work over interval

* prevent hang
2021-09-29 07:42:54 -05:00
Jeff Washington (jwash) 46263d405e
fix math error with stats calc (#20253) 2021-09-27 11:55:19 -05:00
Jeff Washington (jwash) d8f98b3dcb
AcctIdx: avoid crazy #s on ms_per_age stat (#20184) 2021-09-27 08:25:23 -05:00
Jeff Washington (jwash) c6d189dac2
bg threads wait until next expected event (#20126) 2021-09-24 16:00:41 -05:00
Jeff Washington (jwash) 093dd68214
AcctIdx: stats for how much time bg threads wait (#20127) 2021-09-23 18:37:14 +00:00
Jeff Washington (jwash) 5bbb0da7b8
AcctIdx: stats for in-mem (#20114) 2021-09-23 08:14:53 -05:00
Jeff Washington (jwash) cb9d93525b
AcctIdx: metrics for loading from disk (#20124) 2021-09-23 08:14:08 -05:00
Jeff Washington (jwash) 476124de50
AcctIdx: report disk bucket stats (#20120) 2021-09-22 18:17:20 -05:00
Jeff Washington (jwash) 4fb77183ef
AcctIdx: move background() to bucket holder (#20085) 2021-09-22 17:40:30 +00:00
Jeff Washington (jwash) deab944b30
AcctIdx: flush inserts to disk (#20074) 2021-09-22 09:55:40 -05:00