Skip to content

[minmax] 8-bit benchmark #959

Open
arkrishn94 wants to merge 4 commits intomainfrom
u/adkrishnan/minmax-8bit-benchmark
Open

[minmax] 8-bit benchmark #959
arkrishn94 wants to merge 4 commits intomainfrom
u/adkrishnan/minmax-8bit-benchmark

Conversation

@arkrishn94
Copy link
Copy Markdown
Contributor

Add support for 8-bit queries in the exhaustive benchmark for minmax in diskann-benchmark. Here is a sample result -

Dataset:  958,152 vectors, dim=896, float16
Distance: cosine    Transform: double_hadamard(same)    Threads: 12

+----------+----------------+--------------------+-------------+
| num_bits | query_layout   | mean search (us)   | R@10/10 (%) |
+----------+----------------+--------------------+-------------+
|    2     | full_precision |       233,416      |    56.58    |
|    2     | eight_bit      |       222,594      |    56.60    |
|    4     | full_precision |       337,161      |    88.84    |
|    4     | eight_bit      |       195,933      |    88.87    |
+----------+----------------+--------------------+-------------+

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds an eight_bit query layout option for the MinMax exhaustive benchmark in diskann-benchmark, enabling benchmarking where queries are compressed to 8-bit even when the indexed data uses a different MinMax bit-rate.

Changes:

  • Extend MinMaxQuery with a new EightBit variant and include it in the MinMax exhaustive example input defaults.
  • Update the MinMax exhaustive backend to build distance computers for 8-bit-compressed queries (Data<8>).
  • Update the minmax-exhaustive.json example to include "eight_bit" in some jobs and adjust several job parameters.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 5 comments.

File Description
diskann-benchmark/src/inputs/exhaustive.rs Adds MinMaxQuery::EightBit and includes it in the MinMax example query_layouts.
diskann-benchmark/src/backend/exhaustive/minmax.rs Adds support for 8-bit query compression in the exhaustive MinMax distance computer creation path.
diskann-benchmark/example/minmax-exhaustive.json Updates example jobs to include eight_bit in some cases and changes several job distances/query_layout sets.
Comments suppressed due to low confidence (1)

diskann-benchmark/src/backend/exhaustive/minmax.rs:624

  • The new MinMaxQuery::EightBit path supports multiple distance measures at runtime, but the updated example/minmax-exhaustive.json only exercises eight_bit for squared_l2. To avoid regressions in the inner_product/cosine branches (which are easy to break with trait-bound changes), add at least one integration example job that uses query_layouts: ["eight_bit", ...] with inner_product and/or cosine, or add a focused test that runs those combinations.
                MinMaxQuery::EightBit => {
                    let mut compressed = Data::<8>::new_boxed(output_dim);
                    quantizer.compress_into(query, compressed.reborrow_mut())?;

                    match self.measure {
                        SimilarityMeasure::SquaredL2 => {
                            let inner: MinMaxL2Squared = quantizer.as_functor();
                            Ok(Boxed::new(Curried::new(inner, compressed)))

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread diskann-benchmark/example/minmax-exhaustive.json
Comment thread diskann-benchmark/example/minmax-exhaustive.json Outdated
Comment thread diskann-benchmark/example/minmax-exhaustive.json Outdated
Comment thread diskann-benchmark/src/backend/exhaustive/minmax.rs Outdated
Comment thread diskann-benchmark/example/minmax-exhaustive.json Outdated
@codecov-commenter
Copy link
Copy Markdown

codecov-commenter commented Apr 19, 2026

Codecov Report

❌ Patch coverage is 0% with 6 lines in your changes missing coverage. Please review.
✅ Project coverage is 89.32%. Comparing base (20295a4) to head (37874d2).
⚠️ Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
diskann-benchmark/src/inputs/exhaustive.rs 0.00% 6 Missing ⚠️
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #959      +/-   ##
==========================================
- Coverage   89.32%   89.32%   -0.01%     
==========================================
  Files         448      447       -1     
  Lines       83563    83260     -303     
==========================================
- Hits        74645    74369     -276     
+ Misses       8918     8891      -27     
Flag Coverage Δ
miri 89.32% <0.00%> (-0.01%) ⬇️
unittests 89.16% <0.00%> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
diskann-benchmark/src/backend/exhaustive/minmax.rs 100.00% <ø> (ø)
diskann-benchmark/src/inputs/exhaustive.rs 28.92% <0.00%> (-0.53%) ⬇️

... and 11 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants