Skip to content

Randomise the query order in the MS Marco V2 param supplier#1025

Merged
davidkyle merged 3 commits intoelastic:masterfrom
davidkyle:shuffle
May 8, 2026
Merged

Randomise the query order in the MS Marco V2 param supplier#1025
davidkyle merged 3 commits intoelastic:masterfrom
davidkyle:shuffle

Conversation

@davidkyle
Copy link
Copy Markdown
Member

If a challenge contains many search operations using this supplier and the supplier is recreated each time then the same hot vectors at the beginning of the file will be used every time. This change shuffles the vectors read from the file.

Comment thread msmarco-v2-vector/track.py Outdated
Copy link
Copy Markdown
Contributor

@john-wagster john-wagster left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@davidkyle davidkyle merged commit b304cd4 into elastic:master May 8, 2026
15 checks passed
@esbenchmachine esbenchmachine added the backport pending Awaiting backport to stable release branch label May 8, 2026
@esbenchmachine
Copy link
Copy Markdown
Collaborator

@davidkyle
A backport is pending for this PR.
Apply all the labels that correspond to Elasticsearch minor versions expected to work with this PR, but select only from the available ones.
If intended for future releases, apply label for next minor

If the PR is only applicable for serverless, add the vServerless label.

When a vX.Y label is added, a new pull request will be automatically created, unless merge conflicts are detected or if the label supplied points to the next Elasticsearch minor version. If successful, a link to the newly opened backport PR will be provided in a comment.

In case of merge conflicts during backporting, create the backport PR manually following the steps from README:
Final steps to complete the backporting process:

  1. Ensure the correct version labels exist in this PR.
  2. Ensure each backport pull request is labeled with backport.
  3. Review and merge each backport pull request into the appropriate version branch.
  4. Remove backport pending label from this PR once all backport PRs are merged. Note: a version label must be added, otherwise the bot will add back the ´backport pending´ label.

Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport pending Awaiting backport to stable release branch

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants