Pack terms once when `terms` query is created for indexed and docValues… by mkhludnev · Pull Request #19350 · opensearch-project/OpenSearch

mkhludnev · 2025-09-20T21:20:52Z

… keyword field.

Description

Pack terms once when terms query is created for indexed and docValues keywords field.

Related Issues

It's a continuation of #17714 made possible by lucene#14425

Check List

[v] Functionality includes testing.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

… keyword field. Signed-off-by: Mikhail Khludnev <mkhl@apache.org>

Signed-off-by: Mikhail Khludnev <mkhludnev@users.noreply.github.com>

mkhludnev · 2025-09-20T21:36:30Z

fwiw this optimization doesn't work for flattened field type.

Signed-off-by: Mikhail Khludnev <mkhl@apache.org>

github-actions · 2025-09-20T23:54:54Z

❌ Gradle check result for b8819fd: null

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

mkhludnev · 2025-09-21T07:49:15Z

something is wrong with failed job

{"pr_from_sha": "b8819fd815e4082ca79c799602369fccb10452a8", "pr_from_clone_url": "https://github.com/mkhludnev/OpenSearch.git", "pr_to_clone_url": "https://github.com/opensearch-project/OpenSearch.git", "pr_title": "pack terms once when terms query is created for indexed and docvalues", "pr_number": "19350", "post_merge_action": "false", "pr_owner": "mkhludnev"}
<html>
 <head><title>403 Forbidden</title></head>
 <body>
 <center><h1>403 Forbidden</h1></center>
 </body>
 </html>
jq: parse error: Invalid numeric literal at line 1, column 7
QUEUE_URL

Signed-off-by: Mikhail Khludnev <mkhl@apache.org>

github-actions · 2025-09-22T21:01:58Z

❌ Gradle check result for c5d23c9: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

mkhludnev · 2025-09-23T07:32:25Z

reproduced failure StreamStringTermsAggregatorTests.testReduceWithSizeLimit locally. looking into

UPD ok. it seems unrelated.

Signed-off-by: Mikhail Khludnev <mkhludnev@users.noreply.github.com>

github-actions · 2025-09-26T09:30:23Z

❌ Gradle check result for e65a5c1: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Signed-off-by: Mikhail Khludnev <mkhludnev@users.noreply.github.com>

github-actions · 2025-10-08T08:54:10Z

✅ Gradle check result for e2828ae: SUCCESS

codecov · 2025-10-08T08:54:50Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 73.01%. Comparing base (39b7a59) to head (e2828ae).
⚠️ Report is 8 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff            @@
##               main   #19350   +/-   ##
=========================================
  Coverage     73.00%   73.01%           
- Complexity    70534    70542    +8     
=========================================
  Files          5719     5719           
  Lines        323260   323272   +12     
  Branches      46816    46821    +5     
=========================================
+ Hits         235993   236030   +37     
+ Misses        68224    68188   -36     
- Partials      19043    19054   +11

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

jainankitk

Thanks @mkhludnev for patiently waiting for apache/lucene#14435 to be available as part of 10.3 upgrade and adding this improvement!

server/src/main/java/org/opensearch/index/mapper/KeywordFieldMapper.java

…es… (opensearch-project#19350) Signed-off-by: Mikhail Khludnev <mkhl@apache.org> Signed-off-by: Mikhail Khludnev <mkhludnev@users.noreply.github.com>

mkhludnev requested review from Bukhtawar, CEHENKLE, anasalkouz, andrross, ashking94, cwperks, dbwiddis, gbbafna, jed326 and kotwanikunal as code owners September 20, 2025 21:20

opensearch-infra bot added the lucene label Sep 20, 2025

mkhludnev requested review from a team, Rishikesh1159, mch2, msfroh, owaiskazi19, reta, sachinpkale, saratvemulapalli, shwetathareja and sohami as code owners September 20, 2025 21:20

mkhludnev force-pushed the keyword-idx-dv-termsQ-sort-once branch from 7f1cd7e to 9e66c4a Compare September 20, 2025 21:32

Sort terms once when terms query is created for indexed and docValues…

e8f9e61

… keyword field. Signed-off-by: Mikhail Khludnev <mkhl@apache.org>

mkhludnev force-pushed the keyword-idx-dv-termsQ-sort-once branch from 9e66c4a to e8f9e61 Compare September 20, 2025 21:33

Merge branch 'main' into keyword-idx-dv-termsQ-sort-once

b146489

Signed-off-by: Mikhail Khludnev <mkhludnev@users.noreply.github.com>

mkhludnev changed the title ~~Sort terms once when terms query is created for indexed and docValues…~~ Pack pre-sorted terms once when terms query is created for indexed and docValues… Sep 20, 2025

mkhludnev changed the title ~~Pack pre-sorted terms once when terms query is created for indexed and docValues…~~ Pack terms once when terms query is created for indexed and docValues… Sep 20, 2025

Clarify CHANGELOG.md

b8819fd

Signed-off-by: Mikhail Khludnev <mkhl@apache.org>

opensearch-ci-bot mentioned this pull request Sep 20, 2025

[AUTOCUT] Gradle Check Flaky Test Report for IngestFromKinesisIT #17678

Open

more testing

c5d23c9

Signed-off-by: Mikhail Khludnev <mkhl@apache.org>

This was referenced Sep 22, 2025

[AUTOCUT] Gradle Check Flaky Test Report for StreamStringTermsAggregatorTests #18956

Closed

[AUTOCUT] Gradle Check Flaky Test Report for WlmAutoTaggingIT #19354

Closed

Merge branch 'main' into keyword-idx-dv-termsQ-sort-once

e65a5c1

Signed-off-by: Mikhail Khludnev <mkhludnev@users.noreply.github.com>

Merge branch 'main' into keyword-idx-dv-termsQ-sort-once

e2828ae

Signed-off-by: Mikhail Khludnev <mkhludnev@users.noreply.github.com>

jainankitk approved these changes Oct 9, 2025

View reviewed changes

server/src/main/java/org/opensearch/index/mapper/KeywordFieldMapper.java Show resolved Hide resolved

jainankitk merged commit 9f7c73b into opensearch-project:main Oct 9, 2025
33 checks passed

opensearch-ci-bot mentioned this pull request Oct 22, 2025

[AUTOCUT] Gradle Check Flaky Test Report for AllocationConstraintsTests #15831

Closed

BrewTestBot mentioned this pull request Dec 17, 2025

opensearch 3.4.0 Homebrew/homebrew-core#258986

Merged

This was referenced Jan 11, 2026

[feature] Terms Query Optimization tkykenmt/opensearch-feature-explorer#1694

Closed

docs: add Terms Query Optimization report for v3.4.0 tkykenmt/opensearch-feature-explorer#1788

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Pack terms once when `terms` query is created for indexed and docValues…#19350

Pack terms once when `terms` query is created for indexed and docValues…#19350
jainankitk merged 6 commits intoopensearch-project:mainfrom
mkhludnev:keyword-idx-dv-termsQ-sort-once

mkhludnev commented Sep 20, 2025 •

edited

Loading

Uh oh!

mkhludnev commented Sep 20, 2025

Uh oh!

github-actions bot commented Sep 20, 2025

Uh oh!

mkhludnev commented Sep 21, 2025

Uh oh!

github-actions bot commented Sep 22, 2025

Uh oh!

mkhludnev commented Sep 23, 2025 •

edited

Loading

Uh oh!

github-actions bot commented Sep 26, 2025

Uh oh!

github-actions bot commented Oct 8, 2025

Uh oh!

codecov bot commented Oct 8, 2025 •

edited

Loading

Uh oh!

jainankitk left a comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

mkhludnev commented Sep 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Related Issues

Check List

Uh oh!

mkhludnev commented Sep 20, 2025

Uh oh!

github-actions bot commented Sep 20, 2025

Uh oh!

mkhludnev commented Sep 21, 2025

Uh oh!

github-actions bot commented Sep 22, 2025

Uh oh!

mkhludnev commented Sep 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Sep 26, 2025

Uh oh!

github-actions bot commented Oct 8, 2025

Uh oh!

codecov bot commented Oct 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

jainankitk left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

mkhludnev commented Sep 20, 2025 •

edited

Loading

mkhludnev commented Sep 23, 2025 •

edited

Loading

codecov bot commented Oct 8, 2025 •

edited

Loading