Skip to content

[Backport 2.x] Adds various Query overrides to Keyword Field (#10425)#11031

Merged
msfroh merged 3 commits intoopensearch-project:2.xfrom
harshavamsi:backport/backport-10425-to-2.x
Nov 1, 2023
Merged

[Backport 2.x] Adds various Query overrides to Keyword Field (#10425)#11031
msfroh merged 3 commits intoopensearch-project:2.xfrom
harshavamsi:backport/backport-10425-to-2.x

Conversation

@harshavamsi
Copy link
Copy Markdown
Contributor

@harshavamsi harshavamsi commented Oct 31, 2023 •
edited
Loading

Backport of #10425

The keywordfield mapper provides access to various query types, e.g. the termsQuery, fuzzyQuery. These are inherited as is from the StringType. But we do not take into account the fact that keyword fields can have doc_values enabled. This PR adds the ability for various queries to first check if doc_values are enabled and if so out-source the work to lucene to decide if it's better to use index values or doc_values when running queries.

Signed-off-by: Harsha Vamsi Kalluri harshavamsi096@gmail.com
(cherry picked from commit 63aff16)

Description

[Describe what this change achieves]

Related Issues

Resolves #[Issue number to be closed when this PR is merged]

Check List

  • New functionality includes testing.
    • All tests pass
  • New functionality has been documented.
    • New functionality has javadoc added
  • Failing checks are inspected and point to the corresponding known issue(s) (See: Troubleshooting Failing Builds)
  • Commits are signed per the DCO using --signoff
  • Commit changes are listed out in CHANGELOG.md file (See: Changelog)
  • Public documentation issue/PR created

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@harshavamsi
Adds various Query overrides to Keyword Field (opensearch-project#10425)
53db98a 
The keywordfield mapper provides access to various query types, e.g. the termsQuery, fuzzyQuery. These are inherited as is from the StringType. But we do not take into account the fact that keyword fields can have doc_values enabled. This PR adds the ability for various queries to first check if doc_values are enabled and if so out-source the work to lucene to decide if it's better to use index values or doc_values when running queries.

Signed-off-by: Harsha Vamsi Kalluri <harshavamsi096@gmail.com>
(cherry picked from commit 63aff16)
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Oct 31, 2023

Gradle Check (Jenkins) Run Completed with:

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Oct 31, 2023 •
edited
Loading

Compatibility status:

Checks if related components are compatible with change 41d1e4f

Incompatible components

Incompatible components: [https://github.com/opensearch-project/performance-analyzer.git]

Skipped components

Compatible components

Compatible components: [https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/reporting.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/security-analytics.git, https://github.com/opensearch-project/cross-cluster-replication.git, https://github.com/opensearch-project/custom-codecs.git, https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/performance-analyzer-rca.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/neural-search.git]

@harshavamsi
Copy link
Copy Markdown
Contributor Author

harshavamsi commented Oct 31, 2023 

com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.opensearch.repositories.s3.S3BlobContainerRetriesTests: 
   1) Thread[id=38, name=sdk-ScheduledExecutor-2-0, state=TIMED_WAITING, group=TGRP-S3BlobContainerRetriesTests]
        at java.base@17.0.8/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@17.0.8/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:252)
        at java.base@17.0.8/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:1672)
        at java.base@17.0.8/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182)
        at java.base@17.0.8/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)
        at java.base@17.0.8/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1062)
        at java.base@17.0.8/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1122)
        at java.base@17.0.8/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
        at java.base@17.0.8/java.lang.Thread.run(Thread.java:833)

@harshavamsi
Copy link
Copy Markdown
Contributor Author

harshavamsi commented Oct 31, 2023

@msfroh backport PR.

msfroh
msfroh reviewed Oct 31, 2023
View reviewed changes
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Oct 31, 2023

Gradle Check (Jenkins) Run Completed with:

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Oct 31, 2023

Gradle Check (Jenkins) Run Completed with:

  • RESULT: UNSTABLE ❕
  • TEST FAILURES:
      1 org.opensearch.search.SearchWeightedRoutingIT.testMultiGetWithNetworkDisruption_FailOpenEnabled

@codecov
Copy link
Copy Markdown

codecov bot commented Oct 31, 2023 •
edited
Loading

Codecov Report

Merging #11031 (41d1e4f) into 2.x (e297b21) will increase coverage by 0.02%.
The diff coverage is 67.00%.

@@             Coverage Diff              @@
##                2.x   #11031      +/-   ##
============================================
+ Coverage     70.86%   70.88%   +0.02%     
+ Complexity    58864    58837      -27     
============================================
  Files          4850     4850              
  Lines        277900   277998      +98     
  Branches      40735    40765      +30     
============================================
+ Hits         196937   197065     +128     
+ Misses        64238    64159      -79     
- Partials      16725    16774      +49
Files Coverage Δ
...a/org/opensearch/index/mapper/MappedFieldType.java 74.28% <66.66%> (-2.49%) ⬇️
...ch/fetch/subphase/highlight/CustomQueryScorer.java 29.03% <0.00%> (-2.01%) ⬇️
...rg/opensearch/index/mapper/KeywordFieldMapper.java 80.53% <68.47%> (-6.10%) ⬇️

... and 465 files with indirect coverage changes

@harshavamsi
Merge branch '2.x' into backport/backport-10425-to-2.x
41d1e4f 
Signed-off-by: Harsha Vamsi Kalluri <harshavamsi096@gmail.com>
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Nov 1, 2023

Gradle Check (Jenkins) Run Completed with:

@msfroh msfroh merged commit 0d30590 into opensearch-project:2.x Nov 1, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@msfroh msfroh msfroh approved these changes

@reta reta Awaiting requested review from reta reta is a code owner

@anasalkouz anasalkouz Awaiting requested review from anasalkouz anasalkouz is a code owner

@andrross andrross Awaiting requested review from andrross andrross is a code owner

@Bukhtawar Bukhtawar Awaiting requested review from Bukhtawar Bukhtawar is a code owner

@CEHENKLE CEHENKLE Awaiting requested review from CEHENKLE CEHENKLE is a code owner

@dblock dblock Awaiting requested review from dblock

@gbbafna gbbafna Awaiting requested review from gbbafna gbbafna is a code owner

@setiah setiah Awaiting requested review from setiah

@kartg kartg Awaiting requested review from kartg

@kotwanikunal kotwanikunal Awaiting requested review from kotwanikunal kotwanikunal is a code owner

@mch2 mch2 Awaiting requested review from mch2 mch2 is a code owner

@nknize nknize Awaiting requested review from nknize

@owaiskazi19 owaiskazi19 Awaiting requested review from owaiskazi19 owaiskazi19 is a code owner

@peternied peternied Awaiting requested review from peternied

@Rishikesh1159 Rishikesh1159 Awaiting requested review from Rishikesh1159 Rishikesh1159 is a code owner

@ryanbogan ryanbogan Awaiting requested review from ryanbogan

@saratvemulapalli saratvemulapalli Awaiting requested review from saratvemulapalli saratvemulapalli is a code owner

@shwetathareja shwetathareja Awaiting requested review from shwetathareja shwetathareja is a code owner

@dreamer-89 dreamer-89 Awaiting requested review from dreamer-89

@VachaShah VachaShah Awaiting requested review from VachaShah

@dbwiddis dbwiddis Awaiting requested review from dbwiddis dbwiddis is a code owner

@sachinpkale sachinpkale Awaiting requested review from sachinpkale sachinpkale is a code owner

@sohami sohami Awaiting requested review from sohami sohami is a code owner

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@harshavamsi @msfroh