Skip to content

Introduce ApproximateRangeQuery and ApproximateableQuery#13788

Merged
msfroh merged 20 commits into
opensearch-project:mainfrom
harshavamsi:pointrange_optimization
Sep 2, 2024
Merged

Introduce ApproximateRangeQuery and ApproximateableQuery#13788
msfroh merged 20 commits into
opensearch-project:mainfrom
harshavamsi:pointrange_optimization

Conversation

@harshavamsi
Copy link
Copy Markdown
Contributor

@harshavamsi harshavamsi commented May 22, 2024

Description

Most of the logic is as per #13566. I've introduced a new ApproximateableQuery that is virtually similar to what IndexOrDocValues does today. It returns either an originalQuery or an approximateQuery. During search time we evaluate if a query matches a particular requirement for it to be rewritten from originalQuery to approximateQuery. Here I started off with just converting the DateRangeQuery to use the approximation. If we have a top level range query on a date field, we will approximate the results by only scoring 10K or size.

Related Issues

Resolves #11251 #9541 #13566

Check List

  • New functionality includes testing.
    • All tests pass
  • New functionality has been documented.
    • New functionality has javadoc added
  • API changes companion pull request created.
  • Failing checks are inspected and point to the corresponding known issue(s) (See: Troubleshooting Failing Builds)
  • Commits are signed per the DCO using --signoff
  • Commit changes are listed out in CHANGELOG.md file (See: Changelog)
  • Public documentation issue/PR created

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Harsha Vamsi Kalluri <harshavamsi096@gmail.com>
@github-actions github-actions Bot added enhancement Enhancement or improvement to existing feature or request Search:Performance labels May 22, 2024
@github-actions
Copy link
Copy Markdown
Contributor

❌ Gradle check result for 95236d6: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@github-actions
Copy link
Copy Markdown
Contributor

❌ Gradle check result for c98b56c: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@harshavamsi harshavamsi force-pushed the pointrange_optimization branch from c98b56c to 76b4abe Compare May 22, 2024 22:31
@github-actions
Copy link
Copy Markdown
Contributor

❌ Gradle check result for 76b4abe: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@github-actions
Copy link
Copy Markdown
Contributor

❌ Gradle check result for 76b4abe: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@harshavamsi harshavamsi force-pushed the pointrange_optimization branch from 2cf5e27 to 9ac309a Compare May 23, 2024 17:38
@github-actions
Copy link
Copy Markdown
Contributor

❌ Gradle check result for 2cf5e27: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@github-actions
Copy link
Copy Markdown
Contributor

❌ Gradle check result for 090ddc6: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@github-actions
Copy link
Copy Markdown
Contributor

❌ Gradle check result for 9ac309a: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Comment thread server/src/main/java/org/opensearch/index/mapper/DateFieldMapper.java Outdated
akolarkunnu pushed a commit to akolarkunnu/OpenSearch that referenced this pull request Sep 10, 2024
…ect#13788)

This introduces a basic "approximation" framework that improves
query performance by modifying the query in a way that should be
functionally equivalent.

To start, we can reduce the bounds of a range query in order to
satisfy the `track_total_hits` value (which defaults to 10,000).

---------

Signed-off-by: Harsha Vamsi Kalluri <harshavamsi096@gmail.com>
Signed-off-by: Michael Froh <froh@amazon.com>
Co-authored-by: Michael Froh <froh@amazon.com>
dk2k pushed a commit to dk2k/OpenSearch that referenced this pull request Oct 16, 2024
…ect#13788)

This introduces a basic "approximation" framework that improves
query performance by modifying the query in a way that should be
functionally equivalent.

To start, we can reduce the bounds of a range query in order to
satisfy the `track_total_hits` value (which defaults to 10,000).

---------

Signed-off-by: Harsha Vamsi Kalluri <harshavamsi096@gmail.com>
Signed-off-by: Michael Froh <froh@amazon.com>
Co-authored-by: Michael Froh <froh@amazon.com>
dk2k pushed a commit to dk2k/OpenSearch that referenced this pull request Oct 17, 2024
…ect#13788)

This introduces a basic "approximation" framework that improves
query performance by modifying the query in a way that should be
functionally equivalent.

To start, we can reduce the bounds of a range query in order to
satisfy the `track_total_hits` value (which defaults to 10,000).

---------

Signed-off-by: Harsha Vamsi Kalluri <harshavamsi096@gmail.com>
Signed-off-by: Michael Froh <froh@amazon.com>
Co-authored-by: Michael Froh <froh@amazon.com>
dk2k pushed a commit to dk2k/OpenSearch that referenced this pull request Oct 21, 2024
…ect#13788)

This introduces a basic "approximation" framework that improves
query performance by modifying the query in a way that should be
functionally equivalent.

To start, we can reduce the bounds of a range query in order to
satisfy the `track_total_hits` value (which defaults to 10,000).

---------

Signed-off-by: Harsha Vamsi Kalluri <harshavamsi096@gmail.com>
Signed-off-by: Michael Froh <froh@amazon.com>
Co-authored-by: Michael Froh <froh@amazon.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport 2.x Backport to 2.x branch backport-failed enhancement Enhancement or improvement to existing feature or request Search:Performance v2.17.0 v3.0.0 Issues and PRs related to version 3.0.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Understand/Improve the performance of range queries

9 participants