[Backport 2.x] Add support for scored named queries (#11626)#12427
Conversation
Opensearch already support labelling the queries, that returns as a list in the returned results, of which query it matched. However one of the use case while doing hybrid search with query text and dense vector is to determine individual scores for each query type. This is very useful in further analysis and building offline model to generate better weights for ranking score. Hence adding this feature that sends the client to add the score for each matched query. --------- Signed-off-by: Dharin Shah <8616130+Dharin-shah@users.noreply.github.com> Signed-off-by: Dharin Shah <Dharin-shah@users.noreply.github.com> Co-authored-by: Dharin Shah <8616130+Dharin-shah@users.noreply.github.com> (cherry picked from commit 52b27f4) Signed-off-by: Andrew Ross <andrross@amazon.com>
andrross
requested review from
Bukhtawar,
CEHENKLE,
Rishikesh1159,
VachaShah,
abbashus,
adnapibar,
anasalkouz,
dblock,
dbwiddis,
dreamer-89,
gbbafna,
kartg,
kotwanikunal,
mch2,
msfroh,
nknize,
owaiskazi19,
reta,
ryanbogan,
sachinpkale,
saratvemulapalli,
shwetathareja,
sohami and
tlfeng
as code owners
Compatibility status:Checks if related components are compatible with change 56d080a Incompatible componentsSkipped componentsCompatible componentsCompatible components: [https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/performance-analyzer-rca.git, https://github.com/opensearch-project/cross-cluster-replication.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/reporting.git, https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/custom-codecs.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/flow-framework.git, https://github.com/opensearch-project/performance-analyzer.git, https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/neural-search.git, https://github.com/opensearch-project/security-analytics.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/sql.git] |
|
❌ Gradle check result for ccbd26b: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
Ugh...some version guards need to be updated. This also means the guards on main will have to be changed as soon as this PR is committed otherwise bwc tests will break. |
Signed-off-by: Andrew Ross <andrross@amazon.com>
|
❌ Gradle check result for 0aae97b: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
|
Thanks @andrross ! |
andrross
force-pushed
the
backport/backport-11626-to-2.x
branch
from
346583f to
5cc2675
Compare
|
❌ Gradle check result for 346583f: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
|
❌ Gradle check result for 5cc2675: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
Co-authored-by: Andriy Redko <drreta@gmail.com> Signed-off-by: Andrew Ross <andrross@amazon.com>
andrross
force-pushed
the
backport/backport-11626-to-2.x
branch
from
5cc2675 to
56d080a
Compare
|
❌ Gradle check result for 56d080a: null Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
|
This check actually succeeded, but the GHA poller failed on one of its requests to Jenkins: https://build.ci.opensearch.org/job/gradle-check/34086/ |
|
❕ Gradle check result for 56d080a: UNSTABLE
Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure. |
| @@ -0,0 +1,103 @@ | |||
| setup: | |||
| - skip: | |||
| version: " - 2.12.99" | |||
There was a problem hiding this comment.
@andrross my apologies, could you please bring this change to main? thank you
|
Sorry i couldn't pick this up, thanks for handling it @andrross |
4 participants
Opensearch already support labelling the queries, that returns as a list in the returned results, of which query it matched. However one of the use case while doing hybrid search with query text and dense vector is to determine individual scores for each query type. This is very useful in further analysis and building offline model to generate better weights for ranking score. Hence adding this feature that sends the client to add the score for each matched query.
(cherry picked from commit 52b27f4)
Check List
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.