Skip to content

Fixing the regression in inner hits aggregation#13488

Closed
jainankitk wants to merge 4 commits intoopensearch-project:mainfrom
jainankitk:inner-hits-fix
Closed

Fixing the regression in inner hits aggregation#13488
jainankitk wants to merge 4 commits intoopensearch-project:mainfrom
jainankitk:inner-hits-fix

Conversation

@jainankitk
Copy link
Copy Markdown
Contributor

@jainankitk jainankitk commented May 1, 2024

This commit fixing the regression introduced by da5b205.

Description

Reverts #12503 that introduced regression in 2.13.0. Thanks a lot @martijnbolhuis for spotting it and supplying the tests, @dblock for working with the contributor.
Related Issues

Closes #13467

Check List

  • New functionality includes testing.
    • All tests pass
  • New functionality has been documented.
    • New functionality has javadoc added
  • Failing checks are inspected and point to the corresponding known issue(s) (See: Troubleshooting Failing Builds)
  • Commits are signed per the DCO using --signoff
  • Commit changes are listed out in CHANGELOG.md file (See: Changelog)
    - [ ] Public documentation issue/PR created

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

jainankitk added 2 commits May 1, 2024 10:35
Signed-off-by: Ankit Jain <akjain@amazon.com>
Signed-off-by: Ankit Jain <akjain@amazon.com>
@jainankitk
Copy link
Copy Markdown
Contributor Author

Related to #13486

@reta
Copy link
Copy Markdown
Contributor

reta commented May 1, 2024

Related to #13486

I am not comfortable with this change as is - we need to add more tests, not just one, to cover all uncovered flows and catch the regressions.

jainankitk added 2 commits May 1, 2024 10:44
Signed-off-by: Ankit Jain <akjain@amazon.com>
Signed-off-by: Ankit Jain <akjain@amazon.com>
@jainankitk
Copy link
Copy Markdown
Contributor Author

I am not comfortable with this change as is - we need to add more tests, not just one, to cover all uncovered flows and catch the regressions.

@dblock @reta - Ideally the original commit should have been 2 separate PRs one for ScriptFields and InnerHits. My suggestion is to not press panic button and revert everything. Instead make more cognitive choice.

to cover all uncovered flows and catch the regressions.

all is very hard in my opinion, but I am open to suggestions

@reta
Copy link
Copy Markdown
Contributor

reta commented May 1, 2024

all is very hard in my opinion, but I am open to suggestions

hence we have such regressions, I prefer taking more time to understand were we have misses instead of patching in a rush

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented May 1, 2024

❌ Gradle check result for bbdbc08: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented May 1, 2024

❌ Gradle check result for 45ee240: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented May 1, 2024

❌ Gradle check result for 23c45d5: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@jainankitk
Copy link
Copy Markdown
Contributor Author

jainankitk commented May 1, 2024

all is very hard in my opinion, but I am open to suggestions

hence we have such regressions, I prefer taking more time to understand were we have misses instead of patching in a rush

As previously mentioned, those 2 changes should have gone as part of separate commits, and makes sense to revert only one of those. But, I am okay if you really prefer to revert both the changes.

@reta
Copy link
Copy Markdown
Contributor

reta commented May 1, 2024

@dblock @msfroh wdyt folks? I would go with full revert #13486 and reintroducing those in 2.15.0 once we have sufficient test coverage for both.

@msfroh
Copy link
Copy Markdown
Contributor

msfroh commented May 1, 2024

@dblock @msfroh wdyt folks? I would go with full revert #13486 and reintroducing those in 2.15.0 once we have sufficient test coverage for both.

Yeah... I think that makes sense. Let's get back to the known-okay behavior -- as far as I know, we don't have any evidence that initializing the FetchSubPhaseProcessor was causing any noticeable pain(*). We should treat this regression as a helpful sign that we didn't have enough test coverage for these kinds of requests. Once we have the test coverage, we can move forward with the change to avoid unnecessary object creation.

(*) Taking a look at the FetchSubPhaseProcessor implementation in InnerHitsPhase, it looks like it only captures references to searchContext and innerHits, so those would be the fields of the anonymous class. Combined with standard object overhead, I'm guessing a useless FetchPhaseSubProcessor probably wastes less than 20 bytes (assuming 32-bit object references).

@dblock
Copy link
Copy Markdown
Member

dblock commented May 1, 2024

@msfroh @reta decide in #13486 (comment)?

@jainankitk
Copy link
Copy Markdown
Contributor Author

Closing this PR in favor of #13486

@jainankitk jainankitk closed this May 1, 2024
@jainankitk jainankitk deleted the inner-hits-fix branch May 3, 2024 01:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working Search:Aggregations Severity-Critical v2.14.0 v3.0.0 Issues and PRs related to version 3.0.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] Missing inner hits in top hits of an aggregation results since upgrade to 2.13.0

4 participants