Allow mmap to use new preview JDK-19 APIs in Apache Lucene 9.4+ by reta · Pull Request #5151 · opensearch-project/OpenSearch

reta · 2022-11-08T19:13:36Z

Signed-off-by: Andriy Redko andriy.redko@aiven.io

Description

Allow mmap to use new preview JDK-19 APIs in Apache Lucene 9.4+. When --enable-preview is not present:

[2022-11-08T14:03:56,502][WARN ][o.a.l.s.MMapDirectory    ] [runTask-0] You are running with Java 19. To make full use of MMapDirectory, please pass '--enable-preview' to the Java command line.

With --enable-preview:

[2022-11-08T14:10:36,009][INFO ][o.a.l.s.MemorySegmentIndexInputProvider] [runTask-0] Using MemorySegmentIndexInput with Java 19

Issues Resolved

Final part of #4637

Check List

New functionality includes testing.
- All tests pass
New functionality has been documented.
- New functionality has javadoc added
Commits are signed per the DCO using --signoff
Commit changes are listed out in CHANGELOG.md file (See: Changelog)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

github-actions · 2022-11-08T19:18:28Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/6605/
CommitID: 9391076
Please examine the workflow log, locate, and copy-paste the failure below, then iterate to green.
Is the failure a flaky test unrelated to your change?

andrross · 2022-11-08T19:26:40Z

CHANGELOG.md

 - Fail weight update when decommission ongoing and fail decommission when attribute not weighed away ([#4839](https://github.com/opensearch-project/OpenSearch/pull/4839))
 - Skip SymbolicLinkPreservingTarIT when running on Windows ([#5023](https://github.com/opensearch-project/OpenSearch/pull/5023))
 - Change the output error message back to use OpenSearchException in the cause chain. ([#5081](https://github.com/opensearch-project/OpenSearch/pull/5081))
+- Allow mmap to use new preview JDK-19 APIs in Apache Lucene 9.4+ ([#5151](https://github.com/opensearch-project/OpenSearch/pull/5151))


We just pushed an update to the changelog policy that applies the learnings from the recent 2.4 release.

This is probably an example that can skip the changelog? Whatever changes make use of the new APIs would be relevant, but this is probably too low level.

Thanks @andrross , I could be mistaken but it falls into A newly added feature: we activate a new Apache Lucene feature by changing the JVM parameters, what do you think from this perspective? (I am fine removing changelog line as well)

~~Ah, does Lucene detect this setting and change its behavior at runtime accordingly? If so then you're right :)~~

I should have read the description more carefully. You're right this is a newly added feature.

Thanks @andrross !

github-actions · 2022-11-08T19:46:33Z

Gradle Check (Jenkins) Run Completed with:

RESULT: UNSTABLE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/6606/
CommitID: fec2ee5
Please examine the workflow log, locate, and copy-paste the failure below, then iterate to green.
Is the failure a flaky test unrelated to your change?

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

github-actions · 2022-11-08T19:59:36Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/6611/
CommitID: f381e50
Please examine the workflow log, locate, and copy-paste the failure below, then iterate to green.
Is the failure a flaky test unrelated to your change?

uschindler · 2022-11-08T20:13:10Z

Technically the preview feature would only work with exactly java 19. But it won't hurt if you pass it with 20. Lucenes will ignore it.

reta · 2022-11-08T20:15:13Z

Technically the preview feature would only work with exactly java 19. But it won't hurt if you pass it with 20. Lucenes will ignore it.

Thanks @uschindler , I started with 19:--enable-preview (exact) but replaced it with 19-:--enable-preview

github-actions · 2022-11-08T20:21:44Z

Gradle Check (Jenkins) Run Completed with:

RESULT: SUCCESS ✅
URL: https://build.ci.opensearch.org/job/gradle-check/6610/
CommitID: 2869696

github-actions · 2022-11-08T20:28:13Z

Gradle Check (Jenkins) Run Completed with:

RESULT: SUCCESS ✅
URL: https://build.ci.opensearch.org/job/gradle-check/6612/
CommitID: 79c1170

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

reta · 2022-11-08T20:50:48Z

build.gradle

-      javadoc.options.addBooleanOption("-enable-preview", true)
-      javadoc.options.addStringOption("-release", BuildParams.runtimeJavaVersion.majorVersion)
-    }
+    javadoc.options.addStringOption("-release", targetCompatibility.majorVersion)


Align javadoc release with target compatibility settings

github-actions · 2022-11-08T21:19:29Z

Gradle Check (Jenkins) Run Completed with:

RESULT: SUCCESS ✅
URL: https://build.ci.opensearch.org/job/gradle-check/6618/
CommitID: a046505

codecov-commenter · 2022-11-08T21:21:56Z

Codecov Report

Merging #5151 (a046505) into main (fab4336) will increase coverage by 0.09%.
The diff coverage is 66.34%.

@@             Coverage Diff              @@
##               main    #5151      +/-   ##
============================================
+ Coverage     70.84%   70.94%   +0.09%     
- Complexity    57867    58141     +274     
============================================
  Files          4692     4708      +16     
  Lines        276650   277557     +907     
  Branches      40155    40188      +33     
============================================
+ Hits         196002   196921     +919     
+ Misses        64500    64482      -18     
- Partials      16148    16154       +6

Impacted Files	Coverage Δ
...java/org/opensearch/upgrade/ValidateInputTask.java	`84.61% <0.00%> (ø)`
...ch/analysis/common/CommonAnalysisModulePlugin.java	`92.33% <0.00%> (+0.05%)`	⬆️
...mon/HyphenationCompoundWordTokenFilterFactory.java	`0.00% <0.00%> (ø)`
...index/analysis/IcuCollationTokenFilterFactory.java	`59.25% <0.00%> (-2.28%)`	⬇️
.../src/main/java/org/opensearch/LegacyESVersion.java	`59.34% <ø> (-2.37%)`	⬇️
...ecommission/awareness/put/DecommissionRequest.java	`85.71% <0.00%> (-6.60%)`	⬇️
...min/cluster/stats/TransportClusterStatsAction.java	`70.83% <ø> (ø)`
...ensearch/action/search/SearchTransportService.java	`83.75% <ø> (-3.25%)`	⬇️
...va/org/opensearch/action/update/UpdateRequest.java	`46.72% <0.00%> (+2.28%)`	⬆️
...arch/cluster/decommission/DecommissionService.java	`35.27% <0.00%> (ø)`
... and 634 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

reta requested a review from a team as a code owner November 8, 2022 19:13

reta force-pushed the allow.preview branch from 9391076 to fec2ee5 Compare November 8, 2022 19:14

reta added the v3.0.0 Issues and PRs related to version 3.0.0 label Nov 8, 2022

andrross reviewed Nov 8, 2022

View reviewed changes

reta force-pushed the allow.preview branch 2 times, most recently from 2869696 to f381e50 Compare November 8, 2022 19:54

Allow mmap to use new preview JDK-19 APIs in Apache Lucene 9.4+

79c1170

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

reta force-pushed the allow.preview branch from f381e50 to 79c1170 Compare November 8, 2022 19:55

Align javadoc release with target compatibility settings

a046505

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

reta mentioned this pull request Nov 8, 2022

After update to Lucene 9.4 use --enable-preview on Java==19 (exact) to allow mmap use new JDK APIs #4637

Closed

reta commented Nov 8, 2022

View reviewed changes

reta requested a review from dblock November 8, 2022 21:14

dblock approved these changes Nov 8, 2022

View reviewed changes

dblock merged commit fc47572 into opensearch-project:main Nov 8, 2022

BrewTestBot mentioned this pull request May 6, 2025

opensearch 3.0.0 Homebrew/homebrew-core#222665

Merged

This was referenced Jan 8, 2026

docs: add java-runtime-and-jpms report for v3.0.0 tkykenmt/opensearch-feature-explorer#290

Merged

[breaking] Java Runtime & JPMS tkykenmt/opensearch-feature-explorer#136

Closed

Conversation

reta commented Nov 8, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Issues Resolved

Check List

Uh oh!

github-actions bot commented Nov 8, 2022

Gradle Check (Jenkins) Run Completed with:

Uh oh!

andrross Nov 8, 2022

Choose a reason for hiding this comment

Uh oh!

reta Nov 8, 2022

Choose a reason for hiding this comment

Uh oh!

andrross Nov 8, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

reta Nov 8, 2022

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Nov 8, 2022

Gradle Check (Jenkins) Run Completed with:

Uh oh!

github-actions bot commented Nov 8, 2022

Gradle Check (Jenkins) Run Completed with:

Uh oh!

uschindler commented Nov 8, 2022

Uh oh!

reta commented Nov 8, 2022

Uh oh!

github-actions bot commented Nov 8, 2022

Gradle Check (Jenkins) Run Completed with:

Uh oh!

github-actions bot commented Nov 8, 2022

Gradle Check (Jenkins) Run Completed with:

Uh oh!

reta Nov 8, 2022

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Nov 8, 2022

Gradle Check (Jenkins) Run Completed with:

Uh oh!

codecov-commenter commented Nov 8, 2022

Codecov Report

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

reta commented Nov 8, 2022 •

edited

Loading

andrross Nov 8, 2022 •

edited

Loading