We need to define the JPMS strategy for EE10 and beyond

[starksm64]

Is your feature request related to a problem? Please describe.
We have a problem with inconsistent JPMS usage for the platform APIs jars.

Describe the solution you'd like
We need to clearly define the requirements for supporting JPMS.

Describe alternatives you've considered
The current approach of allowing specification projects to define the level of JPMS support has produced an inconsistent experience for users/servers attempting to use JPMS.

[starksm64]

Previous issues:
#174

[starksm64]

Ideas on various levels of details that are needed based on internal discussions:

• Simply establishing a module naming convention for Jakarta EE APIs
• Requiring automodules for Jakarta EE APIs which expose those names
• Requiring (or allowing) actual module descriptors for Jakarta EE API artifacts
• Requiring specific content within module descriptors for Jakarta EE API artifacts (for example, version information, annotations, dependency information, etc.)
• Requiring (or allowing) Jakarta EE API artifacts to be deployed on the module path or in another JPMS-compatible environment
• Requiring (or allowing) programs which consume Jakarta EE API artifacts to be deployed on the module path or in another JPMS-compatible environment
• Establishing behaviors or conventions around usage of module mechanisms in a Jakarta EE environment (for example, special behavior specifications around service loading)

[keilw]

It may be a special niche but based on Bugs at least until Java 13 not fully addressed (so if the minimum JDK version was 11-13 there is a great risk of that) JPMS and Multi-Release-JAR mechanism are incompatible and may cause serious problems if used together without safety-measures.

• This issue for the JSR 385 RI: can't use from jpms, maven 3.6.3, openjdk 11 unitsofmeasurement/indriya#301 could only be solved by adding both a module-info (especially in a Multi-Release-JAR structure) and an Automatic-Module-Name as a fallback, since Java 11 (and potentially still 12, it looks resolved after 13) can ignore the Multi-Release-JAR and simply call the initial structure if JPMS and Multi-Release-JAR get combined

Most of that is hidden very well in OpenJDK Bugs like JDK-8207162 and similar ones, but even the statement it affects only Java 9-11 seems incomplete because at the very least 12 is also still affected from our experience. Therefore as long as the minimum version for either Specs or implementations is Java 11 or 12 and not higher, there should be some form of caveat about Multi-Release-JAR usage in combination with JPMS.

[tomas-langer]

Is there a plan to upgrade minimal Java version requirement? Considering Java 8 is end of life, if we base Jakarta 10 on Java 11, we do not need to have multi-release jars, and can fully utilize JPMS using module-info.java.

[keilw]

That's a good point, I am also not entirely sure, if OpenJDK some day pruned or eliminated the ServiceLoader mechanism via META-INF\services only to use definitions in module-info.java? From what I know, if you define them as provides in the latter you should not need to define the services folder as well (unless you need a fallback for JDKs before 9)

Should any project plan to use "fancy new stuff" like records, etc. (maybe not all of it likely for an API but who knows what they plan to use) then even with a minimum Java SE version like 10, 11 or 12 there could still be a temptation and need to use these via Multi-Release-JAR although I must admit, for specs there may not be such a benefit, given that for the time being I don't think a spec may offer element X but Y only if you are on Java 16 or above? ;-)

[anthonyvdotbe]

tl;dr Java SE 11 for module descriptors (and Java SE 8 or 11 for source code, but that question is irrelevant w.r.t. modules)

@keilw I don't see any evidence that proves there are unresolved JDK issues w.r.t. the module system and/or MR JARs:

• you say "OpenJDK Bugs like JDK-8207162 and similar ones", but there's not a single link to an unresolved issue to back up your claims (JDK-8207162 was already fixed in JDK 12 and backported to JDK 11.0.2. Moreover, it's jdeps-specific and thus irrelevant anyway)
• even if there would be existing or yet-undiscovered issues: Java SE 11 is the only LTS release to date with module support, and thus also the only one for which backports would be done

@tomas-langer MR JARs are not a prerequisite for Java SE 8-compatible modular JARs: the module-info.class can simply be placed in the JAR's root directory

I don't think a spec may offer element X but Y only if you are on Java 16 or above?

This question is irrelevant, because the JAR spec itself doesn't currently allow it:

The public API exported by the classes in a multi-release JAR file must be exactly the same across versions [...] A future release of this specification may relax the exact same API constraint to support careful evolution.

(edit: @keilw I'm disappointed by your response and will not engage any further in this discussion)

[keilw]

@anthonyvdotbe Thanks but indriya#346 is meaningless and I already closed it as invalid.
Frankly I don't care if the JDK team did not do their homework or papertrail. What I and users care is that it works and it doesn't between Java 9 and 12, there is neither a JDK 11.0.2 this works with (we tested it, it fails with every JDK 11 up to jdk-11.0.8) nor any released Java 12, it was only fixed from 13 and they simply forgot or were too busy with other things to backport it. Or it was only made available to paying customers but not the common OpenJDK builds available under AdoptOpenJDK or similar.

You are right however, that only adding a module-info unless new services or packages only making sense for a higher JDK (unlikely for a spec but an implementation like Indriya is free to support certain functionality only when it makes sense for that JDK version, the JSR 363 RI also did not support LocalDateTime or Temporal while the then inofficial forward-port for Java 8 did) require you to define more than one module-info in a Multi-Release-JAR structure, it can be placed in the top-level and Java 8 applications will just ignore it.
What you quoted from a JEP first of all isn't a binding specification, the JEP is merely a feature request that may or may not be done and the actual Java specifications may be very different from what the spec writes about
Plus it sounds reasonable for the public API of Jakarta Specifications, but this ticket is not only about (Jakarta) Specification documents and you may well see an E.class in an implementation that is specific to Java 12 or 16 e.g. it may be using a record that makes no sense in any of the prior Java versions. And again there is no need to declare the same class as an empty shell or throw an UnsupportedOperationException in an implementation if that feature makes no sense, don't offer it before the JDK version that does.

[lukasj]

Requiring automodules for Jakarta EE APIs which expose those names

This should be no-go for spec projects. There are more problems with them but the one I mean here is that jlink does not support them. Even though that can be seen as not-an-issue from the EE/appserver perspective, it can be problem in various standalone implementations and there were users who were quite loud about this in few projects (ie activation can be an example) and Jakarta EE definitely wants to allow and also support usage of various specifications out of the EE container.

Establishing behaviors or conventions around usage of module mechanisms in a Jakarta EE environment (for example, special behavior specifications around service loading)

open module propagation to allow reflection access is another thing. What I as an end user want is to open pkg.entities to jakarta.persistence and let the persistence API figure out to which implementation this open should be propagated to. I do not want to rely on particular persistence implementation be it EclipseLink or Hibernate or whatever else. This can be generalized as sth like To allow reflection access from the CI to user defined classes, user code must be open to specJar which propagates this openess to an implementation found by lookup mechanism defined in spec (where user defined classes can be ie entities, jaxb/jsonb annotated classes etc)

Is there a plan to upgrade minimal Java version requirement? Considering Java 8 is end of life,..

Do not take me wrong from the text bellow - I vote for SE 11 as the base should my vote be the binding one ;-)
While Java SE 8 is EOL, support for it is going to end by December 2030, so in almost 10 years from now. I'd be personally very happy to jump directly over to SE 17 right now all over the place but I think, EE 10 should still run on 8 and min Java SE version should move to 11 in Jakarta EE 11. (Should there be any 10.X releases, all of them should keep the same min Java SE level as initial 10.0 version - but I think this is clear).

if OpenJDK some day pruned or eliminated the ServiceLoader mechanism via META-INF\services only to use definitions in module-info.java? From what I know, if you define them as provides in the latter you should not need to define the services folder as well (unless you need a fallback for JDKs before 9)

https://docs.oracle.com/en/java/javase/11/docs/specs/jar/jar.html#modular-jar-files gives clear answer to that:

If the modular JAR file is deployed on the class path then it behaves as if a non-modular JAR file.

Which, in the other words means that module-info is ignored if the jar is on the classpath. So as long as class-path mode is to be supported, META-INF/services must be present.

Should any project plan to use "fancy new stuff" like records, etc.

At the spec project level, usage of "fancy new stuff" should be discouraged (or not recommended) - but probably not strictly prohibited, unless it comes from an LTS Java SE version (ie support for SE 15 would not be recommended, for SE 17 it would be OK). My thinking here is that implementations/vendors should not be forced to provide support for possibly changing features. If they want to support them in their implementations, they can do so even under current rules.

the module-info.class can simply be placed in the JAR's root directory

or under the versioned directory. It is possible to have slightly different descriptors for different SE versions. This is defined at https://docs.oracle.com/en/java/javase/11/docs/specs/jar/jar.html#modular-multi-release-jar-files - this also proves that the statement JPMS and Multi-Release-JAR mechanism are incompatible is false as long as defined rules are followed.

I'm not sure it is feasible to have this completely covered, defined and implemented in EE 10. But it would be definitely good to start with spec and CI can support JPMS by following TBD rules/recommendations and switching it over to spec must and CI can support JPMS in EE 11 together with further adjustments and bumping up min SE version to 11 there. It would be nice to say must also for the CIs in EE 11 but I'm afraid there will be implementations not willing to jump on the JPMS train.

[nipafx]

Hi there, sorry to barge in but I heard people are discussing modules and I'm always up for that. :) Caveat: I don't know the Jakarta EE space well, so I may be missing context - let me know if you think I do.

A few random replies:

We need to clearly define the requirements for supporting JPMS

👍🏾 Maybe this helps. (Note that it has no official character and didn't catch on, but I still think it's good. 😊 )

Requiring automodules for Jakarta EE APIs which expose those names

Technically, the automatic module name is only that: a name for the JAR if it's used on the module path. But socially, it also signals to users that the JAR works fine on the module path. I recommend to only define a name once that has been established.

It may be a special niche but based on Bugs at least until Java 13 not fully addressed (so if the minimum JDK version was 11-13 there is a great risk of that) JPMS and Multi-Release-JAR mechanism are incompatible and may cause serious problems if used together without safety-measures.

As far as I can tell, this analysis is based on a malformed JAR (see my comment in unitsofmeasurement/indriya#346) and unrelated jdeps and Maven issues. I don't think it holds up.

I am also not entirely sure, if OpenJDK some day pruned or eliminated the ServiceLoader mechanism via META-INF\services only to use definitions in module-info.java?

I have no particular insight into plans for the service loader, but I'd be extremely surprised by such a change as it would severely neuter the class path. So unless the class path itself is going away (would be equally surprising), I wouldn't worry about this. And @lukasj got it right re service declarations: If a JAR is supposed to work on class path and module path, it needs to define services in META-INF\services and module-info.java.

What I and users care is that it works and it doesn't between Java 9 and 12, there is neither a JDK 11.0.2 this works with (we tested it, it fails with every JDK 11 up to jdk-11.0.8) nor any released Java 12, it was only fixed from 13 and they simply forgot or were too busy with other things to backport it. Or it was only made available to paying customers but not the common OpenJDK builds available under AdoptOpenJDK or similar.

Re fixes in 13 not being backported in 12, that's because 12 is no LTS version. As far as I can tell nobody in the OpenJDK community is backporting fixes to it and neither does anybody offer commercial support for it.

At the spec project level, usage of "fancy new stuff" should be discouraged (or not recommended) - but probably not strictly prohibited, unless it comes from an LTS Java SE version (ie support for SE 15 would not be recommended, for SE 17 it would be OK). My thinking here is that implementations/vendors should not be forced to provide support for possibly changing features.

I may well have misunderstood the last sentence, so this might be a non-sequitur, but I want to point out that only preview features (enabled with --enable-preview) will change between releases. Once they're out of preview (e.g. text blocks in 15), they're set in stone regardless of whether that version is LTS or not. More to the point LTS is a commercial concept that the JLS doesn't give two Guy Fawkes about. 😁

[keilw]

@nipafx Thanks for your input here. I just literelly at this moment saw the mail, that DWX' 21 also has to happen remotely, so while I anticipate your JDK talk, we (including @thodorisbais, @ivargrimstad, you and myself) won't meet in person then, because after the recent Corona "Emergency Break" in Germany they certainly did not see a chance to hold it in person, especially not in the home town of Markus Söder ;-/

You made most of the good points here, especially about --enable-preview which the initial JSR 376 spec (that remained unchanged ever since a vast number of us here who were also in the JCP EC back then voted against it in 2017 and some, but only minimal changes were applied but enough to get us approve it in the end) did not even know back then, but especially problems and bugs related to jdepts or jlink like JDK-8207162 are going to be a problem as long as the mimimal Java version stays at 9-11 or even 12.

@anthonyvdotbe Sorry to dissapoint you, but I was also quite surprised to even learn of your ticket #174 which only @kwsutter had ever exchanged a few thoughts with until @starksm64 linked it here and some had a look at it. I was quite actively involved in shaping a minumum level of modularity and JPMS in Jakarta EE 9 already, but there were some compromises (as everywhere also in other specs especially in the JDK itself ;-) and a few specs or APIs that did not feel ready at the time especially CDI while others had done that already.
It seems you made at least one commit to microprofile, have you ever tried approaching it about JPMS because while 2/3 or more of Jakarta EE specs and APIs already support at least Automatic-Module-Name since Jakarta EE 9 or earlier, there is zero evidence the official MP APIs ever got a single JPMS definition of their own, the only one seems to be added later by projects like Helidon (@lukasj probably knows a lot more about that) and unfortunately not even some key stakeholders to Eclipse MicroProfile are always completely aware of using JPMS and problems that may arise if you do it wrong, especially Split Packages? Did you ever raise those kinds of tickets with MicroProfile like you did here and we just don't know about or did you not suggest it there @anthonyvdotbe, I think while I cannot promise it'll succeed with every single contributor you should try to raise awareness there if you care about this topic.

[pron]

support for it is going to end by December 2030, so in almost 10 years from now

Support is going to continue at least until December 2030. Oracle might decide to extend it further as long as there are enough paying customers.

The question is, what does this have to do with support for Java 8 in new versions of libraries/specifications? Java 8 is essentially EOL and has been very much discouraged for new projects for a while, now. It is now in pure legacy maintenance mode, and new specifications targeting it might confuse people further about this.

[starksm64]

Is there a plan to upgrade minimal Java version requirement? Considering Java 8 is end of life, if we base Jakarta 10 on Java 11, we do not need to have multi-release jars, and can fully utilize JPMS using module-info.java.

The expectation is that at least Java 11 will be required. This is being tracked in #331 where the question of Java 17 support is also raised.

[keilw]

@pron For completely new services I agree and I also helped a customer introduce a green-field system based on Java 11+ around 2 years ago, but right now I have another case where "Legacy" systems like Tibco, CORBA or even Host systems are involved in a very big company and none of those are being replaced before at least the mid 2020s. Thus they add new services using Java 11 and Jakarta EE or Spring Boot, etc. but they still have those old systems and for SOAP or other protocols many don't get touched until eventually they might be replaced by something like a new REST based API and only then when all the services are upgraded, the last Java 8 client or server is going to disappear.
And keep in mind, Java 8 has a longer Extended Service guarantee than even the upcoming Java 17, see Oracle Java SE Support Roadmap, so some customers including the ones I deal with at the moment are more likely to switch from 8 to 23 in Fall 2024 when that seems like the next LTS following a 3-year cycle.

@tomas-langer For the spec the need for multi-release-jar support could be close to zero, for implementations I would not outrule that however as I cannot speak for all the projects and their project leads, that we may only recommend on a platform level but it's up to each of them and their products what they do, so it can't be outruled. Even less for applications consuming it, #331 had a good case by someone who said they already use Records, thus unless they force all of their users of the system to immediately upgrade to Java 16 or 17 they may need a multi-release-jar while the spec or even implementation could do without. And as @lukasj mentioned a little earlier, if the module-info was ignored in that case, the module could be lost unless you also provide an Automatic-Module-Name as a last resort.