Skip to content

Comments

Remove -nouses directive from maven-bundle-plugin#581

Merged
garydgregory merged 1 commit intoapache:masterfrom
laeubi:remove_nouse
Jan 25, 2025
Merged

Remove -nouses directive from maven-bundle-plugin#581
garydgregory merged 1 commit intoapache:masterfrom
laeubi:remove_nouse

Conversation

@laeubi
Copy link
Contributor

@laeubi laeubi commented Jan 23, 2025

In commit 740bcf1 the instruction '-nouses' was added (but without any rationale) but using that is extremely dangerous. Without any use clause, all packages are treated as independent from each other, that means the OSGi resolver is free to wire these packages to different classloaders if used by a consumer (or its dependencies).

This now completely removes the offending clause from the instructions, if any project has a real usecase for this (even though I can't think of cases it would be useful) they still can enable it on their demands.

Thanks for your contribution to Apache Commons! Your help is appreciated!

Before you push a pull request, review this list:

  • Read the contribution guidelines for this project.
  • Run a successful build using the default Maven goal with mvn; that's mvn on the command line by itself.
  • Write unit tests that match behavioral changes, where the tests fail if the changes to the runtime are not applied. This may not always be possible but is a best-practice.
  • Write a pull request description that is detailed enough to understand what the pull request does, how, and why.
  • Each commit in the pull request should have a meaningful subject line and body. Note that commits might be squashed by a maintainer on merge.

In commit 740bcf1 the instruction
'-nouses' was added (but without any rationale) but using that is
extremely dangerous. Without any use clause, all packages are treated as
independent from each other, that means the OSGi resolver is free to
wire these packages to different classloaders if used by a consumer (or
its dependencies).

This now completely removes the offending clause from the instructions,
if any project has a real usecase for this (even though I can't think of
cases it would be useful) they still can enable it on their demands.
@ppkarwasz
Copy link

I would rather migrate from maven-bundle-plugin + moditect-maven-plugin to bnd-maven-plugin for several reasons:

  • The maven-bundle-plugin has a history of sporadic updates. Its releases are not synchronized with BND Tools.
  • The uses clauses in an OSGi descriptor should translate into requires transitive directives in a JPMS descriptor. As far as I can see, the Moditect plugin is not able to detect those: commons-compress has some types from commons-codec in its public API, but I don't see any requires transitive.
  • There should be a correspondence between an OSGi resolution:=optional and a requires static directive. I don't see this in currently published artifacts.

@laeubi
Copy link
Contributor Author

laeubi commented Jan 23, 2025

I would rather migrate from maven-bundle-plugin + moditect-maven-plugin to bnd-maven-plugin for several reasons:

Feel free to propose such a change, but this will break all current users of the common-parent so maybe better make this additional.

In the end both use the same underlying library and felix-maven plugin has some features that bnd-maven plugin do not offer so it is not a 1:1 replacement, in practice there is often no advantage in using one over the other.

In any way, using the -nouses would be bad for any of those and JPMS data can be generated with felix bundle plugin as well (just add <_jpms-module-info>), see https://bnd.bndtools.org/chapters/330-jpms.html so this is really not bound to what maven wrapper is used :-)

@ppkarwasz
Copy link

In any way, using the -nouses would be bad for any of those and JPMS data can be generated with felix bundle plugin as well (just add <_jpms-module-info>), see https://bnd.bndtools.org/chapters/330-jpms.html so this is really not bound to what maven wrapper is used :-)

Good point 💯

Copy link

@ppkarwasz ppkarwasz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@garydgregory
Copy link
Member

It's one thing to say it looks good to you, but did you test it against components? 😉

@ppkarwasz
Copy link

Before this change the exports of commons-compress where:

org.apache.commons.compress;version="1.28.0.SNAPSHOT"
org.apache.commons.compress.archivers;version="1.28.0.SNAPSHOT"
org.apache.commons.compress.archivers.ar;version="1.28.0.SNAPSHOT"
org.apache.commons.compress.archivers.arj;version="1.28.0.SNAPSHOT"
org.apache.commons.compress.archivers.cpio;version="1.28.0.SNAPSHOT"
org.apache.commons.compress.archivers.dump;version="1.28.0.SNAPSHOT"
org.apache.commons.compress.archivers.examples;version="1.28.0.SNAPSHOT"
org.apache.commons.compress.archivers.jar;version="1.28.0.SNAPSHOT"
org.apache.commons.compress.archivers.sevenz;version="1.28.0.SNAPSHOT"
org.apache.commons.compress.archivers.tar;version="1.28.0.SNAPSHOT"
org.apache.commons.compress.archivers.zip;version="1.28.0.SNAPSHOT"
org.apache.commons.compress.changes;version="1.28.0.SNAPSHOT"
org.apache.commons.compress.compressors;version="1.28.0.SNAPSHOT"
org.apache.commons.compress.compressors.brotli;version="1.28.0.SNAPSHOT"
org.apache.commons.compress.compressors.bzip2;version="1.28.0.SNAPSHOT"
org.apache.commons.compress.compressors.deflate;version="1.28.0.SNAPSHOT"
org.apache.commons.compress.compressors.deflate64;version="1.28.0.SNAPSHOT"
org.apache.commons.compress.compressors.gzip;version="1.28.0.SNAPSHOT"
org.apache.commons.compress.compressors.lz4;version="1.28.0.SNAPSHOT"
org.apache.commons.compress.compressors.lz77support;version="1.28.0.SNAPSHOT"
org.apache.commons.compress.compressors.lzma;version="1.28.0.SNAPSHOT"
org.apache.commons.compress.compressors.lzw;version="1.28.0.SNAPSHOT"
org.apache.commons.compress.compressors.pack200;version="1.28.0.SNAPSHOT"
org.apache.commons.compress.compressors.snappy;version="1.28.0.SNAPSHOT"
org.apache.commons.compress.compressors.xz;version="1.28.0.SNAPSHOT"
org.apache.commons.compress.compressors.z;version="1.28.0.SNAPSHOT"
org.apache.commons.compress.compressors.zstandard;version="1.28.0.SNAPSHOT"
org.apache.commons.compress.harmony;version="1.28.0.SNAPSHOT"
org.apache.commons.compress.harmony.archive.internal.nls;version="1.28.0.SNAPSHOT"
org.apache.commons.compress.harmony.pack200;version="1.28.0.SNAPSHOT"
org.apache.commons.compress.harmony.unpack200;version="1.28.0.SNAPSHOT"
org.apache.commons.compress.harmony.unpack200.bytecode;version="1.28.0.SNAPSHOT"
org.apache.commons.compress.harmony.unpack200.bytecode.forms;version="1.28.0.SNAPSHOT"
org.apache.commons.compress.java.util.jar;version="1.28.0.SNAPSHOT"
org.apache.commons.compress.parallel;version="1.28.0.SNAPSHOT"
org.apache.commons.compress.utils;version="1.28.0.SNAPSHOT"

now they are:

org.apache.commons.compress;version="1.28.0.SNAPSHOT"
org.apache.commons.compress.archivers;version="1.28.0.SNAPSHOT";uses:="org.apache.commons.compress,org.apache.commons.io.function"
org.apache.commons.compress.archivers.ar;version="1.28.0.SNAPSHOT";uses:="org.apache.commons.compress.archivers"
org.apache.commons.compress.archivers.arj;version="1.28.0.SNAPSHOT";uses:="org.apache.commons.compress.archivers"
org.apache.commons.compress.archivers.cpio;version="1.28.0.SNAPSHOT";uses:="org.apache.commons.compress.archivers"
org.apache.commons.compress.archivers.dump;version="1.28.0.SNAPSHOT";uses:="org.apache.commons.compress.archivers"
org.apache.commons.compress.archivers.examples;version="1.28.0.SNAPSHOT";uses:="org.apache.commons.compress.archivers,org.apache.commons.compress.archivers.sevenz,org.apache.commons.compress.archivers.tar,org.apache.commons.compress.archivers.zip"
org.apache.commons.compress.archivers.jar;version="1.28.0.SNAPSHOT";uses:="org.apache.commons.compress.archivers,org.apache.commons.compress.archivers.zip"
org.apache.commons.compress.archivers.sevenz;version="1.28.0.SNAPSHOT";uses:="org.apache.commons.compress.archivers,org.apache.commons.compress.utils,org.apache.commons.io.build"
org.apache.commons.compress.archivers.tar;version="1.28.0.SNAPSHOT";uses:="org.apache.commons.compress.archivers,org.apache.commons.compress.archivers.zip"
org.apache.commons.compress.archivers.zip;version="1.28.0.SNAPSHOT";uses:="org.apache.commons.compress.archivers,org.apache.commons.compress.parallel,org.apache.commons.compress.utils,org.apache.commons.io.build,org.apache.commons.io.function"
org.apache.commons.compress.changes;version="1.28.0.SNAPSHOT";uses:="org.apache.commons.compress.archivers,org.apache.commons.compress.archivers.zip"
org.apache.commons.compress.compressors;version="1.28.0.SNAPSHOT";uses:="org.apache.commons.compress"
org.apache.commons.compress.compressors.brotli;version="1.28.0.SNAPSHOT";uses:="org.apache.commons.compress.compressors,org.apache.commons.compress.utils"
org.apache.commons.compress.compressors.bzip2;version="1.28.0.SNAPSHOT";uses:="org.apache.commons.compress.compressors,org.apache.commons.compress.utils"
org.apache.commons.compress.compressors.deflate;version="1.28.0.SNAPSHOT";uses:="org.apache.commons.compress.compressors,org.apache.commons.compress.utils"
org.apache.commons.compress.compressors.deflate64;version="1.28.0.SNAPSHOT";uses:="org.apache.commons.compress.compressors,org.apache.commons.compress.utils"
org.apache.commons.compress.compressors.gzip;version="1.28.0.SNAPSHOT";uses:="org.apache.commons.compress.compressors,org.apache.commons.compress.utils,org.apache.commons.io.build,org.apache.commons.io.function"
org.apache.commons.compress.compressors.lz4;version="1.28.0.SNAPSHOT";uses:="org.apache.commons.codec.digest,org.apache.commons.compress.compressors,org.apache.commons.compress.compressors.lz77support,org.apache.commons.compress.utils"
org.apache.commons.compress.compressors.lz77support;version="1.28.0.SNAPSHOT";uses:="org.apache.commons.compress.compressors,org.apache.commons.compress.utils"
org.apache.commons.compress.compressors.lzma;version="1.28.0.SNAPSHOT";uses:="org.apache.commons.compress.compressors,org.apache.commons.compress.utils,org.tukaani.xz"
org.apache.commons.compress.compressors.lzw;version="1.28.0.SNAPSHOT";uses:="org.apache.commons.compress,org.apache.commons.compress.compressors,org.apache.commons.compress.utils"
org.apache.commons.compress.compressors.pack200;version="1.28.0.SNAPSHOT";uses:="org.apache.commons.compress.compressors"
org.apache.commons.compress.compressors.snappy;version="1.28.0.SNAPSHOT";uses:="org.apache.commons.compress.compressors,org.apache.commons.compress.compressors.lz77support,org.apache.commons.compress.utils"
org.apache.commons.compress.compressors.xz;version="1.28.0.SNAPSHOT";uses:="org.apache.commons.compress.compressors,org.apache.commons.compress.utils,org.tukaani.xz"
org.apache.commons.compress.compressors.z;version="1.28.0.SNAPSHOT";uses:="org.apache.commons.compress.compressors.lzw"
org.apache.commons.compress.compressors.zstandard;version="1.28.0.SNAPSHOT";uses:="com.github.luben.zstd,org.apache.commons.compress.compressors,org.apache.commons.compress.utils"
org.apache.commons.compress.harmony;version="1.28.0.SNAPSHOT"
org.apache.commons.compress.harmony.archive.internal.nls;version="1.28.0.SNAPSHOT"
org.apache.commons.compress.harmony.pack200;version="1.28.0.SNAPSHOT";uses:="org.apache.commons.compress.java.util.jar,org.objectweb.asm"
org.apache.commons.compress.harmony.unpack200;version="1.28.0.SNAPSHOT";uses:="org.apache.commons.compress.harmony.pack200,org.apache.commons.compress.harmony.unpack200.bytecode,org.apache.commons.compress.java.util.jar"
org.apache.commons.compress.harmony.unpack200.bytecode;version="1.28.0.SNAPSHOT";uses:="org.apache.commons.compress.harmony.pack200,org.apache.commons.compress.harmony.unpack200,org.apache.commons.compress.harmony.unpack200.bytecode.forms"
org.apache.commons.compress.harmony.unpack200.bytecode.forms;version="1.28.0.SNAPSHOT";uses:="org.apache.commons.compress.harmony.pack200,org.apache.commons.compress.harmony.unpack200.bytecode"
org.apache.commons.compress.java.util.jar;version="1.28.0.SNAPSHOT"
org.apache.commons.compress.parallel;version="1.28.0.SNAPSHOT"
org.apache.commons.compress.utils;version="1.28.0.SNAPSHOT";uses:="org.apache.commons.compress.archivers,org.apache.commons.io.input"

@garydgregory
Copy link
Member

Maybe someone should ask Niall on the dev list why he needed the "nouses" clause...

@ecki
Copy link
Contributor

ecki commented Jan 23, 2025

In our codebase it’s customary to remove them as they are hard to read and can lead to some broken resolves (i dont recall the details), so from me it would be apreciated to keep them off - but I dont know this particular case/reason.

@laeubi
Copy link
Contributor Author

laeubi commented Jan 24, 2025

I just wanted to quote the spec might be a good first source for derive some normative things:

Classes can depend on classes in other packages. For example, when they extend classes from another package, or these other classes appear in method signatures. It can therefore be said that a package uses other packages. These inter-package dependencies are modeled with the uses directive on the Export-Package header.

The most important part is this one here

Class space consistency can only be ensured if a bundle has only one exporter for each package.

That means that omitting them as "they are hard to read" is really worst thing one can do as it opens the bundle to all kind of class space inconsistencies (and these are really hard to debug). A Manifest is not meant to be read by users, so one better use some tools if really interested in the results.

@garydgregory
Copy link
Member

Maybe this is a basic over-specification problem baked in OSGi, but because a class depends on one class in a package doesn't necessarily mean it depends on the whole package.

@laeubi
Copy link
Contributor Author

laeubi commented Jan 24, 2025

Maybe this is a basic over-specification problem baked in OSGi, but because a class depends on one class in a package doesn't necessarily mean it depends on the whole package.

I don't understand this... you can't depend on a part of a package in java, so if you reference a class X from package a you depend on that package and java do only allows to load the same package from another classloader. So there can't be an a.Y that came from another classloader and that is what use constraint describes. There is even a nice picture that describes this.

@garydgregory
Copy link
Member

Of course you can. You might be conflating this with using JPMS and/or OSGi.

@laeubi
Copy link
Contributor Author

laeubi commented Jan 24, 2025

Of course you can. You might be conflating this with using JPMS and/or OSGi.

I'm not sure what you are talking about here, in any way this is about OSGi metadata that is ignored by all other java code so it is quite useless to discuss what you theoretically can do in special cases, but then you are responsible for ClassCastException yourself.

So the problem is not that you can't load the classes from different classlaoders, they will just not work together what is the whole purpose of Export-Package to allow other use it.

So if you take the example from the spec

grafik

and there is no use clause then A will get 2.1 (Classloader 1) and B 2.4 (Classloader 2) then calling any code working with the package javax.servlet will result in ClassCastException (even without OSGi) for example javax.Servlet instances of A and B cant be cast to each other.

@ecki
Copy link
Contributor

ecki commented Jan 24, 2025

you can't depend on a part of a package in java,

In OSGi you can depend on a single package, so you might get another package from the same bundle (jar) via other classloaders (in OSGi you can get one classloader per "wire" aka import relationship).

(Not sure why JPMS is brought into this issue?)

@ecki
Copy link
Contributor

ecki commented Jan 24, 2025

but because a class depends on one class in a package doesn't necessarily mean it depends on the whole package.

The package imnport in the meta data is always on the whole package. But the problem is different packages might not come the same way, even when they are provided by one bundle (however in practice if you declare most bundles toplevel thats a seldom issue - thats why we avoid the uses, but I can understand that the commons artifacts might want to be strict in what they declare. Just make sure to write a release note when artifacts use the new parent).

@garydgregory
Copy link
Member

@ecki
How would you phrase this warning in the release notes so it doesn't alarm non-OSGi users and gives OSGi folks the information they need?

@ecki
Copy link
Contributor

ecki commented Jan 24, 2025

@ecki How would you phrase this warning in the release notes so it doesn't alarm non-OSGi users and gives OSGi folks the information they need?

Maybe:

Due to new OSGi packaging policy (introduced by cp:x.x.x) the OSGi package imports now state uses definitions for package imports. This should restore proper behavior in OSGi environemnts and not affect different deployment models (including JPMS).

Not sure if we also want to mention that it had that before.

@ecki
Copy link
Contributor

ecki commented Jan 24, 2025 via email

@laeubi
Copy link
Contributor Author

laeubi commented Jan 24, 2025

In OSGi you can depend on a single package, so you might get another package from the same bundle (jar) via other classloaders (in OSGi you can get one classloader per "wire" aka import relationship).

I think it might getting to detailed here, but yes that's exactly the problem. If you have no use-clause then this "wire" will probably use another classpace --> ClassCastException. With use-clause the OSGI resolver ensures that only compatible wires are established.

JPMS is just "similar" that each module (classpath) has similar restrictions (for good reasons), so the so called "split package" (e.g. having package A+B from different classlaoders/modules) is not allowed.

Not sure if we also want to mention that it had that before.

This is nothing new, the earliest mention in the specification I could found was Release 4 from August 2005, I would just add a note that it is no longer removed by default and if all risks are properly known might be enabled individually.

@ecki
Copy link
Contributor

ecki commented Jan 24, 2025

Not sure if we also want to mention that it had that before.

This is nothing new, the earliest mention in the specification I could found was Release 4 from August 2005, I would just add a note that it is no longer removed by default and if all risks are properly known might be enabled individually.

No I mean i am not sure if in the release note warning about this revert if we also want to mention that it was present before (after all the -nouses is a fairly recent (unexplained) addition to CP).

@laeubi
Copy link
Contributor Author

laeubi commented Jan 24, 2025

It was added here at 2008 so if you can call this recent I'm not sure :-)

So it might be that 2005 it was added by the OSGi spec, then it needed some time to show up in manifests and the people wanted to get rid of it because it was "different than before"... so 2008 this was disabled for commons.... and now 20 years later we can only guess.

@ecki
Copy link
Contributor

ecki commented Jan 24, 2025

so 2008 this was disabled for commons.... and now 20 years later we can only guess.

Yeah at least nobody had an issue with it since then (asuming that config from the parent is used in for all commons artifacts). But yes we don't need to mention that detail.

@ecki
Copy link
Contributor

ecki commented Jan 24, 2025

Btw I just remembered a concrete problem, if a package has a wide surface its uses might drag in (depend on) packages which are in the actual environment not used, therefore increasing the dependency graph. (Not sure if it would fail if not provided). It’s arguable bad design to have such optional dependencies but as a dependent party you can’t be picky.

I can imagine for the usual commons bundle that should be quite seldom, though.

And also, in the case of our applications with a few thousand packages the resolver gets very slow and resource intensive if all of them had multiple uses declarations,

@laeubi
Copy link
Contributor Author

laeubi commented Jan 24, 2025

That should not be a problem, as a package in use is just a hint to the resolver that says:

  • If package is imported by consumer
  • And consumer (or any of its dependencies) also require any package that is mentioned
  • Then they need to share the same provider for that package

So if the package is not there (because its optional) or not used (because its only used by the provider) then nothing will be required.

@garydgregory
Copy link
Member

It sounds like we have concensus to bring this in. Let's wait another day or so, and if no one objects, I'll merge this in and create a release candidate.

@garydgregory garydgregory merged commit 68c49c2 into apache:master Jan 25, 2025
9 checks passed
garydgregory added a commit that referenced this pull request Jan 25, 2025
now state 'uses' definitions for package imports, this doesn't affect
JPMS #581
@garydgregory
Copy link
Member

I created the release candidate, please see the dev mailing list https://lists.apache.org/thread/3soxzd1bpg6z8vozww68phdcvj4mto0w

@laeubi
Copy link
Contributor Author

laeubi commented Jan 26, 2025

Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants