Skip to content

Update iceberg to 0.14.0#13206

Merged
electrum merged 1 commit intotrinodb:masterfrom
danielcweeks:iceberg-0.14.0
Jul 20, 2022
Merged

Update iceberg to 0.14.0#13206
electrum merged 1 commit intotrinodb:masterfrom
danielcweeks:iceberg-0.14.0

Conversation

@danielcweeks
Copy link
Contributor

@danielcweeks danielcweeks commented Jul 18, 2022

Description

Iceberg client library and necessary changes to update to 0.14.0

Is this change a fix, improvement, new feature, refactoring, or other?

Dependency update

Is this a change to the core query engine, a connector, client library, or the SPI interfaces? (be specific)

Client library

How would you describe this change to a non-technical end user or system administrator?

Related issues, pull requests, and links

Documentation

(X) No documentation is needed.
( ) Sufficient documentation is included in this PR.
( ) Documentation PR is available with #prnumber.
( ) Documentation issue #issuenumber is filed, and can be handled later.

Release notes

(X) No release notes entries required.
( ) Release notes entries required with the following suggested text:

# Section
*  ({issue}`issuenumber`)

@electrum
Copy link
Member

We don't mention internal changes in release notes, so this shouldn't need one. If there are any user visible changes, we should mention those changes explicitly in a language the end user or administrator would understand.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this due to a version conflict?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it appears there is a patch-level version dependency drift and the dependency enforcer caught it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure if there's a better way to resolve this. This was the error (repeated for all iceberg dependencies):

Require upper bound dependencies error for org.slf4j:slf4j-api:1.7.32 paths to dependency are:
+-io.trino:trino-iceberg:391-SNAPSHOT
  +-org.apache.iceberg:iceberg-api:0.14.0
    +-org.slf4j:slf4j-api:1.7.32 (managed) <-- org.slf4j:slf4j-api:1.7.25

Iceberg is currently on 1.7.25

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's weird that it complains when a dependency has an older patch version. Note that it would be good to update Iceberg to the latest 1.7.36, since some of the SLF4J components have CVEs in 1.7.25 and stupid security scanners will flag the API JAR (even though it's not affected).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can definitely update in the next release (I'll create a PR for updating). There was a gradle update and it may have changed how these are declared which triggered the enforcer, but the dependency is valid and hasn't changed.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we file an issue for this in Iceberg? This resource should be namespaced to avoid conflicts, or drop it entirely (since Iceberg doesn't use the public suffix functions).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll bring this up because the iceberg-build.properties was recently introduced and is included in all the artifact jars so that it's clear where any given jar came from.

The suffix list is a conflict from http client libraries, I believe.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, I was thinking of the Guava com/google/thirdparty/publicsuffix package which is easy to miss since it's different than com/google/common. If Iceberg isn't the one shading the HTTP client, then there's nothing we can fix here.

For the iceberg-build.properties, you might consider putting it in META-INF, similar to how Maven does it with e.g., META-INF/maven/com.google.guava/guava/pom.properties.

<properties>
<air.main.basedir>${project.parent.basedir}</air.main.basedir>
<dep.iceberg.version>0.13.2</dep.iceberg.version>
<dep.iceberg.version>0.14.0</dep.iceberg.version>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for pointing this out @ebyhr, I'll open a PR

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ebyhr docker image update is here #127

@electrum electrum merged commit b9e0a40 into trinodb:master Jul 20, 2022
@electrum
Copy link
Member

Thanks!

@github-actions github-actions bot added this to the 391 milestone Jul 20, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Development

Successfully merging this pull request may close these issues.

3 participants