Hive: Add View support for HIVE catalog#9852

Merged

nastra merged 17 commits intoapache:mainfrom

nk1506:hive_view_new

Sep 13, 2024

Contributor

nk1506 commented Mar 2, 2024

This changes include:

Introduction of common metadata interface(BaseMetadata) for table and view.
Refactor for HiveTableOperation to have common code for table and view commits.

Ref: #9682 , #9461 , #8907

github-actions bot added API core hive labels

nk1506 mentioned this pull request

Core: Common metadata for TableMetadata and ViewMetadata #9682

Closed

nk1506 marked this pull request as ready for review

March 4, 2024 18:45

szehon-ho reviewed

View reviewed changes

Member

szehon-ho left a comment

Thanks for combining the earlier prs. It's clearer now.

Some preliminary comments.

@nastra please take a look as well to see if there are any major concerns.

hive-metastore/src/main/java/org/apache/iceberg/hive/HiveViewOperations.java Outdated Show resolved Hide resolved

core/src/main/java/org/apache/iceberg/BaseMetastoreOperations.java Outdated Show resolved Hide resolved

core/src/main/java/org/apache/iceberg/BaseMetastoreOperations.java Outdated Show resolved Hide resolved

core/src/main/java/org/apache/iceberg/BaseMetastoreTableOperations.java Outdated Show resolved Hide resolved

core/src/main/java/org/apache/iceberg/BaseMetastoreTableOperations.java Outdated Show resolved Hide resolved

hive-metastore/src/main/java/org/apache/iceberg/hive/HiveViewOperations.java Outdated Show resolved Hide resolved

core/src/main/java/org/apache/iceberg/BaseMetastoreTableOperations.java Outdated Show resolved Hide resolved

hive-metastore/src/main/java/org/apache/iceberg/hive/HiveViewOperations.java Outdated Show resolved Hide resolved

hive-metastore/src/main/java/org/apache/iceberg/hive/HiveOperationsBase.java Outdated Show resolved Hide resolved

hive-metastore/src/main/java/org/apache/iceberg/hive/HiveOperationsBase.java Outdated Show resolved Hide resolved

szehon-ho reviewed

View reviewed changes

hive-metastore/src/main/java/org/apache/iceberg/hive/HiveCatalog.java Outdated Show resolved Hide resolved

hive-metastore/src/main/java/org/apache/iceberg/hive/HiveCatalog.java Outdated Show resolved Hide resolved

hive-metastore/src/main/java/org/apache/iceberg/hive/HiveCatalog.java Outdated Show resolved Hide resolved

hive-metastore/src/main/java/org/apache/iceberg/hive/HiveCatalog.java Outdated Show resolved Hide resolved

hive-metastore/src/main/java/org/apache/iceberg/hive/HiveCatalog.java Outdated Show resolved Hide resolved

hive-metastore/src/main/java/org/apache/iceberg/hive/HiveCatalog.java Outdated Show resolved Hide resolved

hive-metastore/src/main/java/org/apache/iceberg/hive/HiveOperationsBase.java Outdated Show resolved Hide resolved

nk1506 force-pushed the hive_view_new branch from 8969987 to e9d1660 Compare

March 5, 2024 10:53

szehon-ho reviewed

View reviewed changes

hive-metastore/src/main/java/org/apache/iceberg/hive/HiveCatalog.java Show resolved Hide resolved

szehon-ho reviewed

View reviewed changes

hive-metastore/src/main/java/org/apache/iceberg/hive/HiveCatalog.java Outdated Show resolved Hide resolved

nk1506 force-pushed the hive_view_new branch from e9d1660 to bb2efa4 Compare

March 6, 2024 05:56

nastra reviewed

View reviewed changes

.palantir/revapi.yml Outdated Show resolved Hide resolved

nastra reviewed

View reviewed changes

core/src/main/java/org/apache/iceberg/CatalogUtil.java Outdated Show resolved Hide resolved

nastra reviewed

View reviewed changes

core/src/main/java/org/apache/iceberg/CatalogUtil.java Show resolved Hide resolved

nastra reviewed

View reviewed changes

core/src/main/java/org/apache/iceberg/MetadataTableUtils.java Outdated Show resolved Hide resolved

nastra reviewed

View reviewed changes

core/src/main/java/org/apache/iceberg/inmemory/InMemoryCatalog.java Outdated Show resolved Hide resolved

nastra reviewed

View reviewed changes

hive-metastore/src/main/java/org/apache/iceberg/hive/HiveCatalog.java Outdated Show resolved Hide resolved

nastra reviewed

View reviewed changes

hive-metastore/src/main/java/org/apache/iceberg/hive/HiveCatalog.java Outdated Show resolved Hide resolved

nastra reviewed

View reviewed changes

hive-metastore/src/main/java/org/apache/iceberg/hive/HiveCatalog.java Outdated Show resolved Hide resolved

nastra reviewed

View reviewed changes

hive-metastore/src/main/java/org/apache/iceberg/hive/HiveCatalog.java Outdated Show resolved Hide resolved

nastra reviewed

View reviewed changes

hive-metastore/src/main/java/org/apache/iceberg/hive/HiveCatalog.java Outdated Show resolved Hide resolved

nastra reviewed

View reviewed changes

hive-metastore/src/main/java/org/apache/iceberg/hive/HiveCatalog.java Outdated Show resolved Hide resolved

nastra reviewed

View reviewed changes

hive-metastore/src/main/java/org/apache/iceberg/hive/HiveCatalog.java Outdated Show resolved Hide resolved

nastra reviewed

View reviewed changes

hive-metastore/src/main/java/org/apache/iceberg/hive/HiveCatalog.java Outdated Show resolved Hide resolved

nastra reviewed

View reviewed changes

hive-metastore/src/main/java/org/apache/iceberg/hive/HiveCatalog.java Outdated Show resolved Hide resolved

nastra reviewed

View reviewed changes

hive-metastore/src/main/java/org/apache/iceberg/hive/HiveOperationsBase.java Outdated Show resolved Hide resolved

nastra reviewed

View reviewed changes

hive-metastore/src/main/java/org/apache/iceberg/hive/HiveTableOperations.java Outdated Show resolved Hide resolved

nastra reviewed

View reviewed changes

.palantir/revapi.yml Outdated Show resolved Hide resolved

nastra reviewed

View reviewed changes

core/src/main/java/org/apache/iceberg/BaseMetadata.java Outdated Show resolved Hide resolved

nastra reviewed

View reviewed changes

core/src/main/java/org/apache/iceberg/BaseMetastoreOperations.java Outdated Show resolved Hide resolved

nastra reviewed

View reviewed changes

hive-metastore/src/test/java/org/apache/iceberg/hive/TestHiveViewCommits.java Outdated Show resolved Hide resolved


          Addressed comments: Fix incorrect tests from TestHiveViewCommits.

b8c0446

nastra reviewed

View reviewed changes

hive-metastore/src/test/java/org/apache/iceberg/hive/TestHiveViewCommits.java Show resolved Hide resolved

nastra reviewed

View reviewed changes

hive-metastore/src/test/java/org/apache/iceberg/hive/TestHiveViewCommits.java Show resolved Hide resolved

nastra reviewed

View reviewed changes

hive-metastore/src/test/java/org/apache/iceberg/hive/TestHiveViewCommits.java Outdated Show resolved Hide resolved

nastra reviewed

View reviewed changes

hive-metastore/src/test/java/org/apache/iceberg/hive/TestHiveViewCommits.java Show resolved Hide resolved

nastra reviewed

View reviewed changes

hive-metastore/src/test/java/org/apache/iceberg/hive/TestHiveViewCommits.java Show resolved Hide resolved

nk1506 force-pushed the hive_view_new branch from dd82f4b to 7da5812 Compare

September 13, 2024 13:25


          Addressed comments: added metadata file counts for tests of TestHiveV…

2b88e13

…iewCommits

nk1506 force-pushed the hive_view_new branch from 7da5812 to 2b88e13 Compare

September 13, 2024 14:02

Contributor Author

nk1506 commented Sep 13, 2024

Build is failing because of flaky issue #11046

nastra approved these changes

View reviewed changes

Contributor

nastra left a comment

will wait a bit with merging in case @danielcweeks has some additional comments

nastra closed this

nastra reopened this

danielcweeks approved these changes

View reviewed changes

Contributor

nastra commented Sep 13, 2024

thanks @nk1506 for your patience here. Also thanks for everyone that helped out with reviews

nastra merged commit e449d34 into apache:main

Contributor Author

nk1506 commented Sep 13, 2024

Thanks @nastra and @danielcweeks for your reviews and feedback.

nqvuong1998 commented Sep 17, 2024

Hi @nastra @danielcweeks , Is this pull request scheduled for inclusion in the v1.7 release?

Contributor

nastra commented Sep 17, 2024

@nqvuong1998 yes that will be shipped with 1.7.0

ajantha-bhat mentioned this pull request

Add view support for Hive catalog #8698

Closed

nqvuong1998 mentioned this pull request

Add Iceberg view support for Hive catalog in Iceberg connector trinodb/trino#24093

Open

zachdisc pushed a commit to zachdisc/iceberg that referenced this pull request


          Hive: Add View support for HIVE catalog (apache#9852)

100faa5

ajantha-bhat mentioned this pull request

Add Support for Views in the HIVE Catalog #12118

Closed

3 tasks

FANNG1 mentioned this pull request

Iceberg Rest Unable to list tables for Hive Metastore with mixed Hive and Iceberg apache/gravitino#6382

Open

hsiang-c mentioned this pull request

Core: JDBCCatalog's dropView() should purge metadata files if GC is enabled #12511

Merged

monologuist mentioned this pull request

Spark: Support creating views via SQL #9423

Merged

zhongyujiang pushed a commit to zhongyujiang/iceberg that referenced this pull request


          Hive: Add View support for HIVE catalog (apache#9852)

1b1b2f6

xxzhky pushed a commit to xxzhky/iceberg that referenced this pull request


          Hive: Add View support for HIVE catalog (apache#9852)

ae43f56

(cherry picked from commit e449d34)

nastra mentioned this pull request

Issues caused by adding view functionality to SparkSessionCatalog #14557

Open

3 tasks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

pvary pvary left review comments

szehon-ho szehon-ho left review comments

nastra nastra approved these changes

danielcweeks danielcweeks approved these changes

+1 more reviewer

vteja11 vteja11 left review comments

Labels