Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-28229

HiveTableOperations - Fasterxml Shading Problem (hive-iceberg-shading)



    • Bug
    • Status: Open
    • Minor
    • Resolution: Unresolved
    • 4.0.0, 4.0.0-beta-1
    • None
    • HCatalog, Hive
    • None
    • openjdk 18.0.2-ea 2022-07-19
      OpenJDK Runtime Environment (build 18.0.2-ea+9-Ubuntu-222.04)
      OpenJDK 64-Bit Server VM (build 18.0.2-ea+9-Ubuntu-222.04, mixed mode, sharing)




      I've been using apache.hive with apache.iceberg in order to communicate with a Hive MetaStore. Since the 4.0.0-beta-1 changes, underscores for all environment variables have been introduced. Instead of using the iceberg-hive-metastore dependency with including HiveCatalog, HiveTableOperations I had to switch to hive-iceberg-catalog from apache.hive dependency because of the now missing underscores in the apache.iceberg HiveCatalog class version.

      Normally, this shouldn't be a problem but I figured that the HiveTableOperations class of apache.hive is using the relocated fasterxml dependency and then calls the JsonUtil class when using setSnapshotSummary, where fasterxml is being imported by the non-relocated fasterxml dependency path.

      Now the Hive Metastore Client breaks because I get the error message that the ObjectMapper method can't be found:

      DEBUG [log] io.grpc.netty.shaded.io.netty.util.internal.logging.AbstractInternalLogger (AbstractInternalLogger.java:214) : [id: 0xc34d5ce5, L:/ - R:/] OUTBOUND HEADERS: streamId=3 headers=GrpcHttp2OutboundHeaders[:status: 200, content-type: application/grpc, grpc-status: 2, grpc-message: Application error processing RPC] padding=0 endStream=true java.lang.NoSuchMethodError: 'org.apache.hive.iceberg.com.fasterxml.jackson.databind.ObjectMapper org.apache.iceberg.util.JsonUtil.mapper()' at org.apache.iceberg.hive.HiveTableOperations.setSnapshotSummary(HiveTableOperations.java:435) at org.apache.iceberg.hive.HiveTableOperations.setSnapshotStats(HiveTableOperations.java:426) at org.apache.iceberg.hive.HiveTableOperations.setHmsTableParameters(HiveTableOperations.java:408) at org.apache.iceberg.hive.HiveTableOperations.doCommit(HiveTableOperations.java:222) at org.apache.iceberg.BaseMetastoreTableOperations.commit(BaseMetastoreTableOperations.java:135) at org.apache.iceberg.BaseTransaction.lambda$commitSimpleTransaction$3(BaseTransaction.java:417) at org.apache.iceberg.util.Tasks$Builder.runTaskWithRetry(Tasks.java:413) at org.apache.iceberg.util.Tasks$Builder.runSingleThreaded(Tasks.java:219) at org.apache.iceberg.util.Tasks$Builder.run(Tasks.java:203) at org.apache.iceberg.util.Tasks$Builder.run(Tasks.java:196) at org.apache.iceberg.BaseTransaction.commitSimpleTransaction(BaseTransaction.java:413) at org.apache.iceberg.BaseTransaction.commitTransaction(BaseTransaction.java:308) at

      The only workound I could find is to manually add the Shadings of package hive-iceberg-shading from apache.hive to my own pom. But now I need to always build an uber jar.

      Did I find a bug here or is this expected behavior? If so, how can this be fixed?




            Unassigned Unassigned
            dom93dddd Dominik Diedrich
            0 Vote for this issue
            1 Start watching this issue