[DRILL-6604] Upgrade Drill Hive client to Hive3.1 version

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.14.0
• Fix Version/s: 1.18.0
• Labels: ready-to-commit
• Assignee: Vova Vysotskyi
• Reporter: Vitalii Diravka
• Reviewer: Igor Guzenko
• Created: 12/Jul/18 21:52
• Updated: 30/Mar/20 20:44
• Resolved: 30/Mar/20 20:44

Description

Recently the Hive3.1 version is released.
3.1 versions of hive-exec and hive-metastore (main Drill Hive client libraries) are the last available artifacts in maven repo for now:
https://mvnrepository.com/artifact/org.apache.hive/hive-exec/3.1.0
https://mvnrepository.com/artifact/org.apache.hive/hive-metastore/3.1.0

It is necessary to investigate what impact of update onto the newer major Hive client version.

The initial conflicts after changing the hive.version in Drill root pom file from 2.3.2 to 3.1.0 are:

[ERROR] COMPILATION ERROR : 
[INFO] -------------------------------------------------------------
[ERROR] /home/vitalii/IdeaProjects/drill-fork/contrib/storage-hive/core/src/main/java/org/apache/drill/exec/store/hive/HiveUtilities.java:[58,39] error: cannot find symbol
[ERROR]   symbol:   class MetaStoreUtils
  location: package org.apache.hadoop.hive.metastore
/home/vitalii/IdeaProjects/drill-fork/contrib/storage-hive/core/src/main/java/org/apache/drill/exec/store/hive/HiveMetadataProvider.java:[34,39] error: cannot find symbol
[INFO] 2 errors

[ERROR] COMPILATION ERROR : 
[INFO] -------------------------------------------------------------
[ERROR] /home/vitalii/IdeaProjects/drill-fork/contrib/storage-hive/core/src/main/java/org/apache/drill/exec/store/hive/HiveUtilities.java:[575,15] error: cannot find symbol
[ERROR]   symbol:   method setTransactionalTableScan(JobConf,boolean)
  location: class AcidUtils
/home/vitalii/IdeaProjects/drill-fork/contrib/storage-hive/core/src/main/java/org/apache/drill/exec/store/hive/HiveFieldConverter.java:[216,92] error: incompatible types: org.apache.hadoop.hive.common.type.Timestamp cannot be converted to java.sql.Timestamp
[ERROR] /home/vitalii/IdeaProjects/drill-fork/contrib/storage-hive/core/src/main/java/org/apache/drill/exec/store/hive/HiveFieldConverter.java:[225,82] error: incompatible types: org.apache.hadoop.hive.common.type.Date cannot be converted to java.sql.Date
[ERROR] /home/vitalii/IdeaProjects/drill-fork/contrib/storage-hive/core/src/main/java/org/apache/drill/exec/store/hive/DrillHiveMetaStoreClient.java:[100,17] error: cannot find symbol
[ERROR]   symbol:   method setTokenStr(UserGroupInformation,String,String)
  location: class Utils
/home/vitalii/IdeaProjects/drill-fork/contrib/storage-hive/core/target/generated-sources/org/apache/drill/exec/expr/fn/impl/hive/DrillTimeStampTimestampObjectInspector.java:[29,16] error: Required is not abstract and does not override abstract method getPrimitiveJavaObject(Object) in TimestampObjectInspector
[ERROR] /home/vitalii/IdeaProjects/drill-fork/contrib/storage-hive/core/target/generated-sources/org/apache/drill/exec/expr/fn/impl/hive/DrillTimeStampTimestampObjectInspector.java:[35,30] error: getPrimitiveJavaObject(Object) in Required cannot implement getPrimitiveJavaObject(Object) in TimestampObjectInspector
[ERROR]   return type java.sql.Timestamp is not compatible with org.apache.hadoop.hive.common.type.Timestamp
/home/vitalii/IdeaProjects/drill-fork/contrib/storage-hive/core/target/generated-sources/org/apache/drill/exec/expr/fn/impl/hive/DrillTimeStampTimestampObjectInspector.java:[44,29] error: getPrimitiveWritableObject(Object) in Required cannot implement getPrimitiveWritableObject(Object) in TimestampObjectInspector
[ERROR]   return type TimestampWritable is not compatible with TimestampWritableV2
/home/vitalii/IdeaProjects/drill-fork/contrib/storage-hive/core/target/generated-sources/org/apache/drill/exec/expr/fn/impl/hive/DrillTimeStampTimestampObjectInspector.java:[54,16] error: Optional is not abstract and does not override abstract method getPrimitiveJavaObject(Object) in TimestampObjectInspector
[ERROR] /home/vitalii/IdeaProjects/drill-fork/contrib/storage-hive/core/target/generated-sources/org/apache/drill/exec/expr/fn/impl/hive/DrillTimeStampTimestampObjectInspector.java:[60,30] error: getPrimitiveJavaObject(Object) in Optional cannot implement getPrimitiveJavaObject(Object) in TimestampObjectInspector
[ERROR]   return type java.sql.Timestamp is not compatible with org.apache.hadoop.hive.common.type.Timestamp
/home/vitalii/IdeaProjects/drill-fork/contrib/storage-hive/core/target/generated-sources/org/apache/drill/exec/expr/fn/impl/hive/DrillTimeStampTimestampObjectInspector.java:[72,29] error: getPrimitiveWritableObject(Object) in Optional cannot implement getPrimitiveWritableObject(Object) in TimestampObjectInspector
[ERROR]   return type TimestampWritable is not compatible with TimestampWritableV2
/home/vitalii/IdeaProjects/drill-fork/contrib/storage-hive/core/target/generated-sources/org/apache/drill/exec/expr/fn/impl/hive/DrillDateDateObjectInspector.java:[29,16] error: Required is not abstract and does not override abstract method getPrimitiveJavaObject(Object) in DateObjectInspector
[ERROR] /home/vitalii/IdeaProjects/drill-fork/contrib/storage-hive/core/target/generated-sources/org/apache/drill/exec/expr/fn/impl/hive/DrillDateDateObjectInspector.java:[35,25] error: getPrimitiveJavaObject(Object) in Required cannot implement getPrimitiveJavaObject(Object) in DateObjectInspector
[ERROR]   return type java.sql.Date is not compatible with org.apache.hadoop.hive.common.type.Date
/home/vitalii/IdeaProjects/drill-fork/contrib/storage-hive/core/target/generated-sources/org/apache/drill/exec/expr/fn/impl/hive/DrillDateDateObjectInspector.java:[44,24] error: getPrimitiveWritableObject(Object) in Required cannot implement getPrimitiveWritableObject(Object) in DateObjectInspector
[ERROR]   return type DateWritable is not compatible with DateWritableV2
/home/vitalii/IdeaProjects/drill-fork/contrib/storage-hive/core/target/generated-sources/org/apache/drill/exec/expr/fn/impl/hive/DrillDateDateObjectInspector.java:[54,16] error: Optional is not abstract and does not override abstract method getPrimitiveJavaObject(Object) in DateObjectInspector
[ERROR] /home/vitalii/IdeaProjects/drill-fork/contrib/storage-hive/core/target/generated-sources/org/apache/drill/exec/expr/fn/impl/hive/DrillDateDateObjectInspector.java:[60,25] error: getPrimitiveJavaObject(Object) in Optional cannot implement getPrimitiveJavaObject(Object) in DateObjectInspector
[ERROR]   return type java.sql.Date is not compatible with org.apache.hadoop.hive.common.type.Date
/home/vitalii/IdeaProjects/drill-fork/contrib/storage-hive/core/target/generated-sources/org/apache/drill/exec/expr/fn/impl/hive/DrillDateDateObjectInspector.java:[72,24] error: getPrimitiveWritableObject(Object) in Optional cannot implement getPrimitiveWritableObject(Object) in DateObjectInspector
[INFO] 16 errors

Issue Links

• is required by: DRILL-6552 Drill Metadata management "Drill Metastore"
• requires: DRILL-6540 Upgrade to HADOOP-3.0 libraries
• links to: GitHub Pull Request #2038