Uploaded image for project: 'Apache Drill'
  1. Apache Drill
  2. DRILL-6604

Upgrade Drill Hive client to Hive3.1 version

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 1.14.0
    • Fix Version/s: Future
    • Component/s: None
    • Labels:
      None

      Description

      Recently the Hive3.1 version is released.
      3.1 versions of hive-exec and hive-metastore (main Drill Hive client libraries) are the last available artifacts in maven repo for now:
      https://mvnrepository.com/artifact/org.apache.hive/hive-exec/3.1.0
      https://mvnrepository.com/artifact/org.apache.hive/hive-metastore/3.1.0

      It is necessary to investigate what impact of update onto the newer major Hive client version.

      The initial conflicts after changing the hive.version in Drill root pom file from 2.3.2 to 3.1.0 are:

      [ERROR] COMPILATION ERROR : 
      [INFO] -------------------------------------------------------------
      [ERROR] /home/vitalii/IdeaProjects/drill-fork/contrib/storage-hive/core/src/main/java/org/apache/drill/exec/store/hive/HiveUtilities.java:[58,39] error: cannot find symbol
      [ERROR]   symbol:   class MetaStoreUtils
        location: package org.apache.hadoop.hive.metastore
      /home/vitalii/IdeaProjects/drill-fork/contrib/storage-hive/core/src/main/java/org/apache/drill/exec/store/hive/HiveMetadataProvider.java:[34,39] error: cannot find symbol
      [INFO] 2 errors
      
      [ERROR] COMPILATION ERROR : 
      [INFO] -------------------------------------------------------------
      [ERROR] /home/vitalii/IdeaProjects/drill-fork/contrib/storage-hive/core/src/main/java/org/apache/drill/exec/store/hive/HiveUtilities.java:[575,15] error: cannot find symbol
      [ERROR]   symbol:   method setTransactionalTableScan(JobConf,boolean)
        location: class AcidUtils
      /home/vitalii/IdeaProjects/drill-fork/contrib/storage-hive/core/src/main/java/org/apache/drill/exec/store/hive/HiveFieldConverter.java:[216,92] error: incompatible types: org.apache.hadoop.hive.common.type.Timestamp cannot be converted to java.sql.Timestamp
      [ERROR] /home/vitalii/IdeaProjects/drill-fork/contrib/storage-hive/core/src/main/java/org/apache/drill/exec/store/hive/HiveFieldConverter.java:[225,82] error: incompatible types: org.apache.hadoop.hive.common.type.Date cannot be converted to java.sql.Date
      [ERROR] /home/vitalii/IdeaProjects/drill-fork/contrib/storage-hive/core/src/main/java/org/apache/drill/exec/store/hive/DrillHiveMetaStoreClient.java:[100,17] error: cannot find symbol
      [ERROR]   symbol:   method setTokenStr(UserGroupInformation,String,String)
        location: class Utils
      /home/vitalii/IdeaProjects/drill-fork/contrib/storage-hive/core/target/generated-sources/org/apache/drill/exec/expr/fn/impl/hive/DrillTimeStampTimestampObjectInspector.java:[29,16] error: Required is not abstract and does not override abstract method getPrimitiveJavaObject(Object) in TimestampObjectInspector
      [ERROR] /home/vitalii/IdeaProjects/drill-fork/contrib/storage-hive/core/target/generated-sources/org/apache/drill/exec/expr/fn/impl/hive/DrillTimeStampTimestampObjectInspector.java:[35,30] error: getPrimitiveJavaObject(Object) in Required cannot implement getPrimitiveJavaObject(Object) in TimestampObjectInspector
      [ERROR]   return type java.sql.Timestamp is not compatible with org.apache.hadoop.hive.common.type.Timestamp
      /home/vitalii/IdeaProjects/drill-fork/contrib/storage-hive/core/target/generated-sources/org/apache/drill/exec/expr/fn/impl/hive/DrillTimeStampTimestampObjectInspector.java:[44,29] error: getPrimitiveWritableObject(Object) in Required cannot implement getPrimitiveWritableObject(Object) in TimestampObjectInspector
      [ERROR]   return type TimestampWritable is not compatible with TimestampWritableV2
      /home/vitalii/IdeaProjects/drill-fork/contrib/storage-hive/core/target/generated-sources/org/apache/drill/exec/expr/fn/impl/hive/DrillTimeStampTimestampObjectInspector.java:[54,16] error: Optional is not abstract and does not override abstract method getPrimitiveJavaObject(Object) in TimestampObjectInspector
      [ERROR] /home/vitalii/IdeaProjects/drill-fork/contrib/storage-hive/core/target/generated-sources/org/apache/drill/exec/expr/fn/impl/hive/DrillTimeStampTimestampObjectInspector.java:[60,30] error: getPrimitiveJavaObject(Object) in Optional cannot implement getPrimitiveJavaObject(Object) in TimestampObjectInspector
      [ERROR]   return type java.sql.Timestamp is not compatible with org.apache.hadoop.hive.common.type.Timestamp
      /home/vitalii/IdeaProjects/drill-fork/contrib/storage-hive/core/target/generated-sources/org/apache/drill/exec/expr/fn/impl/hive/DrillTimeStampTimestampObjectInspector.java:[72,29] error: getPrimitiveWritableObject(Object) in Optional cannot implement getPrimitiveWritableObject(Object) in TimestampObjectInspector
      [ERROR]   return type TimestampWritable is not compatible with TimestampWritableV2
      /home/vitalii/IdeaProjects/drill-fork/contrib/storage-hive/core/target/generated-sources/org/apache/drill/exec/expr/fn/impl/hive/DrillDateDateObjectInspector.java:[29,16] error: Required is not abstract and does not override abstract method getPrimitiveJavaObject(Object) in DateObjectInspector
      [ERROR] /home/vitalii/IdeaProjects/drill-fork/contrib/storage-hive/core/target/generated-sources/org/apache/drill/exec/expr/fn/impl/hive/DrillDateDateObjectInspector.java:[35,25] error: getPrimitiveJavaObject(Object) in Required cannot implement getPrimitiveJavaObject(Object) in DateObjectInspector
      [ERROR]   return type java.sql.Date is not compatible with org.apache.hadoop.hive.common.type.Date
      /home/vitalii/IdeaProjects/drill-fork/contrib/storage-hive/core/target/generated-sources/org/apache/drill/exec/expr/fn/impl/hive/DrillDateDateObjectInspector.java:[44,24] error: getPrimitiveWritableObject(Object) in Required cannot implement getPrimitiveWritableObject(Object) in DateObjectInspector
      [ERROR]   return type DateWritable is not compatible with DateWritableV2
      /home/vitalii/IdeaProjects/drill-fork/contrib/storage-hive/core/target/generated-sources/org/apache/drill/exec/expr/fn/impl/hive/DrillDateDateObjectInspector.java:[54,16] error: Optional is not abstract and does not override abstract method getPrimitiveJavaObject(Object) in DateObjectInspector
      [ERROR] /home/vitalii/IdeaProjects/drill-fork/contrib/storage-hive/core/target/generated-sources/org/apache/drill/exec/expr/fn/impl/hive/DrillDateDateObjectInspector.java:[60,25] error: getPrimitiveJavaObject(Object) in Optional cannot implement getPrimitiveJavaObject(Object) in DateObjectInspector
      [ERROR]   return type java.sql.Date is not compatible with org.apache.hadoop.hive.common.type.Date
      /home/vitalii/IdeaProjects/drill-fork/contrib/storage-hive/core/target/generated-sources/org/apache/drill/exec/expr/fn/impl/hive/DrillDateDateObjectInspector.java:[72,24] error: getPrimitiveWritableObject(Object) in Optional cannot implement getPrimitiveWritableObject(Object) in DateObjectInspector
      [INFO] 16 errors
      

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                vitalii Vitalii Diravka
                Reporter:
                vitalii Vitalii Diravka
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated: