Uploaded image for project: 'Sqoop'
  1. Sqoop
  2. SQOOP-1165

When import data into Hive, sqoop pick up the jar files from $HBASE_HOME/lib, not from $HIVE_HOME/lib

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 1.4.3
    • Fix Version/s: None
    • Component/s: tools
    • Labels:
      None
    • Environment:

      RHEL 5.4 x86_64

      Description

      My env includes $HADOOP_HOME, $HBASE_HOME and $HIVE_HOME. But I failed to execute command './sqoop import --connect jdbc:db2://hostname:50000/sample --username user --password pwd --table STAFF --hive-table staff --hive-import --m 1', and it returned exception:
      13/08/05 10:33:35 INFO hive.HiveImport: Exception in thread "main" java.lang.NoSuchMethodError: org/apache/thrift/EncodingUtils.setBit(BIZ)B
      13/08/05 10:33:35 INFO hive.HiveImport: at org.apache.hadoop.hive.metastore.api.PrivilegeGrantInfo.setCreateTimeIsSet(PrivilegeGrantInfo.java:245)
      13/08/05 10:33:35 INFO hive.HiveImport: at org.apache.hadoop.hive.metastore.api.PrivilegeGrantInfo.<init>(PrivilegeGrantInfo.java:163)

      I believe the root cause is that: when import data into hive, this command leveraged the libthrift-0.8.0.jar under folder $HBASE_HOME/lib, but not use the libthrift-0.9.0.jar under folder $HIVE_HOME/lib. This is incorrect, and caused the failure of hive import.

      In the $SQOOP_HOME/bin/configure-sqoop file, it will add the classpath of HBASE into the classpath of SQOOP, but won't add the classpath of HIVE into classpath of SQOOP. The correct way is to add the classpath of HBASE into the classpath of SQOOP when the command relates to hbase, but to add the classpath of HIVE when the command relates to hive.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              sam liu sam liu
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated: