Uploaded image for project: 'Sqoop'
  1. Sqoop
  2. SQOOP-481

Sqoop import with --hive-import using wrong column names in --columns throws a NPE

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.4.2
    • Component/s: None
    • Labels:
      None

      Description

      To reproduce the error,

      1) Create a table "foo" with a column name "I" on Oracle DB
      2) Run sqoop import --connect jdbc:oracle:thin:@//localhost/xe --username **** --password **** --verbose --table foo --split-by i --columns i --hive-import

      This generates the following call stack:

      12/05/01 16:12:00 ERROR sqoop.Sqoop: Got exception running Sqoop: java.lang.NullPointerException
      java.lang.NullPointerException
      	at com.cloudera.sqoop.hive.TableDefWriter.getCreateTableStmt(TableDefWriter.java:162)
      	at com.cloudera.sqoop.hive.HiveImport.importTable(HiveImport.java:195)
      	at com.cloudera.sqoop.tool.ImportTool.importTable(ImportTool.java:394)
      	at com.cloudera.sqoop.tool.ImportTool.run(ImportTool.java:455)
      	at com.cloudera.sqoop.Sqoop.run(Sqoop.java:146)
      	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
      	at com.cloudera.sqoop.Sqoop.runSqoop(Sqoop.java:182)
      	at com.cloudera.sqoop.Sqoop.runTool(Sqoop.java:221)
      	at com.cloudera.sqoop.Sqoop.runTool(Sqoop.java:230)
      	at com.cloudera.sqoop.Sqoop.main(Sqoop.java:239)
      

      The reason is simple. In the following lines of code:

      Integer colType = columnTypes.get(col);
      ...
      tring hiveColType = connManager.toHiveType(colType);
      

      colType is null because column "i" does not exist in the table "foo" but "I" exists. Now toHiveType(int colType) tries to autocast a null to a primitive int, resulting a NPE.

      It would be better if more informative message is provided rather than a random NPE.

        Attachments

        1. SQOOP_481.patch
          6 kB
          Cheolsoo Park
        2. SQOOP_481-2.patch
          6 kB
          Cheolsoo Park

          Activity

            People

            • Assignee:
              cheolsoo Cheolsoo Park
              Reporter:
              cheolsoo Cheolsoo Park
            • Votes:
              1 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: