Sqoop
  1. Sqoop
  2. SQOOP-1453

Sqoop2: Suport export of null from hdfs text files to RDBMS numeric columns

    Details

    • Type: Bug Bug
    • Status: Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: sqoop2-framework
    • Labels:
      None

      Description

      Suppose you have an RDBMS table with number column containing nulls.

      When you Sqoop the data to HDFS text file, you get:
      ,null,

      When you try to Sqoop this file back to RDBMS, you get:
      2014-08-19 15:12:42,552 [OutputFormatLoader-consumer] ERROR org.apache.sqoop.job.mr.SqoopOutputFormatLoadExecutor - Error while loading data out of MR job.
      java.lang.NumberFormatException
      at java.math.BigDecimal.<init>(BigDecimal.java:470)
      at java.math.BigDecimal.<init>(BigDecimal.java:739)
      at org.apache.sqoop.connector.idf.CSVIntermediateDataFormat.getObjectData(CSVIntermediateDataFormat.java:219)
      at org.apache.sqoop.job.mr.SqoopOutputFormatLoadExecutor$OutputFormatDataReader.readArrayRecord(SqoopOutputFormatLoadExecutor.java:157)
      at org.apache.sqoop.connector.jdbc.GenericJdbcLoader.load(GenericJdbcLoader.java:48)
      at org.apache.sqoop.connector.jdbc.GenericJdbcLoader.load(GenericJdbcLoader.java:25)
      at org.apache.sqoop.job.mr.SqoopOutputFormatLoadExecutor$ConsumerThread.run(SqoopOutputFormatLoadExecutor.java:248)
      at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
      at java.util.concurrent.FutureTask.run(FutureTask.java:262)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      at java.lang.Thread.run(Thread.java:745)

      Because "null" doesn't convert to BigDecimal.

      We need to detect handle these cases.

        Activity

        There are no comments yet on this issue.

          People

          • Assignee:
            Unassigned
            Reporter:
            Gwen Shapira
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:

              Development