Uploaded image for project: 'Sqoop'
  1. Sqoop
  2. SQOOP-1423

hcatalog export with --map-column-java fails

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 1.4.4, 1.4.6
    • Fix Version/s: 1.4.6
    • Component/s: None
    • Labels:

      Description

      hcat export does not use mappings provided using --map-column-java and crashes with NPE when target table has a non standard sql type.

      Test when exporting to postgres

      Hive table schema

      CREATE TABLE hive_table(col1 String);
      

      Hive table data

      127.0.0.1
      182.27.181.11
      ...
      

      Postgres Table Schema

      CREATE TABLE postgres_table (col1 inet)
      

      Sqoop Command for Exporting

      sqoop export --verbose --connect jdbc:postgresql://pg_server:5432/pg_db?stringtype=unspecified --username user --password xxxxx --table postgres_table --hcatalog-table hive_table --map-column-java col1=String
      

      exception

      14/08/08 12:51:04 INFO hcat.SqoopHCatUtilities: Configuring HCatalog for export job
      14/08/08 12:51:04 INFO hcat.SqoopHCatUtilities: Ignoring configuration request for HCatalog info
      14/08/08 12:51:04 ERROR sqoop.Sqoop: Got exception running Sqoop: java.lang.NullPointerException
      java.lang.NullPointerException
      	at org.apache.hadoop.io.Text.encode(Text.java:432)
      	at org.apache.hadoop.io.Text.set(Text.java:196)
      	at org.apache.hadoop.io.Text.<init>(Text.java:88)
      	at org.apache.sqoop.mapreduce.hcat.SqoopHCatUtilities.configureExportInputFormat(SqoopHCatUtilities.java:708)
      	at org.apache.sqoop.mapreduce.JdbcExportJob.configureInputFormat(JdbcExportJob.java:70)
      	at org.apache.sqoop.mapreduce.ExportJobBase.runExport(ExportJobBase.java:387)
      	at org.apache.sqoop.manager.SqlManager.exportTable(SqlManager.java:828)
      	at org.apache.sqoop.tool.ExportTool.exportTable(ExportTool.java:81)
      	at org.apache.sqoop.tool.ExportTool.run(ExportTool.java:100)
      	at org.apache.sqoop.Sqoop.run(Sqoop.java:145)
      	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
      	at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181)
      	at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220)
      	at org.apache.sqoop.Sqoop.runTool(Sqoop.java:229)
      	at org.apache.sqoop.Sqoop.main(Sqoop.java:238)
      

        Attachments

        1. SQOOP-1423_1.4.4.patch
          2 kB
          Muhammad Ehsan ul Haque
        2. SQOOP-1423_1.4.6.patch
          2 kB
          Muhammad Ehsan ul Haque

          Activity

            People

            • Assignee:
              ehsan Muhammad Ehsan ul Haque
              Reporter:
              ehsan Muhammad Ehsan ul Haque
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: