Sqoop
  1. Sqoop
  2. SQOOP-1117

when failed to import a non-existing table, the failure information includes NullPointerException

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 1.4.3
    • Fix Version/s: 1.4.5
    • Component/s: tools
    • Labels:
      None
    • Environment:

      RHEL 5.4

      Description

      when failed to import a non-existing table, the returned failure information includes java.lang.NullPointerException which customer does not care about. These exception might confuse customers.

      a) command:

      sqoop import --connect jdbc:db2://hostname:50000/SAMPLE --table DB2ADMIN.DB2TEST_TBL1001 --username sam --password pwd -m 1 --target-dir /tmp/DB2TEST_TBL001_0620_f
      

      b) result:

      13/06/26 22:39:50 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
      13/06/26 22:39:50 INFO manager.SqlManager: Using default fetchSize of 1000
      13/06/26 22:39:50 INFO tool.CodeGenTool: Beginning code generation
      13/06/26 22:39:53 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM DB2ADMIN.DB2TEST_TBL1001 AS t WHERE 1=0
      13/06/26 22:39:53 ERROR manager.SqlManager: Error executing statement: com.ibm.db2.jcc.am.SqlSyntaxErrorException: DB2 SQL Error: SQLCODE=-204, SQLSTATE=42704, SQLERRMC=DB2ADMIN.DB2TEST_TBL1001, DRIVER=4.12.55
      com.ibm.db2.jcc.am.SqlSyntaxErrorException: DB2 SQL Error: SQLCODE=-204, SQLSTATE=42704, SQLERRMC=DB2ADMIN.DB2TEST_TBL1001, DRIVER=4.12.55
              at com.ibm.db2.jcc.am.hd.a(hd.java:676)
              at com.ibm.db2.jcc.am.hd.a(hd.java:60)
              at com.ibm.db2.jcc.am.hd.a(hd.java:127)
              at com.ibm.db2.jcc.am.mn.c(mn.java:2621)
              at com.ibm.db2.jcc.am.mn.d(mn.java:2609)
              at com.ibm.db2.jcc.am.mn.a(mn.java:2085)
              at com.ibm.db2.jcc.am.nn.a(nn.java:7054)
              at com.ibm.db2.jcc.t4.cb.g(cb.java:141)
              at com.ibm.db2.jcc.t4.cb.a(cb.java:41)
              at com.ibm.db2.jcc.t4.q.a(q.java:32)
              at com.ibm.db2.jcc.t4.rb.i(rb.java:135)
              at com.ibm.db2.jcc.am.mn.ib(mn.java:2055)
              at com.ibm.db2.jcc.am.nn.rc(nn.java:3219)
              at com.ibm.db2.jcc.am.nn.b(nn.java:4002)
              at com.ibm.db2.jcc.am.nn.ec(nn.java:728)
              at com.ibm.db2.jcc.am.nn.executeQuery(nn.java:698)
              at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:539)
              at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:548)
              at org.apache.sqoop.manager.SqlManager.getColumnTypesForRawQuery(SqlManager.java:191)
              at org.apache.sqoop.manager.SqlManager.getColumnTypes(SqlManager.java:175)
              at org.apache.sqoop.manager.ConnManager.getColumnTypes(ConnManager.java:262)
              at org.apache.sqoop.orm.ClassWriter.getColumnTypes(ClassWriter.java:1235)
              at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1060)
              at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:82)
              at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:390)
              at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:476)
              at org.apache.sqoop.Sqoop.run(Sqoop.java:145)
              at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
              at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181)
              at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220)
              at org.apache.sqoop.Sqoop.runTool(Sqoop.java:229)
              at org.apache.sqoop.Sqoop.main(Sqoop.java:238)
              at com.cloudera.sqoop.Sqoop.main(Sqoop.java:57)
      13/06/26 22:39:53 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM DB2ADMIN.DB2TEST_TBL1001 AS t WHERE 1=0
      13/06/26 22:39:53 ERROR manager.SqlManager: Error executing statement: com.ibm.db2.jcc.am.SqlSyntaxErrorException: DB2 SQL Error: SQLCODE=-204, SQLSTATE=42704, SQLERRMC=DB2ADMIN.DB2TEST_TBL1001, DRIVER=4.12.55
      com.ibm.db2.jcc.am.SqlSyntaxErrorException: DB2 SQL Error: SQLCODE=-204, SQLSTATE=42704, SQLERRMC=DB2ADMIN.DB2TEST_TBL1001, DRIVER=4.12.55
              at com.ibm.db2.jcc.am.hd.a(hd.java:676)
              at com.ibm.db2.jcc.am.hd.a(hd.java:60)
              at com.ibm.db2.jcc.am.hd.a(hd.java:127)
              at com.ibm.db2.jcc.am.mn.c(mn.java:2621)
              at com.ibm.db2.jcc.am.mn.d(mn.java:2609)
              at com.ibm.db2.jcc.am.mn.a(mn.java:2085)
              at com.ibm.db2.jcc.am.nn.a(nn.java:7054)
              at com.ibm.db2.jcc.t4.cb.g(cb.java:141)
              at com.ibm.db2.jcc.t4.cb.a(cb.java:41)
              at com.ibm.db2.jcc.t4.q.a(q.java:32)
              at com.ibm.db2.jcc.t4.rb.i(rb.java:135)
              at com.ibm.db2.jcc.am.mn.ib(mn.java:2055)
              at com.ibm.db2.jcc.am.nn.rc(nn.java:3219)
              at com.ibm.db2.jcc.am.nn.b(nn.java:4002)
              at com.ibm.db2.jcc.am.nn.ec(nn.java:728)
              at com.ibm.db2.jcc.am.nn.executeQuery(nn.java:698)
              at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:539)
              at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:548)
              at org.apache.sqoop.manager.SqlManager.getColumnNamesForRawQuery(SqlManager.java:126)
              at org.apache.sqoop.manager.SqlManager.getColumnNames(SqlManager.java:110)
              at org.apache.sqoop.orm.ClassWriter.getColumnNames(ClassWriter.java:1207)
              at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1062)
              at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:82)
              at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:390)
              at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:476)
              at org.apache.sqoop.Sqoop.run(Sqoop.java:145)
              at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
              at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181)
              at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220)
              at org.apache.sqoop.Sqoop.runTool(Sqoop.java:229)
              at org.apache.sqoop.Sqoop.main(Sqoop.java:238)
              at com.cloudera.sqoop.Sqoop.main(Sqoop.java:57)
      13/06/26 22:39:53 WARN tool.BaseSqoopTool: Error while closing connection: com.ibm.db2.jcc.am.SqlException: [jcc][t4][10251][10308][4.12.55] java.sql.Connection.close() requested while a transaction is in progress on the connection.
      The transaction remains active, and the connection cannot be closed. ERRORCODE=-4471, SQLSTATE=null
      13/06/26 22:39:53 ERROR sqoop.Sqoop: Got exception running Sqoop: java.lang.NullPointerException
      java.lang.NullPointerException
              at org.apache.sqoop.orm.ClassWriter.cleanColNames(ClassWriter.java:1045)
              at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1066)
              at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:82)
              at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:390)
              at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:476)
              at org.apache.sqoop.Sqoop.run(Sqoop.java:145)
              at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
              at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181)
              at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220)
              at org.apache.sqoop.Sqoop.runTool(Sqoop.java:229)
              at org.apache.sqoop.Sqoop.main(Sqoop.java:238)
              at com.cloudera.sqoop.Sqoop.main(Sqoop.java:57)
      

        Activity

        Hide
        Hudson added a comment -

        SUCCESS: Integrated in Sqoop-ant-jdk-1.6-hadoop23 #1087 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop23/1087/)
        SQOOP-1117: when failed to import a non-existing table, the failure information includes NullPointerException (jarcec: https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=72e3bfdd6677f8630678d8aa94da7a8652692033)

        • src/java/org/apache/sqoop/orm/ClassWriter.java
        Show
        Hudson added a comment - SUCCESS: Integrated in Sqoop-ant-jdk-1.6-hadoop23 #1087 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop23/1087/ ) SQOOP-1117 : when failed to import a non-existing table, the failure information includes NullPointerException (jarcec: https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=72e3bfdd6677f8630678d8aa94da7a8652692033 ) src/java/org/apache/sqoop/orm/ClassWriter.java
        Hide
        Hudson added a comment -

        SUCCESS: Integrated in Sqoop-ant-jdk-1.6-hadoop20 #884 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop20/884/)
        SQOOP-1117: when failed to import a non-existing table, the failure information includes NullPointerException (jarcec: https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=72e3bfdd6677f8630678d8aa94da7a8652692033)

        • src/java/org/apache/sqoop/orm/ClassWriter.java
        Show
        Hudson added a comment - SUCCESS: Integrated in Sqoop-ant-jdk-1.6-hadoop20 #884 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop20/884/ ) SQOOP-1117 : when failed to import a non-existing table, the failure information includes NullPointerException (jarcec: https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=72e3bfdd6677f8630678d8aa94da7a8652692033 ) src/java/org/apache/sqoop/orm/ClassWriter.java
        Hide
        Hudson added a comment -

        SUCCESS: Integrated in Sqoop-ant-jdk-1.6-hadoop200 #890 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop200/890/)
        SQOOP-1117: when failed to import a non-existing table, the failure information includes NullPointerException (jarcec: https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=72e3bfdd6677f8630678d8aa94da7a8652692033)

        • src/java/org/apache/sqoop/orm/ClassWriter.java
        Show
        Hudson added a comment - SUCCESS: Integrated in Sqoop-ant-jdk-1.6-hadoop200 #890 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop200/890/ ) SQOOP-1117 : when failed to import a non-existing table, the failure information includes NullPointerException (jarcec: https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=72e3bfdd6677f8630678d8aa94da7a8652692033 ) src/java/org/apache/sqoop/orm/ClassWriter.java
        Hide
        Hudson added a comment -

        SUCCESS: Integrated in Sqoop-ant-jdk-1.6-hadoop100 #849 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop100/849/)
        SQOOP-1117: when failed to import a non-existing table, the failure information includes NullPointerException (jarcec: https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=72e3bfdd6677f8630678d8aa94da7a8652692033)

        • src/java/org/apache/sqoop/orm/ClassWriter.java
        Show
        Hudson added a comment - SUCCESS: Integrated in Sqoop-ant-jdk-1.6-hadoop100 #849 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop100/849/ ) SQOOP-1117 : when failed to import a non-existing table, the failure information includes NullPointerException (jarcec: https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=72e3bfdd6677f8630678d8aa94da7a8652692033 ) src/java/org/apache/sqoop/orm/ClassWriter.java
        Hide
        Jarek Jarcec Cecho added a comment -

        Committed, thank you for your contribution Sam Liu!

        Show
        Jarek Jarcec Cecho added a comment - Committed, thank you for your contribution Sam Liu!
        Hide
        ASF subversion and git services added a comment -

        Commit 72e3bfdd6677f8630678d8aa94da7a8652692033 in sqoop's branch refs/heads/trunk from Jarek Jarcec Cecho
        [ https://git-wip-us.apache.org/repos/asf?p=sqoop.git;h=72e3bfd ]

        SQOOP-1117: when failed to import a non-existing table, the failure information includes NullPointerException

        (sam liu via Jarek Jarcec Cecho)

        Show
        ASF subversion and git services added a comment - Commit 72e3bfdd6677f8630678d8aa94da7a8652692033 in sqoop's branch refs/heads/trunk from Jarek Jarcec Cecho [ https://git-wip-us.apache.org/repos/asf?p=sqoop.git;h=72e3bfd ] SQOOP-1117 : when failed to import a non-existing table, the failure information includes NullPointerException (sam liu via Jarek Jarcec Cecho)
        Hide
        Jarek Jarcec Cecho added a comment -

        +1

        Show
        Jarek Jarcec Cecho added a comment - +1
        Hide
        sam liu added a comment -

        As the solution mentioned above, with the attached patch, the NPE will not happen any more, but with a more meaningful reminder as '13/07/01 20:06:39 ERROR tool.ImportTool: Imported Failed: There is no column found in the target table DB2ADMIN.DB2TEST_TBL1001. Please ensure your table name is correct.'

        Show
        sam liu added a comment - As the solution mentioned above, with the attached patch, the NPE will not happen any more, but with a more meaningful reminder as '13/07/01 20:06:39 ERROR tool.ImportTool: Imported Failed: There is no column found in the target table DB2ADMIN.DB2TEST_TBL1001. Please ensure your table name is correct.'
        Hide
        sam liu added a comment -

        This issue happens on line 'at org.apache.sqoop.orm.ClassWriter.cleanColNames(ClassWriter.java:1057)': when it wants to get the length of the String array object colNames, the NPE occurs as the object colNames is null if the target table is not existing at all.

        So the resolution is to verify the object colNames first, and then will throw an exception to user for reminder if the input target table is not existing.

        Show
        sam liu added a comment - This issue happens on line 'at org.apache.sqoop.orm.ClassWriter.cleanColNames(ClassWriter.java:1057)': when it wants to get the length of the String array object colNames, the NPE occurs as the object colNames is null if the target table is not existing at all. So the resolution is to verify the object colNames first, and then will throw an exception to user for reminder if the input target table is not existing.

          People

          • Assignee:
            sam liu
            Reporter:
            sam liu
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development