Sqoop
  1. Sqoop
  2. SQOOP-1130

Sqoop2: Sqoop MapReduce Job Failed when max and min boundary are both null

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 1.99.2
    • Fix Version/s: 1.99.3
    • Component/s: connectors/generic
    • Labels:
      None
    • Environment:

      Mysql import data to HDFS

      Description

      It happens that all the values in the partition column are null. In this case, when Sqoop is partitioning data, it will find the max and min boundary are both null. Sqoop handles this case, but there is a bug.

      Currently, the Map task throw the following exception in this case:

      org.apache.sqoop.common.SqoopException: MAPRED_EXEC_0017:Error occurs during extractor run
      	at org.apache.sqoop.job.mr.SqoopMapper.run(SqoopMapper.java:94)
      	at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:672)
      	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:330)
      	at org.apache.hadoop.mapred.Child$4.run(Child.java:268)
      	at java.security.AccessController.doPrivileged(Native Method)
      	at javax.security.auth.Subject.doAs(Subject.java:396)
      	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408)
      	at org.apache.hadoop.mapred.Child.main(Child.java:262)
      Caused by: org.apache.sqoop.common.SqoopException: GENERIC_JDBC_CONNECTOR_0002:Unable to execute the SQL statement
      	at org.apache.sqoop.connector.jdbc.GenericJdbcExecutor.executeQuery(GenericJdbcExecutor.java:59)
      	at org.apache.sqoop.connector.jdbc.GenericJdbcImportExtractor.extract(GenericJdbcImportExtractor.java:51)
      	at org.apache.sqoop.connector.jdbc.GenericJdbcImportExtractor.extract(GenericJdbcImportExtractor.java:31)
      	at org.apache.sqoop.job.mr.SqoopMapper.run(SqoopMapper.java:89)
      	... 7 more
      Caused by: com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'NULL' at line 1
      	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
      	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
      	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
      	at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
      	at com.mysql.jdbc.Util.handleNewInstance(Util.java:411)
      	at com.mysql.jdbc.Util.getInstance(Util.java:386)
      	at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1054)
      	at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:4187)
      	at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:4119)
      	at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2570)
      	at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2731)
      	at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2809)
      	at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2758)
      	at com.mysql.jdbc.StatementImpl.executeQuery(StatementImpl.java:1612)
      	at org.apache.sqoop.connector.jdbc.GenericJdbcExecutor.executeQuery(GenericJdbcExecutor.java:56)
      	... 10 more
      
      1. SQOOP-1130.1.patch
        0.9 kB
        Mengwei Ding

        Issue Links

          Activity

          No work has yet been logged on this issue.

            People

            • Assignee:
              Mengwei Ding
              Reporter:
              Mengwei Ding
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development