Uploaded image for project: 'Sqoop'
  1. Sqoop
  2. SQOOP-1279

Sqoop connection resiliency option breaks older Mysql versions that don't have JDBC 4 methods

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.4.5
    • Component/s: None
    • Labels:
      None
    • Environment:

      Mysql 5.0.8 connector/J

      Description

      The connection resiliency changes (SQOOP-1056 and SQOOP-1057) add additional checks on the preparedStatement (Statement.isClosed()) that is introduced in JDBC 4 as part of JDK 6. That method is not available in older drivers that are not JDBC4 compliant. Since this is a backward breaking change proposing to fix it.

      1. SQOOP-1279.patch
        1 kB
        Venkat Ranganathan

        Issue Links

          Activity

          Hide
          mkempanna Mala Chikka Kempanna added a comment -

          Sample symptom of the problem when connecting to Informix looks like below
          14/06/25 10:13:32 INFO mapreduce.Job: Task Id : attempt_1403182618659_0156_m_000000_2, Status : FAILED
          Error: java.io.IOException: java.sql.SQLException: Method not supported with this server. : IfxPreparedStatement.isClosed()
          at org.apache.sqoop.mapreduce.db.DBRecordReader.close(DBRecordReader.java:171)
          at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.close(MapTask.java:500)
          at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:768)
          at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
          at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
          at java.security.AccessController.doPrivileged(Native Method)
          at javax.security.auth.Subject.doAs(Subject.java:415)
          at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
          at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)
          Caused by: java.sql.SQLException: Method not supported with this server. : IfxPreparedStatement.isClosed()
          at com.informix.util.IfxErrMsg.getSQLException(IfxErrMsg.java:488)
          at com.informix.jdbc.IfxPreparedStatement.isClosed(IfxPreparedStatement.java:5504)
          at org.apache.sqoop.mapreduce.db.DBRecordReader.close(DBRecordReader.java:163)
          ... 8 more

          Show
          mkempanna Mala Chikka Kempanna added a comment - Sample symptom of the problem when connecting to Informix looks like below 14/06/25 10:13:32 INFO mapreduce.Job: Task Id : attempt_1403182618659_0156_m_000000_2, Status : FAILED Error: java.io.IOException: java.sql.SQLException: Method not supported with this server. : IfxPreparedStatement.isClosed() at org.apache.sqoop.mapreduce.db.DBRecordReader.close(DBRecordReader.java:171) at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.close(MapTask.java:500) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:768) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340) at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548) at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163) Caused by: java.sql.SQLException: Method not supported with this server. : IfxPreparedStatement.isClosed() at com.informix.util.IfxErrMsg.getSQLException(IfxErrMsg.java:488) at com.informix.jdbc.IfxPreparedStatement.isClosed(IfxPreparedStatement.java:5504) at org.apache.sqoop.mapreduce.db.DBRecordReader.close(DBRecordReader.java:163) ... 8 more
          Hide
          venkatnrangan Venkat Ranganathan added a comment -

          Thanks Hari & Abe for the reviews and committing it.

          Show
          venkatnrangan Venkat Ranganathan added a comment - Thanks Hari & Abe for the reviews and committing it.
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Sqoop-ant-jdk-1.6-hadoop100 #841 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop100/841/)
          SQOOP-1279. Sqoop connection resiliency option breaks older Mysql versions that don't have JDBC 4 methods (hshreedharan: https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=c913f77f284cb27ce5aa43fd433b59a52d3032b3)

          • src/java/org/apache/sqoop/mapreduce/db/DBRecordReader.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Sqoop-ant-jdk-1.6-hadoop100 #841 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop100/841/ ) SQOOP-1279 . Sqoop connection resiliency option breaks older Mysql versions that don't have JDBC 4 methods (hshreedharan: https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=c913f77f284cb27ce5aa43fd433b59a52d3032b3 ) src/java/org/apache/sqoop/mapreduce/db/DBRecordReader.java
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Sqoop-ant-jdk-1.6-hadoop20 #876 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop20/876/)
          SQOOP-1279. Sqoop connection resiliency option breaks older Mysql versions that don't have JDBC 4 methods (hshreedharan: https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=c913f77f284cb27ce5aa43fd433b59a52d3032b3)

          • src/java/org/apache/sqoop/mapreduce/db/DBRecordReader.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Sqoop-ant-jdk-1.6-hadoop20 #876 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop20/876/ ) SQOOP-1279 . Sqoop connection resiliency option breaks older Mysql versions that don't have JDBC 4 methods (hshreedharan: https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=c913f77f284cb27ce5aa43fd433b59a52d3032b3 ) src/java/org/apache/sqoop/mapreduce/db/DBRecordReader.java
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Sqoop-ant-jdk-1.6-hadoop23 #1078 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop23/1078/)
          SQOOP-1279. Sqoop connection resiliency option breaks older Mysql versions that don't have JDBC 4 methods (hshreedharan: https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=c913f77f284cb27ce5aa43fd433b59a52d3032b3)

          • src/java/org/apache/sqoop/mapreduce/db/DBRecordReader.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Sqoop-ant-jdk-1.6-hadoop23 #1078 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop23/1078/ ) SQOOP-1279 . Sqoop connection resiliency option breaks older Mysql versions that don't have JDBC 4 methods (hshreedharan: https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=c913f77f284cb27ce5aa43fd433b59a52d3032b3 ) src/java/org/apache/sqoop/mapreduce/db/DBRecordReader.java
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Sqoop-ant-jdk-1.6-hadoop200 #882 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop200/882/)
          SQOOP-1279. Sqoop connection resiliency option breaks older Mysql versions that don't have JDBC 4 methods (hshreedharan: https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=c913f77f284cb27ce5aa43fd433b59a52d3032b3)

          • src/java/org/apache/sqoop/mapreduce/db/DBRecordReader.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Sqoop-ant-jdk-1.6-hadoop200 #882 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop200/882/ ) SQOOP-1279 . Sqoop connection resiliency option breaks older Mysql versions that don't have JDBC 4 methods (hshreedharan: https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=c913f77f284cb27ce5aa43fd433b59a52d3032b3 ) src/java/org/apache/sqoop/mapreduce/db/DBRecordReader.java
          Hide
          hshreedharan Hari Shreedharan added a comment -

          Committed. Thanks Venkat!

          Show
          hshreedharan Hari Shreedharan added a comment - Committed. Thanks Venkat!
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit c913f77f284cb27ce5aa43fd433b59a52d3032b3 in branch refs/heads/trunk from Hari Shreedharan
          [ https://git-wip-us.apache.org/repos/asf?p=sqoop.git;h=c913f77 ]

          SQOOP-1279. Sqoop connection resiliency option breaks older Mysql versions that don't have JDBC 4 methods

          (Venkat Ranganathan via Hari Shreedharan)

          Show
          jira-bot ASF subversion and git services added a comment - Commit c913f77f284cb27ce5aa43fd433b59a52d3032b3 in branch refs/heads/trunk from Hari Shreedharan [ https://git-wip-us.apache.org/repos/asf?p=sqoop.git;h=c913f77 ] SQOOP-1279 . Sqoop connection resiliency option breaks older Mysql versions that don't have JDBC 4 methods (Venkat Ranganathan via Hari Shreedharan)
          Hide
          hshreedharan Hari Shreedharan added a comment -

          +1. Looks good. Will run tests and commit it later today

          Show
          hshreedharan Hari Shreedharan added a comment - +1. Looks good. Will run tests and commit it later today

            People

            • Assignee:
              venkatnrangan Venkat Ranganathan
              Reporter:
              venkatnrangan Venkat Ranganathan
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development