Sqoop
  1. Sqoop
  2. SQOOP-843

Sqoop2: Generic JDBC connector is committing transaction on export with autoCommit on

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 1.99.1
    • Fix Version/s: 1.99.2
    • Component/s: None
    • Labels:
      None

      Description

      I've noticed that Generic JDBC Connector is not explicitly setting autoCommit and yet is committing transactions. This might lead to issues with for example MySQL JDBC driver that sets the autoCommit to true by default and calling commit will result in following exception being thrown:

      2013-01-23 09:05:02,316 ERROR org.apache.sqoop.job.mr.SqoopOutputFormatLoadExecutor: Error while loading data out of MR job.
      org.apache.sqoop.common.SqoopException: GENERIC_JDBC_CONNECTOR_0002:Unable to execute the SQL statement
      	at org.apache.sqoop.connector.jdbc.GenericJdbcExecutor.executeBatch(GenericJdbcExecutor.java:106)
      	at org.apache.sqoop.connector.jdbc.GenericJdbcExportLoader.load(GenericJdbcExportLoader.java:66)
      	at org.apache.sqoop.connector.jdbc.GenericJdbcExportLoader.load(GenericJdbcExportLoader.java:26)
      	at org.apache.sqoop.job.mr.SqoopOutputFormatLoadExecutor$ConsumerThread.run(SqoopOutputFormatLoadExecutor.java:212)
      	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
      	at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
      	at java.util.concurrent.FutureTask.run(FutureTask.java:138)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
      	at java.lang.Thread.run(Thread.java:662)
      Caused by: java.sql.SQLException: Can't call commit when autocommit=true
      	at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:926)
      	at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:923)
      	at com.mysql.jdbc.ConnectionImpl.commit(ConnectionImpl.java:1683)
      	at org.apache.sqoop.connector.jdbc.GenericJdbcExecutor.executeBatch(GenericJdbcExecutor.java:103)
      	... 9 more
      
      1. bugSQOOP-843.patch
        2 kB
        Jarek Jarcec Cecho

        Issue Links

          Activity

          Hide
          Cheolsoo Park added a comment -

          +1.

          Show
          Cheolsoo Park added a comment - +1.
          Hide
          Cheolsoo Park added a comment -

          Committed to sqoop2. Thanks Jarcec!

          Show
          Cheolsoo Park added a comment - Committed to sqoop2. Thanks Jarcec!
          Hide
          Hudson added a comment -

          Integrated in Sqoop2-hadoop200 #171 (See https://builds.apache.org/job/Sqoop2-hadoop200/171/)
          SQOOP-843: Generic JDBC connector is committing transaction on export with autoCommit on (Revision eef227a5f9eee8a4dd04f9016387bbed59c96995)

          Result = SUCCESS
          cheolsoo : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=eef227a5f9eee8a4dd04f9016387bbed59c96995
          Files :

          • connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/GenericJdbcExecutor.java
          • connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/GenericJdbcExportLoader.java
          Show
          Hudson added a comment - Integrated in Sqoop2-hadoop200 #171 (See https://builds.apache.org/job/Sqoop2-hadoop200/171/ ) SQOOP-843 : Generic JDBC connector is committing transaction on export with autoCommit on (Revision eef227a5f9eee8a4dd04f9016387bbed59c96995) Result = SUCCESS cheolsoo : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=eef227a5f9eee8a4dd04f9016387bbed59c96995 Files : connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/GenericJdbcExecutor.java connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/GenericJdbcExportLoader.java
          Hide
          Hudson added a comment -

          Integrated in Sqoop2-hadoop100 #157 (See https://builds.apache.org/job/Sqoop2-hadoop100/157/)
          SQOOP-843: Generic JDBC connector is committing transaction on export with autoCommit on (Revision eef227a5f9eee8a4dd04f9016387bbed59c96995)

          Result = SUCCESS
          cheolsoo : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=eef227a5f9eee8a4dd04f9016387bbed59c96995
          Files :

          • connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/GenericJdbcExportLoader.java
          • connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/GenericJdbcExecutor.java
          Show
          Hudson added a comment - Integrated in Sqoop2-hadoop100 #157 (See https://builds.apache.org/job/Sqoop2-hadoop100/157/ ) SQOOP-843 : Generic JDBC connector is committing transaction on export with autoCommit on (Revision eef227a5f9eee8a4dd04f9016387bbed59c96995) Result = SUCCESS cheolsoo : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=eef227a5f9eee8a4dd04f9016387bbed59c96995 Files : connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/GenericJdbcExportLoader.java connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/GenericJdbcExecutor.java

            People

            • Assignee:
              Jarek Jarcec Cecho
              Reporter:
              Jarek Jarcec Cecho
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development