Sqoop
  1. Sqoop
  2. SQOOP-1056

Implement connection resiliency in Sqoop using pluggable failure handlers

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.4.5
    • Component/s: connectors/sqlserver
    • Labels:
      None

      Description

      Implement a pluggable way for handling connection failures, and/or intermittent errors in Sqoop. This is especially crucial in environments where the probability of connections getting reset or throttled is high.
      In case of intermittent failures in Sqoop, due to connection losses or server throttling, Sqoop does not recover from those failures. As a result, the running Sqoop task would eventually fail, and a new task is started. In those cases, Sqoop does not always guarantee that tasks can safely be restarted. For example, if part of the records is already committed to the database, then restarting the task would result in some failures like primary key violations. Even for Sqoop jobs which commit the records only at the end of the task, any failures towards the end of the task would involve reprocessing the whole range of split owned by the task, and any progress is lost.

      1. SQOOP-1056-1057-combo.patch
        106 kB
        Venkat Ranganathan
      2. SQOOP-1056.3.patch
        73 kB
        Shuaishuai Nie
      3. SQOOP-1056.2.patch
        73 kB
        Shuaishuai Nie
      4. Sqoop Connection Resiliency.docx
        24 kB
        Shuaishuai Nie
      5. SQOOP-1056.1.patch
        74 kB
        Shuaishuai Nie

        Issue Links

          Activity

          Hide
          Shuaishuai Nie added a comment -

          Re-based the patch for SQLServerManager due to recent code change

          Show
          Shuaishuai Nie added a comment - Re-based the patch for SQLServerManager due to recent code change
          Hide
          Venkat Ranganathan added a comment -

          Canceling patch as it is too old. Updating patch to trunk. Sorry about the long delay in getting this through Shuaishuai Nie

          Show
          Venkat Ranganathan added a comment - Canceling patch as it is too old. Updating patch to trunk. Sorry about the long delay in getting this through Shuaishuai Nie
          Hide
          Venkat Ranganathan added a comment -

          Attaching a patch for SQOOP-1056 and SQOOP-1057 after updating to trunk and fixing a few checkstyle issues.

          Show
          Venkat Ranganathan added a comment - Attaching a patch for SQOOP-1056 and SQOOP-1057 after updating to trunk and fixing a few checkstyle issues.
          Hide
          Venkat Ranganathan added a comment -

          Shuaishuai Nie Can you please look through the patch and if you are OK update the review board with the patch and we can work from there.

          Show
          Venkat Ranganathan added a comment - Shuaishuai Nie Can you please look through the patch and if you are OK update the review board with the patch and we can work from there.
          Hide
          ASF subversion and git services added a comment -

          Commit 03fa9c53024671edb8807b3deb31b104c38a6a07 in branch refs/heads/trunk from Venkat Ranganathan
          [ https://git-wip-us.apache.org/repos/asf?p=sqoop.git;h=03fa9c5 ]

          SQOOP-1056: Implement connection resiliency in Sqoop using pluggable failure handlers
          SQOOP-1057: Introduce fault injection framework to test connection resiliency

          (Shuaishuai Nie via Venkat Ranganathan)

          Show
          ASF subversion and git services added a comment - Commit 03fa9c53024671edb8807b3deb31b104c38a6a07 in branch refs/heads/trunk from Venkat Ranganathan [ https://git-wip-us.apache.org/repos/asf?p=sqoop.git;h=03fa9c5 ] SQOOP-1056 : Implement connection resiliency in Sqoop using pluggable failure handlers SQOOP-1057 : Introduce fault injection framework to test connection resiliency (Shuaishuai Nie via Venkat Ranganathan)
          Hide
          Venkat Ranganathan added a comment -

          Thanks Shuaishuai Nie for your contribution and validating the changes to trunk.

          Show
          Venkat Ranganathan added a comment - Thanks Shuaishuai Nie for your contribution and validating the changes to trunk.
          Hide
          Hudson added a comment -

          SUCCESS: Integrated in Sqoop-ant-jdk-1.6-hadoop200 #880 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop200/880/)
          SQOOP-1056: Implement connection resiliency in Sqoop using pluggable failure handlers (venkat: https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=03fa9c53024671edb8807b3deb31b104c38a6a07)

          • src/java/org/apache/sqoop/mapreduce/db/SQLServerDBInputFormat.java
          • src/java/org/apache/sqoop/mapreduce/db/DBRecordReader.java
          • src/java/org/apache/sqoop/mapreduce/db/DataDrivenDBRecordReader.java
          • src/java/org/apache/sqoop/mapreduce/SQLServerExportDBExecThread.java
          • src/java/org/apache/sqoop/mapreduce/SQLServerResilientUpdateOutputFormat.java
          • src/java/org/apache/sqoop/mapreduce/sqlserver/SqlServerRecordReader.java
          • ivy.xml
          • src/java/org/apache/sqoop/mapreduce/SQLServerUpdateDBExecThread.java
          • src/java/org/apache/sqoop/manager/SqlManager.java
          • src/test/fi-site.xml
          • src/java/org/apache/sqoop/manager/SQLServerManager.java
          • src/test/aop/org/apache/sqoop/mapreduce/db/SqlServerImportAspect.aj
          • src/java/org/apache/sqoop/mapreduce/db/SQLFailureHandler.java
          • src/test/aop/org/apache/sqoop/mapreduce/SqlServerExportAspect.aj
          • src/java/org/apache/sqoop/mapreduce/db/BasicRetrySQLFailureHandler.java
          • ivy/libraries.properties
          • src/java/org/apache/sqoop/mapreduce/db/SQLServerDBRecordReader.java
          • src/java/org/apache/sqoop/mapreduce/SQLServerResilientExportOutputFormat.java
          • src/java/org/apache/sqoop/mapreduce/SQLServerAsyncDBExecThread.java
          • src/test/aop/org/apache/sqoop/fi/ProbabilityModel.java
          • src/test/aop/org/apache/sqoop/fi/FiConfig.java
          • src/java/org/apache/sqoop/mapreduce/db/SQLServerConnectionFailureHandler.java
          • src/test/aop/build/aop.xml
          • build.xml
          Show
          Hudson added a comment - SUCCESS: Integrated in Sqoop-ant-jdk-1.6-hadoop200 #880 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop200/880/ ) SQOOP-1056 : Implement connection resiliency in Sqoop using pluggable failure handlers (venkat: https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=03fa9c53024671edb8807b3deb31b104c38a6a07 ) src/java/org/apache/sqoop/mapreduce/db/SQLServerDBInputFormat.java src/java/org/apache/sqoop/mapreduce/db/DBRecordReader.java src/java/org/apache/sqoop/mapreduce/db/DataDrivenDBRecordReader.java src/java/org/apache/sqoop/mapreduce/SQLServerExportDBExecThread.java src/java/org/apache/sqoop/mapreduce/SQLServerResilientUpdateOutputFormat.java src/java/org/apache/sqoop/mapreduce/sqlserver/SqlServerRecordReader.java ivy.xml src/java/org/apache/sqoop/mapreduce/SQLServerUpdateDBExecThread.java src/java/org/apache/sqoop/manager/SqlManager.java src/test/fi-site.xml src/java/org/apache/sqoop/manager/SQLServerManager.java src/test/aop/org/apache/sqoop/mapreduce/db/SqlServerImportAspect.aj src/java/org/apache/sqoop/mapreduce/db/SQLFailureHandler.java src/test/aop/org/apache/sqoop/mapreduce/SqlServerExportAspect.aj src/java/org/apache/sqoop/mapreduce/db/BasicRetrySQLFailureHandler.java ivy/libraries.properties src/java/org/apache/sqoop/mapreduce/db/SQLServerDBRecordReader.java src/java/org/apache/sqoop/mapreduce/SQLServerResilientExportOutputFormat.java src/java/org/apache/sqoop/mapreduce/SQLServerAsyncDBExecThread.java src/test/aop/org/apache/sqoop/fi/ProbabilityModel.java src/test/aop/org/apache/sqoop/fi/FiConfig.java src/java/org/apache/sqoop/mapreduce/db/SQLServerConnectionFailureHandler.java src/test/aop/build/aop.xml build.xml
          Hide
          Hudson added a comment -

          SUCCESS: Integrated in Sqoop-ant-jdk-1.6-hadoop20 #874 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop20/874/)
          SQOOP-1056: Implement connection resiliency in Sqoop using pluggable failure handlers (venkat: https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=03fa9c53024671edb8807b3deb31b104c38a6a07)

          • src/java/org/apache/sqoop/mapreduce/SQLServerExportDBExecThread.java
          • src/java/org/apache/sqoop/mapreduce/SQLServerResilientUpdateOutputFormat.java
          • src/java/org/apache/sqoop/mapreduce/db/DBRecordReader.java
          • src/test/aop/build/aop.xml
          • src/java/org/apache/sqoop/mapreduce/SQLServerUpdateDBExecThread.java
          • ivy/libraries.properties
          • src/java/org/apache/sqoop/manager/SQLServerManager.java
          • src/java/org/apache/sqoop/manager/SqlManager.java
          • src/java/org/apache/sqoop/mapreduce/db/DataDrivenDBRecordReader.java
          • src/java/org/apache/sqoop/mapreduce/db/SQLServerDBInputFormat.java
          • src/test/fi-site.xml
          • src/test/aop/org/apache/sqoop/mapreduce/db/SqlServerImportAspect.aj
          • src/test/aop/org/apache/sqoop/mapreduce/SqlServerExportAspect.aj
          • ivy.xml
          • src/test/aop/org/apache/sqoop/fi/FiConfig.java
          • src/java/org/apache/sqoop/mapreduce/db/SQLFailureHandler.java
          • src/java/org/apache/sqoop/mapreduce/SQLServerResilientExportOutputFormat.java
          • build.xml
          • src/java/org/apache/sqoop/mapreduce/db/SQLServerConnectionFailureHandler.java
          • src/java/org/apache/sqoop/mapreduce/sqlserver/SqlServerRecordReader.java
          • src/java/org/apache/sqoop/mapreduce/db/SQLServerDBRecordReader.java
          • src/test/aop/org/apache/sqoop/fi/ProbabilityModel.java
          • src/java/org/apache/sqoop/mapreduce/db/BasicRetrySQLFailureHandler.java
          • src/java/org/apache/sqoop/mapreduce/SQLServerAsyncDBExecThread.java
          Show
          Hudson added a comment - SUCCESS: Integrated in Sqoop-ant-jdk-1.6-hadoop20 #874 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop20/874/ ) SQOOP-1056 : Implement connection resiliency in Sqoop using pluggable failure handlers (venkat: https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=03fa9c53024671edb8807b3deb31b104c38a6a07 ) src/java/org/apache/sqoop/mapreduce/SQLServerExportDBExecThread.java src/java/org/apache/sqoop/mapreduce/SQLServerResilientUpdateOutputFormat.java src/java/org/apache/sqoop/mapreduce/db/DBRecordReader.java src/test/aop/build/aop.xml src/java/org/apache/sqoop/mapreduce/SQLServerUpdateDBExecThread.java ivy/libraries.properties src/java/org/apache/sqoop/manager/SQLServerManager.java src/java/org/apache/sqoop/manager/SqlManager.java src/java/org/apache/sqoop/mapreduce/db/DataDrivenDBRecordReader.java src/java/org/apache/sqoop/mapreduce/db/SQLServerDBInputFormat.java src/test/fi-site.xml src/test/aop/org/apache/sqoop/mapreduce/db/SqlServerImportAspect.aj src/test/aop/org/apache/sqoop/mapreduce/SqlServerExportAspect.aj ivy.xml src/test/aop/org/apache/sqoop/fi/FiConfig.java src/java/org/apache/sqoop/mapreduce/db/SQLFailureHandler.java src/java/org/apache/sqoop/mapreduce/SQLServerResilientExportOutputFormat.java build.xml src/java/org/apache/sqoop/mapreduce/db/SQLServerConnectionFailureHandler.java src/java/org/apache/sqoop/mapreduce/sqlserver/SqlServerRecordReader.java src/java/org/apache/sqoop/mapreduce/db/SQLServerDBRecordReader.java src/test/aop/org/apache/sqoop/fi/ProbabilityModel.java src/java/org/apache/sqoop/mapreduce/db/BasicRetrySQLFailureHandler.java src/java/org/apache/sqoop/mapreduce/SQLServerAsyncDBExecThread.java
          Hide
          Hudson added a comment -

          SUCCESS: Integrated in Sqoop-ant-jdk-1.6-hadoop23 #1076 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop23/1076/)
          SQOOP-1056: Implement connection resiliency in Sqoop using pluggable failure handlers (venkat: https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=03fa9c53024671edb8807b3deb31b104c38a6a07)

          • src/test/aop/org/apache/sqoop/mapreduce/SqlServerExportAspect.aj
          • src/java/org/apache/sqoop/mapreduce/db/SQLServerConnectionFailureHandler.java
          • src/java/org/apache/sqoop/mapreduce/db/BasicRetrySQLFailureHandler.java
          • src/java/org/apache/sqoop/mapreduce/SQLServerUpdateDBExecThread.java
          • src/test/aop/org/apache/sqoop/mapreduce/db/SqlServerImportAspect.aj
          • src/test/aop/build/aop.xml
          • src/java/org/apache/sqoop/mapreduce/sqlserver/SqlServerRecordReader.java
          • src/java/org/apache/sqoop/mapreduce/db/SQLFailureHandler.java
          • src/java/org/apache/sqoop/mapreduce/db/DataDrivenDBRecordReader.java
          • src/test/aop/org/apache/sqoop/fi/FiConfig.java
          • src/java/org/apache/sqoop/mapreduce/db/DBRecordReader.java
          • src/java/org/apache/sqoop/mapreduce/SQLServerAsyncDBExecThread.java
          • src/java/org/apache/sqoop/mapreduce/db/SQLServerDBInputFormat.java
          • src/java/org/apache/sqoop/mapreduce/SQLServerExportDBExecThread.java
          • src/java/org/apache/sqoop/mapreduce/db/SQLServerDBRecordReader.java
          • src/java/org/apache/sqoop/manager/SqlManager.java
          • src/java/org/apache/sqoop/mapreduce/SQLServerResilientUpdateOutputFormat.java
          • ivy.xml
          • build.xml
          • src/test/fi-site.xml
          • src/test/aop/org/apache/sqoop/fi/ProbabilityModel.java
          • src/java/org/apache/sqoop/manager/SQLServerManager.java
          • ivy/libraries.properties
          • src/java/org/apache/sqoop/mapreduce/SQLServerResilientExportOutputFormat.java
          Show
          Hudson added a comment - SUCCESS: Integrated in Sqoop-ant-jdk-1.6-hadoop23 #1076 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop23/1076/ ) SQOOP-1056 : Implement connection resiliency in Sqoop using pluggable failure handlers (venkat: https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=03fa9c53024671edb8807b3deb31b104c38a6a07 ) src/test/aop/org/apache/sqoop/mapreduce/SqlServerExportAspect.aj src/java/org/apache/sqoop/mapreduce/db/SQLServerConnectionFailureHandler.java src/java/org/apache/sqoop/mapreduce/db/BasicRetrySQLFailureHandler.java src/java/org/apache/sqoop/mapreduce/SQLServerUpdateDBExecThread.java src/test/aop/org/apache/sqoop/mapreduce/db/SqlServerImportAspect.aj src/test/aop/build/aop.xml src/java/org/apache/sqoop/mapreduce/sqlserver/SqlServerRecordReader.java src/java/org/apache/sqoop/mapreduce/db/SQLFailureHandler.java src/java/org/apache/sqoop/mapreduce/db/DataDrivenDBRecordReader.java src/test/aop/org/apache/sqoop/fi/FiConfig.java src/java/org/apache/sqoop/mapreduce/db/DBRecordReader.java src/java/org/apache/sqoop/mapreduce/SQLServerAsyncDBExecThread.java src/java/org/apache/sqoop/mapreduce/db/SQLServerDBInputFormat.java src/java/org/apache/sqoop/mapreduce/SQLServerExportDBExecThread.java src/java/org/apache/sqoop/mapreduce/db/SQLServerDBRecordReader.java src/java/org/apache/sqoop/manager/SqlManager.java src/java/org/apache/sqoop/mapreduce/SQLServerResilientUpdateOutputFormat.java ivy.xml build.xml src/test/fi-site.xml src/test/aop/org/apache/sqoop/fi/ProbabilityModel.java src/java/org/apache/sqoop/manager/SQLServerManager.java ivy/libraries.properties src/java/org/apache/sqoop/mapreduce/SQLServerResilientExportOutputFormat.java
          Hide
          Hudson added a comment -

          SUCCESS: Integrated in Sqoop-ant-jdk-1.6-hadoop100 #839 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop100/839/)
          SQOOP-1056: Implement connection resiliency in Sqoop using pluggable failure handlers (venkat: https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=03fa9c53024671edb8807b3deb31b104c38a6a07)

          • ivy/libraries.properties
          • src/test/aop/build/aop.xml
          • src/java/org/apache/sqoop/mapreduce/db/SQLFailureHandler.java
          • src/java/org/apache/sqoop/manager/SqlManager.java
          • src/java/org/apache/sqoop/mapreduce/sqlserver/SqlServerRecordReader.java
          • src/java/org/apache/sqoop/mapreduce/SQLServerExportDBExecThread.java
          • src/test/aop/org/apache/sqoop/mapreduce/SqlServerExportAspect.aj
          • src/test/aop/org/apache/sqoop/fi/FiConfig.java
          • src/java/org/apache/sqoop/mapreduce/SQLServerResilientUpdateOutputFormat.java
          • src/java/org/apache/sqoop/mapreduce/SQLServerResilientExportOutputFormat.java
          • src/java/org/apache/sqoop/mapreduce/SQLServerUpdateDBExecThread.java
          • src/java/org/apache/sqoop/mapreduce/db/SQLServerDBInputFormat.java
          • src/java/org/apache/sqoop/mapreduce/db/DBRecordReader.java
          • src/java/org/apache/sqoop/mapreduce/SQLServerAsyncDBExecThread.java
          • src/java/org/apache/sqoop/mapreduce/db/BasicRetrySQLFailureHandler.java
          • build.xml
          • src/java/org/apache/sqoop/manager/SQLServerManager.java
          • ivy.xml
          • src/java/org/apache/sqoop/mapreduce/db/SQLServerDBRecordReader.java
          • src/test/aop/org/apache/sqoop/mapreduce/db/SqlServerImportAspect.aj
          • src/test/aop/org/apache/sqoop/fi/ProbabilityModel.java
          • src/java/org/apache/sqoop/mapreduce/db/DataDrivenDBRecordReader.java
          • src/java/org/apache/sqoop/mapreduce/db/SQLServerConnectionFailureHandler.java
          • src/test/fi-site.xml
          Show
          Hudson added a comment - SUCCESS: Integrated in Sqoop-ant-jdk-1.6-hadoop100 #839 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop100/839/ ) SQOOP-1056 : Implement connection resiliency in Sqoop using pluggable failure handlers (venkat: https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=03fa9c53024671edb8807b3deb31b104c38a6a07 ) ivy/libraries.properties src/test/aop/build/aop.xml src/java/org/apache/sqoop/mapreduce/db/SQLFailureHandler.java src/java/org/apache/sqoop/manager/SqlManager.java src/java/org/apache/sqoop/mapreduce/sqlserver/SqlServerRecordReader.java src/java/org/apache/sqoop/mapreduce/SQLServerExportDBExecThread.java src/test/aop/org/apache/sqoop/mapreduce/SqlServerExportAspect.aj src/test/aop/org/apache/sqoop/fi/FiConfig.java src/java/org/apache/sqoop/mapreduce/SQLServerResilientUpdateOutputFormat.java src/java/org/apache/sqoop/mapreduce/SQLServerResilientExportOutputFormat.java src/java/org/apache/sqoop/mapreduce/SQLServerUpdateDBExecThread.java src/java/org/apache/sqoop/mapreduce/db/SQLServerDBInputFormat.java src/java/org/apache/sqoop/mapreduce/db/DBRecordReader.java src/java/org/apache/sqoop/mapreduce/SQLServerAsyncDBExecThread.java src/java/org/apache/sqoop/mapreduce/db/BasicRetrySQLFailureHandler.java build.xml src/java/org/apache/sqoop/manager/SQLServerManager.java ivy.xml src/java/org/apache/sqoop/mapreduce/db/SQLServerDBRecordReader.java src/test/aop/org/apache/sqoop/mapreduce/db/SqlServerImportAspect.aj src/test/aop/org/apache/sqoop/fi/ProbabilityModel.java src/java/org/apache/sqoop/mapreduce/db/DataDrivenDBRecordReader.java src/java/org/apache/sqoop/mapreduce/db/SQLServerConnectionFailureHandler.java src/test/fi-site.xml

            People

            • Assignee:
              Shuaishuai Nie
              Reporter:
              Shuaishuai Nie
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development