Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 1.4.4
    • Fix Version/s: 1.4.4
    • Component/s: None
    • Labels:
      None

      Description

      Validation of the data copied, either import or export, is implemented today by comparing the row counts from the source and the target post copy. This issue adds a handler that aborts the processing notifying the user of the anomaly.

      Makes this the default implementation.

      1. SQOOP-916.patch
        16 kB
        Venkatesh Seetharam

        Activity

        Hide
        Hudson added a comment -

        Integrated in Sqoop-ant-jdk-1.6-hadoop100 #617 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop100/617/)
        SQOOP-916: Add an abort validation handler (Revision ef093399bf01fed4a7c17caebd05676f1e566b8a)

        Result = SUCCESS
        jarcec : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=ef093399bf01fed4a7c17caebd05676f1e566b8a
        Files :

        • src/java/org/apache/sqoop/validation/AbortOnFailureHandler.java
        • src/test/org/apache/sqoop/validation/RowCountValidatorImportTest.java
        • src/java/org/apache/sqoop/SqoopOptions.java
        • src/docs/user/validation.txt
        • src/java/org/apache/sqoop/validation/ValidationException.java
        • src/java/org/apache/sqoop/validation/RowCountValidator.java
        • src/java/org/apache/sqoop/validation/LogOnFailureHandler.java
        • src/test/org/apache/sqoop/validation/AbsoluteValidationThresholdTest.java
        • src/test/org/apache/sqoop/validation/AbortOnFailureHandlerTest.java
        Show
        Hudson added a comment - Integrated in Sqoop-ant-jdk-1.6-hadoop100 #617 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop100/617/ ) SQOOP-916 : Add an abort validation handler (Revision ef093399bf01fed4a7c17caebd05676f1e566b8a) Result = SUCCESS jarcec : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=ef093399bf01fed4a7c17caebd05676f1e566b8a Files : src/java/org/apache/sqoop/validation/AbortOnFailureHandler.java src/test/org/apache/sqoop/validation/RowCountValidatorImportTest.java src/java/org/apache/sqoop/SqoopOptions.java src/docs/user/validation.txt src/java/org/apache/sqoop/validation/ValidationException.java src/java/org/apache/sqoop/validation/RowCountValidator.java src/java/org/apache/sqoop/validation/LogOnFailureHandler.java src/test/org/apache/sqoop/validation/AbsoluteValidationThresholdTest.java src/test/org/apache/sqoop/validation/AbortOnFailureHandlerTest.java
        Hide
        Hudson added a comment -

        Integrated in Sqoop-ant-jdk-1.6-hadoop200 #606 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop200/606/)
        SQOOP-916: Add an abort validation handler (Revision ef093399bf01fed4a7c17caebd05676f1e566b8a)

        Result = SUCCESS
        jarcec : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=ef093399bf01fed4a7c17caebd05676f1e566b8a
        Files :

        • src/java/org/apache/sqoop/validation/RowCountValidator.java
        • src/java/org/apache/sqoop/validation/LogOnFailureHandler.java
        • src/test/org/apache/sqoop/validation/AbortOnFailureHandlerTest.java
        • src/java/org/apache/sqoop/SqoopOptions.java
        • src/test/org/apache/sqoop/validation/AbsoluteValidationThresholdTest.java
        • src/java/org/apache/sqoop/validation/AbortOnFailureHandler.java
        • src/docs/user/validation.txt
        • src/test/org/apache/sqoop/validation/RowCountValidatorImportTest.java
        • src/java/org/apache/sqoop/validation/ValidationException.java
        Show
        Hudson added a comment - Integrated in Sqoop-ant-jdk-1.6-hadoop200 #606 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop200/606/ ) SQOOP-916 : Add an abort validation handler (Revision ef093399bf01fed4a7c17caebd05676f1e566b8a) Result = SUCCESS jarcec : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=ef093399bf01fed4a7c17caebd05676f1e566b8a Files : src/java/org/apache/sqoop/validation/RowCountValidator.java src/java/org/apache/sqoop/validation/LogOnFailureHandler.java src/test/org/apache/sqoop/validation/AbortOnFailureHandlerTest.java src/java/org/apache/sqoop/SqoopOptions.java src/test/org/apache/sqoop/validation/AbsoluteValidationThresholdTest.java src/java/org/apache/sqoop/validation/AbortOnFailureHandler.java src/docs/user/validation.txt src/test/org/apache/sqoop/validation/RowCountValidatorImportTest.java src/java/org/apache/sqoop/validation/ValidationException.java
        Hide
        Hudson added a comment -

        Integrated in Sqoop-ant-jdk-1.6-hadoop23 #818 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop23/818/)
        SQOOP-916: Add an abort validation handler (Revision ef093399bf01fed4a7c17caebd05676f1e566b8a)

        Result = SUCCESS
        jarcec : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=ef093399bf01fed4a7c17caebd05676f1e566b8a
        Files :

        • src/java/org/apache/sqoop/validation/ValidationException.java
        • src/test/org/apache/sqoop/validation/AbsoluteValidationThresholdTest.java
        • src/docs/user/validation.txt
        • src/test/org/apache/sqoop/validation/AbortOnFailureHandlerTest.java
        • src/java/org/apache/sqoop/validation/RowCountValidator.java
        • src/java/org/apache/sqoop/validation/AbortOnFailureHandler.java
        • src/java/org/apache/sqoop/validation/LogOnFailureHandler.java
        • src/test/org/apache/sqoop/validation/RowCountValidatorImportTest.java
        • src/java/org/apache/sqoop/SqoopOptions.java
        Show
        Hudson added a comment - Integrated in Sqoop-ant-jdk-1.6-hadoop23 #818 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop23/818/ ) SQOOP-916 : Add an abort validation handler (Revision ef093399bf01fed4a7c17caebd05676f1e566b8a) Result = SUCCESS jarcec : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=ef093399bf01fed4a7c17caebd05676f1e566b8a Files : src/java/org/apache/sqoop/validation/ValidationException.java src/test/org/apache/sqoop/validation/AbsoluteValidationThresholdTest.java src/docs/user/validation.txt src/test/org/apache/sqoop/validation/AbortOnFailureHandlerTest.java src/java/org/apache/sqoop/validation/RowCountValidator.java src/java/org/apache/sqoop/validation/AbortOnFailureHandler.java src/java/org/apache/sqoop/validation/LogOnFailureHandler.java src/test/org/apache/sqoop/validation/RowCountValidatorImportTest.java src/java/org/apache/sqoop/SqoopOptions.java
        Hide
        Hudson added a comment -

        Integrated in Sqoop-ant-jdk-1.6-hadoop20 #609 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop20/609/)
        SQOOP-916: Add an abort validation handler (Revision ef093399bf01fed4a7c17caebd05676f1e566b8a)

        Result = SUCCESS
        jarcec : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=ef093399bf01fed4a7c17caebd05676f1e566b8a
        Files :

        • src/java/org/apache/sqoop/SqoopOptions.java
        • src/test/org/apache/sqoop/validation/AbortOnFailureHandlerTest.java
        • src/java/org/apache/sqoop/validation/AbortOnFailureHandler.java
        • src/java/org/apache/sqoop/validation/ValidationException.java
        • src/java/org/apache/sqoop/validation/RowCountValidator.java
        • src/java/org/apache/sqoop/validation/LogOnFailureHandler.java
        • src/docs/user/validation.txt
        • src/test/org/apache/sqoop/validation/AbsoluteValidationThresholdTest.java
        • src/test/org/apache/sqoop/validation/RowCountValidatorImportTest.java
        Show
        Hudson added a comment - Integrated in Sqoop-ant-jdk-1.6-hadoop20 #609 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop20/609/ ) SQOOP-916 : Add an abort validation handler (Revision ef093399bf01fed4a7c17caebd05676f1e566b8a) Result = SUCCESS jarcec : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=ef093399bf01fed4a7c17caebd05676f1e566b8a Files : src/java/org/apache/sqoop/SqoopOptions.java src/test/org/apache/sqoop/validation/AbortOnFailureHandlerTest.java src/java/org/apache/sqoop/validation/AbortOnFailureHandler.java src/java/org/apache/sqoop/validation/ValidationException.java src/java/org/apache/sqoop/validation/RowCountValidator.java src/java/org/apache/sqoop/validation/LogOnFailureHandler.java src/docs/user/validation.txt src/test/org/apache/sqoop/validation/AbsoluteValidationThresholdTest.java src/test/org/apache/sqoop/validation/RowCountValidatorImportTest.java
        Hide
        Jarek Jarcec Cecho added a comment -

        The path is in: https://git-wip-us.apache.org/repos/asf?p=sqoop.git;a=commit;h=ef093399bf01fed4a7c17caebd05676f1e566b8a

        Thank you Venkatesh for your time and effort!

        Jarcec

        Show
        Jarek Jarcec Cecho added a comment - The path is in: https://git-wip-us.apache.org/repos/asf?p=sqoop.git;a=commit;h=ef093399bf01fed4a7c17caebd05676f1e566b8a Thank you Venkatesh for your time and effort! Jarcec
        Hide
        Venkatesh Seetharam added a comment -

        Attaching the patch.

        Show
        Venkatesh Seetharam added a comment - Attaching the patch.
        Show
        Venkatesh Seetharam added a comment - https://reviews.apache.org/r/10252/

          People

          • Assignee:
            Venkatesh Seetharam
            Reporter:
            Venkatesh Seetharam
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development