Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 1.4.4
    • Fix Version/s: 1.4.4
    • Component/s: None
    • Labels:
      None

      Description

      Validation of the data copied, either import or export, is implemented today by comparing the row counts from the source and the target post copy. This issue adds a handler that aborts the processing notifying the user of the anomaly.

      Makes this the default implementation.

      1. SQOOP-916.patch
        16 kB
        Venkatesh Seetharam

        Activity

        Venkatesh Seetharam created issue -
        Venkatesh Seetharam made changes -
        Field Original Value New Value
        Issue Type Bug [ 1 ] Improvement [ 4 ]
        Venkatesh Seetharam made changes -
        Description Validation of the data copied, either import or export, is implemented today by comparing the row counts from the source and the target post copy. This issue adds a handler that aborts the processing notifying the user of the anomaly.
        Venkatesh Seetharam made changes -
        Description Validation of the data copied, either import or export, is implemented today by comparing the row counts from the source and the target post copy. This issue adds a handler that aborts the processing notifying the user of the anomaly. Validation of the data copied, either import or export, is implemented today by comparing the row counts from the source and the target post copy. This issue adds a handler that aborts the processing notifying the user of the anomaly.

        Makes this the default implementation.
        Show
        Venkatesh Seetharam added a comment - https://reviews.apache.org/r/10252/
        Venkatesh Seetharam made changes -
        Status Open [ 1 ] In Progress [ 3 ]
        Hide
        Venkatesh Seetharam added a comment -

        Attaching the patch.

        Show
        Venkatesh Seetharam added a comment - Attaching the patch.
        Venkatesh Seetharam made changes -
        Attachment SQOOP-916.patch [ 12577244 ]
        Hide
        Jarek Jarcec Cecho added a comment -

        The path is in: https://git-wip-us.apache.org/repos/asf?p=sqoop.git;a=commit;h=ef093399bf01fed4a7c17caebd05676f1e566b8a

        Thank you Venkatesh for your time and effort!

        Jarcec

        Show
        Jarek Jarcec Cecho added a comment - The path is in: https://git-wip-us.apache.org/repos/asf?p=sqoop.git;a=commit;h=ef093399bf01fed4a7c17caebd05676f1e566b8a Thank you Venkatesh for your time and effort! Jarcec
        Jarek Jarcec Cecho made changes -
        Status In Progress [ 3 ] Resolved [ 5 ]
        Fix Version/s 1.4.4 [ 12324082 ]
        Resolution Fixed [ 1 ]
        Hide
        Hudson added a comment -

        Integrated in Sqoop-ant-jdk-1.6-hadoop20 #609 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop20/609/)
        SQOOP-916: Add an abort validation handler (Revision ef093399bf01fed4a7c17caebd05676f1e566b8a)

        Result = SUCCESS
        jarcec : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=ef093399bf01fed4a7c17caebd05676f1e566b8a
        Files :

        • src/java/org/apache/sqoop/SqoopOptions.java
        • src/test/org/apache/sqoop/validation/AbortOnFailureHandlerTest.java
        • src/java/org/apache/sqoop/validation/AbortOnFailureHandler.java
        • src/java/org/apache/sqoop/validation/ValidationException.java
        • src/java/org/apache/sqoop/validation/RowCountValidator.java
        • src/java/org/apache/sqoop/validation/LogOnFailureHandler.java
        • src/docs/user/validation.txt
        • src/test/org/apache/sqoop/validation/AbsoluteValidationThresholdTest.java
        • src/test/org/apache/sqoop/validation/RowCountValidatorImportTest.java
        Show
        Hudson added a comment - Integrated in Sqoop-ant-jdk-1.6-hadoop20 #609 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop20/609/ ) SQOOP-916 : Add an abort validation handler (Revision ef093399bf01fed4a7c17caebd05676f1e566b8a) Result = SUCCESS jarcec : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=ef093399bf01fed4a7c17caebd05676f1e566b8a Files : src/java/org/apache/sqoop/SqoopOptions.java src/test/org/apache/sqoop/validation/AbortOnFailureHandlerTest.java src/java/org/apache/sqoop/validation/AbortOnFailureHandler.java src/java/org/apache/sqoop/validation/ValidationException.java src/java/org/apache/sqoop/validation/RowCountValidator.java src/java/org/apache/sqoop/validation/LogOnFailureHandler.java src/docs/user/validation.txt src/test/org/apache/sqoop/validation/AbsoluteValidationThresholdTest.java src/test/org/apache/sqoop/validation/RowCountValidatorImportTest.java
        Hide
        Hudson added a comment -

        Integrated in Sqoop-ant-jdk-1.6-hadoop23 #818 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop23/818/)
        SQOOP-916: Add an abort validation handler (Revision ef093399bf01fed4a7c17caebd05676f1e566b8a)

        Result = SUCCESS
        jarcec : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=ef093399bf01fed4a7c17caebd05676f1e566b8a
        Files :

        • src/java/org/apache/sqoop/validation/ValidationException.java
        • src/test/org/apache/sqoop/validation/AbsoluteValidationThresholdTest.java
        • src/docs/user/validation.txt
        • src/test/org/apache/sqoop/validation/AbortOnFailureHandlerTest.java
        • src/java/org/apache/sqoop/validation/RowCountValidator.java
        • src/java/org/apache/sqoop/validation/AbortOnFailureHandler.java
        • src/java/org/apache/sqoop/validation/LogOnFailureHandler.java
        • src/test/org/apache/sqoop/validation/RowCountValidatorImportTest.java
        • src/java/org/apache/sqoop/SqoopOptions.java
        Show
        Hudson added a comment - Integrated in Sqoop-ant-jdk-1.6-hadoop23 #818 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop23/818/ ) SQOOP-916 : Add an abort validation handler (Revision ef093399bf01fed4a7c17caebd05676f1e566b8a) Result = SUCCESS jarcec : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=ef093399bf01fed4a7c17caebd05676f1e566b8a Files : src/java/org/apache/sqoop/validation/ValidationException.java src/test/org/apache/sqoop/validation/AbsoluteValidationThresholdTest.java src/docs/user/validation.txt src/test/org/apache/sqoop/validation/AbortOnFailureHandlerTest.java src/java/org/apache/sqoop/validation/RowCountValidator.java src/java/org/apache/sqoop/validation/AbortOnFailureHandler.java src/java/org/apache/sqoop/validation/LogOnFailureHandler.java src/test/org/apache/sqoop/validation/RowCountValidatorImportTest.java src/java/org/apache/sqoop/SqoopOptions.java
        Hide
        Hudson added a comment -

        Integrated in Sqoop-ant-jdk-1.6-hadoop200 #606 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop200/606/)
        SQOOP-916: Add an abort validation handler (Revision ef093399bf01fed4a7c17caebd05676f1e566b8a)

        Result = SUCCESS
        jarcec : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=ef093399bf01fed4a7c17caebd05676f1e566b8a
        Files :

        • src/java/org/apache/sqoop/validation/RowCountValidator.java
        • src/java/org/apache/sqoop/validation/LogOnFailureHandler.java
        • src/test/org/apache/sqoop/validation/AbortOnFailureHandlerTest.java
        • src/java/org/apache/sqoop/SqoopOptions.java
        • src/test/org/apache/sqoop/validation/AbsoluteValidationThresholdTest.java
        • src/java/org/apache/sqoop/validation/AbortOnFailureHandler.java
        • src/docs/user/validation.txt
        • src/test/org/apache/sqoop/validation/RowCountValidatorImportTest.java
        • src/java/org/apache/sqoop/validation/ValidationException.java
        Show
        Hudson added a comment - Integrated in Sqoop-ant-jdk-1.6-hadoop200 #606 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop200/606/ ) SQOOP-916 : Add an abort validation handler (Revision ef093399bf01fed4a7c17caebd05676f1e566b8a) Result = SUCCESS jarcec : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=ef093399bf01fed4a7c17caebd05676f1e566b8a Files : src/java/org/apache/sqoop/validation/RowCountValidator.java src/java/org/apache/sqoop/validation/LogOnFailureHandler.java src/test/org/apache/sqoop/validation/AbortOnFailureHandlerTest.java src/java/org/apache/sqoop/SqoopOptions.java src/test/org/apache/sqoop/validation/AbsoluteValidationThresholdTest.java src/java/org/apache/sqoop/validation/AbortOnFailureHandler.java src/docs/user/validation.txt src/test/org/apache/sqoop/validation/RowCountValidatorImportTest.java src/java/org/apache/sqoop/validation/ValidationException.java
        Hide
        Hudson added a comment -

        Integrated in Sqoop-ant-jdk-1.6-hadoop100 #617 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop100/617/)
        SQOOP-916: Add an abort validation handler (Revision ef093399bf01fed4a7c17caebd05676f1e566b8a)

        Result = SUCCESS
        jarcec : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=ef093399bf01fed4a7c17caebd05676f1e566b8a
        Files :

        • src/java/org/apache/sqoop/validation/AbortOnFailureHandler.java
        • src/test/org/apache/sqoop/validation/RowCountValidatorImportTest.java
        • src/java/org/apache/sqoop/SqoopOptions.java
        • src/docs/user/validation.txt
        • src/java/org/apache/sqoop/validation/ValidationException.java
        • src/java/org/apache/sqoop/validation/RowCountValidator.java
        • src/java/org/apache/sqoop/validation/LogOnFailureHandler.java
        • src/test/org/apache/sqoop/validation/AbsoluteValidationThresholdTest.java
        • src/test/org/apache/sqoop/validation/AbortOnFailureHandlerTest.java
        Show
        Hudson added a comment - Integrated in Sqoop-ant-jdk-1.6-hadoop100 #617 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop100/617/ ) SQOOP-916 : Add an abort validation handler (Revision ef093399bf01fed4a7c17caebd05676f1e566b8a) Result = SUCCESS jarcec : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=ef093399bf01fed4a7c17caebd05676f1e566b8a Files : src/java/org/apache/sqoop/validation/AbortOnFailureHandler.java src/test/org/apache/sqoop/validation/RowCountValidatorImportTest.java src/java/org/apache/sqoop/SqoopOptions.java src/docs/user/validation.txt src/java/org/apache/sqoop/validation/ValidationException.java src/java/org/apache/sqoop/validation/RowCountValidator.java src/java/org/apache/sqoop/validation/LogOnFailureHandler.java src/test/org/apache/sqoop/validation/AbsoluteValidationThresholdTest.java src/test/org/apache/sqoop/validation/AbortOnFailureHandlerTest.java
        Transition Time In Source Status Execution Times Last Executer Last Execution Date
        Open Open In Progress In Progress
        33d 11h 43m 1 Venkatesh Seetharam 03/Apr/13 11:55
        In Progress In Progress Resolved Resolved
        2d 7h 13m 1 Jarek Jarcec Cecho 05/Apr/13 19:09

          People

          • Assignee:
            Venkatesh Seetharam
            Reporter:
            Venkatesh Seetharam
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development