Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-12469

distcp should not ignore the ignoreFailures option

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 2.7.1
    • Fix Version/s: 2.9.0, 3.0.0-alpha1
    • Component/s: tools/distcp
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      RetriableFileCopyCommand.CopyReadException is double-wrapped via

      1. via RetriableCommand::execute
      2. via CopyMapper#copyFileWithRetry

      before CopyMapper::handleFailure tests

      if (ignoreFailures && exception.getCause() instanceof
                  RetriableFileCopyCommand.CopyReadException
      

      which is always false.

      Orthogonally, ignoring failures should be mutually exclusive with the atomic option otherwise an incomplete dir is eligible for commit defeating the purpose.

      1. HADOOP-12469.000.patch
        2 kB
        Mingliang Liu
      2. HADOOP-12469.001.patch
        7 kB
        Mingliang Liu
      3. HADOOP-12469.002.patch
        7 kB
        Mingliang Liu
      4. HADOOP-12469.003.patch
        8 kB
        Mingliang Liu
      5. HADOOP-12469.004.patch
        7 kB
        Mingliang Liu
      6. HADOOP-12469.005.patch
        7 kB
        Mingliang Liu
      7. HADOOP-12469.005.patch
        7 kB
        Mingliang Liu
      8. HADOOP-12469.006.patch
        6 kB
        Mingliang Liu

        Issue Links

          Activity

          Hide
          liuml07 Mingliang Liu added a comment -

          Thanks Jing Zhao for your helpful discussion, review and commit!

          Show
          liuml07 Mingliang Liu added a comment - Thanks Jing Zhao for your helpful discussion, review and commit!
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #9714 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9714/)
          HADOOP-12469. distcp should not ignore the ignoreFailures option. (jing9: rev af942585a108d70e0946f6dd4c465a54d068eabf)

          • hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestCopyMapper.java
          • hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyMapper.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #9714 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9714/ ) HADOOP-12469 . distcp should not ignore the ignoreFailures option. (jing9: rev af942585a108d70e0946f6dd4c465a54d068eabf) hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestCopyMapper.java hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyMapper.java
          Hide
          jingzhao Jing Zhao added a comment -

          +1 on the latest patch. I've committed this to trunk and branch-2. Thanks Mingliang Liu for the contribution!

          Show
          jingzhao Jing Zhao added a comment - +1 on the latest patch. I've committed this to trunk and branch-2. Thanks Mingliang Liu for the contribution!
          Hide
          hadoopqa Hadoop QA added a comment -
          +1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 11s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 7m 14s trunk passed
          +1 compile 0m 16s trunk passed with JDK v1.8.0_91
          +1 compile 0m 18s trunk passed with JDK v1.7.0_95
          +1 checkstyle 0m 15s trunk passed
          +1 mvnsite 0m 24s trunk passed
          +1 mvneclipse 1m 32s trunk passed
          +1 findbugs 0m 31s trunk passed
          +1 javadoc 0m 13s trunk passed with JDK v1.8.0_91
          +1 javadoc 0m 15s trunk passed with JDK v1.7.0_95
          +1 mvninstall 0m 18s the patch passed
          +1 compile 0m 13s the patch passed with JDK v1.8.0_91
          +1 javac 0m 13s the patch passed
          +1 compile 0m 16s the patch passed with JDK v1.7.0_95
          +1 javac 0m 16s the patch passed
          +1 checkstyle 0m 12s the patch passed
          +1 mvnsite 0m 23s the patch passed
          +1 mvneclipse 0m 11s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 0m 41s the patch passed
          +1 javadoc 0m 11s the patch passed with JDK v1.8.0_91
          +1 javadoc 0m 13s the patch passed with JDK v1.7.0_95
          +1 unit 8m 37s hadoop-distcp in the patch passed with JDK v1.8.0_91.
          +1 unit 8m 9s hadoop-distcp in the patch passed with JDK v1.7.0_95.
          +1 asflicense 0m 17s The patch does not generate ASF License warnings.
          31m 51s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:cf2ee45
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12802090/HADOOP-12469.006.patch
          JIRA Issue HADOOP-12469
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 680afd66f367 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 6d77d6e
          Default Java 1.7.0_95
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_91 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
          findbugs v3.0.0
          JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9269/testReport/
          modules C: hadoop-tools/hadoop-distcp U: hadoop-tools/hadoop-distcp
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9269/console
          Powered by Apache Yetus 0.3.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 11s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 7m 14s trunk passed +1 compile 0m 16s trunk passed with JDK v1.8.0_91 +1 compile 0m 18s trunk passed with JDK v1.7.0_95 +1 checkstyle 0m 15s trunk passed +1 mvnsite 0m 24s trunk passed +1 mvneclipse 1m 32s trunk passed +1 findbugs 0m 31s trunk passed +1 javadoc 0m 13s trunk passed with JDK v1.8.0_91 +1 javadoc 0m 15s trunk passed with JDK v1.7.0_95 +1 mvninstall 0m 18s the patch passed +1 compile 0m 13s the patch passed with JDK v1.8.0_91 +1 javac 0m 13s the patch passed +1 compile 0m 16s the patch passed with JDK v1.7.0_95 +1 javac 0m 16s the patch passed +1 checkstyle 0m 12s the patch passed +1 mvnsite 0m 23s the patch passed +1 mvneclipse 0m 11s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 0m 41s the patch passed +1 javadoc 0m 11s the patch passed with JDK v1.8.0_91 +1 javadoc 0m 13s the patch passed with JDK v1.7.0_95 +1 unit 8m 37s hadoop-distcp in the patch passed with JDK v1.8.0_91. +1 unit 8m 9s hadoop-distcp in the patch passed with JDK v1.7.0_95. +1 asflicense 0m 17s The patch does not generate ASF License warnings. 31m 51s Subsystem Report/Notes Docker Image:yetus/hadoop:cf2ee45 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12802090/HADOOP-12469.006.patch JIRA Issue HADOOP-12469 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 680afd66f367 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 6d77d6e Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_91 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9269/testReport/ modules C: hadoop-tools/hadoop-distcp U: hadoop-tools/hadoop-distcp Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9269/console Powered by Apache Yetus 0.3.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          liuml07 Mingliang Liu added a comment -

          Thanks Jing Zhao for the comments. The v6 patch is to address this.

          Show
          liuml07 Mingliang Liu added a comment - Thanks Jing Zhao for the comments. The v6 patch is to address this.
          Hide
          jingzhao Jing Zhao added a comment -

          Instead of replacing the old unit test (which deletes some source files to generate failures), maybe we can add a new unit test. +1 after addressing the comment.

          Show
          jingzhao Jing Zhao added a comment - Instead of replacing the old unit test (which deletes some source files to generate failures), maybe we can add a new unit test. +1 after addressing the comment.
          Hide
          hadoopqa Hadoop QA added a comment -
          +1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 16s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 9m 48s trunk passed
          +1 compile 0m 23s trunk passed with JDK v1.8.0_92
          +1 compile 0m 19s trunk passed with JDK v1.7.0_95
          +1 checkstyle 0m 19s trunk passed
          +1 mvnsite 0m 30s trunk passed
          +1 mvneclipse 1m 9s trunk passed
          +1 findbugs 0m 45s trunk passed
          +1 javadoc 0m 22s trunk passed with JDK v1.8.0_92
          +1 javadoc 0m 17s trunk passed with JDK v1.7.0_95
          +1 mvninstall 0m 24s the patch passed
          +1 compile 0m 22s the patch passed with JDK v1.8.0_92
          +1 javac 0m 22s the patch passed
          +1 compile 0m 19s the patch passed with JDK v1.7.0_95
          +1 javac 0m 19s the patch passed
          +1 checkstyle 0m 13s the patch passed
          +1 mvnsite 0m 25s the patch passed
          +1 mvneclipse 0m 13s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 0m 47s the patch passed
          +1 javadoc 0m 16s the patch passed with JDK v1.8.0_92
          +1 javadoc 0m 15s the patch passed with JDK v1.7.0_95
          +1 unit 9m 21s hadoop-distcp in the patch passed with JDK v1.8.0_92.
          +1 unit 8m 50s hadoop-distcp in the patch passed with JDK v1.7.0_95.
          +1 asflicense 0m 19s The patch does not generate ASF License warnings.
          37m 1s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:cf2ee45
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12801351/HADOOP-12469.005.patch
          JIRA Issue HADOOP-12469
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux a059b13a3f1d 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 6243eab
          Default Java 1.7.0_95
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_92 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
          findbugs v3.0.0
          JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9221/testReport/
          modules C: hadoop-tools/hadoop-distcp U: hadoop-tools/hadoop-distcp
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9221/console
          Powered by Apache Yetus 0.3.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 16s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 9m 48s trunk passed +1 compile 0m 23s trunk passed with JDK v1.8.0_92 +1 compile 0m 19s trunk passed with JDK v1.7.0_95 +1 checkstyle 0m 19s trunk passed +1 mvnsite 0m 30s trunk passed +1 mvneclipse 1m 9s trunk passed +1 findbugs 0m 45s trunk passed +1 javadoc 0m 22s trunk passed with JDK v1.8.0_92 +1 javadoc 0m 17s trunk passed with JDK v1.7.0_95 +1 mvninstall 0m 24s the patch passed +1 compile 0m 22s the patch passed with JDK v1.8.0_92 +1 javac 0m 22s the patch passed +1 compile 0m 19s the patch passed with JDK v1.7.0_95 +1 javac 0m 19s the patch passed +1 checkstyle 0m 13s the patch passed +1 mvnsite 0m 25s the patch passed +1 mvneclipse 0m 13s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 0m 47s the patch passed +1 javadoc 0m 16s the patch passed with JDK v1.8.0_92 +1 javadoc 0m 15s the patch passed with JDK v1.7.0_95 +1 unit 9m 21s hadoop-distcp in the patch passed with JDK v1.8.0_92. +1 unit 8m 50s hadoop-distcp in the patch passed with JDK v1.7.0_95. +1 asflicense 0m 19s The patch does not generate ASF License warnings. 37m 1s Subsystem Report/Notes Docker Image:yetus/hadoop:cf2ee45 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12801351/HADOOP-12469.005.patch JIRA Issue HADOOP-12469 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux a059b13a3f1d 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 6243eab Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_92 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9221/testReport/ modules C: hadoop-tools/hadoop-distcp U: hadoop-tools/hadoop-distcp Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9221/console Powered by Apache Yetus 0.3.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          liuml07 Mingliang Liu added a comment -

          Re-uploading the same patch v5 to trigger Jenkins.

          Show
          liuml07 Mingliang Liu added a comment - Re-uploading the same patch v5 to trigger Jenkins.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 0s Docker mode activated.
          -1 docker 0m 1s Docker failed to build yetus/hadoop:7b1c37a.



          Subsystem Report/Notes
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12801168/HADOOP-12469.005.patch
          JIRA Issue HADOOP-12469
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9212/console
          Powered by Apache Yetus 0.3.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. -1 docker 0m 1s Docker failed to build yetus/hadoop:7b1c37a. Subsystem Report/Notes JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12801168/HADOOP-12469.005.patch JIRA Issue HADOOP-12469 Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9212/console Powered by Apache Yetus 0.3.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          liuml07 Mingliang Liu added a comment -

          Per-offline discussion with Jing Zhao, the v5 patch relaxes the need of wrapping a CopyReadException exception, which happens in multiple nested call paths, and checks if any Throwable in the exception chain matches CopyReadException exception type.

          Show
          liuml07 Mingliang Liu added a comment - Per-offline discussion with Jing Zhao , the v5 patch relaxes the need of wrapping a CopyReadException exception, which happens in multiple nested call paths, and checks if any Throwable in the exception chain matches CopyReadException exception type.
          Hide
          hadoopqa Hadoop QA added a comment -
          +1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 9s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 6m 43s trunk passed
          +1 compile 0m 14s trunk passed with JDK v1.8.0_92
          +1 compile 0m 17s trunk passed with JDK v1.7.0_95
          +1 checkstyle 0m 13s trunk passed
          +1 mvnsite 0m 22s trunk passed
          +1 mvneclipse 2m 13s trunk passed
          +1 findbugs 0m 30s trunk passed
          +1 javadoc 0m 13s trunk passed with JDK v1.8.0_92
          +1 javadoc 0m 15s trunk passed with JDK v1.7.0_95
          +1 mvninstall 0m 18s the patch passed
          +1 compile 0m 13s the patch passed with JDK v1.8.0_92
          +1 javac 0m 13s the patch passed
          +1 compile 0m 15s the patch passed with JDK v1.7.0_95
          +1 javac 0m 15s the patch passed
          +1 checkstyle 0m 11s the patch passed
          +1 mvnsite 0m 20s the patch passed
          +1 mvneclipse 0m 11s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 0m 39s the patch passed
          +1 javadoc 0m 10s the patch passed with JDK v1.8.0_92
          +1 javadoc 0m 13s the patch passed with JDK v1.7.0_95
          +1 unit 8m 19s hadoop-distcp in the patch passed with JDK v1.8.0_92.
          +1 unit 7m 23s hadoop-distcp in the patch passed with JDK v1.7.0_95.
          +1 asflicense 0m 18s The patch does not generate ASF License warnings.
          30m 37s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:fbe3e86
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12784743/HADOOP-12469.004.patch
          JIRA Issue HADOOP-12469
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 0c5233f3895e 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 68b4564
          Default Java 1.7.0_95
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_92 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
          findbugs v3.0.0
          JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9190/testReport/
          modules C: hadoop-tools/hadoop-distcp U: hadoop-tools/hadoop-distcp
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9190/console
          Powered by Apache Yetus 0.3.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 9s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 6m 43s trunk passed +1 compile 0m 14s trunk passed with JDK v1.8.0_92 +1 compile 0m 17s trunk passed with JDK v1.7.0_95 +1 checkstyle 0m 13s trunk passed +1 mvnsite 0m 22s trunk passed +1 mvneclipse 2m 13s trunk passed +1 findbugs 0m 30s trunk passed +1 javadoc 0m 13s trunk passed with JDK v1.8.0_92 +1 javadoc 0m 15s trunk passed with JDK v1.7.0_95 +1 mvninstall 0m 18s the patch passed +1 compile 0m 13s the patch passed with JDK v1.8.0_92 +1 javac 0m 13s the patch passed +1 compile 0m 15s the patch passed with JDK v1.7.0_95 +1 javac 0m 15s the patch passed +1 checkstyle 0m 11s the patch passed +1 mvnsite 0m 20s the patch passed +1 mvneclipse 0m 11s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 0m 39s the patch passed +1 javadoc 0m 10s the patch passed with JDK v1.8.0_92 +1 javadoc 0m 13s the patch passed with JDK v1.7.0_95 +1 unit 8m 19s hadoop-distcp in the patch passed with JDK v1.8.0_92. +1 unit 7m 23s hadoop-distcp in the patch passed with JDK v1.7.0_95. +1 asflicense 0m 18s The patch does not generate ASF License warnings. 30m 37s Subsystem Report/Notes Docker Image:yetus/hadoop:fbe3e86 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12784743/HADOOP-12469.004.patch JIRA Issue HADOOP-12469 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 0c5233f3895e 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 68b4564 Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_92 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9190/testReport/ modules C: hadoop-tools/hadoop-distcp U: hadoop-tools/hadoop-distcp Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9190/console Powered by Apache Yetus 0.3.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          liuml07 Mingliang Liu added a comment -

          Can anyone commit this reviewed patch? Thanks.

          Show
          liuml07 Mingliang Liu added a comment - Can anyone commit this reviewed patch? Thanks.
          Hide
          liuml07 Mingliang Liu added a comment -

          Hi Gera Shegalov, any further comments? Thanks.

          Show
          liuml07 Mingliang Liu added a comment - Hi Gera Shegalov , any further comments? Thanks.
          Hide
          hadoopqa Hadoop QA added a comment -
          +1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 0s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 6m 42s trunk passed
          +1 compile 0m 14s trunk passed with JDK v1.8.0_66
          +1 compile 0m 17s trunk passed with JDK v1.7.0_91
          +1 checkstyle 0m 14s trunk passed
          +1 mvnsite 0m 23s trunk passed
          +1 mvneclipse 0m 13s trunk passed
          +1 findbugs 0m 30s trunk passed
          +1 javadoc 0m 12s trunk passed with JDK v1.8.0_66
          +1 javadoc 0m 15s trunk passed with JDK v1.7.0_91
          +1 mvninstall 0m 18s the patch passed
          +1 compile 0m 12s the patch passed with JDK v1.8.0_66
          +1 javac 0m 12s the patch passed
          +1 compile 0m 14s the patch passed with JDK v1.7.0_91
          +1 javac 0m 14s the patch passed
          +1 checkstyle 0m 12s the patch passed
          +1 mvnsite 0m 20s the patch passed
          +1 mvneclipse 0m 10s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 findbugs 0m 36s the patch passed
          +1 javadoc 0m 10s the patch passed with JDK v1.8.0_66
          +1 javadoc 0m 12s the patch passed with JDK v1.7.0_91
          +1 unit 6m 55s hadoop-distcp in the patch passed with JDK v1.8.0_66.
          +1 unit 6m 47s hadoop-distcp in the patch passed with JDK v1.7.0_91.
          +1 asflicense 0m 17s Patch does not generate ASF License warnings.
          26m 23s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:0ca8df7
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12784743/HADOOP-12469.004.patch
          JIRA Issue HADOOP-12469
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 2033d7dde036 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / fb238d7
          Default Java 1.7.0_91
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91
          findbugs v3.0.0
          JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8485/testReport/
          modules C: hadoop-tools/hadoop-distcp U: hadoop-tools/hadoop-distcp
          Max memory used 76MB
          Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8485/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 6m 42s trunk passed +1 compile 0m 14s trunk passed with JDK v1.8.0_66 +1 compile 0m 17s trunk passed with JDK v1.7.0_91 +1 checkstyle 0m 14s trunk passed +1 mvnsite 0m 23s trunk passed +1 mvneclipse 0m 13s trunk passed +1 findbugs 0m 30s trunk passed +1 javadoc 0m 12s trunk passed with JDK v1.8.0_66 +1 javadoc 0m 15s trunk passed with JDK v1.7.0_91 +1 mvninstall 0m 18s the patch passed +1 compile 0m 12s the patch passed with JDK v1.8.0_66 +1 javac 0m 12s the patch passed +1 compile 0m 14s the patch passed with JDK v1.7.0_91 +1 javac 0m 14s the patch passed +1 checkstyle 0m 12s the patch passed +1 mvnsite 0m 20s the patch passed +1 mvneclipse 0m 10s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 0m 36s the patch passed +1 javadoc 0m 10s the patch passed with JDK v1.8.0_66 +1 javadoc 0m 12s the patch passed with JDK v1.7.0_91 +1 unit 6m 55s hadoop-distcp in the patch passed with JDK v1.8.0_66. +1 unit 6m 47s hadoop-distcp in the patch passed with JDK v1.7.0_91. +1 asflicense 0m 17s Patch does not generate ASF License warnings. 26m 23s Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12784743/HADOOP-12469.004.patch JIRA Issue HADOOP-12469 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 2033d7dde036 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / fb238d7 Default Java 1.7.0_91 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91 findbugs v3.0.0 JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8485/testReport/ modules C: hadoop-tools/hadoop-distcp U: hadoop-tools/hadoop-distcp Max memory used 76MB Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8485/console This message was automatically generated.
          Hide
          liuml07 Mingliang Liu added a comment -

          Thanks Gera Shegalov for your review and comments. The v4 patch moves the doTestIgnoreFailures() method back to its old position.

          The patch mainly changes helper method doTestIgnoreFailures by creating a file and making it unusable by setting 000 permission. The unit test testIgnoreFailures can not pass in trunk code without this patch, as it should wrap the CopyReadException only once.

          Show
          liuml07 Mingliang Liu added a comment - Thanks Gera Shegalov for your review and comments. The v4 patch moves the doTestIgnoreFailures() method back to its old position. The patch mainly changes helper method doTestIgnoreFailures by creating a file and making it unusable by setting 000 permission. The unit test testIgnoreFailures can not pass in trunk code without this patch, as it should wrap the CopyReadException only once.
          Hide
          jira.shegalov Gera Shegalov added a comment -

          Hi Mingliang Liu, can you move doTestIgnoreFailures back underneath testDirToFile so it's easier to see what you changed?

          Show
          jira.shegalov Gera Shegalov added a comment - Hi Mingliang Liu , can you move doTestIgnoreFailures back underneath testDirToFile so it's easier to see what you changed?
          Hide
          hadoopqa Hadoop QA added a comment -
          +1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 0s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 7m 37s trunk passed
          +1 compile 0m 15s trunk passed with JDK v1.8.0_66
          +1 compile 0m 17s trunk passed with JDK v1.7.0_91
          +1 checkstyle 0m 13s trunk passed
          +1 mvnsite 0m 23s trunk passed
          +1 mvneclipse 0m 13s trunk passed
          +1 findbugs 0m 28s trunk passed
          +1 javadoc 0m 12s trunk passed with JDK v1.8.0_66
          +1 javadoc 0m 15s trunk passed with JDK v1.7.0_91
          +1 mvninstall 0m 17s the patch passed
          +1 compile 0m 12s the patch passed with JDK v1.8.0_66
          +1 javac 0m 12s the patch passed
          +1 compile 0m 14s the patch passed with JDK v1.7.0_91
          +1 javac 0m 14s the patch passed
          +1 checkstyle 0m 11s the patch passed
          +1 mvnsite 0m 21s the patch passed
          +1 mvneclipse 0m 10s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 findbugs 0m 37s the patch passed
          +1 javadoc 0m 9s the patch passed with JDK v1.8.0_66
          +1 javadoc 0m 13s the patch passed with JDK v1.7.0_91
          +1 unit 6m 54s hadoop-distcp in the patch passed with JDK v1.8.0_66.
          +1 unit 6m 46s hadoop-distcp in the patch passed with JDK v1.7.0_91.
          +1 asflicense 0m 17s Patch does not generate ASF License warnings.
          27m 14s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:0ca8df7
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12784479/HADOOP-12469.003.patch
          JIRA Issue HADOOP-12469
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux f766c0f0e797 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / cf8af7b
          Default Java 1.7.0_91
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91
          findbugs v3.0.0
          JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8473/testReport/
          modules C: hadoop-tools/hadoop-distcp U: hadoop-tools/hadoop-distcp
          Max memory used 77MB
          Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8473/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 7m 37s trunk passed +1 compile 0m 15s trunk passed with JDK v1.8.0_66 +1 compile 0m 17s trunk passed with JDK v1.7.0_91 +1 checkstyle 0m 13s trunk passed +1 mvnsite 0m 23s trunk passed +1 mvneclipse 0m 13s trunk passed +1 findbugs 0m 28s trunk passed +1 javadoc 0m 12s trunk passed with JDK v1.8.0_66 +1 javadoc 0m 15s trunk passed with JDK v1.7.0_91 +1 mvninstall 0m 17s the patch passed +1 compile 0m 12s the patch passed with JDK v1.8.0_66 +1 javac 0m 12s the patch passed +1 compile 0m 14s the patch passed with JDK v1.7.0_91 +1 javac 0m 14s the patch passed +1 checkstyle 0m 11s the patch passed +1 mvnsite 0m 21s the patch passed +1 mvneclipse 0m 10s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 0m 37s the patch passed +1 javadoc 0m 9s the patch passed with JDK v1.8.0_66 +1 javadoc 0m 13s the patch passed with JDK v1.7.0_91 +1 unit 6m 54s hadoop-distcp in the patch passed with JDK v1.8.0_66. +1 unit 6m 46s hadoop-distcp in the patch passed with JDK v1.7.0_91. +1 asflicense 0m 17s Patch does not generate ASF License warnings. 27m 14s Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12784479/HADOOP-12469.003.patch JIRA Issue HADOOP-12469 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux f766c0f0e797 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / cf8af7b Default Java 1.7.0_91 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91 findbugs v3.0.0 JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8473/testReport/ modules C: hadoop-tools/hadoop-distcp U: hadoop-tools/hadoop-distcp Max memory used 77MB Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8473/console This message was automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -
          +1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 0s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 9m 1s trunk passed
          +1 compile 0m 27s trunk passed with JDK v1.8.0_66
          +1 compile 0m 17s trunk passed with JDK v1.7.0_91
          +1 checkstyle 0m 10s trunk passed
          +1 mvnsite 0m 23s trunk passed
          +1 mvneclipse 0m 13s trunk passed
          +1 findbugs 0m 30s trunk passed
          +1 javadoc 0m 13s trunk passed with JDK v1.8.0_66
          +1 javadoc 0m 15s trunk passed with JDK v1.7.0_91
          +1 mvninstall 0m 18s the patch passed
          +1 compile 0m 11s the patch passed with JDK v1.8.0_66
          +1 javac 0m 11s the patch passed
          +1 compile 0m 14s the patch passed with JDK v1.7.0_91
          +1 javac 0m 14s the patch passed
          +1 checkstyle 0m 9s the patch passed
          +1 mvnsite 0m 20s the patch passed
          +1 mvneclipse 0m 10s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 findbugs 0m 37s the patch passed
          +1 javadoc 0m 10s the patch passed with JDK v1.8.0_66
          +1 javadoc 0m 12s the patch passed with JDK v1.7.0_91
          +1 unit 7m 19s hadoop-distcp in the patch passed with JDK v1.8.0_66.
          +1 unit 7m 31s hadoop-distcp in the patch passed with JDK v1.7.0_91.
          +1 asflicense 0m 27s Patch does not generate ASF License warnings.
          30m 32s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:0ca8df7
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12783685/HADOOP-12469.002.patch
          JIRA Issue HADOOP-12469
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 7c4655b03f00 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / f3427d3
          Default Java 1.7.0_91
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91
          findbugs v3.0.0
          JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8447/testReport/
          modules C: hadoop-tools/hadoop-distcp U: hadoop-tools/hadoop-distcp
          Max memory used 77MB
          Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8447/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 9m 1s trunk passed +1 compile 0m 27s trunk passed with JDK v1.8.0_66 +1 compile 0m 17s trunk passed with JDK v1.7.0_91 +1 checkstyle 0m 10s trunk passed +1 mvnsite 0m 23s trunk passed +1 mvneclipse 0m 13s trunk passed +1 findbugs 0m 30s trunk passed +1 javadoc 0m 13s trunk passed with JDK v1.8.0_66 +1 javadoc 0m 15s trunk passed with JDK v1.7.0_91 +1 mvninstall 0m 18s the patch passed +1 compile 0m 11s the patch passed with JDK v1.8.0_66 +1 javac 0m 11s the patch passed +1 compile 0m 14s the patch passed with JDK v1.7.0_91 +1 javac 0m 14s the patch passed +1 checkstyle 0m 9s the patch passed +1 mvnsite 0m 20s the patch passed +1 mvneclipse 0m 10s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 0m 37s the patch passed +1 javadoc 0m 10s the patch passed with JDK v1.8.0_66 +1 javadoc 0m 12s the patch passed with JDK v1.7.0_91 +1 unit 7m 19s hadoop-distcp in the patch passed with JDK v1.8.0_66. +1 unit 7m 31s hadoop-distcp in the patch passed with JDK v1.7.0_91. +1 asflicense 0m 27s Patch does not generate ASF License warnings. 30m 32s Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12783685/HADOOP-12469.002.patch JIRA Issue HADOOP-12469 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 7c4655b03f00 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / f3427d3 Default Java 1.7.0_91 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91 findbugs v3.0.0 JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8447/testReport/ modules C: hadoop-tools/hadoop-distcp U: hadoop-tools/hadoop-distcp Max memory used 77MB Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8447/console This message was automatically generated.
          Hide
          liuml07 Mingliang Liu added a comment -

          The v2 patch rebases from trunk branch to resolve trivial conflicts.

          Show
          liuml07 Mingliang Liu added a comment - The v2 patch rebases from trunk branch to resolve trivial conflicts.
          Hide
          mithun Mithun Radhakrishnan added a comment -

          Ah, I see what you did there. +1.

          Show
          mithun Mithun Radhakrishnan added a comment - Ah, I see what you did there. +1.
          Hide
          hadoopqa Hadoop QA added a comment -
          +1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 7s docker + precommit patch detected.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 3m 22s trunk passed
          +1 compile 0m 14s trunk passed with JDK v1.8.0_60
          +1 compile 0m 15s trunk passed with JDK v1.7.0_79
          +1 checkstyle 0m 9s trunk passed
          +1 mvneclipse 0m 26s trunk passed
          +1 findbugs 0m 33s trunk passed
          +1 javadoc 0m 14s trunk passed with JDK v1.8.0_60
          +1 javadoc 0m 14s trunk passed with JDK v1.7.0_79
          +1 mvninstall 0m 19s the patch passed
          +1 compile 0m 13s the patch passed with JDK v1.8.0_60
          +1 javac 0m 13s the patch passed
          +1 compile 0m 14s the patch passed with JDK v1.7.0_79
          +1 javac 0m 14s the patch passed
          +1 checkstyle 0m 9s the patch passed
          +1 mvneclipse 0m 13s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 findbugs 0m 40s the patch passed
          +1 javadoc 0m 13s the patch passed with JDK v1.8.0_60
          +1 javadoc 0m 14s the patch passed with JDK v1.7.0_79
          +1 unit 8m 0s hadoop-distcp in the patch passed with JDK v1.8.0_60.
          +1 unit 7m 19s hadoop-distcp in the patch passed with JDK v1.7.0_79.
          +1 asflicense 0m 25s Patch does not generate ASF License warnings.
          24m 30s



          Subsystem Report/Notes
          Docker Client=1.7.1 Server=1.7.1 Image:test-patch-base-hadoop-date2015-11-02
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12769962/HADOOP-12469.001.patch
          JIRA Issue HADOOP-12469
          Optional Tests asflicense javac javadoc mvninstall unit findbugs checkstyle compile
          uname Linux 83e5bfdecf3b 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HADOOP-Build/patchprocess/apache-yetus-b17912e/precommit/personality/hadoop.sh
          git revision trunk / 78d6890
          Default Java 1.7.0_79
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_60 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_79
          findbugs v3.0.0
          JDK v1.7.0_79 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8000/testReport/
          modules C: hadoop-tools/hadoop-distcp U: hadoop-tools/hadoop-distcp
          Max memory used 226MB
          Powered by Apache Yetus http://yetus.apache.org
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8000/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 7s docker + precommit patch detected. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 3m 22s trunk passed +1 compile 0m 14s trunk passed with JDK v1.8.0_60 +1 compile 0m 15s trunk passed with JDK v1.7.0_79 +1 checkstyle 0m 9s trunk passed +1 mvneclipse 0m 26s trunk passed +1 findbugs 0m 33s trunk passed +1 javadoc 0m 14s trunk passed with JDK v1.8.0_60 +1 javadoc 0m 14s trunk passed with JDK v1.7.0_79 +1 mvninstall 0m 19s the patch passed +1 compile 0m 13s the patch passed with JDK v1.8.0_60 +1 javac 0m 13s the patch passed +1 compile 0m 14s the patch passed with JDK v1.7.0_79 +1 javac 0m 14s the patch passed +1 checkstyle 0m 9s the patch passed +1 mvneclipse 0m 13s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 0m 40s the patch passed +1 javadoc 0m 13s the patch passed with JDK v1.8.0_60 +1 javadoc 0m 14s the patch passed with JDK v1.7.0_79 +1 unit 8m 0s hadoop-distcp in the patch passed with JDK v1.8.0_60. +1 unit 7m 19s hadoop-distcp in the patch passed with JDK v1.7.0_79. +1 asflicense 0m 25s Patch does not generate ASF License warnings. 24m 30s Subsystem Report/Notes Docker Client=1.7.1 Server=1.7.1 Image:test-patch-base-hadoop-date2015-11-02 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12769962/HADOOP-12469.001.patch JIRA Issue HADOOP-12469 Optional Tests asflicense javac javadoc mvninstall unit findbugs checkstyle compile uname Linux 83e5bfdecf3b 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HADOOP-Build/patchprocess/apache-yetus-b17912e/precommit/personality/hadoop.sh git revision trunk / 78d6890 Default Java 1.7.0_79 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_60 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_79 findbugs v3.0.0 JDK v1.7.0_79 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8000/testReport/ modules C: hadoop-tools/hadoop-distcp U: hadoop-tools/hadoop-distcp Max memory used 226MB Powered by Apache Yetus http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8000/console This message was automatically generated.
          Hide
          liuml07 Mingliang Liu added a comment -

          As Gera Shegalov suggested, the v1 patch changes unit test helper method doTestIgnoreFailures by creating a file and making it unusable by setting 000 permission. The unit test testIgnoreFailures can not pass in trunk code without this patch, as it should wrap the CopyReadException only once.

          Show
          liuml07 Mingliang Liu added a comment - As Gera Shegalov suggested, the v1 patch changes unit test helper method doTestIgnoreFailures by creating a file and making it unusable by setting 000 permission. The unit test testIgnoreFailures can not pass in trunk code without this patch, as it should wrap the CopyReadException only once.
          Hide
          liuml07 Mingliang Liu added a comment -

          Thanks Haohui Mai for reviewing this patch. I should make it clear that it was not to address the whole issue when I upload the patch. Sorry for the confusion.

          As Gera Shegalov suggested, I'll update the patch with tests enabled. I filed a new issue HADOOP-12473 about the atomic option for ignoring failures.

          Show
          liuml07 Mingliang Liu added a comment - Thanks Haohui Mai for reviewing this patch. I should make it clear that it was not to address the whole issue when I upload the patch. Sorry for the confusion. As Gera Shegalov suggested, I'll update the patch with tests enabled. I filed a new issue HADOOP-12473 about the atomic option for ignoring failures.
          Hide
          jira.shegalov Gera Shegalov added a comment -

          Haohui Mai even if you had posted your +1, I would have appreciated a chance to review the patch before committing it.

          Show
          jira.shegalov Gera Shegalov added a comment - Haohui Mai even if you had posted your +1, I would have appreciated a chance to review the patch before committing it.
          Hide
          jira.shegalov Gera Shegalov added a comment -

          Hi Mingliang Liu, thanks for working on the patch. Regarding the unit test you should consider a full distcp test where you create a file and make it unusable by setting 000 permission. DistCp should be run doAs non-cluster-root user.

          Should we address the orthogonal ignoreFailures mutually exclusive with the atomic option separately?

          Sure, will you file it?

          Show
          jira.shegalov Gera Shegalov added a comment - Hi Mingliang Liu , thanks for working on the patch. Regarding the unit test you should consider a full distcp test where you create a file and make it unusable by setting 000 permission. DistCp should be run doAs non-cluster-root user. Should we address the orthogonal ignoreFailures mutually exclusive with the atomic option separately? Sure, will you file it?
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #481 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/481/)
          Revert "HADOOP-12469. distcp shout not ignore the ignoreFailures option. (wheat9: rev 7e2c971fedd1195a51203ff12fdfdeb4642f2082)

          • hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyMapper.java
          • hadoop-common-project/hadoop-common/CHANGES.txt
          • hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestCopyMapper.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #481 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/481/ ) Revert " HADOOP-12469 . distcp shout not ignore the ignoreFailures option. (wheat9: rev 7e2c971fedd1195a51203ff12fdfdeb4642f2082) hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyMapper.java hadoop-common-project/hadoop-common/CHANGES.txt hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestCopyMapper.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk #2419 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2419/)
          Revert "HADOOP-12469. distcp shout not ignore the ignoreFailures option. (wheat9: rev 7e2c971fedd1195a51203ff12fdfdeb4642f2082)

          • hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyMapper.java
          • hadoop-common-project/hadoop-common/CHANGES.txt
          • hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestCopyMapper.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2419 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2419/ ) Revert " HADOOP-12469 . distcp shout not ignore the ignoreFailures option. (wheat9: rev 7e2c971fedd1195a51203ff12fdfdeb4642f2082) hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyMapper.java hadoop-common-project/hadoop-common/CHANGES.txt hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestCopyMapper.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #521 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/521/)
          Revert "HADOOP-12469. distcp shout not ignore the ignoreFailures option. (wheat9: rev 7e2c971fedd1195a51203ff12fdfdeb4642f2082)

          • hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestCopyMapper.java
          • hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyMapper.java
          • hadoop-common-project/hadoop-common/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #521 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/521/ ) Revert " HADOOP-12469 . distcp shout not ignore the ignoreFailures option. (wheat9: rev 7e2c971fedd1195a51203ff12fdfdeb4642f2082) hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestCopyMapper.java hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyMapper.java hadoop-common-project/hadoop-common/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #509 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/509/)
          Revert "HADOOP-12469. distcp shout not ignore the ignoreFailures option. (wheat9: rev 7e2c971fedd1195a51203ff12fdfdeb4642f2082)

          • hadoop-common-project/hadoop-common/CHANGES.txt
          • hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestCopyMapper.java
          • hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyMapper.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #509 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/509/ ) Revert " HADOOP-12469 . distcp shout not ignore the ignoreFailures option. (wheat9: rev 7e2c971fedd1195a51203ff12fdfdeb4642f2082) hadoop-common-project/hadoop-common/CHANGES.txt hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestCopyMapper.java hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyMapper.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk #1246 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1246/)
          Revert "HADOOP-12469. distcp shout not ignore the ignoreFailures option. (wheat9: rev 7e2c971fedd1195a51203ff12fdfdeb4642f2082)

          • hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestCopyMapper.java
          • hadoop-common-project/hadoop-common/CHANGES.txt
          • hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyMapper.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk #1246 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1246/ ) Revert " HADOOP-12469 . distcp shout not ignore the ignoreFailures option. (wheat9: rev 7e2c971fedd1195a51203ff12fdfdeb4642f2082) hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestCopyMapper.java hadoop-common-project/hadoop-common/CHANGES.txt hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyMapper.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk #2455 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2455/)
          Revert "HADOOP-12469. distcp shout not ignore the ignoreFailures option. (wheat9: rev 7e2c971fedd1195a51203ff12fdfdeb4642f2082)

          • hadoop-common-project/hadoop-common/CHANGES.txt
          • hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyMapper.java
          • hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestCopyMapper.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #2455 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2455/ ) Revert " HADOOP-12469 . distcp shout not ignore the ignoreFailures option. (wheat9: rev 7e2c971fedd1195a51203ff12fdfdeb4642f2082) hadoop-common-project/hadoop-common/CHANGES.txt hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyMapper.java hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestCopyMapper.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #8607 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8607/)
          Revert "HADOOP-12469. distcp shout not ignore the ignoreFailures option. (wheat9: rev 7e2c971fedd1195a51203ff12fdfdeb4642f2082)

          • hadoop-common-project/hadoop-common/CHANGES.txt
          • hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyMapper.java
          • hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestCopyMapper.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #8607 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8607/ ) Revert " HADOOP-12469 . distcp shout not ignore the ignoreFailures option. (wheat9: rev 7e2c971fedd1195a51203ff12fdfdeb4642f2082) hadoop-common-project/hadoop-common/CHANGES.txt hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyMapper.java hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestCopyMapper.java
          Hide
          wheat9 Haohui Mai added a comment -

          Reopening the jira.

          Sorry for the noise – I realized that I haven't posted my +1 on the patch before committing. Taking a closer look the patch, it does not seem to really address the problem. I'm going to revert it now.

          Show
          wheat9 Haohui Mai added a comment - Reopening the jira. Sorry for the noise – I realized that I haven't posted my +1 on the patch before committing. Taking a closer look the patch, it does not seem to really address the problem. I'm going to revert it now.
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk #2418 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2418/)
          HADOOP-12469. distcp shout not ignore the ignoreFailures option. (wheat9: rev fa5a54cac7e0f70ef28ebfe94aee95ffbbaf20e8)

          • hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyMapper.java
          • hadoop-common-project/hadoop-common/CHANGES.txt
          • hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestCopyMapper.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2418 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2418/ ) HADOOP-12469 . distcp shout not ignore the ignoreFailures option. (wheat9: rev fa5a54cac7e0f70ef28ebfe94aee95ffbbaf20e8) hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyMapper.java hadoop-common-project/hadoop-common/CHANGES.txt hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestCopyMapper.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #480 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/480/)
          HADOOP-12469. distcp shout not ignore the ignoreFailures option. (wheat9: rev fa5a54cac7e0f70ef28ebfe94aee95ffbbaf20e8)

          • hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestCopyMapper.java
          • hadoop-common-project/hadoop-common/CHANGES.txt
          • hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyMapper.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #480 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/480/ ) HADOOP-12469 . distcp shout not ignore the ignoreFailures option. (wheat9: rev fa5a54cac7e0f70ef28ebfe94aee95ffbbaf20e8) hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestCopyMapper.java hadoop-common-project/hadoop-common/CHANGES.txt hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyMapper.java
          Hide
          wheat9 Haohui Mai added a comment -

          I've committed the patch to trunk and branch-2. Thanks Mingliang Liu for the contribution.

          Show
          wheat9 Haohui Mai added a comment - I've committed the patch to trunk and branch-2. Thanks Mingliang Liu for the contribution.
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #520 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/520/)
          HADOOP-12469. distcp shout not ignore the ignoreFailures option. (wheat9: rev fa5a54cac7e0f70ef28ebfe94aee95ffbbaf20e8)

          • hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyMapper.java
          • hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestCopyMapper.java
          • hadoop-common-project/hadoop-common/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #520 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/520/ ) HADOOP-12469 . distcp shout not ignore the ignoreFailures option. (wheat9: rev fa5a54cac7e0f70ef28ebfe94aee95ffbbaf20e8) hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyMapper.java hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestCopyMapper.java hadoop-common-project/hadoop-common/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk #1245 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1245/)
          HADOOP-12469. distcp shout not ignore the ignoreFailures option. (wheat9: rev fa5a54cac7e0f70ef28ebfe94aee95ffbbaf20e8)

          • hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestCopyMapper.java
          • hadoop-common-project/hadoop-common/CHANGES.txt
          • hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyMapper.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk #1245 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1245/ ) HADOOP-12469 . distcp shout not ignore the ignoreFailures option. (wheat9: rev fa5a54cac7e0f70ef28ebfe94aee95ffbbaf20e8) hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestCopyMapper.java hadoop-common-project/hadoop-common/CHANGES.txt hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyMapper.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk #2454 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2454/)
          HADOOP-12469. distcp shout not ignore the ignoreFailures option. (wheat9: rev fa5a54cac7e0f70ef28ebfe94aee95ffbbaf20e8)

          • hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyMapper.java
          • hadoop-common-project/hadoop-common/CHANGES.txt
          • hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestCopyMapper.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #2454 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2454/ ) HADOOP-12469 . distcp shout not ignore the ignoreFailures option. (wheat9: rev fa5a54cac7e0f70ef28ebfe94aee95ffbbaf20e8) hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyMapper.java hadoop-common-project/hadoop-common/CHANGES.txt hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestCopyMapper.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #508 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/508/)
          HADOOP-12469. distcp shout not ignore the ignoreFailures option. (wheat9: rev fa5a54cac7e0f70ef28ebfe94aee95ffbbaf20e8)

          • hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestCopyMapper.java
          • hadoop-common-project/hadoop-common/CHANGES.txt
          • hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyMapper.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #508 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/508/ ) HADOOP-12469 . distcp shout not ignore the ignoreFailures option. (wheat9: rev fa5a54cac7e0f70ef28ebfe94aee95ffbbaf20e8) hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestCopyMapper.java hadoop-common-project/hadoop-common/CHANGES.txt hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyMapper.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #8606 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8606/)
          HADOOP-12469. distcp shout not ignore the ignoreFailures option. (wheat9: rev fa5a54cac7e0f70ef28ebfe94aee95ffbbaf20e8)

          • hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyMapper.java
          • hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestCopyMapper.java
          • hadoop-common-project/hadoop-common/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #8606 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8606/ ) HADOOP-12469 . distcp shout not ignore the ignoreFailures option. (wheat9: rev fa5a54cac7e0f70ef28ebfe94aee95ffbbaf20e8) hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyMapper.java hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestCopyMapper.java hadoop-common-project/hadoop-common/CHANGES.txt
          Hide
          liuml07 Mingliang Liu added a comment -

          The release audit warning is unrelated.

          Show
          liuml07 Mingliang Liu added a comment - The release audit warning is unrelated.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 21m 4s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
          +1 javac 9m 32s There were no new javac warning messages.
          +1 javadoc 12m 24s There were no new javadoc warning messages.
          -1 release audit 0m 35s The applied patch generated 1 release audit warnings.
          +1 checkstyle 0m 38s There were no new checkstyle issues.
          +1 whitespace 0m 0s The patch has no lines that end in whitespace.
          +1 install 2m 11s mvn install still works.
          +1 eclipse:eclipse 0m 43s The patch built with eclipse:eclipse.
          +1 findbugs 1m 5s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          +1 tools/hadoop tests 8m 32s Tests passed in hadoop-distcp.
              56m 49s  



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12765932/HADOOP-12469.000.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / def374e
          Release Audit https://builds.apache.org/job/PreCommit-HADOOP-Build/7791/artifact/patchprocess/patchReleaseAuditProblems.txt
          hadoop-distcp test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7791/artifact/patchprocess/testrun_hadoop-distcp.txt
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/7791/testReport/
          Java 1.7.0_55
          uname Linux asf906.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/7791/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 21m 4s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 javac 9m 32s There were no new javac warning messages. +1 javadoc 12m 24s There were no new javadoc warning messages. -1 release audit 0m 35s The applied patch generated 1 release audit warnings. +1 checkstyle 0m 38s There were no new checkstyle issues. +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 2m 11s mvn install still works. +1 eclipse:eclipse 0m 43s The patch built with eclipse:eclipse. +1 findbugs 1m 5s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 tools/hadoop tests 8m 32s Tests passed in hadoop-distcp.     56m 49s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12765932/HADOOP-12469.000.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / def374e Release Audit https://builds.apache.org/job/PreCommit-HADOOP-Build/7791/artifact/patchprocess/patchReleaseAuditProblems.txt hadoop-distcp test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7791/artifact/patchprocess/testrun_hadoop-distcp.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/7791/testReport/ Java 1.7.0_55 uname Linux asf906.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/7791/console This message was automatically generated.
          Hide
          liuml07 Mingliang Liu added a comment -

          Hi Gera Shegalov, thanks for reporting this.

          I'm working on a patch to wrap the CopyReadException only once. However, I found it's hard to write a unit test which will fail. The current unit tests are not able to detect this bug because the map method throws an CopyReadException when calling getFileStatus, before it calls the copyFileWithRetry. This makes sense as the file is deleted before the copy operation. This way the unit tests pass with the following condition true (in case of ignoreFailures):

          if (ignoreFailures && exception.getCause() instanceof
                      RetriableFileCopyCommand.CopyReadException
          

          Should we address the orthogonal ignoreFailures mutually exclusive with the atomic option separately?

          Show
          liuml07 Mingliang Liu added a comment - Hi Gera Shegalov , thanks for reporting this. I'm working on a patch to wrap the CopyReadException only once. However, I found it's hard to write a unit test which will fail. The current unit tests are not able to detect this bug because the map method throws an CopyReadException when calling getFileStatus , before it calls the copyFileWithRetry . This makes sense as the file is deleted before the copy operation. This way the unit tests pass with the following condition true (in case of ignoreFailures): if (ignoreFailures && exception.getCause() instanceof RetriableFileCopyCommand.CopyReadException Should we address the orthogonal ignoreFailures mutually exclusive with the atomic option separately?

            People

            • Assignee:
              liuml07 Mingliang Liu
              Reporter:
              jira.shegalov Gera Shegalov
            • Votes:
              1 Vote for this issue
              Watchers:
              14 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development