Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-5485

Allow repeating job commit by extending OutputCommitter API

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 2.1.0-beta
    • Fix Version/s: 2.8.0, 2.7.3, 3.0.0-alpha1
    • Component/s: None
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Hide
      Previously, the MR job will get failed if AM get restarted for some reason (like node failure, etc.) during its doing commit job no matter if AM attempts reach to the maximum attempts.
      In this improvement, we add a new API isCommitJobRepeatable() to OutputCommitter interface which to indicate if job's committer can do commitJob again if previous commit work is interrupted by NM/AM failures, etc. The instance of OutputCommitter, which support repeatable job commit (like FileOutputCommitter in algorithm 2), can allow AM to continue the commitJob() after AM restart as a new attempt.
      Show
      Previously, the MR job will get failed if AM get restarted for some reason (like node failure, etc.) during its doing commit job no matter if AM attempts reach to the maximum attempts. In this improvement, we add a new API isCommitJobRepeatable() to OutputCommitter interface which to indicate if job's committer can do commitJob again if previous commit work is interrupted by NM/AM failures, etc. The instance of OutputCommitter, which support repeatable job commit (like FileOutputCommitter in algorithm 2), can allow AM to continue the commitJob() after AM restart as a new attempt.

      Description

      There are chances MRAppMaster crush during job committing,or NodeManager restart cause the committing AM exit due to container expire.In these cases ,the job will fail.
      However,some jobs can redo commit so failing the job becomes unnecessary.
      Let clients tell AM to allow redo commit or not is a better choice.
      This idea comes from Jason Lowe's comments in MAPREDUCE-4819

      1. MAPREDUCE-5485-demo.patch
        14 kB
        Junping Du
      2. MAPREDUCE-5485-demo-2.patch
        18 kB
        Junping Du
      3. MAPREDUCE-5485-v1.patch
        29 kB
        Junping Du
      4. MAPREDUCE-5485-v2.patch
        33 kB
        Junping Du
      5. MAPREDUCE-5485-v3.1.patch
        24 kB
        Junping Du
      6. MAPREDUCE-5485-v3.patch
        24 kB
        Junping Du
      7. MAPREDUCE-5485-v4.1.patch
        32 kB
        Junping Du
      8. MAPREDUCE-5485-v4.patch
        32 kB
        Junping Du
      9. MAPREDUCE-5485-v5.patch
        33 kB
        Junping Du
      10. MAPREDUCE-5485-v5-branch-2.7.patch
        33 kB
        Junping Du

        Issue Links

          Activity

          Hide
          nemon Nemon Lou added a comment -

          Some initial thoughts:
          1,Adding a method : boolean isCommitJobRepeatable() for outputCommitters.
          The abstract class OutputCommitter will return false for it.
          And FileOutputCommitter returns true,as FileOutputCommitter's commitJob will remove exists files that will be committed and then do the rename.

          For commit repeatable jobs,we have 2,3,4:
          2,When commitJob method throws exception,AM will retry commit directly with a times limit.
          3,When AM has an error during committing(error is not from the commitJob method),it will not reach a job final state ,but just exit and leave work to another AM.
          4,For secondly started AM,it will check the phase the job has reached.If the phase is commit failed,then it's state will reach job committing after recovery and start commit again.

          Show
          nemon Nemon Lou added a comment - Some initial thoughts: 1,Adding a method : boolean isCommitJobRepeatable() for outputCommitters. The abstract class OutputCommitter will return false for it. And FileOutputCommitter returns true,as FileOutputCommitter's commitJob will remove exists files that will be committed and then do the rename. For commit repeatable jobs,we have 2,3,4: 2,When commitJob method throws exception,AM will retry commit directly with a times limit. 3,When AM has an error during committing(error is not from the commitJob method),it will not reach a job final state ,but just exit and leave work to another AM. 4,For secondly started AM,it will check the phase the job has reached.If the phase is commit failed,then it's state will reach job committing after recovery and start commit again.
          Hide
          djp Junping Du added a comment -

          The proposal above sounds good to me. Nemon Lou, thanks for filing this JIRA which is quite useful in some scenarios. If you don't mind, I would like to work on it and move it forward. Thanks!

          Show
          djp Junping Du added a comment - The proposal above sounds good to me. Nemon Lou , thanks for filing this JIRA which is quite useful in some scenarios. If you don't mind, I would like to work on it and move it forward. Thanks!
          Hide
          nemon Nemon Lou added a comment -

          Feel free to take it. And thanks for your interest.

          Show
          nemon Nemon Lou added a comment - Feel free to take it. And thanks for your interest.
          Hide
          djp Junping Du added a comment -

          Upload a demo patch first. More unit test will be added later.
          BTW, it adopt some code in MAPREDUCE-5718 with similar purpose, so please share the credit to the contributors of MAPREDUCE-5718 if we want to commit the following patches of this JIRA in future.

          Show
          djp Junping Du added a comment - Upload a demo patch first. More unit test will be added later. BTW, it adopt some code in MAPREDUCE-5718 with similar purpose, so please share the credit to the contributors of MAPREDUCE-5718 if we want to commit the following patches of this JIRA in future.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 18m 5s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          +1 javac 8m 2s There were no new javac warning messages.
          +1 javadoc 10m 26s There were no new javadoc warning messages.
          +1 release audit 0m 24s The applied patch does not increase the total number of release audit warnings.
          -1 checkstyle 1m 10s The applied patch generated 2 new checkstyle issues (total was 566, now 567).
          -1 whitespace 0m 1s The patch has 3 line(s) that end in whitespace. Use git apply --whitespace=fix.
          +1 install 1m 33s mvn install still works.
          +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse.
          +1 findbugs 2m 36s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          -1 mapreduce tests 9m 41s Tests failed in hadoop-mapreduce-client-app.
          +1 mapreduce tests 1m 55s Tests passed in hadoop-mapreduce-client-core.
              54m 31s  



          Reason Tests
          Failed unit tests hadoop.mapreduce.v2.app.TestMRAppMaster



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12768071/MAPREDUCE-5485-demo.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 123b3db
          checkstyle https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6089/artifact/patchprocess/diffcheckstylehadoop-mapreduce-client-core.txt
          whitespace https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6089/artifact/patchprocess/whitespace.txt
          hadoop-mapreduce-client-app test log https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6089/artifact/patchprocess/testrun_hadoop-mapreduce-client-app.txt
          hadoop-mapreduce-client-core test log https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6089/artifact/patchprocess/testrun_hadoop-mapreduce-client-core.txt
          Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6089/testReport/
          Java 1.7.0_55
          uname Linux asf902.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6089/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 18m 5s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac 8m 2s There were no new javac warning messages. +1 javadoc 10m 26s There were no new javadoc warning messages. +1 release audit 0m 24s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 1m 10s The applied patch generated 2 new checkstyle issues (total was 566, now 567). -1 whitespace 0m 1s The patch has 3 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 install 1m 33s mvn install still works. +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse. +1 findbugs 2m 36s The patch does not introduce any new Findbugs (version 3.0.0) warnings. -1 mapreduce tests 9m 41s Tests failed in hadoop-mapreduce-client-app. +1 mapreduce tests 1m 55s Tests passed in hadoop-mapreduce-client-core.     54m 31s   Reason Tests Failed unit tests hadoop.mapreduce.v2.app.TestMRAppMaster Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12768071/MAPREDUCE-5485-demo.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 123b3db checkstyle https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6089/artifact/patchprocess/diffcheckstylehadoop-mapreduce-client-core.txt whitespace https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6089/artifact/patchprocess/whitespace.txt hadoop-mapreduce-client-app test log https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6089/artifact/patchprocess/testrun_hadoop-mapreduce-client-app.txt hadoop-mapreduce-client-core test log https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6089/artifact/patchprocess/testrun_hadoop-mapreduce-client-core.txt Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6089/testReport/ Java 1.7.0_55 uname Linux asf902.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6089/console This message was automatically generated.
          Hide
          djp Junping Du added a comment -

          Update 2nd demo patch, with fixing:
          1. Make sure FileOutputCommitter only repeatable when using algorithm 2 (algorithm 1 is not support yet).
          2. Make the temporary directory delete operation idempotent by allowing temporary directory is deleted because the directory does not exist (since it may have been deleted by the first AM).
          3. Make the SUCCESS file marker creation operation idempotent by allowing for the file to exist (since it may have been created by the first AM).
          Test is still ongoing, will add unit test in next patch.

          Show
          djp Junping Du added a comment - Update 2nd demo patch, with fixing: 1. Make sure FileOutputCommitter only repeatable when using algorithm 2 (algorithm 1 is not support yet). 2. Make the temporary directory delete operation idempotent by allowing temporary directory is deleted because the directory does not exist (since it may have been deleted by the first AM). 3. Make the SUCCESS file marker creation operation idempotent by allowing for the file to exist (since it may have been created by the first AM). Test is still ongoing, will add unit test in next patch.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 7s docker + precommit patch detected.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          +1 mvninstall 3m 5s trunk passed
          +1 compile 2m 14s trunk passed with JDK v1.8.0_60
          +1 compile 2m 14s trunk passed with JDK v1.7.0_79
          +1 checkstyle 0m 19s trunk passed
          +1 mvneclipse 0m 26s trunk passed
          +1 findbugs 1m 44s trunk passed
          +1 javadoc 0m 35s trunk passed with JDK v1.8.0_60
          +1 javadoc 0m 40s trunk passed with JDK v1.7.0_79
          -1 mvninstall 0m 14s hadoop-mapreduce-client-app in the patch failed.
          +1 compile 2m 14s the patch passed with JDK v1.8.0_60
          +1 javac 2m 14s the patch passed
          +1 compile 2m 16s the patch passed with JDK v1.7.0_79
          +1 javac 2m 16s the patch passed
          -1 checkstyle 0m 18s Patch generated 6 new checkstyle issues in hadoop-mapreduce-project/hadoop-mapreduce-client (total was 648, now 649).
          +1 mvneclipse 0m 27s the patch passed
          -1 whitespace 0m 0s The patch has 4 line(s) that end in whitespace. Use git apply --whitespace=fix.
          +1 xml 0m 1s The patch has no ill-formed XML file.
          -1 findbugs 1m 16s hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core introduced 2 new FindBugs issues.
          +1 javadoc 0m 35s the patch passed with JDK v1.8.0_60
          +1 javadoc 0m 43s the patch passed with JDK v1.7.0_79
          -1 unit 9m 26s hadoop-mapreduce-client-app in the patch failed with JDK v1.8.0_60.
          -1 unit 1m 32s hadoop-mapreduce-client-core in the patch failed with JDK v1.8.0_60.
          -1 unit 10m 4s hadoop-mapreduce-client-app in the patch failed with JDK v1.7.0_79.
          -1 unit 1m 54s hadoop-mapreduce-client-core in the patch failed with JDK v1.7.0_79.
          -1 asflicense 0m 20s Patch generated 1 ASF License warnings.
          45m 7s



          Reason Tests
          FindBugs module:hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core
            org.apache.hadoop.mapred.FileOutputCommitter.isCommitJobRepeatable(JobContext) doesn't override method in superclass because parameter type org.apache.hadoop.mapred.JobContext doesn't match superclass parameter type org.apache.hadoop.mapreduce.JobContext At FileOutputCommitter.java:superclass because parameter type org.apache.hadoop.mapred.JobContext doesn't match superclass parameter type org.apache.hadoop.mapreduce.JobContext At FileOutputCommitter.java:[line 194]
            org.apache.hadoop.mapred.OutputCommitter.isCommitJobRepeatable(JobContext) doesn't override method in superclass because parameter type org.apache.hadoop.mapred.JobContext doesn't match superclass parameter type org.apache.hadoop.mapreduce.JobContext At OutputCommitter.java:superclass because parameter type org.apache.hadoop.mapred.JobContext doesn't match superclass parameter type org.apache.hadoop.mapreduce.JobContext At OutputCommitter.java:[line 224]
          JDK v1.7.0_79 Failed junit tests hadoop.mapreduce.jobhistory.TestJobHistoryEventHandler
            hadoop.mapreduce.filecache.TestClientDistributedCacheManager
            hadoop.mapreduce.jobhistory.TestJobHistoryEventHandler
            hadoop.mapreduce.filecache.TestClientDistributedCacheManager



          Subsystem Report/Notes
          Docker Client=1.7.1 Server=1.7.1 Image:test-patch-base-hadoop-date2015-10-29
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12769599/MAPREDUCE-5485-demo-2.patch
          JIRA Issue MAPREDUCE-5485
          Optional Tests asflicense javac javadoc mvninstall unit findbugs checkstyle compile xml
          uname Linux cc2d6c263958 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /home/jenkins/jenkins-slave/workspace/PreCommit-MAPREDUCE-Build@2/patchprocess/apache-yetus-c3a2069/precommit/personality/hadoop.sh
          git revision trunk / 58d1df5
          Default Java 1.7.0_79
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_60 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_79
          findbugs v3.0.0
          mvninstall https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6100/artifact/patchprocess/patch-mvninstall-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app.txt
          checkstyle https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6100/artifact/patchprocess/diff-checkstyle-hadoop-mapreduce-project_hadoop-mapreduce-client.txt
          whitespace https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6100/artifact/patchprocess/whitespace-eol.txt
          findbugs https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6100/artifact/patchprocess/new-findbugs-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core.html
          unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6100/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.8.0_60.txt
          unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6100/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.8.0_60.txt
          unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6100/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.7.0_79.txt
          unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6100/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.7.0_79.txt
          unit test logs https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6100/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.8.0_60.txt https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6100/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.8.0_60.txt https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6100/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.7.0_79.txt https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6100/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.7.0_79.txt
          JDK v1.7.0_79 Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6100/testReport/
          asflicense https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6100/artifact/patchprocess/patch-asflicense-problems.txt
          modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core U: hadoop-mapreduce-project/hadoop-mapreduce-client
          Max memory used 225MB
          Powered by Apache Yetus http://yetus.apache.org
          Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6100/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 7s docker + precommit patch detected. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 3m 5s trunk passed +1 compile 2m 14s trunk passed with JDK v1.8.0_60 +1 compile 2m 14s trunk passed with JDK v1.7.0_79 +1 checkstyle 0m 19s trunk passed +1 mvneclipse 0m 26s trunk passed +1 findbugs 1m 44s trunk passed +1 javadoc 0m 35s trunk passed with JDK v1.8.0_60 +1 javadoc 0m 40s trunk passed with JDK v1.7.0_79 -1 mvninstall 0m 14s hadoop-mapreduce-client-app in the patch failed. +1 compile 2m 14s the patch passed with JDK v1.8.0_60 +1 javac 2m 14s the patch passed +1 compile 2m 16s the patch passed with JDK v1.7.0_79 +1 javac 2m 16s the patch passed -1 checkstyle 0m 18s Patch generated 6 new checkstyle issues in hadoop-mapreduce-project/hadoop-mapreduce-client (total was 648, now 649). +1 mvneclipse 0m 27s the patch passed -1 whitespace 0m 0s The patch has 4 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 xml 0m 1s The patch has no ill-formed XML file. -1 findbugs 1m 16s hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core introduced 2 new FindBugs issues. +1 javadoc 0m 35s the patch passed with JDK v1.8.0_60 +1 javadoc 0m 43s the patch passed with JDK v1.7.0_79 -1 unit 9m 26s hadoop-mapreduce-client-app in the patch failed with JDK v1.8.0_60. -1 unit 1m 32s hadoop-mapreduce-client-core in the patch failed with JDK v1.8.0_60. -1 unit 10m 4s hadoop-mapreduce-client-app in the patch failed with JDK v1.7.0_79. -1 unit 1m 54s hadoop-mapreduce-client-core in the patch failed with JDK v1.7.0_79. -1 asflicense 0m 20s Patch generated 1 ASF License warnings. 45m 7s Reason Tests FindBugs module:hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core   org.apache.hadoop.mapred.FileOutputCommitter.isCommitJobRepeatable(JobContext) doesn't override method in superclass because parameter type org.apache.hadoop.mapred.JobContext doesn't match superclass parameter type org.apache.hadoop.mapreduce.JobContext At FileOutputCommitter.java:superclass because parameter type org.apache.hadoop.mapred.JobContext doesn't match superclass parameter type org.apache.hadoop.mapreduce.JobContext At FileOutputCommitter.java: [line 194]   org.apache.hadoop.mapred.OutputCommitter.isCommitJobRepeatable(JobContext) doesn't override method in superclass because parameter type org.apache.hadoop.mapred.JobContext doesn't match superclass parameter type org.apache.hadoop.mapreduce.JobContext At OutputCommitter.java:superclass because parameter type org.apache.hadoop.mapred.JobContext doesn't match superclass parameter type org.apache.hadoop.mapreduce.JobContext At OutputCommitter.java: [line 224] JDK v1.7.0_79 Failed junit tests hadoop.mapreduce.jobhistory.TestJobHistoryEventHandler   hadoop.mapreduce.filecache.TestClientDistributedCacheManager   hadoop.mapreduce.jobhistory.TestJobHistoryEventHandler   hadoop.mapreduce.filecache.TestClientDistributedCacheManager Subsystem Report/Notes Docker Client=1.7.1 Server=1.7.1 Image:test-patch-base-hadoop-date2015-10-29 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12769599/MAPREDUCE-5485-demo-2.patch JIRA Issue MAPREDUCE-5485 Optional Tests asflicense javac javadoc mvninstall unit findbugs checkstyle compile xml uname Linux cc2d6c263958 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /home/jenkins/jenkins-slave/workspace/PreCommit-MAPREDUCE-Build@2/patchprocess/apache-yetus-c3a2069/precommit/personality/hadoop.sh git revision trunk / 58d1df5 Default Java 1.7.0_79 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_60 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_79 findbugs v3.0.0 mvninstall https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6100/artifact/patchprocess/patch-mvninstall-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app.txt checkstyle https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6100/artifact/patchprocess/diff-checkstyle-hadoop-mapreduce-project_hadoop-mapreduce-client.txt whitespace https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6100/artifact/patchprocess/whitespace-eol.txt findbugs https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6100/artifact/patchprocess/new-findbugs-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core.html unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6100/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.8.0_60.txt unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6100/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.8.0_60.txt unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6100/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.7.0_79.txt unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6100/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.7.0_79.txt unit test logs https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6100/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.8.0_60.txt https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6100/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.8.0_60.txt https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6100/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.7.0_79.txt https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6100/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.7.0_79.txt JDK v1.7.0_79 Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6100/testReport/ asflicense https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6100/artifact/patchprocess/patch-asflicense-problems.txt modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core U: hadoop-mapreduce-project/hadoop-mapreduce-client Max memory used 225MB Powered by Apache Yetus http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6100/console This message was automatically generated.
          Hide
          bikassaha Bikas Saha added a comment -

          The repeatable commit API seems useful. However, I dont understand why in this patch we are also changing the AM code to retry commits upon exception in commitJob() itself.

          From an offline conversation with Junping Du my understanding is that some commit operations can timeout etc. (e.g. delete of many nested dirs) and so a retry can prevent job failures.

          This is where making delete repeatable in the committer code (by catching the file does not exist exception) will help. This will make the commit operation actually repeatable. Perhaps we can do that in a separate jira which includes making the different commit steps repeatable. E.g. also creating the success marker file with an over-write option so that it does not fail if the file exists.

          More important though, the retry of the commit itself should probably be inside the committer itself. Moving it all the way up to the AM is leaking abstractions and also requires the implementation to be repeated across AM’s (MR, Tez, Spark etc.) And we cannot just retry in the AM on any exception because we don’t understand the semantics of the user land commit. IMO, this should move into the committer itself so that it retries the internal commit operations (based on a retry-able config) and based on the semantics of those operations.

          Show
          bikassaha Bikas Saha added a comment - The repeatable commit API seems useful. However, I dont understand why in this patch we are also changing the AM code to retry commits upon exception in commitJob() itself. From an offline conversation with Junping Du my understanding is that some commit operations can timeout etc. (e.g. delete of many nested dirs) and so a retry can prevent job failures. This is where making delete repeatable in the committer code (by catching the file does not exist exception) will help. This will make the commit operation actually repeatable. Perhaps we can do that in a separate jira which includes making the different commit steps repeatable. E.g. also creating the success marker file with an over-write option so that it does not fail if the file exists. More important though, the retry of the commit itself should probably be inside the committer itself. Moving it all the way up to the AM is leaking abstractions and also requires the implementation to be repeated across AM’s (MR, Tez, Spark etc.) And we cannot just retry in the AM on any exception because we don’t understand the semantics of the user land commit. IMO, this should move into the committer itself so that it retries the internal commit operations (based on a retry-able config) and based on the semantics of those operations.
          Hide
          djp Junping Du added a comment -

          Thanks Bikas Saha for the comments! I agree it makes more sense to move retry logic into committer.commitJob() if it support repeatable. My original thinking is to combine this retry for committer.commitJob() with other AM exceptions in handleJobCommit (outside of committer), like: failed to write endCommitSuccessFile, etc. But now I think we should separate committer retry with AM specific handling for the reason you mentioned above. For this case, I would prefer we just let AM exit directly instead of fail the job (if commit job is repeatable). Most like the same as proposed above by Nemon Lou, but a slightly different is: we should apply AM fail (not job fail) even for commiter.commitJob() failed after retry for handling some corner cases, i.e. something goes wrong with related to committer in this AM but still get chance to success in another AM if we support repeatable in commit job.
          I will update a patch soon.

          Show
          djp Junping Du added a comment - Thanks Bikas Saha for the comments! I agree it makes more sense to move retry logic into committer.commitJob() if it support repeatable. My original thinking is to combine this retry for committer.commitJob() with other AM exceptions in handleJobCommit (outside of committer), like: failed to write endCommitSuccessFile, etc. But now I think we should separate committer retry with AM specific handling for the reason you mentioned above. For this case, I would prefer we just let AM exit directly instead of fail the job (if commit job is repeatable). Most like the same as proposed above by Nemon Lou , but a slightly different is: we should apply AM fail (not job fail) even for commiter.commitJob() failed after retry for handling some corner cases, i.e. something goes wrong with related to committer in this AM but still get chance to success in another AM if we support repeatable in commit job. I will update a patch soon.
          Hide
          djp Junping Du added a comment -

          Attach a new patch to address Bikas comments above, include:
          1. Make retry logic go to committer.commitJob() rather than MRAppMaster
          2. It will fail AM instead of Job when exception happens during jobCommit if commitJob() is repeatable.
          3. Add related unit tests.
          Verify this feature works well on a small scale cluster that kill AM during job committing stage, and the job can continue and succeed after AM restarted.

          Show
          djp Junping Du added a comment - Attach a new patch to address Bikas comments above, include: 1. Make retry logic go to committer.commitJob() rather than MRAppMaster 2. It will fail AM instead of Job when exception happens during jobCommit if commitJob() is repeatable. 3. Add related unit tests. Verify this feature works well on a small scale cluster that kill AM during job committing stage, and the job can continue and succeed after AM restarted.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 11s docker + precommit patch detected.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 3 new or modified test files.
          +1 mvninstall 4m 3s trunk passed
          +1 compile 2m 52s trunk passed with JDK v1.8.0_66
          +1 compile 2m 44s trunk passed with JDK v1.7.0_79
          +1 checkstyle 0m 25s trunk passed
          +1 mvneclipse 0m 31s trunk passed
          +1 findbugs 2m 9s trunk passed
          +1 javadoc 0m 49s trunk passed with JDK v1.8.0_66
          +1 javadoc 0m 55s trunk passed with JDK v1.7.0_79
          -1 mvninstall 0m 17s hadoop-mapreduce-client-app in the patch failed.
          +1 compile 2m 55s the patch passed with JDK v1.8.0_66
          +1 javac 2m 55s the patch passed
          +1 compile 2m 57s the patch passed with JDK v1.7.0_79
          +1 javac 2m 57s the patch passed
          -1 checkstyle 0m 25s Patch generated 4 new checkstyle issues in hadoop-mapreduce-project/hadoop-mapreduce-client (total was 129, now 129).
          +1 mvneclipse 0m 33s the patch passed
          -1 whitespace 0m 0s The patch has 6 line(s) that end in whitespace. Use git apply --whitespace=fix.
          -1 findbugs 1m 35s hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core introduced 2 new FindBugs issues.
          -1 javadoc 2m 47s hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.8.0_66 with JDK v1.8.0_66 generated 2 new issues (was 100, now 100).
          +1 javadoc 0m 49s the patch passed with JDK v1.8.0_66
          +1 javadoc 0m 49s the patch passed with JDK v1.7.0_79
          +1 unit 10m 14s hadoop-mapreduce-client-app in the patch passed with JDK v1.8.0_66.
          -1 unit 2m 6s hadoop-mapreduce-client-core in the patch failed with JDK v1.8.0_66.
          +1 unit 10m 49s hadoop-mapreduce-client-app in the patch passed with JDK v1.7.0_79.
          -1 unit 2m 25s hadoop-mapreduce-client-core in the patch failed with JDK v1.7.0_79.
          -1 asflicense 0m 31s Patch generated 1 ASF License warnings.
          54m 4s



          Reason Tests
          FindBugs module:hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core
            org.apache.hadoop.mapred.FileOutputCommitter.isCommitJobRepeatable(JobContext) doesn't override method in superclass because parameter type org.apache.hadoop.mapred.JobContext doesn't match superclass parameter type org.apache.hadoop.mapreduce.JobContext At FileOutputCommitter.java:superclass because parameter type org.apache.hadoop.mapred.JobContext doesn't match superclass parameter type org.apache.hadoop.mapreduce.JobContext At FileOutputCommitter.java:[line 194]
            org.apache.hadoop.mapred.OutputCommitter.isCommitJobRepeatable(JobContext) doesn't override method in superclass because parameter type org.apache.hadoop.mapred.JobContext doesn't match superclass parameter type org.apache.hadoop.mapreduce.JobContext At OutputCommitter.java:superclass because parameter type org.apache.hadoop.mapred.JobContext doesn't match superclass parameter type org.apache.hadoop.mapreduce.JobContext At OutputCommitter.java:[line 224]
          JDK v1.8.0_66 Failed junit tests hadoop.mapreduce.filecache.TestClientDistributedCacheManager
          JDK v1.7.0_79 Failed junit tests hadoop.mapreduce.filecache.TestClientDistributedCacheManager



          Subsystem Report/Notes
          Docker Client=1.7.0 Server=1.7.0 Image:test-patch-base-hadoop-date2015-11-04
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12770634/MAPREDUCE-5485-v1.patch
          JIRA Issue MAPREDUCE-5485
          Optional Tests asflicense javac javadoc mvninstall unit findbugs checkstyle compile
          uname Linux 737907cbfcaa 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /home/jenkins/jenkins-slave/workspace/PreCommit-MAPREDUCE-Build/patchprocess/apache-yetus-d0f6847/precommit/personality/hadoop.sh
          git revision trunk / 5667129
          Default Java 1.7.0_79
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_79
          findbugs v3.0.0
          mvninstall https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6114/artifact/patchprocess/patch-mvninstall-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app.txt
          checkstyle https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6114/artifact/patchprocess/diff-checkstyle-hadoop-mapreduce-project_hadoop-mapreduce-client.txt
          whitespace https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6114/artifact/patchprocess/whitespace-eol.txt
          findbugs https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6114/artifact/patchprocess/new-findbugs-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core.html
          javadoc hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.8.0_66: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6114/artifact/patchprocess/diff-javadoc-javadoc-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.8.0_66.txt
          unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6114/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.8.0_66.txt
          unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6114/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.7.0_79.txt
          unit test logs https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6114/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6114/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.7.0_79.txt
          JDK v1.7.0_79 Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6114/testReport/
          asflicense https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6114/artifact/patchprocess/patch-asflicense-problems.txt
          modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core U: hadoop-mapreduce-project/hadoop-mapreduce-client
          Max memory used 226MB
          Powered by Apache Yetus http://yetus.apache.org
          Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6114/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 11s docker + precommit patch detected. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 3 new or modified test files. +1 mvninstall 4m 3s trunk passed +1 compile 2m 52s trunk passed with JDK v1.8.0_66 +1 compile 2m 44s trunk passed with JDK v1.7.0_79 +1 checkstyle 0m 25s trunk passed +1 mvneclipse 0m 31s trunk passed +1 findbugs 2m 9s trunk passed +1 javadoc 0m 49s trunk passed with JDK v1.8.0_66 +1 javadoc 0m 55s trunk passed with JDK v1.7.0_79 -1 mvninstall 0m 17s hadoop-mapreduce-client-app in the patch failed. +1 compile 2m 55s the patch passed with JDK v1.8.0_66 +1 javac 2m 55s the patch passed +1 compile 2m 57s the patch passed with JDK v1.7.0_79 +1 javac 2m 57s the patch passed -1 checkstyle 0m 25s Patch generated 4 new checkstyle issues in hadoop-mapreduce-project/hadoop-mapreduce-client (total was 129, now 129). +1 mvneclipse 0m 33s the patch passed -1 whitespace 0m 0s The patch has 6 line(s) that end in whitespace. Use git apply --whitespace=fix. -1 findbugs 1m 35s hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core introduced 2 new FindBugs issues. -1 javadoc 2m 47s hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.8.0_66 with JDK v1.8.0_66 generated 2 new issues (was 100, now 100). +1 javadoc 0m 49s the patch passed with JDK v1.8.0_66 +1 javadoc 0m 49s the patch passed with JDK v1.7.0_79 +1 unit 10m 14s hadoop-mapreduce-client-app in the patch passed with JDK v1.8.0_66. -1 unit 2m 6s hadoop-mapreduce-client-core in the patch failed with JDK v1.8.0_66. +1 unit 10m 49s hadoop-mapreduce-client-app in the patch passed with JDK v1.7.0_79. -1 unit 2m 25s hadoop-mapreduce-client-core in the patch failed with JDK v1.7.0_79. -1 asflicense 0m 31s Patch generated 1 ASF License warnings. 54m 4s Reason Tests FindBugs module:hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core   org.apache.hadoop.mapred.FileOutputCommitter.isCommitJobRepeatable(JobContext) doesn't override method in superclass because parameter type org.apache.hadoop.mapred.JobContext doesn't match superclass parameter type org.apache.hadoop.mapreduce.JobContext At FileOutputCommitter.java:superclass because parameter type org.apache.hadoop.mapred.JobContext doesn't match superclass parameter type org.apache.hadoop.mapreduce.JobContext At FileOutputCommitter.java: [line 194]   org.apache.hadoop.mapred.OutputCommitter.isCommitJobRepeatable(JobContext) doesn't override method in superclass because parameter type org.apache.hadoop.mapred.JobContext doesn't match superclass parameter type org.apache.hadoop.mapreduce.JobContext At OutputCommitter.java:superclass because parameter type org.apache.hadoop.mapred.JobContext doesn't match superclass parameter type org.apache.hadoop.mapreduce.JobContext At OutputCommitter.java: [line 224] JDK v1.8.0_66 Failed junit tests hadoop.mapreduce.filecache.TestClientDistributedCacheManager JDK v1.7.0_79 Failed junit tests hadoop.mapreduce.filecache.TestClientDistributedCacheManager Subsystem Report/Notes Docker Client=1.7.0 Server=1.7.0 Image:test-patch-base-hadoop-date2015-11-04 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12770634/MAPREDUCE-5485-v1.patch JIRA Issue MAPREDUCE-5485 Optional Tests asflicense javac javadoc mvninstall unit findbugs checkstyle compile uname Linux 737907cbfcaa 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /home/jenkins/jenkins-slave/workspace/PreCommit-MAPREDUCE-Build/patchprocess/apache-yetus-d0f6847/precommit/personality/hadoop.sh git revision trunk / 5667129 Default Java 1.7.0_79 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_79 findbugs v3.0.0 mvninstall https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6114/artifact/patchprocess/patch-mvninstall-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app.txt checkstyle https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6114/artifact/patchprocess/diff-checkstyle-hadoop-mapreduce-project_hadoop-mapreduce-client.txt whitespace https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6114/artifact/patchprocess/whitespace-eol.txt findbugs https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6114/artifact/patchprocess/new-findbugs-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core.html javadoc hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.8.0_66: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6114/artifact/patchprocess/diff-javadoc-javadoc-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.8.0_66.txt unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6114/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.8.0_66.txt unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6114/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.7.0_79.txt unit test logs https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6114/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6114/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.7.0_79.txt JDK v1.7.0_79 Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6114/testReport/ asflicense https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6114/artifact/patchprocess/patch-asflicense-problems.txt modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core U: hadoop-mapreduce-project/hadoop-mapreduce-client Max memory used 226MB Powered by Apache Yetus http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6114/console This message was automatically generated.
          Hide
          bikassaha Bikas Saha added a comment -

          I dont think AM should reboot to retry after commit failure. Commit repeatable implies that a commit can be retried if it was stopped while in progress. If the commit actually failed then there does not seem any reason to assume that retrying it will succeed. IMO if the commit reports a failure then AM should fail. Similarly, if a commit failure file exists (from a previous AM version) then the new version of the AM should respect that and fail since the commit has been reported to be failed.

          So, IMO, we are looking to change the code here. Thoughts?

                    //The commit is still pending, commit error
                    shutDownMessage =
                        "Job commit from a prior MRAppMaster attempt is " +
                        "potentially in progress. Preventing multiple commit executions";
                    forcedState = JobStateInternal.ERROR;
          

          Javadoc could be improved. Inline

             /**
          +   * Is repeatable job commit supported so that job commit can be repeated again
          +   * from previous failures. <<<< Returns true if an in-progress job commit can be retried. If the MR AM is re-run then it will check this value to determine if it can retry an in-progress commit that was started by a previous version. Note that in rare scenarios, the previous AM version might still be running at that time, due to system anomalies. Hence if this method returns true then the retry commit operation should be able to run concurrently with the previous operation.
          +   *
          +   * If repeatable job commit is supported, job restart can tolerant previous 
          +   * failures during job is committing.
          +   *
          +   * By default, it is not supported. Extended classes (like: 
          +   * FileOutputCommitter) should explicitly override it if provide support. << if they support it
          +   *
          +   * @param jobContext
          +   *          Context of the job whose output is being written.
          +   * @return <code>true</code> repeatable job commit is supported,
          +   *         <code>false</code> otherwise
          +   */
          

          num-retries instead of retries? Also, if its num-retries then default should be 0. If its num-attempts then default should be 1.

          +  public static final String FILEOUTPUTCOMMITTER_FAILURE_RETRIES =
          +      "mapreduce.fileoutputcommitter.failures.retry";

          Retry count checking code in the catch block subsumes the check retry count check in the while block?

          +    while (jobCommitNotFinished && (retries++ < retriesOnFailure)) {
          ...
          +      } catch (Exception e) {
          +        if (retries >= retriesOnFailure) {

          The previous operation could delete the path after the if check has succeeded. So we probably also need to catch FileNotFoundException exception class here and ignore it if repeatableCommit is true.

          +      // if job allow repeatable commit and pendingJobAttemptsPath could be
          +      // deleted by previous AM, we do nothing here.
          +      if (isCommitJobRepeatable(context) && !fs.exists(pendingJobAttemptsPath)) {
          +        return;
          +      }

          Do testcases need an @Test annotation?

          +  public void testCommitterWithFailureV1() throws Exception {
          +    testCommitterWithFailureInternal(1);

          firstTimeFail is probably a more clear name for what its doing - failing on the first attempt.

          +    boolean firstTimeCommit = true;

          Would be good to have a test that version 2 and retry = 1 will fail also.

          Testcases missing for specific changes in FileOutputCommitter for create/delete operation changes?

          Show
          bikassaha Bikas Saha added a comment - I dont think AM should reboot to retry after commit failure. Commit repeatable implies that a commit can be retried if it was stopped while in progress. If the commit actually failed then there does not seem any reason to assume that retrying it will succeed. IMO if the commit reports a failure then AM should fail. Similarly, if a commit failure file exists (from a previous AM version) then the new version of the AM should respect that and fail since the commit has been reported to be failed. So, IMO, we are looking to change the code here. Thoughts? //The commit is still pending, commit error shutDownMessage = "Job commit from a prior MRAppMaster attempt is " + "potentially in progress. Preventing multiple commit executions" ; forcedState = JobStateInternal.ERROR; Javadoc could be improved. Inline /** + * Is repeatable job commit supported so that job commit can be repeated again + * from previous failures. <<<< Returns true if an in-progress job commit can be retried. If the MR AM is re-run then it will check this value to determine if it can retry an in-progress commit that was started by a previous version. Note that in rare scenarios, the previous AM version might still be running at that time, due to system anomalies. Hence if this method returns true then the retry commit operation should be able to run concurrently with the previous operation. + * + * If repeatable job commit is supported, job restart can tolerant previous + * failures during job is committing. + * + * By default , it is not supported. Extended classes (like: + * FileOutputCommitter) should explicitly override it if provide support. << if they support it + * + * @param jobContext + * Context of the job whose output is being written. + * @ return <code> true </code> repeatable job commit is supported, + * <code> false </code> otherwise + */ num-retries instead of retries? Also, if its num-retries then default should be 0. If its num-attempts then default should be 1. + public static final String FILEOUTPUTCOMMITTER_FAILURE_RETRIES = + "mapreduce.fileoutputcommitter.failures.retry" ; Retry count checking code in the catch block subsumes the check retry count check in the while block? + while (jobCommitNotFinished && (retries++ < retriesOnFailure)) { ... + } catch (Exception e) { + if (retries >= retriesOnFailure) { The previous operation could delete the path after the if check has succeeded. So we probably also need to catch FileNotFoundException exception class here and ignore it if repeatableCommit is true. + // if job allow repeatable commit and pendingJobAttemptsPath could be + // deleted by previous AM, we do nothing here. + if (isCommitJobRepeatable(context) && !fs.exists(pendingJobAttemptsPath)) { + return ; + } Do testcases need an @Test annotation? + public void testCommitterWithFailureV1() throws Exception { + testCommitterWithFailureInternal(1); firstTimeFail is probably a more clear name for what its doing - failing on the first attempt. + boolean firstTimeCommit = true ; Would be good to have a test that version 2 and retry = 1 will fail also. Testcases missing for specific changes in FileOutputCommitter for create/delete operation changes?
          Hide
          djp Junping Du added a comment -

          Thanks Bikas for review and comments!

          If the commit actually failed then there does not seem any reason to assume that retrying it will succeed. IMO if the commit reports a failure then AM should fail. Similarly, if a commit failure file exists (from a previous AM version) then the new version of the AM should respect that and fail since the commit has been reported to be failed.

          There are still reasons that related to AM specific, i.e. previous AM cannot connect to FS (FS or other CloudFS), committer mis-behavior because of getting loaded incorrect (due to classpath or other defect), etc. I think it make sense to do the best effort to retry the commit failure (like other reason to cause AM failure) given the commit is repeatable and all tasks are done successfully.

          Javadoc could be improved. Inline

          Yes. I will.

          num-retries instead of retries? Also, if its num-retries then default should be 0. If its num-attempts then default should be 1.

          Ok. I will update to attempts. default to be 1 means no retry to keep consistent with previous behavior.

          Retry count checking code in the catch block subsumes the check retry count check in the while block?

          I don't think so. Can you take a look at it again?

          The previous operation could delete the path after the if check has succeeded. So we probably also need to catch FileNotFoundException exception class here and ignore it if repeatableCommit is true.

          That's good point. Will fix it.

          Do testcases need an @Test annotation?

          No. The test class extends TestCase, so all method start with "test" will be executed automatically.

          firstTimeFail is probably a more clear name for what its doing - failing on the first attempt. Would be good to have a test that version 2 and retry = 1 will fail also. Testcases missing for specific changes in FileOutputCommitter for create/delete operation changes?

          Sounds good. Will fix/add later.

          Show
          djp Junping Du added a comment - Thanks Bikas for review and comments! If the commit actually failed then there does not seem any reason to assume that retrying it will succeed. IMO if the commit reports a failure then AM should fail. Similarly, if a commit failure file exists (from a previous AM version) then the new version of the AM should respect that and fail since the commit has been reported to be failed. There are still reasons that related to AM specific, i.e. previous AM cannot connect to FS (FS or other CloudFS), committer mis-behavior because of getting loaded incorrect (due to classpath or other defect), etc. I think it make sense to do the best effort to retry the commit failure (like other reason to cause AM failure) given the commit is repeatable and all tasks are done successfully. Javadoc could be improved. Inline Yes. I will. num-retries instead of retries? Also, if its num-retries then default should be 0. If its num-attempts then default should be 1. Ok. I will update to attempts. default to be 1 means no retry to keep consistent with previous behavior. Retry count checking code in the catch block subsumes the check retry count check in the while block? I don't think so. Can you take a look at it again? The previous operation could delete the path after the if check has succeeded. So we probably also need to catch FileNotFoundException exception class here and ignore it if repeatableCommit is true. That's good point. Will fix it. Do testcases need an @Test annotation? No. The test class extends TestCase, so all method start with "test" will be executed automatically. firstTimeFail is probably a more clear name for what its doing - failing on the first attempt. Would be good to have a test that version 2 and retry = 1 will fail also. Testcases missing for specific changes in FileOutputCommitter for create/delete operation changes? Sounds good. Will fix/add later.
          Hide
          bikassaha Bikas Saha added a comment -

          I don't think so. Can you take a look at it again?

          Inline

          +    while (jobCommitNotFinished && (retries++ < retriesOnFailure)) {
          +      try {
          +        commitJobInternal(context);
          +        jobCommitNotFinished = false;
          +      } catch (Exception e) {
          +        if (retries >= retriesOnFailure) { <<<<< doing ++retries here can remove code duplication for the < check in the while?
          +          throw e;
          +        } else {
          +          LOG.warn("Exception get thrown in job commit, retry (" + retries +
          +              ") time.", e);
          +        }
          +      }
          +    }

          There are still reasons that related to AM specific, i.e. previous AM cannot connect to FS (FS or other CloudFS), committer mis-behavior because of getting loaded incorrect (due to classpath or other defect), etc. I think it make sense to do the best effort to retry the commit failure (like other reason to cause AM failure) given the commit is repeatable and all tasks are done successfully.

          Sure. But then for such cases commitIsRepeatable may not be strictly needed. Even for a non-repeatable committer, if there is a classpath issue (which can get fixed by retrying the AM) then the AM should retry, right? The scope of that change seems related to this but is perhaps large enough to deserve its own jira as a follow up to this one. E.g. if the committer has written a failed file then commit is failed. Maybe we need an extension or API exception that allows us to know if the committer error was fatal or non-fatal and write a retry/failed file based on that?

          Show
          bikassaha Bikas Saha added a comment - I don't think so. Can you take a look at it again? Inline + while (jobCommitNotFinished && (retries++ < retriesOnFailure)) { + try { + commitJobInternal(context); + jobCommitNotFinished = false ; + } catch (Exception e) { + if (retries >= retriesOnFailure) { <<<<< doing ++retries here can remove code duplication for the < check in the while ? + throw e; + } else { + LOG.warn( "Exception get thrown in job commit, retry (" + retries + + ") time." , e); + } + } + } There are still reasons that related to AM specific, i.e. previous AM cannot connect to FS (FS or other CloudFS), committer mis-behavior because of getting loaded incorrect (due to classpath or other defect), etc. I think it make sense to do the best effort to retry the commit failure (like other reason to cause AM failure) given the commit is repeatable and all tasks are done successfully. Sure. But then for such cases commitIsRepeatable may not be strictly needed. Even for a non-repeatable committer, if there is a classpath issue (which can get fixed by retrying the AM) then the AM should retry, right? The scope of that change seems related to this but is perhaps large enough to deserve its own jira as a follow up to this one. E.g. if the committer has written a failed file then commit is failed. Maybe we need an extension or API exception that allows us to know if the committer error was fatal or non-fatal and write a retry/failed file based on that?
          Hide
          djp Junping Du added a comment -

          Update patch according to review comments from Bikas.

          Show
          djp Junping Du added a comment - Update patch according to review comments from Bikas.
          Hide
          djp Junping Du added a comment -

          doing ++retries here can remove code duplication for the < check in the while?

          Sorry. I miss this comment in my patch just uploaded. Will update in next patch.

          Even for a non-repeatable committer, if there is a classpath issue (which can get fixed by retrying the AM) then the AM should retry, right?

          I agree this could be a potentially separated topic. However, it could take more time and effort to make sure the retry on non-repeatable committer won't bring risk to cause a successl commit which is not right for result and should get failed earlier. For repeatable committer, it seems no such risk but it could paid price of unnecessary retry in some cases but earn more chance for succeed in commit stage in other cases, especially you cannot differentiate the case belongs to former or later. Just like the exception of deleting temp directory failed, it could due to AM connection with HDFS (we should retry) or HDFS down permanently (we shouldn't retry). I would prefer the current trade-off: simple and best effort for commit success in repeatable case.

          Show
          djp Junping Du added a comment - doing ++retries here can remove code duplication for the < check in the while? Sorry. I miss this comment in my patch just uploaded. Will update in next patch. Even for a non-repeatable committer, if there is a classpath issue (which can get fixed by retrying the AM) then the AM should retry, right? I agree this could be a potentially separated topic. However, it could take more time and effort to make sure the retry on non-repeatable committer won't bring risk to cause a successl commit which is not right for result and should get failed earlier. For repeatable committer, it seems no such risk but it could paid price of unnecessary retry in some cases but earn more chance for succeed in commit stage in other cases, especially you cannot differentiate the case belongs to former or later. Just like the exception of deleting temp directory failed, it could due to AM connection with HDFS (we should retry) or HDFS down permanently (we shouldn't retry). I would prefer the current trade-off: simple and best effort for commit success in repeatable case.
          Hide
          bikassaha Bikas Saha added a comment -

          I am not disagreeing with the AM retry in an absolute sense. However, it does not seem to belong to this jira and is likely better done as a follow up.

          Show
          bikassaha Bikas Saha added a comment - I am not disagreeing with the AM retry in an absolute sense. However, it does not seem to belong to this jira and is likely better done as a follow up.
          Hide
          djp Junping Du added a comment -

          Upload v3 patch to address following comments:

          I am not disagreeing with the AM retry in an absolute sense. However, it does not seem to belong to this jira and is likely better done as a follow up.

          Make sense. We can separate this part (AM retry after commit failure) out.

          Show
          djp Junping Du added a comment - Upload v3 patch to address following comments: I am not disagreeing with the AM retry in an absolute sense. However, it does not seem to belong to this jira and is likely better done as a follow up. Make sense. We can separate this part (AM retry after commit failure) out.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 8s docker + precommit patch detected.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 3m 23s trunk passed
          +1 compile 2m 24s trunk passed with JDK v1.8.0_60
          +1 compile 2m 19s trunk passed with JDK v1.7.0_79
          +1 checkstyle 0m 20s trunk passed
          +1 mvneclipse 0m 27s trunk passed
          +1 findbugs 1m 46s trunk passed
          +1 javadoc 0m 38s trunk passed with JDK v1.8.0_60
          +1 javadoc 0m 42s trunk passed with JDK v1.7.0_79
          -1 mvninstall 0m 15s hadoop-mapreduce-client-app in the patch failed.
          +1 compile 2m 17s the patch passed with JDK v1.8.0_60
          +1 javac 2m 17s the patch passed
          +1 compile 2m 19s the patch passed with JDK v1.7.0_79
          +1 javac 2m 19s the patch passed
          -1 checkstyle 0m 20s Patch generated 5 new checkstyle issues in hadoop-mapreduce-project/hadoop-mapreduce-client (total was 128, now 129).
          +1 mvneclipse 0m 27s the patch passed
          -1 whitespace 0m 0s The patch has 7 line(s) that end in whitespace. Use git apply --whitespace=fix.
          -1 findbugs 1m 16s hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core introduced 2 new FindBugs issues.
          -1 javadoc 2m 21s hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.8.0_60 with JDK v1.8.0_60 generated 2 new issues (was 100, now 100).
          +1 javadoc 0m 46s the patch passed with JDK v1.8.0_60
          +1 javadoc 0m 51s the patch passed with JDK v1.7.0_79
          +1 unit 9m 9s hadoop-mapreduce-client-app in the patch passed with JDK v1.8.0_60.
          -1 unit 1m 35s hadoop-mapreduce-client-core in the patch failed with JDK v1.8.0_60.
          +1 unit 9m 42s hadoop-mapreduce-client-app in the patch passed with JDK v1.7.0_79.
          -1 unit 1m 59s hadoop-mapreduce-client-core in the patch failed with JDK v1.7.0_79.
          -1 asflicense 0m 21s Patch generated 1 ASF License warnings.
          45m 52s



          Reason Tests
          FindBugs module:hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core
            org.apache.hadoop.mapred.FileOutputCommitter.isCommitJobRepeatable(JobContext) doesn't override method in superclass because parameter type org.apache.hadoop.mapred.JobContext doesn't match superclass parameter type org.apache.hadoop.mapreduce.JobContext At FileOutputCommitter.java:superclass because parameter type org.apache.hadoop.mapred.JobContext doesn't match superclass parameter type org.apache.hadoop.mapreduce.JobContext At FileOutputCommitter.java:[line 194]
            org.apache.hadoop.mapred.OutputCommitter.isCommitJobRepeatable(JobContext) doesn't override method in superclass because parameter type org.apache.hadoop.mapred.JobContext doesn't match superclass parameter type org.apache.hadoop.mapreduce.JobContext At OutputCommitter.java:superclass because parameter type org.apache.hadoop.mapred.JobContext doesn't match superclass parameter type org.apache.hadoop.mapreduce.JobContext At OutputCommitter.java:[line 224]
          JDK v1.8.0_60 Failed junit tests hadoop.mapreduce.filecache.TestClientDistributedCacheManager
          JDK v1.7.0_79 Failed junit tests hadoop.mapreduce.filecache.TestClientDistributedCacheManager



          Subsystem Report/Notes
          Docker Client=1.7.1 Server=1.7.1 Image:test-patch-base-hadoop-date2015-11-09
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12771463/MAPREDUCE-5485-v3.patch
          JIRA Issue MAPREDUCE-5485
          Optional Tests asflicense javac javadoc mvninstall unit findbugs checkstyle compile
          uname Linux ac813784eb45 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /home/jenkins/jenkins-slave/workspace/PreCommit-MAPREDUCE-Build/patchprocess/apache-yetus-ee5baeb/precommit/personality/hadoop.sh
          git revision trunk / 94a1833
          Default Java 1.7.0_79
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_60 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_79
          findbugs v3.0.0
          mvninstall https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6135/artifact/patchprocess/patch-mvninstall-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app.txt
          checkstyle https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6135/artifact/patchprocess/diff-checkstyle-hadoop-mapreduce-project_hadoop-mapreduce-client.txt
          whitespace https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6135/artifact/patchprocess/whitespace-eol.txt
          findbugs https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6135/artifact/patchprocess/new-findbugs-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core.html
          javadoc hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.8.0_60: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6135/artifact/patchprocess/diff-javadoc-javadoc-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.8.0_60.txt
          unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6135/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.8.0_60.txt
          unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6135/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.7.0_79.txt
          unit test logs https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6135/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.8.0_60.txt https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6135/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.7.0_79.txt
          JDK v1.7.0_79 Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6135/testReport/
          asflicense https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6135/artifact/patchprocess/patch-asflicense-problems.txt
          modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core U: hadoop-mapreduce-project/hadoop-mapreduce-client
          Max memory used 230MB
          Powered by Apache Yetus http://yetus.apache.org
          Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6135/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 8s docker + precommit patch detected. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 3m 23s trunk passed +1 compile 2m 24s trunk passed with JDK v1.8.0_60 +1 compile 2m 19s trunk passed with JDK v1.7.0_79 +1 checkstyle 0m 20s trunk passed +1 mvneclipse 0m 27s trunk passed +1 findbugs 1m 46s trunk passed +1 javadoc 0m 38s trunk passed with JDK v1.8.0_60 +1 javadoc 0m 42s trunk passed with JDK v1.7.0_79 -1 mvninstall 0m 15s hadoop-mapreduce-client-app in the patch failed. +1 compile 2m 17s the patch passed with JDK v1.8.0_60 +1 javac 2m 17s the patch passed +1 compile 2m 19s the patch passed with JDK v1.7.0_79 +1 javac 2m 19s the patch passed -1 checkstyle 0m 20s Patch generated 5 new checkstyle issues in hadoop-mapreduce-project/hadoop-mapreduce-client (total was 128, now 129). +1 mvneclipse 0m 27s the patch passed -1 whitespace 0m 0s The patch has 7 line(s) that end in whitespace. Use git apply --whitespace=fix. -1 findbugs 1m 16s hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core introduced 2 new FindBugs issues. -1 javadoc 2m 21s hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.8.0_60 with JDK v1.8.0_60 generated 2 new issues (was 100, now 100). +1 javadoc 0m 46s the patch passed with JDK v1.8.0_60 +1 javadoc 0m 51s the patch passed with JDK v1.7.0_79 +1 unit 9m 9s hadoop-mapreduce-client-app in the patch passed with JDK v1.8.0_60. -1 unit 1m 35s hadoop-mapreduce-client-core in the patch failed with JDK v1.8.0_60. +1 unit 9m 42s hadoop-mapreduce-client-app in the patch passed with JDK v1.7.0_79. -1 unit 1m 59s hadoop-mapreduce-client-core in the patch failed with JDK v1.7.0_79. -1 asflicense 0m 21s Patch generated 1 ASF License warnings. 45m 52s Reason Tests FindBugs module:hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core   org.apache.hadoop.mapred.FileOutputCommitter.isCommitJobRepeatable(JobContext) doesn't override method in superclass because parameter type org.apache.hadoop.mapred.JobContext doesn't match superclass parameter type org.apache.hadoop.mapreduce.JobContext At FileOutputCommitter.java:superclass because parameter type org.apache.hadoop.mapred.JobContext doesn't match superclass parameter type org.apache.hadoop.mapreduce.JobContext At FileOutputCommitter.java: [line 194]   org.apache.hadoop.mapred.OutputCommitter.isCommitJobRepeatable(JobContext) doesn't override method in superclass because parameter type org.apache.hadoop.mapred.JobContext doesn't match superclass parameter type org.apache.hadoop.mapreduce.JobContext At OutputCommitter.java:superclass because parameter type org.apache.hadoop.mapred.JobContext doesn't match superclass parameter type org.apache.hadoop.mapreduce.JobContext At OutputCommitter.java: [line 224] JDK v1.8.0_60 Failed junit tests hadoop.mapreduce.filecache.TestClientDistributedCacheManager JDK v1.7.0_79 Failed junit tests hadoop.mapreduce.filecache.TestClientDistributedCacheManager Subsystem Report/Notes Docker Client=1.7.1 Server=1.7.1 Image:test-patch-base-hadoop-date2015-11-09 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12771463/MAPREDUCE-5485-v3.patch JIRA Issue MAPREDUCE-5485 Optional Tests asflicense javac javadoc mvninstall unit findbugs checkstyle compile uname Linux ac813784eb45 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /home/jenkins/jenkins-slave/workspace/PreCommit-MAPREDUCE-Build/patchprocess/apache-yetus-ee5baeb/precommit/personality/hadoop.sh git revision trunk / 94a1833 Default Java 1.7.0_79 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_60 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_79 findbugs v3.0.0 mvninstall https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6135/artifact/patchprocess/patch-mvninstall-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app.txt checkstyle https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6135/artifact/patchprocess/diff-checkstyle-hadoop-mapreduce-project_hadoop-mapreduce-client.txt whitespace https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6135/artifact/patchprocess/whitespace-eol.txt findbugs https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6135/artifact/patchprocess/new-findbugs-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core.html javadoc hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.8.0_60: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6135/artifact/patchprocess/diff-javadoc-javadoc-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.8.0_60.txt unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6135/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.8.0_60.txt unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6135/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.7.0_79.txt unit test logs https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6135/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.8.0_60.txt https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6135/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.7.0_79.txt JDK v1.7.0_79 Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6135/testReport/ asflicense https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6135/artifact/patchprocess/patch-asflicense-problems.txt modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core U: hadoop-mapreduce-project/hadoop-mapreduce-client Max memory used 230MB Powered by Apache Yetus http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6135/console This message was automatically generated.
          Hide
          djp Junping Du added a comment -

          v3.1 patch to fix minor issues (whitespace, checkstyle, java doc, etc.) reported by Jenkins. The unit test failure TestClientDistributedCacheManager.testDetermineCacheVisibilities is not related, will file a separated JIRA to fix it.

          Show
          djp Junping Du added a comment - v3.1 patch to fix minor issues (whitespace, checkstyle, java doc, etc.) reported by Jenkins. The unit test failure TestClientDistributedCacheManager.testDetermineCacheVisibilities is not related, will file a separated JIRA to fix it.
          Hide
          djp Junping Du added a comment -

          Actually, the test failure of TestClientDistributedCacheManager.testDetermineCacheVisibilities is already tracked by MAPREDUCE-6533.

          Show
          djp Junping Du added a comment - Actually, the test failure of TestClientDistributedCacheManager.testDetermineCacheVisibilities is already tracked by MAPREDUCE-6533 .
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 5s docker + precommit patch detected.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 3m 17s trunk passed
          +1 compile 2m 15s trunk passed with JDK v1.8.0_66
          +1 compile 2m 18s trunk passed with JDK v1.7.0_85
          +1 checkstyle 0m 20s trunk passed
          +1 mvneclipse 0m 27s trunk passed
          +1 findbugs 1m 53s trunk passed
          +1 javadoc 0m 37s trunk passed with JDK v1.8.0_66
          +1 javadoc 0m 42s trunk passed with JDK v1.7.0_85
          -1 mvninstall 0m 14s hadoop-mapreduce-client-app in the patch failed.
          +1 compile 2m 14s the patch passed with JDK v1.8.0_66
          +1 javac 2m 14s the patch passed
          +1 compile 2m 19s the patch passed with JDK v1.7.0_85
          +1 javac 2m 19s the patch passed
          -1 checkstyle 0m 18s Patch generated 1 new checkstyle issues in hadoop-mapreduce-project/hadoop-mapreduce-client (total was 129, now 126).
          +1 mvneclipse 0m 27s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          -1 findbugs 1m 19s hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core introduced 2 new FindBugs issues.
          +1 javadoc 0m 36s the patch passed with JDK v1.8.0_66
          +1 javadoc 0m 41s the patch passed with JDK v1.7.0_85
          +1 unit 9m 2s hadoop-mapreduce-client-app in the patch passed with JDK v1.8.0_66.
          -1 unit 1m 38s hadoop-mapreduce-client-core in the patch failed with JDK v1.8.0_66.
          +1 unit 9m 38s hadoop-mapreduce-client-app in the patch passed with JDK v1.7.0_85.
          -1 unit 1m 54s hadoop-mapreduce-client-core in the patch failed with JDK v1.7.0_85.
          -1 asflicense 0m 20s Patch generated 1 ASF License warnings.
          44m 54s



          Reason Tests
          FindBugs module:hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core
            org.apache.hadoop.mapred.FileOutputCommitter.isCommitJobRepeatable(JobContext) doesn't override method in superclass because parameter type org.apache.hadoop.mapred.JobContext doesn't match superclass parameter type org.apache.hadoop.mapreduce.JobContext At FileOutputCommitter.java:superclass because parameter type org.apache.hadoop.mapred.JobContext doesn't match superclass parameter type org.apache.hadoop.mapreduce.JobContext At FileOutputCommitter.java:[line 194]
            org.apache.hadoop.mapred.OutputCommitter.isCommitJobRepeatable(JobContext) doesn't override method in superclass because parameter type org.apache.hadoop.mapred.JobContext doesn't match superclass parameter type org.apache.hadoop.mapreduce.JobContext At OutputCommitter.java:superclass because parameter type org.apache.hadoop.mapred.JobContext doesn't match superclass parameter type org.apache.hadoop.mapreduce.JobContext At OutputCommitter.java:[line 225]
          JDK v1.8.0_66 Failed junit tests hadoop.mapreduce.filecache.TestClientDistributedCacheManager
          JDK v1.7.0_85 Failed junit tests hadoop.mapreduce.filecache.TestClientDistributedCacheManager



          Subsystem Report/Notes
          Docker Client=1.7.1 Server=1.7.1 Image:test-patch-base-hadoop-date2015-11-10
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12771475/MAPREDUCE-5485-v3.1.patch
          JIRA Issue MAPREDUCE-5485
          Optional Tests asflicense javac javadoc mvninstall unit findbugs checkstyle compile
          uname Linux 0f150fbd885b 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /home/jenkins/jenkins-slave/workspace/PreCommit-MAPREDUCE-Build/patchprocess/apache-yetus-ee5baeb/precommit/personality/hadoop.sh
          git revision trunk / 94a1833
          Default Java 1.7.0_85
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_85
          findbugs v3.0.0
          mvninstall https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6136/artifact/patchprocess/patch-mvninstall-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app.txt
          checkstyle https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6136/artifact/patchprocess/diff-checkstyle-hadoop-mapreduce-project_hadoop-mapreduce-client.txt
          findbugs https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6136/artifact/patchprocess/new-findbugs-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core.html
          unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6136/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.8.0_66.txt
          unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6136/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.7.0_85.txt
          unit test logs https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6136/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6136/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.7.0_85.txt
          JDK v1.7.0_85 Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6136/testReport/
          asflicense https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6136/artifact/patchprocess/patch-asflicense-problems.txt
          modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core U: hadoop-mapreduce-project/hadoop-mapreduce-client
          Max memory used 230MB
          Powered by Apache Yetus http://yetus.apache.org
          Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6136/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 5s docker + precommit patch detected. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 3m 17s trunk passed +1 compile 2m 15s trunk passed with JDK v1.8.0_66 +1 compile 2m 18s trunk passed with JDK v1.7.0_85 +1 checkstyle 0m 20s trunk passed +1 mvneclipse 0m 27s trunk passed +1 findbugs 1m 53s trunk passed +1 javadoc 0m 37s trunk passed with JDK v1.8.0_66 +1 javadoc 0m 42s trunk passed with JDK v1.7.0_85 -1 mvninstall 0m 14s hadoop-mapreduce-client-app in the patch failed. +1 compile 2m 14s the patch passed with JDK v1.8.0_66 +1 javac 2m 14s the patch passed +1 compile 2m 19s the patch passed with JDK v1.7.0_85 +1 javac 2m 19s the patch passed -1 checkstyle 0m 18s Patch generated 1 new checkstyle issues in hadoop-mapreduce-project/hadoop-mapreduce-client (total was 129, now 126). +1 mvneclipse 0m 27s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. -1 findbugs 1m 19s hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core introduced 2 new FindBugs issues. +1 javadoc 0m 36s the patch passed with JDK v1.8.0_66 +1 javadoc 0m 41s the patch passed with JDK v1.7.0_85 +1 unit 9m 2s hadoop-mapreduce-client-app in the patch passed with JDK v1.8.0_66. -1 unit 1m 38s hadoop-mapreduce-client-core in the patch failed with JDK v1.8.0_66. +1 unit 9m 38s hadoop-mapreduce-client-app in the patch passed with JDK v1.7.0_85. -1 unit 1m 54s hadoop-mapreduce-client-core in the patch failed with JDK v1.7.0_85. -1 asflicense 0m 20s Patch generated 1 ASF License warnings. 44m 54s Reason Tests FindBugs module:hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core   org.apache.hadoop.mapred.FileOutputCommitter.isCommitJobRepeatable(JobContext) doesn't override method in superclass because parameter type org.apache.hadoop.mapred.JobContext doesn't match superclass parameter type org.apache.hadoop.mapreduce.JobContext At FileOutputCommitter.java:superclass because parameter type org.apache.hadoop.mapred.JobContext doesn't match superclass parameter type org.apache.hadoop.mapreduce.JobContext At FileOutputCommitter.java: [line 194]   org.apache.hadoop.mapred.OutputCommitter.isCommitJobRepeatable(JobContext) doesn't override method in superclass because parameter type org.apache.hadoop.mapred.JobContext doesn't match superclass parameter type org.apache.hadoop.mapreduce.JobContext At OutputCommitter.java:superclass because parameter type org.apache.hadoop.mapred.JobContext doesn't match superclass parameter type org.apache.hadoop.mapreduce.JobContext At OutputCommitter.java: [line 225] JDK v1.8.0_66 Failed junit tests hadoop.mapreduce.filecache.TestClientDistributedCacheManager JDK v1.7.0_85 Failed junit tests hadoop.mapreduce.filecache.TestClientDistributedCacheManager Subsystem Report/Notes Docker Client=1.7.1 Server=1.7.1 Image:test-patch-base-hadoop-date2015-11-10 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12771475/MAPREDUCE-5485-v3.1.patch JIRA Issue MAPREDUCE-5485 Optional Tests asflicense javac javadoc mvninstall unit findbugs checkstyle compile uname Linux 0f150fbd885b 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /home/jenkins/jenkins-slave/workspace/PreCommit-MAPREDUCE-Build/patchprocess/apache-yetus-ee5baeb/precommit/personality/hadoop.sh git revision trunk / 94a1833 Default Java 1.7.0_85 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_85 findbugs v3.0.0 mvninstall https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6136/artifact/patchprocess/patch-mvninstall-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app.txt checkstyle https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6136/artifact/patchprocess/diff-checkstyle-hadoop-mapreduce-project_hadoop-mapreduce-client.txt findbugs https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6136/artifact/patchprocess/new-findbugs-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core.html unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6136/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.8.0_66.txt unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6136/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.7.0_85.txt unit test logs https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6136/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6136/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.7.0_85.txt JDK v1.7.0_85 Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6136/testReport/ asflicense https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6136/artifact/patchprocess/patch-asflicense-problems.txt modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core U: hadoop-mapreduce-project/hadoop-mapreduce-client Max memory used 230MB Powered by Apache Yetus http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6136/console This message was automatically generated.
          Hide
          bikassaha Bikas Saha added a comment -

          Separate jira sounds good. We may need to add a new exception that can be thrown by the committer that clearly signals that the failure can be retried (e.g. NN in safemode) vs a fatal error (e.g. missing task files).

          About the patch.

          +      // cleanup previous half done commits if committer supports repeatable job
          +      // commit.
          +      if (isCommitJobRepeatable()) {
          +        cleanupInterruptedCommit(conf);
          +      }

          Is the above check too early in the code. E.g. IIRC, at this point we have not checked whether the previous job commit was succeeded or failed - in which case we cannot recover and there is nothing to do. We should be doing this check and cleaning up files only when we detect an in-progress commit - ie. startCommit file is present and finish/failedCommit file is not present. Else with this patch, we may end up redoing commit for an already successful commit.
          Is this the code we need to change?

          //The commit is still pending, commit error
                    shutDownMessage =
                        "Job commit from a prior MRAppMaster attempt is " +
                        "potentially in progress. Preventing multiple commit executions";
                    forcedState = JobStateInternal.ERROR;

          Also, we have already changed the startCommit operation to be repeatable via the overwrite flag. After that is there a need to delete the files upfront. Delete may be an expensive operation on some cloud stores.

          Mapred javadoc fixes are missing. Also there are some typos in there. E.g.

          +   * If repeatable job commit is supported, job restart can tolerant previous <<< tolerate ?
          +   * failures during job is committing. <<< during job commit ?
          

          This part of the code change could use some tests.

          Tests for repeatable success marker file and FileExistsException for repeatable deletes would be good to have.

          Show
          bikassaha Bikas Saha added a comment - Separate jira sounds good. We may need to add a new exception that can be thrown by the committer that clearly signals that the failure can be retried (e.g. NN in safemode) vs a fatal error (e.g. missing task files). About the patch. + // cleanup previous half done commits if committer supports repeatable job + // commit. + if (isCommitJobRepeatable()) { + cleanupInterruptedCommit(conf); + } Is the above check too early in the code. E.g. IIRC, at this point we have not checked whether the previous job commit was succeeded or failed - in which case we cannot recover and there is nothing to do. We should be doing this check and cleaning up files only when we detect an in-progress commit - ie. startCommit file is present and finish/failedCommit file is not present. Else with this patch, we may end up redoing commit for an already successful commit. Is this the code we need to change? //The commit is still pending, commit error shutDownMessage = "Job commit from a prior MRAppMaster attempt is " + "potentially in progress. Preventing multiple commit executions" ; forcedState = JobStateInternal.ERROR; Also, we have already changed the startCommit operation to be repeatable via the overwrite flag. After that is there a need to delete the files upfront. Delete may be an expensive operation on some cloud stores. Mapred javadoc fixes are missing. Also there are some typos in there. E.g. + * If repeatable job commit is supported, job restart can tolerant previous <<< tolerate ? + * failures during job is committing. <<< during job commit ? This part of the code change could use some tests. Tests for repeatable success marker file and FileExistsException for repeatable deletes would be good to have.
          Hide
          djp Junping Du added a comment -

          Thanks Bikas Saha for comments!

          Is the above check too early in the code. E.g. IIRC, at this point we have not checked whether the previous job commit was succeeded or failed - in which case we cannot recover and there is nothing to do.

          The cleanupInterruptedCommit() already check previous job commit succeed or failed. Am I missing anything here?

          Also, we have already changed the startCommit operation to be repeatable via the overwrite flag. After that is there a need to delete the files upfront. Delete may be an expensive operation on some cloud stores.

          I don't see much different with deletion/write a small or empty file with overwrite an existing file (updating timestamp, contents) in any cloud stores. I just prefer not to add additional if-else cases to existing ones that is already complicated to me. If we do observe the performance differences in real cluster, we can optimize it then. What do you think?

          Mapred javadoc fixes are missing. Also there are some typos in there. E.g.

          Nice catch! Will fix it in v4 patch.

          This part of the code change could use some tests.

          Ok. Add TestFileOutputCommitter for class in Mapred package.

          Tests for repeatable success marker file and FileExistsException for repeatable deletes would be good to have.

          Previously, success marker file is being added as overwrite (fs.create() default to be overwrite), so no much different here. Add additional tests on duplicated job commit in v4 patch.

          Show
          djp Junping Du added a comment - Thanks Bikas Saha for comments! Is the above check too early in the code. E.g. IIRC, at this point we have not checked whether the previous job commit was succeeded or failed - in which case we cannot recover and there is nothing to do. The cleanupInterruptedCommit() already check previous job commit succeed or failed. Am I missing anything here? Also, we have already changed the startCommit operation to be repeatable via the overwrite flag. After that is there a need to delete the files upfront. Delete may be an expensive operation on some cloud stores. I don't see much different with deletion/write a small or empty file with overwrite an existing file (updating timestamp, contents) in any cloud stores. I just prefer not to add additional if-else cases to existing ones that is already complicated to me. If we do observe the performance differences in real cluster, we can optimize it then. What do you think? Mapred javadoc fixes are missing. Also there are some typos in there. E.g. Nice catch! Will fix it in v4 patch. This part of the code change could use some tests. Ok. Add TestFileOutputCommitter for class in Mapred package. Tests for repeatable success marker file and FileExistsException for repeatable deletes would be good to have. Previously, success marker file is being added as overwrite (fs.create() default to be overwrite), so no much different here. Add additional tests on duplicated job commit in v4 patch.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 10s docker + precommit patch detected.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
          +1 mvninstall 3m 32s trunk passed
          +1 compile 2m 52s trunk passed with JDK v1.8.0_66
          +1 compile 2m 51s trunk passed with JDK v1.7.0_79
          +1 checkstyle 0m 21s trunk passed
          +1 mvneclipse 0m 29s trunk passed
          +1 findbugs 2m 9s trunk passed
          +1 javadoc 0m 48s trunk passed with JDK v1.8.0_66
          +1 javadoc 0m 48s trunk passed with JDK v1.7.0_79
          -1 mvninstall 0m 15s hadoop-mapreduce-client-app in the patch failed.
          +1 compile 2m 42s the patch passed with JDK v1.8.0_66
          +1 javac 2m 42s the patch passed
          +1 compile 2m 38s the patch passed with JDK v1.7.0_79
          +1 javac 2m 38s the patch passed
          -1 checkstyle 0m 25s Patch generated 1 new checkstyle issues in hadoop-mapreduce-project/hadoop-mapreduce-client (total was 129, now 126).
          +1 mvneclipse 0m 29s the patch passed
          -1 whitespace 0m 0s The patch has 4 line(s) that end in whitespace. Use git apply --whitespace=fix.
          -1 findbugs 1m 33s hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core introduced 2 new FindBugs issues.
          -1 javadoc 2m 43s hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.8.0_66 with JDK v1.8.0_66 generated 2 new issues (was 100, now 100).
          +1 javadoc 0m 49s the patch passed with JDK v1.8.0_66
          +1 javadoc 0m 48s the patch passed with JDK v1.7.0_79
          +1 unit 10m 14s hadoop-mapreduce-client-app in the patch passed with JDK v1.8.0_66.
          -1 unit 1m 59s hadoop-mapreduce-client-core in the patch failed with JDK v1.8.0_66.
          +1 unit 10m 51s hadoop-mapreduce-client-app in the patch passed with JDK v1.7.0_79.
          -1 unit 2m 20s hadoop-mapreduce-client-core in the patch failed with JDK v1.7.0_79.
          -1 asflicense 0m 25s Patch generated 1 ASF License warnings.
          52m 20s



          Reason Tests
          FindBugs module:hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core
            org.apache.hadoop.mapred.FileOutputCommitter.isCommitJobRepeatable(JobContext) doesn't override method in superclass because parameter type org.apache.hadoop.mapred.JobContext doesn't match superclass parameter type org.apache.hadoop.mapreduce.JobContext At FileOutputCommitter.java:superclass because parameter type org.apache.hadoop.mapred.JobContext doesn't match superclass parameter type org.apache.hadoop.mapreduce.JobContext At FileOutputCommitter.java:[line 194]
            org.apache.hadoop.mapred.OutputCommitter.isCommitJobRepeatable(JobContext) doesn't override method in superclass because parameter type org.apache.hadoop.mapred.JobContext doesn't match superclass parameter type org.apache.hadoop.mapreduce.JobContext At OutputCommitter.java:superclass because parameter type org.apache.hadoop.mapred.JobContext doesn't match superclass parameter type org.apache.hadoop.mapreduce.JobContext At OutputCommitter.java:[line 230]
          JDK v1.8.0_66 Failed junit tests hadoop.mapreduce.filecache.TestClientDistributedCacheManager
          JDK v1.7.0_79 Failed junit tests hadoop.mapreduce.filecache.TestClientDistributedCacheManager



          Subsystem Report/Notes
          Docker Client=1.7.0 Server=1.7.0 Image:test-patch-base-hadoop-date2015-11-10
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12771591/MAPREDUCE-5485-v4.patch
          JIRA Issue MAPREDUCE-5485
          Optional Tests asflicense javac javadoc mvninstall unit findbugs checkstyle compile
          uname Linux 7d03274dc7e2 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /home/jenkins/jenkins-slave/workspace/PreCommit-MAPREDUCE-Build/patchprocess/apache-yetus-ee5baeb/precommit/personality/hadoop.sh
          git revision trunk / 0eb9c60
          Default Java 1.7.0_79
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_79
          findbugs v3.0.0
          mvninstall https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6138/artifact/patchprocess/patch-mvninstall-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app.txt
          checkstyle https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6138/artifact/patchprocess/diff-checkstyle-hadoop-mapreduce-project_hadoop-mapreduce-client.txt
          whitespace https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6138/artifact/patchprocess/whitespace-eol.txt
          findbugs https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6138/artifact/patchprocess/new-findbugs-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core.html
          javadoc hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.8.0_66: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6138/artifact/patchprocess/diff-javadoc-javadoc-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.8.0_66.txt
          unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6138/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.8.0_66.txt
          unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6138/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.7.0_79.txt
          unit test logs https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6138/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6138/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.7.0_79.txt
          JDK v1.7.0_79 Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6138/testReport/
          asflicense https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6138/artifact/patchprocess/patch-asflicense-problems.txt
          modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core U: hadoop-mapreduce-project/hadoop-mapreduce-client
          Max memory used 227MB
          Powered by Apache Yetus http://yetus.apache.org
          Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6138/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 10s docker + precommit patch detected. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. +1 mvninstall 3m 32s trunk passed +1 compile 2m 52s trunk passed with JDK v1.8.0_66 +1 compile 2m 51s trunk passed with JDK v1.7.0_79 +1 checkstyle 0m 21s trunk passed +1 mvneclipse 0m 29s trunk passed +1 findbugs 2m 9s trunk passed +1 javadoc 0m 48s trunk passed with JDK v1.8.0_66 +1 javadoc 0m 48s trunk passed with JDK v1.7.0_79 -1 mvninstall 0m 15s hadoop-mapreduce-client-app in the patch failed. +1 compile 2m 42s the patch passed with JDK v1.8.0_66 +1 javac 2m 42s the patch passed +1 compile 2m 38s the patch passed with JDK v1.7.0_79 +1 javac 2m 38s the patch passed -1 checkstyle 0m 25s Patch generated 1 new checkstyle issues in hadoop-mapreduce-project/hadoop-mapreduce-client (total was 129, now 126). +1 mvneclipse 0m 29s the patch passed -1 whitespace 0m 0s The patch has 4 line(s) that end in whitespace. Use git apply --whitespace=fix. -1 findbugs 1m 33s hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core introduced 2 new FindBugs issues. -1 javadoc 2m 43s hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.8.0_66 with JDK v1.8.0_66 generated 2 new issues (was 100, now 100). +1 javadoc 0m 49s the patch passed with JDK v1.8.0_66 +1 javadoc 0m 48s the patch passed with JDK v1.7.0_79 +1 unit 10m 14s hadoop-mapreduce-client-app in the patch passed with JDK v1.8.0_66. -1 unit 1m 59s hadoop-mapreduce-client-core in the patch failed with JDK v1.8.0_66. +1 unit 10m 51s hadoop-mapreduce-client-app in the patch passed with JDK v1.7.0_79. -1 unit 2m 20s hadoop-mapreduce-client-core in the patch failed with JDK v1.7.0_79. -1 asflicense 0m 25s Patch generated 1 ASF License warnings. 52m 20s Reason Tests FindBugs module:hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core   org.apache.hadoop.mapred.FileOutputCommitter.isCommitJobRepeatable(JobContext) doesn't override method in superclass because parameter type org.apache.hadoop.mapred.JobContext doesn't match superclass parameter type org.apache.hadoop.mapreduce.JobContext At FileOutputCommitter.java:superclass because parameter type org.apache.hadoop.mapred.JobContext doesn't match superclass parameter type org.apache.hadoop.mapreduce.JobContext At FileOutputCommitter.java: [line 194]   org.apache.hadoop.mapred.OutputCommitter.isCommitJobRepeatable(JobContext) doesn't override method in superclass because parameter type org.apache.hadoop.mapred.JobContext doesn't match superclass parameter type org.apache.hadoop.mapreduce.JobContext At OutputCommitter.java:superclass because parameter type org.apache.hadoop.mapred.JobContext doesn't match superclass parameter type org.apache.hadoop.mapreduce.JobContext At OutputCommitter.java: [line 230] JDK v1.8.0_66 Failed junit tests hadoop.mapreduce.filecache.TestClientDistributedCacheManager JDK v1.7.0_79 Failed junit tests hadoop.mapreduce.filecache.TestClientDistributedCacheManager Subsystem Report/Notes Docker Client=1.7.0 Server=1.7.0 Image:test-patch-base-hadoop-date2015-11-10 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12771591/MAPREDUCE-5485-v4.patch JIRA Issue MAPREDUCE-5485 Optional Tests asflicense javac javadoc mvninstall unit findbugs checkstyle compile uname Linux 7d03274dc7e2 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /home/jenkins/jenkins-slave/workspace/PreCommit-MAPREDUCE-Build/patchprocess/apache-yetus-ee5baeb/precommit/personality/hadoop.sh git revision trunk / 0eb9c60 Default Java 1.7.0_79 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_79 findbugs v3.0.0 mvninstall https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6138/artifact/patchprocess/patch-mvninstall-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app.txt checkstyle https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6138/artifact/patchprocess/diff-checkstyle-hadoop-mapreduce-project_hadoop-mapreduce-client.txt whitespace https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6138/artifact/patchprocess/whitespace-eol.txt findbugs https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6138/artifact/patchprocess/new-findbugs-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core.html javadoc hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.8.0_66: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6138/artifact/patchprocess/diff-javadoc-javadoc-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.8.0_66.txt unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6138/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.8.0_66.txt unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6138/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.7.0_79.txt unit test logs https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6138/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6138/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.7.0_79.txt JDK v1.7.0_79 Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6138/testReport/ asflicense https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6138/artifact/patchprocess/patch-asflicense-problems.txt modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core U: hadoop-mapreduce-project/hadoop-mapreduce-client Max memory used 227MB Powered by Apache Yetus http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6138/console This message was automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 6s docker + precommit patch detected.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
          +1 mvninstall 3m 8s trunk passed
          +1 compile 2m 18s trunk passed with JDK v1.8.0_60
          +1 compile 2m 27s trunk passed with JDK v1.7.0_79
          +1 checkstyle 0m 19s trunk passed
          +1 mvneclipse 0m 26s trunk passed
          +1 findbugs 1m 49s trunk passed
          +1 javadoc 0m 37s trunk passed with JDK v1.8.0_60
          +1 javadoc 0m 42s trunk passed with JDK v1.7.0_79
          -1 mvninstall 0m 15s hadoop-mapreduce-client-app in the patch failed.
          +1 compile 2m 14s the patch passed with JDK v1.8.0_60
          +1 javac 2m 14s the patch passed
          +1 compile 2m 17s the patch passed with JDK v1.7.0_79
          +1 javac 2m 17s the patch passed
          -1 checkstyle 0m 20s Patch generated 1 new checkstyle issues in hadoop-mapreduce-project/hadoop-mapreduce-client (total was 128, now 125).
          +1 mvneclipse 0m 28s the patch passed
          -1 whitespace 0m 0s The patch has 4 line(s) that end in whitespace. Use git apply --whitespace=fix.
          -1 findbugs 1m 19s hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core introduced 2 new FindBugs issues.
          -1 javadoc 2m 15s hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.8.0_60 with JDK v1.8.0_60 generated 2 new issues (was 100, now 100).
          +1 javadoc 0m 37s the patch passed with JDK v1.8.0_60
          +1 javadoc 0m 43s the patch passed with JDK v1.7.0_79
          +1 unit 9m 13s hadoop-mapreduce-client-app in the patch passed with JDK v1.8.0_60.
          -1 unit 1m 36s hadoop-mapreduce-client-core in the patch failed with JDK v1.8.0_60.
          +1 unit 9m 43s hadoop-mapreduce-client-app in the patch passed with JDK v1.7.0_79.
          -1 unit 1m 57s hadoop-mapreduce-client-core in the patch failed with JDK v1.7.0_79.
          -1 asflicense 0m 20s Patch generated 1 ASF License warnings.
          45m 26s



          Reason Tests
          FindBugs module:hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core
            org.apache.hadoop.mapred.FileOutputCommitter.isCommitJobRepeatable(JobContext) doesn't override method in superclass because parameter type org.apache.hadoop.mapred.JobContext doesn't match superclass parameter type org.apache.hadoop.mapreduce.JobContext At FileOutputCommitter.java:superclass because parameter type org.apache.hadoop.mapred.JobContext doesn't match superclass parameter type org.apache.hadoop.mapreduce.JobContext At FileOutputCommitter.java:[line 194]
            org.apache.hadoop.mapred.OutputCommitter.isCommitJobRepeatable(JobContext) doesn't override method in superclass because parameter type org.apache.hadoop.mapred.JobContext doesn't match superclass parameter type org.apache.hadoop.mapreduce.JobContext At OutputCommitter.java:superclass because parameter type org.apache.hadoop.mapred.JobContext doesn't match superclass parameter type org.apache.hadoop.mapreduce.JobContext At OutputCommitter.java:[line 230]
          JDK v1.8.0_60 Failed junit tests hadoop.mapreduce.filecache.TestClientDistributedCacheManager
          JDK v1.7.0_79 Failed junit tests hadoop.mapreduce.filecache.TestClientDistributedCacheManager



          Subsystem Report/Notes
          Docker Client=1.7.1 Server=1.7.1 Image:test-patch-base-hadoop-date2015-11-10
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12771591/MAPREDUCE-5485-v4.patch
          JIRA Issue MAPREDUCE-5485
          Optional Tests asflicense javac javadoc mvninstall unit findbugs checkstyle compile
          uname Linux 9a741af88195 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /home/jenkins/jenkins-slave/workspace/PreCommit-MAPREDUCE-Build/patchprocess/apache-yetus-ee5baeb/precommit/personality/hadoop.sh
          git revision trunk / 73b94d7
          Default Java 1.7.0_79
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_60 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_79
          findbugs v3.0.0
          mvninstall https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6139/artifact/patchprocess/patch-mvninstall-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app.txt
          checkstyle https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6139/artifact/patchprocess/diff-checkstyle-hadoop-mapreduce-project_hadoop-mapreduce-client.txt
          whitespace https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6139/artifact/patchprocess/whitespace-eol.txt
          findbugs https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6139/artifact/patchprocess/new-findbugs-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core.html
          javadoc hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.8.0_60: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6139/artifact/patchprocess/diff-javadoc-javadoc-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.8.0_60.txt
          unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6139/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.8.0_60.txt
          unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6139/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.7.0_79.txt
          unit test logs https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6139/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.8.0_60.txt https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6139/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.7.0_79.txt
          JDK v1.7.0_79 Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6139/testReport/
          asflicense https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6139/artifact/patchprocess/patch-asflicense-problems.txt
          modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core U: hadoop-mapreduce-project/hadoop-mapreduce-client
          Max memory used 228MB
          Powered by Apache Yetus http://yetus.apache.org
          Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6139/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 6s docker + precommit patch detected. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. +1 mvninstall 3m 8s trunk passed +1 compile 2m 18s trunk passed with JDK v1.8.0_60 +1 compile 2m 27s trunk passed with JDK v1.7.0_79 +1 checkstyle 0m 19s trunk passed +1 mvneclipse 0m 26s trunk passed +1 findbugs 1m 49s trunk passed +1 javadoc 0m 37s trunk passed with JDK v1.8.0_60 +1 javadoc 0m 42s trunk passed with JDK v1.7.0_79 -1 mvninstall 0m 15s hadoop-mapreduce-client-app in the patch failed. +1 compile 2m 14s the patch passed with JDK v1.8.0_60 +1 javac 2m 14s the patch passed +1 compile 2m 17s the patch passed with JDK v1.7.0_79 +1 javac 2m 17s the patch passed -1 checkstyle 0m 20s Patch generated 1 new checkstyle issues in hadoop-mapreduce-project/hadoop-mapreduce-client (total was 128, now 125). +1 mvneclipse 0m 28s the patch passed -1 whitespace 0m 0s The patch has 4 line(s) that end in whitespace. Use git apply --whitespace=fix. -1 findbugs 1m 19s hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core introduced 2 new FindBugs issues. -1 javadoc 2m 15s hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.8.0_60 with JDK v1.8.0_60 generated 2 new issues (was 100, now 100). +1 javadoc 0m 37s the patch passed with JDK v1.8.0_60 +1 javadoc 0m 43s the patch passed with JDK v1.7.0_79 +1 unit 9m 13s hadoop-mapreduce-client-app in the patch passed with JDK v1.8.0_60. -1 unit 1m 36s hadoop-mapreduce-client-core in the patch failed with JDK v1.8.0_60. +1 unit 9m 43s hadoop-mapreduce-client-app in the patch passed with JDK v1.7.0_79. -1 unit 1m 57s hadoop-mapreduce-client-core in the patch failed with JDK v1.7.0_79. -1 asflicense 0m 20s Patch generated 1 ASF License warnings. 45m 26s Reason Tests FindBugs module:hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core   org.apache.hadoop.mapred.FileOutputCommitter.isCommitJobRepeatable(JobContext) doesn't override method in superclass because parameter type org.apache.hadoop.mapred.JobContext doesn't match superclass parameter type org.apache.hadoop.mapreduce.JobContext At FileOutputCommitter.java:superclass because parameter type org.apache.hadoop.mapred.JobContext doesn't match superclass parameter type org.apache.hadoop.mapreduce.JobContext At FileOutputCommitter.java: [line 194]   org.apache.hadoop.mapred.OutputCommitter.isCommitJobRepeatable(JobContext) doesn't override method in superclass because parameter type org.apache.hadoop.mapred.JobContext doesn't match superclass parameter type org.apache.hadoop.mapreduce.JobContext At OutputCommitter.java:superclass because parameter type org.apache.hadoop.mapred.JobContext doesn't match superclass parameter type org.apache.hadoop.mapreduce.JobContext At OutputCommitter.java: [line 230] JDK v1.8.0_60 Failed junit tests hadoop.mapreduce.filecache.TestClientDistributedCacheManager JDK v1.7.0_79 Failed junit tests hadoop.mapreduce.filecache.TestClientDistributedCacheManager Subsystem Report/Notes Docker Client=1.7.1 Server=1.7.1 Image:test-patch-base-hadoop-date2015-11-10 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12771591/MAPREDUCE-5485-v4.patch JIRA Issue MAPREDUCE-5485 Optional Tests asflicense javac javadoc mvninstall unit findbugs checkstyle compile uname Linux 9a741af88195 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /home/jenkins/jenkins-slave/workspace/PreCommit-MAPREDUCE-Build/patchprocess/apache-yetus-ee5baeb/precommit/personality/hadoop.sh git revision trunk / 73b94d7 Default Java 1.7.0_79 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_60 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_79 findbugs v3.0.0 mvninstall https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6139/artifact/patchprocess/patch-mvninstall-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app.txt checkstyle https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6139/artifact/patchprocess/diff-checkstyle-hadoop-mapreduce-project_hadoop-mapreduce-client.txt whitespace https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6139/artifact/patchprocess/whitespace-eol.txt findbugs https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6139/artifact/patchprocess/new-findbugs-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core.html javadoc hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.8.0_60: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6139/artifact/patchprocess/diff-javadoc-javadoc-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.8.0_60.txt unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6139/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.8.0_60.txt unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6139/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.7.0_79.txt unit test logs https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6139/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.8.0_60.txt https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6139/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.7.0_79.txt JDK v1.7.0_79 Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6139/testReport/ asflicense https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6139/artifact/patchprocess/patch-asflicense-problems.txt modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core U: hadoop-mapreduce-project/hadoop-mapreduce-client Max memory used 228MB Powered by Apache Yetus http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6139/console This message was automatically generated.
          Hide
          djp Junping Du added a comment -

          Fix minor issues with white spaces, findbugs, etc. in v4.1 patch.

          Show
          djp Junping Du added a comment - Fix minor issues with white spaces, findbugs, etc. in v4.1 patch.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 8s docker + precommit patch detected.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
          +1 mvninstall 3m 21s trunk passed
          +1 compile 2m 29s trunk passed with JDK v1.8.0_60
          +1 compile 2m 28s trunk passed with JDK v1.7.0_79
          +1 checkstyle 0m 23s trunk passed
          +1 mvneclipse 0m 30s trunk passed
          +1 findbugs 2m 2s trunk passed
          +1 javadoc 0m 43s trunk passed with JDK v1.8.0_60
          +1 javadoc 0m 43s trunk passed with JDK v1.7.0_79
          -1 mvninstall 0m 15s hadoop-mapreduce-client-app in the patch failed.
          +1 compile 2m 31s the patch passed with JDK v1.8.0_60
          +1 javac 2m 31s the patch passed
          +1 compile 2m 33s the patch passed with JDK v1.7.0_79
          +1 javac 2m 33s the patch passed
          -1 checkstyle 0m 23s Patch generated 1 new checkstyle issues in hadoop-mapreduce-project/hadoop-mapreduce-client (total was 128, now 125).
          +1 mvneclipse 0m 28s the patch passed
          -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix.
          -1 findbugs 1m 27s hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core introduced 2 new FindBugs issues.
          -1 javadoc 2m 28s hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.8.0_60 with JDK v1.8.0_60 generated 2 new issues (was 100, now 100).
          +1 javadoc 0m 41s the patch passed with JDK v1.8.0_60
          +1 javadoc 0m 45s the patch passed with JDK v1.7.0_79
          -1 unit 9m 50s hadoop-mapreduce-client-app in the patch failed with JDK v1.8.0_60.
          -1 unit 1m 51s hadoop-mapreduce-client-core in the patch failed with JDK v1.8.0_60.
          -1 unit 10m 8s hadoop-mapreduce-client-app in the patch failed with JDK v1.7.0_79.
          -1 unit 2m 6s hadoop-mapreduce-client-core in the patch failed with JDK v1.7.0_79.
          -1 asflicense 0m 22s Patch generated 1 ASF License warnings.
          48m 53s



          Reason Tests
          FindBugs module:hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core
            org.apache.hadoop.mapred.FileOutputCommitter.isCommitJobRepeatable(JobContext) doesn't override method in superclass because parameter type org.apache.hadoop.mapred.JobContext doesn't match superclass parameter type org.apache.hadoop.mapreduce.JobContext At FileOutputCommitter.java:superclass because parameter type org.apache.hadoop.mapred.JobContext doesn't match superclass parameter type org.apache.hadoop.mapreduce.JobContext At FileOutputCommitter.java:[line 194]
            org.apache.hadoop.mapred.OutputCommitter.isCommitJobRepeatable(JobContext) doesn't override method in superclass because parameter type org.apache.hadoop.mapred.JobContext doesn't match superclass parameter type org.apache.hadoop.mapreduce.JobContext At OutputCommitter.java:superclass because parameter type org.apache.hadoop.mapred.JobContext doesn't match superclass parameter type org.apache.hadoop.mapreduce.JobContext At OutputCommitter.java:[line 230]
          JDK v1.8.0_60 Failed junit tests hadoop.mapreduce.v2.app.TestMRAppMaster
            hadoop.mapreduce.v2.app.TestKill
            hadoop.mapreduce.filecache.TestClientDistributedCacheManager
          JDK v1.7.0_79 Failed junit tests hadoop.mapreduce.v2.app.TestMRAppMaster
            hadoop.mapreduce.v2.app.job.impl.TestTaskAttempt
            hadoop.mapreduce.filecache.TestClientDistributedCacheManager



          Subsystem Report/Notes
          Docker Client=1.7.1 Server=1.7.1 Image:test-patch-base-hadoop-date2015-11-11
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12771718/MAPREDUCE-5485-v4.1.patch
          JIRA Issue MAPREDUCE-5485
          Optional Tests asflicense javac javadoc mvninstall unit findbugs checkstyle compile
          uname Linux 539216e1a352 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /home/jenkins/jenkins-slave/workspace/PreCommit-MAPREDUCE-Build/patchprocess/apache-yetus-ee5baeb/precommit/personality/hadoop.sh
          git revision trunk / 23d0db5
          Default Java 1.7.0_79
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_60 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_79
          findbugs v3.0.0
          mvninstall https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6141/artifact/patchprocess/patch-mvninstall-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app.txt
          checkstyle https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6141/artifact/patchprocess/diff-checkstyle-hadoop-mapreduce-project_hadoop-mapreduce-client.txt
          whitespace https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6141/artifact/patchprocess/whitespace-eol.txt
          findbugs https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6141/artifact/patchprocess/new-findbugs-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core.html
          javadoc hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.8.0_60: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6141/artifact/patchprocess/diff-javadoc-javadoc-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.8.0_60.txt
          unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6141/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.8.0_60.txt
          unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6141/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.8.0_60.txt
          unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6141/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.7.0_79.txt
          unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6141/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.7.0_79.txt
          unit test logs https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6141/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.8.0_60.txt https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6141/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.8.0_60.txt https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6141/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.7.0_79.txt https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6141/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.7.0_79.txt
          JDK v1.7.0_79 Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6141/testReport/
          asflicense https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6141/artifact/patchprocess/patch-asflicense-problems.txt
          modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core U: hadoop-mapreduce-project/hadoop-mapreduce-client
          Max memory used 226MB
          Powered by Apache Yetus http://yetus.apache.org
          Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6141/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 8s docker + precommit patch detected. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. +1 mvninstall 3m 21s trunk passed +1 compile 2m 29s trunk passed with JDK v1.8.0_60 +1 compile 2m 28s trunk passed with JDK v1.7.0_79 +1 checkstyle 0m 23s trunk passed +1 mvneclipse 0m 30s trunk passed +1 findbugs 2m 2s trunk passed +1 javadoc 0m 43s trunk passed with JDK v1.8.0_60 +1 javadoc 0m 43s trunk passed with JDK v1.7.0_79 -1 mvninstall 0m 15s hadoop-mapreduce-client-app in the patch failed. +1 compile 2m 31s the patch passed with JDK v1.8.0_60 +1 javac 2m 31s the patch passed +1 compile 2m 33s the patch passed with JDK v1.7.0_79 +1 javac 2m 33s the patch passed -1 checkstyle 0m 23s Patch generated 1 new checkstyle issues in hadoop-mapreduce-project/hadoop-mapreduce-client (total was 128, now 125). +1 mvneclipse 0m 28s the patch passed -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix. -1 findbugs 1m 27s hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core introduced 2 new FindBugs issues. -1 javadoc 2m 28s hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.8.0_60 with JDK v1.8.0_60 generated 2 new issues (was 100, now 100). +1 javadoc 0m 41s the patch passed with JDK v1.8.0_60 +1 javadoc 0m 45s the patch passed with JDK v1.7.0_79 -1 unit 9m 50s hadoop-mapreduce-client-app in the patch failed with JDK v1.8.0_60. -1 unit 1m 51s hadoop-mapreduce-client-core in the patch failed with JDK v1.8.0_60. -1 unit 10m 8s hadoop-mapreduce-client-app in the patch failed with JDK v1.7.0_79. -1 unit 2m 6s hadoop-mapreduce-client-core in the patch failed with JDK v1.7.0_79. -1 asflicense 0m 22s Patch generated 1 ASF License warnings. 48m 53s Reason Tests FindBugs module:hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core   org.apache.hadoop.mapred.FileOutputCommitter.isCommitJobRepeatable(JobContext) doesn't override method in superclass because parameter type org.apache.hadoop.mapred.JobContext doesn't match superclass parameter type org.apache.hadoop.mapreduce.JobContext At FileOutputCommitter.java:superclass because parameter type org.apache.hadoop.mapred.JobContext doesn't match superclass parameter type org.apache.hadoop.mapreduce.JobContext At FileOutputCommitter.java: [line 194]   org.apache.hadoop.mapred.OutputCommitter.isCommitJobRepeatable(JobContext) doesn't override method in superclass because parameter type org.apache.hadoop.mapred.JobContext doesn't match superclass parameter type org.apache.hadoop.mapreduce.JobContext At OutputCommitter.java:superclass because parameter type org.apache.hadoop.mapred.JobContext doesn't match superclass parameter type org.apache.hadoop.mapreduce.JobContext At OutputCommitter.java: [line 230] JDK v1.8.0_60 Failed junit tests hadoop.mapreduce.v2.app.TestMRAppMaster   hadoop.mapreduce.v2.app.TestKill   hadoop.mapreduce.filecache.TestClientDistributedCacheManager JDK v1.7.0_79 Failed junit tests hadoop.mapreduce.v2.app.TestMRAppMaster   hadoop.mapreduce.v2.app.job.impl.TestTaskAttempt   hadoop.mapreduce.filecache.TestClientDistributedCacheManager Subsystem Report/Notes Docker Client=1.7.1 Server=1.7.1 Image:test-patch-base-hadoop-date2015-11-11 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12771718/MAPREDUCE-5485-v4.1.patch JIRA Issue MAPREDUCE-5485 Optional Tests asflicense javac javadoc mvninstall unit findbugs checkstyle compile uname Linux 539216e1a352 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /home/jenkins/jenkins-slave/workspace/PreCommit-MAPREDUCE-Build/patchprocess/apache-yetus-ee5baeb/precommit/personality/hadoop.sh git revision trunk / 23d0db5 Default Java 1.7.0_79 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_60 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_79 findbugs v3.0.0 mvninstall https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6141/artifact/patchprocess/patch-mvninstall-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app.txt checkstyle https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6141/artifact/patchprocess/diff-checkstyle-hadoop-mapreduce-project_hadoop-mapreduce-client.txt whitespace https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6141/artifact/patchprocess/whitespace-eol.txt findbugs https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6141/artifact/patchprocess/new-findbugs-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core.html javadoc hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.8.0_60: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6141/artifact/patchprocess/diff-javadoc-javadoc-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.8.0_60.txt unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6141/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.8.0_60.txt unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6141/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.8.0_60.txt unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6141/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.7.0_79.txt unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6141/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.7.0_79.txt unit test logs https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6141/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.8.0_60.txt https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6141/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.8.0_60.txt https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6141/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.7.0_79.txt https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6141/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.7.0_79.txt JDK v1.7.0_79 Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6141/testReport/ asflicense https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6141/artifact/patchprocess/patch-asflicense-problems.txt modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core U: hadoop-mapreduce-project/hadoop-mapreduce-client Max memory used 226MB Powered by Apache Yetus http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6141/console This message was automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 8s docker + precommit patch detected.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
          +1 mvninstall 3m 21s trunk passed
          +1 compile 2m 33s trunk passed with JDK v1.8.0_60
          +1 compile 2m 32s trunk passed with JDK v1.7.0_79
          +1 checkstyle 0m 20s trunk passed
          +1 mvneclipse 0m 28s trunk passed
          +1 findbugs 1m 55s trunk passed
          +1 javadoc 0m 44s trunk passed with JDK v1.8.0_60
          +1 javadoc 0m 47s trunk passed with JDK v1.7.0_79
          -1 mvninstall 0m 14s hadoop-mapreduce-client-app in the patch failed.
          +1 compile 2m 31s the patch passed with JDK v1.8.0_60
          +1 javac 2m 31s the patch passed
          +1 compile 2m 31s the patch passed with JDK v1.7.0_79
          +1 javac 2m 31s the patch passed
          -1 checkstyle 0m 18s Patch generated 1 new checkstyle issues in hadoop-mapreduce-project/hadoop-mapreduce-client (total was 128, now 125).
          +1 mvneclipse 0m 28s the patch passed
          -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix.
          -1 findbugs 1m 28s hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core introduced 2 new FindBugs issues.
          -1 javadoc 2m 31s hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.8.0_60 with JDK v1.8.0_60 generated 2 new issues (was 100, now 100).
          +1 javadoc 0m 42s the patch passed with JDK v1.8.0_60
          +1 javadoc 0m 44s the patch passed with JDK v1.7.0_79
          -1 unit 9m 47s hadoop-mapreduce-client-app in the patch failed with JDK v1.8.0_60.
          -1 unit 1m 46s hadoop-mapreduce-client-core in the patch failed with JDK v1.8.0_60.
          -1 unit 10m 5s hadoop-mapreduce-client-app in the patch failed with JDK v1.7.0_79.
          -1 unit 2m 4s hadoop-mapreduce-client-core in the patch failed with JDK v1.7.0_79.
          -1 asflicense 0m 22s Patch generated 1 ASF License warnings.
          48m 33s



          Reason Tests
          FindBugs module:hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core
            org.apache.hadoop.mapred.FileOutputCommitter.isCommitJobRepeatable(JobContext) doesn't override method in superclass because parameter type org.apache.hadoop.mapred.JobContext doesn't match superclass parameter type org.apache.hadoop.mapreduce.JobContext At FileOutputCommitter.java:superclass because parameter type org.apache.hadoop.mapred.JobContext doesn't match superclass parameter type org.apache.hadoop.mapreduce.JobContext At FileOutputCommitter.java:[line 194]
            org.apache.hadoop.mapred.OutputCommitter.isCommitJobRepeatable(JobContext) doesn't override method in superclass because parameter type org.apache.hadoop.mapred.JobContext doesn't match superclass parameter type org.apache.hadoop.mapreduce.JobContext At OutputCommitter.java:superclass because parameter type org.apache.hadoop.mapred.JobContext doesn't match superclass parameter type org.apache.hadoop.mapreduce.JobContext At OutputCommitter.java:[line 230]
          JDK v1.8.0_60 Failed junit tests hadoop.mapreduce.v2.app.TestMRAppMaster
            hadoop.mapreduce.v2.app.job.impl.TestJobImpl
            hadoop.mapreduce.filecache.TestClientDistributedCacheManager
          JDK v1.7.0_79 Failed junit tests hadoop.mapreduce.v2.app.TestMRAppMaster
            hadoop.mapreduce.filecache.TestClientDistributedCacheManager



          Subsystem Report/Notes
          Docker Client=1.7.1 Server=1.7.1 Image:test-patch-base-hadoop-date2015-11-11
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12771718/MAPREDUCE-5485-v4.1.patch
          JIRA Issue MAPREDUCE-5485
          Optional Tests asflicense javac javadoc mvninstall unit findbugs checkstyle compile
          uname Linux fd28f8927655 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /home/jenkins/jenkins-slave/workspace/PreCommit-MAPREDUCE-Build@2/patchprocess/apache-yetus-ee5baeb/precommit/personality/hadoop.sh
          git revision trunk / 23d0db5
          Default Java 1.7.0_79
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_60 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_79
          findbugs v3.0.0
          mvninstall https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6142/artifact/patchprocess/patch-mvninstall-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app.txt
          checkstyle https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6142/artifact/patchprocess/diff-checkstyle-hadoop-mapreduce-project_hadoop-mapreduce-client.txt
          whitespace https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6142/artifact/patchprocess/whitespace-eol.txt
          findbugs https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6142/artifact/patchprocess/new-findbugs-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core.html
          javadoc hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.8.0_60: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6142/artifact/patchprocess/diff-javadoc-javadoc-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.8.0_60.txt
          unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6142/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.8.0_60.txt
          unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6142/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.8.0_60.txt
          unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6142/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.7.0_79.txt
          unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6142/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.7.0_79.txt
          unit test logs https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6142/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.8.0_60.txt https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6142/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.8.0_60.txt https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6142/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.7.0_79.txt https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6142/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.7.0_79.txt
          JDK v1.7.0_79 Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6142/testReport/
          asflicense https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6142/artifact/patchprocess/patch-asflicense-problems.txt
          modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core U: hadoop-mapreduce-project/hadoop-mapreduce-client
          Max memory used 227MB
          Powered by Apache Yetus http://yetus.apache.org
          Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6142/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 8s docker + precommit patch detected. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. +1 mvninstall 3m 21s trunk passed +1 compile 2m 33s trunk passed with JDK v1.8.0_60 +1 compile 2m 32s trunk passed with JDK v1.7.0_79 +1 checkstyle 0m 20s trunk passed +1 mvneclipse 0m 28s trunk passed +1 findbugs 1m 55s trunk passed +1 javadoc 0m 44s trunk passed with JDK v1.8.0_60 +1 javadoc 0m 47s trunk passed with JDK v1.7.0_79 -1 mvninstall 0m 14s hadoop-mapreduce-client-app in the patch failed. +1 compile 2m 31s the patch passed with JDK v1.8.0_60 +1 javac 2m 31s the patch passed +1 compile 2m 31s the patch passed with JDK v1.7.0_79 +1 javac 2m 31s the patch passed -1 checkstyle 0m 18s Patch generated 1 new checkstyle issues in hadoop-mapreduce-project/hadoop-mapreduce-client (total was 128, now 125). +1 mvneclipse 0m 28s the patch passed -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix. -1 findbugs 1m 28s hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core introduced 2 new FindBugs issues. -1 javadoc 2m 31s hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.8.0_60 with JDK v1.8.0_60 generated 2 new issues (was 100, now 100). +1 javadoc 0m 42s the patch passed with JDK v1.8.0_60 +1 javadoc 0m 44s the patch passed with JDK v1.7.0_79 -1 unit 9m 47s hadoop-mapreduce-client-app in the patch failed with JDK v1.8.0_60. -1 unit 1m 46s hadoop-mapreduce-client-core in the patch failed with JDK v1.8.0_60. -1 unit 10m 5s hadoop-mapreduce-client-app in the patch failed with JDK v1.7.0_79. -1 unit 2m 4s hadoop-mapreduce-client-core in the patch failed with JDK v1.7.0_79. -1 asflicense 0m 22s Patch generated 1 ASF License warnings. 48m 33s Reason Tests FindBugs module:hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core   org.apache.hadoop.mapred.FileOutputCommitter.isCommitJobRepeatable(JobContext) doesn't override method in superclass because parameter type org.apache.hadoop.mapred.JobContext doesn't match superclass parameter type org.apache.hadoop.mapreduce.JobContext At FileOutputCommitter.java:superclass because parameter type org.apache.hadoop.mapred.JobContext doesn't match superclass parameter type org.apache.hadoop.mapreduce.JobContext At FileOutputCommitter.java: [line 194]   org.apache.hadoop.mapred.OutputCommitter.isCommitJobRepeatable(JobContext) doesn't override method in superclass because parameter type org.apache.hadoop.mapred.JobContext doesn't match superclass parameter type org.apache.hadoop.mapreduce.JobContext At OutputCommitter.java:superclass because parameter type org.apache.hadoop.mapred.JobContext doesn't match superclass parameter type org.apache.hadoop.mapreduce.JobContext At OutputCommitter.java: [line 230] JDK v1.8.0_60 Failed junit tests hadoop.mapreduce.v2.app.TestMRAppMaster   hadoop.mapreduce.v2.app.job.impl.TestJobImpl   hadoop.mapreduce.filecache.TestClientDistributedCacheManager JDK v1.7.0_79 Failed junit tests hadoop.mapreduce.v2.app.TestMRAppMaster   hadoop.mapreduce.filecache.TestClientDistributedCacheManager Subsystem Report/Notes Docker Client=1.7.1 Server=1.7.1 Image:test-patch-base-hadoop-date2015-11-11 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12771718/MAPREDUCE-5485-v4.1.patch JIRA Issue MAPREDUCE-5485 Optional Tests asflicense javac javadoc mvninstall unit findbugs checkstyle compile uname Linux fd28f8927655 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /home/jenkins/jenkins-slave/workspace/PreCommit-MAPREDUCE-Build@2/patchprocess/apache-yetus-ee5baeb/precommit/personality/hadoop.sh git revision trunk / 23d0db5 Default Java 1.7.0_79 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_60 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_79 findbugs v3.0.0 mvninstall https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6142/artifact/patchprocess/patch-mvninstall-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app.txt checkstyle https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6142/artifact/patchprocess/diff-checkstyle-hadoop-mapreduce-project_hadoop-mapreduce-client.txt whitespace https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6142/artifact/patchprocess/whitespace-eol.txt findbugs https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6142/artifact/patchprocess/new-findbugs-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core.html javadoc hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.8.0_60: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6142/artifact/patchprocess/diff-javadoc-javadoc-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.8.0_60.txt unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6142/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.8.0_60.txt unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6142/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.8.0_60.txt unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6142/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.7.0_79.txt unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6142/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.7.0_79.txt unit test logs https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6142/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.8.0_60.txt https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6142/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.8.0_60.txt https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6142/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.7.0_79.txt https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6142/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.7.0_79.txt JDK v1.7.0_79 Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6142/testReport/ asflicense https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6142/artifact/patchprocess/patch-asflicense-problems.txt modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core U: hadoop-mapreduce-project/hadoop-mapreduce-client Max memory used 227MB Powered by Apache Yetus http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6142/console This message was automatically generated.
          Hide
          bikassaha Bikas Saha added a comment -

          The cleanupInterruptedCommit() already check previous job commit succeed or failed. Am I missing anything here?

          This introduces duplication of code for checking commit status and can cause a bug if the logic changes in either place. And also makes extra RPC calls to HDFS for checking file status - which is avoidable. Moving the code to the place where earlier we were failing due to in-progress commit, will allow this method to do exactly as it name suggests - cleanup in progress commit markers. Does that clarify?

          Should we say previous AM failures to be precise?

          +   * If repeatable job commit is supported, job restart can tolerate previous
          +   * failures during job commit.

          To be clear, we should look at adding 2 more tests.
          1) Test MR Appmaster new functionality that allows commit to proceed in a retried AM if commit is repeatable.
          2) Test in FileOutputCommitter that for repeatable commit - a filenotfoundexception is not counted as an error (new behavior).
          Maybe the patch missed adding some new changed file? Sorry if I missed something and the tests already exist.

          Show
          bikassaha Bikas Saha added a comment - The cleanupInterruptedCommit() already check previous job commit succeed or failed. Am I missing anything here? This introduces duplication of code for checking commit status and can cause a bug if the logic changes in either place. And also makes extra RPC calls to HDFS for checking file status - which is avoidable. Moving the code to the place where earlier we were failing due to in-progress commit, will allow this method to do exactly as it name suggests - cleanup in progress commit markers. Does that clarify? Should we say previous AM failures to be precise? + * If repeatable job commit is supported, job restart can tolerate previous + * failures during job commit. To be clear, we should look at adding 2 more tests. 1) Test MR Appmaster new functionality that allows commit to proceed in a retried AM if commit is repeatable. 2) Test in FileOutputCommitter that for repeatable commit - a filenotfoundexception is not counted as an error (new behavior). Maybe the patch missed adding some new changed file? Sorry if I missed something and the tests already exist.
          Hide
          djp Junping Du added a comment -

          This introduces duplication of code for checking commit status and can cause a bug if the logic changes in either place. And also makes extra RPC calls to HDFS for checking file status - which is avoidable. Moving the code to the place where earlier we were failing due to in-progress commit, will allow this method to do exactly as it name suggests - cleanup in progress commit markers. Does that clarify?

          Thanks for clarifying. That sounds good. Will update in v5 patch.

          1) Test MR Appmaster new functionality that allows commit to proceed in a retried AM if commit is repeatable.

          Theoretically, I agree it is nice to have something fully functional. However, I don't think it is easy to have for this case. Do we have other tests on job commit (not retry) with launching AppMaster fully functional? If not, I would prefer to add it later in another JIRA if we have more ideas on how to do it.

          2) Test in FileOutputCommitter that for repeatable commit - a filenotfoundexception is not counted as an error (new behavior).

          Can you check FileOutputCommitter#testCommitterRepeatableV1() and FileOutputCommitter#testCommitterRepeatableV2()?

          Show
          djp Junping Du added a comment - This introduces duplication of code for checking commit status and can cause a bug if the logic changes in either place. And also makes extra RPC calls to HDFS for checking file status - which is avoidable. Moving the code to the place where earlier we were failing due to in-progress commit, will allow this method to do exactly as it name suggests - cleanup in progress commit markers. Does that clarify? Thanks for clarifying. That sounds good. Will update in v5 patch. 1) Test MR Appmaster new functionality that allows commit to proceed in a retried AM if commit is repeatable. Theoretically, I agree it is nice to have something fully functional. However, I don't think it is easy to have for this case. Do we have other tests on job commit (not retry) with launching AppMaster fully functional? If not, I would prefer to add it later in another JIRA if we have more ideas on how to do it. 2) Test in FileOutputCommitter that for repeatable commit - a filenotfoundexception is not counted as an error (new behavior). Can you check FileOutputCommitter#testCommitterRepeatableV1() and FileOutputCommitter#testCommitterRepeatableV2()?
          Hide
          djp Junping Du added a comment -

          Update v5 patch to address Bikas comments above.

          Show
          djp Junping Du added a comment - Update v5 patch to address Bikas comments above.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 6s docker + precommit patch detected.
          +1 @author 0m 1s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
          +1 mvninstall 3m 9s trunk passed
          +1 compile 2m 26s trunk passed with JDK v1.8.0_60
          +1 compile 2m 31s trunk passed with JDK v1.7.0_79
          +1 checkstyle 0m 19s trunk passed
          +1 mvneclipse 0m 28s trunk passed
          +1 findbugs 1m 55s trunk passed
          +1 javadoc 0m 41s trunk passed with JDK v1.8.0_60
          +1 javadoc 0m 45s trunk passed with JDK v1.7.0_79
          -1 mvninstall 0m 15s hadoop-mapreduce-client-app in the patch failed.
          +1 compile 2m 29s the patch passed with JDK v1.8.0_60
          +1 javac 2m 29s the patch passed
          +1 compile 2m 30s the patch passed with JDK v1.7.0_79
          +1 javac 2m 30s the patch passed
          -1 checkstyle 0m 24s Patch generated 2 new checkstyle issues in hadoop-mapreduce-project/hadoop-mapreduce-client (total was 128, now 126).
          +1 mvneclipse 0m 28s the patch passed
          -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix.
          -1 findbugs 1m 21s hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core introduced 2 new FindBugs issues.
          -1 javadoc 2m 20s hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.8.0_60 with JDK v1.8.0_60 generated 2 new issues (was 100, now 100).
          +1 javadoc 0m 40s the patch passed with JDK v1.8.0_60
          +1 javadoc 0m 45s the patch passed with JDK v1.7.0_79
          -1 unit 9m 29s hadoop-mapreduce-client-app in the patch failed with JDK v1.8.0_60.
          -1 unit 1m 41s hadoop-mapreduce-client-core in the patch failed with JDK v1.8.0_60.
          -1 unit 10m 5s hadoop-mapreduce-client-app in the patch failed with JDK v1.7.0_79.
          -1 unit 2m 6s hadoop-mapreduce-client-core in the patch failed with JDK v1.7.0_79.
          -1 asflicense 0m 20s Patch generated 1 ASF License warnings.
          47m 32s



          Reason Tests
          FindBugs module:hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core
            org.apache.hadoop.mapred.FileOutputCommitter.isCommitJobRepeatable(JobContext) doesn't override method in superclass because parameter type org.apache.hadoop.mapred.JobContext doesn't match superclass parameter type org.apache.hadoop.mapreduce.JobContext At FileOutputCommitter.java:superclass because parameter type org.apache.hadoop.mapred.JobContext doesn't match superclass parameter type org.apache.hadoop.mapreduce.JobContext At FileOutputCommitter.java:[line 194]
            org.apache.hadoop.mapred.OutputCommitter.isCommitJobRepeatable(JobContext) doesn't override method in superclass because parameter type org.apache.hadoop.mapred.JobContext doesn't match superclass parameter type org.apache.hadoop.mapreduce.JobContext At OutputCommitter.java:superclass because parameter type org.apache.hadoop.mapred.JobContext doesn't match superclass parameter type org.apache.hadoop.mapreduce.JobContext At OutputCommitter.java:[line 230]
          JDK v1.8.0_60 Failed junit tests hadoop.mapreduce.v2.app.TestMRAppMaster
            hadoop.mapreduce.v2.app.TestRuntimeEstimators
            hadoop.mapreduce.filecache.TestClientDistributedCacheManager
          JDK v1.7.0_79 Failed junit tests hadoop.mapreduce.v2.app.TestMRAppMaster
            hadoop.mapreduce.filecache.TestClientDistributedCacheManager



          Subsystem Report/Notes
          Docker Client=1.7.1 Server=1.7.1 Image:test-patch-base-hadoop-date2015-11-11
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12771767/MAPREDUCE-5485-v5.patch
          JIRA Issue MAPREDUCE-5485
          Optional Tests asflicense javac javadoc mvninstall unit findbugs checkstyle compile
          uname Linux be427943753d 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /home/jenkins/jenkins-slave/workspace/PreCommit-MAPREDUCE-Build/patchprocess/apache-yetus-ee5baeb/precommit/personality/hadoop.sh
          git revision trunk / d907697
          Default Java 1.7.0_79
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_60 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_79
          findbugs v3.0.0
          mvninstall https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6148/artifact/patchprocess/patch-mvninstall-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app.txt
          checkstyle https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6148/artifact/patchprocess/diff-checkstyle-hadoop-mapreduce-project_hadoop-mapreduce-client.txt
          whitespace https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6148/artifact/patchprocess/whitespace-eol.txt
          findbugs https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6148/artifact/patchprocess/new-findbugs-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core.html
          javadoc hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.8.0_60: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6148/artifact/patchprocess/diff-javadoc-javadoc-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.8.0_60.txt
          unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6148/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.8.0_60.txt
          unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6148/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.8.0_60.txt
          unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6148/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.7.0_79.txt
          unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6148/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.7.0_79.txt
          unit test logs https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6148/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.8.0_60.txt https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6148/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.8.0_60.txt https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6148/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.7.0_79.txt https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6148/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.7.0_79.txt
          JDK v1.7.0_79 Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6148/testReport/
          asflicense https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6148/artifact/patchprocess/patch-asflicense-problems.txt
          modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core U: hadoop-mapreduce-project/hadoop-mapreduce-client
          Max memory used 226MB
          Powered by Apache Yetus http://yetus.apache.org
          Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6148/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 6s docker + precommit patch detected. +1 @author 0m 1s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. +1 mvninstall 3m 9s trunk passed +1 compile 2m 26s trunk passed with JDK v1.8.0_60 +1 compile 2m 31s trunk passed with JDK v1.7.0_79 +1 checkstyle 0m 19s trunk passed +1 mvneclipse 0m 28s trunk passed +1 findbugs 1m 55s trunk passed +1 javadoc 0m 41s trunk passed with JDK v1.8.0_60 +1 javadoc 0m 45s trunk passed with JDK v1.7.0_79 -1 mvninstall 0m 15s hadoop-mapreduce-client-app in the patch failed. +1 compile 2m 29s the patch passed with JDK v1.8.0_60 +1 javac 2m 29s the patch passed +1 compile 2m 30s the patch passed with JDK v1.7.0_79 +1 javac 2m 30s the patch passed -1 checkstyle 0m 24s Patch generated 2 new checkstyle issues in hadoop-mapreduce-project/hadoop-mapreduce-client (total was 128, now 126). +1 mvneclipse 0m 28s the patch passed -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix. -1 findbugs 1m 21s hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core introduced 2 new FindBugs issues. -1 javadoc 2m 20s hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.8.0_60 with JDK v1.8.0_60 generated 2 new issues (was 100, now 100). +1 javadoc 0m 40s the patch passed with JDK v1.8.0_60 +1 javadoc 0m 45s the patch passed with JDK v1.7.0_79 -1 unit 9m 29s hadoop-mapreduce-client-app in the patch failed with JDK v1.8.0_60. -1 unit 1m 41s hadoop-mapreduce-client-core in the patch failed with JDK v1.8.0_60. -1 unit 10m 5s hadoop-mapreduce-client-app in the patch failed with JDK v1.7.0_79. -1 unit 2m 6s hadoop-mapreduce-client-core in the patch failed with JDK v1.7.0_79. -1 asflicense 0m 20s Patch generated 1 ASF License warnings. 47m 32s Reason Tests FindBugs module:hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core   org.apache.hadoop.mapred.FileOutputCommitter.isCommitJobRepeatable(JobContext) doesn't override method in superclass because parameter type org.apache.hadoop.mapred.JobContext doesn't match superclass parameter type org.apache.hadoop.mapreduce.JobContext At FileOutputCommitter.java:superclass because parameter type org.apache.hadoop.mapred.JobContext doesn't match superclass parameter type org.apache.hadoop.mapreduce.JobContext At FileOutputCommitter.java: [line 194]   org.apache.hadoop.mapred.OutputCommitter.isCommitJobRepeatable(JobContext) doesn't override method in superclass because parameter type org.apache.hadoop.mapred.JobContext doesn't match superclass parameter type org.apache.hadoop.mapreduce.JobContext At OutputCommitter.java:superclass because parameter type org.apache.hadoop.mapred.JobContext doesn't match superclass parameter type org.apache.hadoop.mapreduce.JobContext At OutputCommitter.java: [line 230] JDK v1.8.0_60 Failed junit tests hadoop.mapreduce.v2.app.TestMRAppMaster   hadoop.mapreduce.v2.app.TestRuntimeEstimators   hadoop.mapreduce.filecache.TestClientDistributedCacheManager JDK v1.7.0_79 Failed junit tests hadoop.mapreduce.v2.app.TestMRAppMaster   hadoop.mapreduce.filecache.TestClientDistributedCacheManager Subsystem Report/Notes Docker Client=1.7.1 Server=1.7.1 Image:test-patch-base-hadoop-date2015-11-11 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12771767/MAPREDUCE-5485-v5.patch JIRA Issue MAPREDUCE-5485 Optional Tests asflicense javac javadoc mvninstall unit findbugs checkstyle compile uname Linux be427943753d 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /home/jenkins/jenkins-slave/workspace/PreCommit-MAPREDUCE-Build/patchprocess/apache-yetus-ee5baeb/precommit/personality/hadoop.sh git revision trunk / d907697 Default Java 1.7.0_79 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_60 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_79 findbugs v3.0.0 mvninstall https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6148/artifact/patchprocess/patch-mvninstall-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app.txt checkstyle https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6148/artifact/patchprocess/diff-checkstyle-hadoop-mapreduce-project_hadoop-mapreduce-client.txt whitespace https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6148/artifact/patchprocess/whitespace-eol.txt findbugs https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6148/artifact/patchprocess/new-findbugs-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core.html javadoc hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.8.0_60: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6148/artifact/patchprocess/diff-javadoc-javadoc-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.8.0_60.txt unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6148/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.8.0_60.txt unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6148/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.8.0_60.txt unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6148/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.7.0_79.txt unit https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6148/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.7.0_79.txt unit test logs https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6148/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.8.0_60.txt https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6148/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.8.0_60.txt https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6148/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.7.0_79.txt https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6148/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.7.0_79.txt JDK v1.7.0_79 Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6148/testReport/ asflicense https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6148/artifact/patchprocess/patch-asflicense-problems.txt modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core U: hadoop-mapreduce-project/hadoop-mapreduce-client Max memory used 226MB Powered by Apache Yetus http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6148/console This message was automatically generated.
          Hide
          djp Junping Du added a comment -

          The test failure is not related and I believe nothing could do with findbugs/checkstyle issues.

          Show
          djp Junping Du added a comment - The test failure is not related and I believe nothing could do with findbugs/checkstyle issues.
          Hide
          bikassaha Bikas Saha added a comment -

          Sorry I missed that the test committer derives from FileOutputCommitter and throws the file not found exception.

          About the overall test. The main overall change is to allow the retry AM to continue after seeing an in-progress commit from the previous AM. It seems incomplete to not have a test for that. However if you think that we dont have existing infra for that code path then we should create a follow up jira to add that infra and relevant tests. I have not followed the MR AM code changes for a while and so I cannot recall of the top of my head about any existing test cases. Maybe other committers may have some ideas.

          With that caveat, the latest patch looks good to me. Thanks for your patience through the reviews

          Show
          bikassaha Bikas Saha added a comment - Sorry I missed that the test committer derives from FileOutputCommitter and throws the file not found exception. About the overall test. The main overall change is to allow the retry AM to continue after seeing an in-progress commit from the previous AM. It seems incomplete to not have a test for that. However if you think that we dont have existing infra for that code path then we should create a follow up jira to add that infra and relevant tests. I have not followed the MR AM code changes for a while and so I cannot recall of the top of my head about any existing test cases. Maybe other committers may have some ideas. With that caveat, the latest patch looks good to me. Thanks for your patience through the reviews
          Hide
          djp Junping Du added a comment -

          About the overall test. The main overall change is to allow the retry AM to continue after seeing an in-progress commit from the previous AM. It seems incomplete to not have a test for that.

          I agree that it is better to add as many cases as possible in unit test. But due to limitations of our current unit test framework, we could miss many functional tests, especially related to MR AM failed/restart, like: in rolling upgrade story, we don't have tests to check AM failed over during NM/RM restart. Instead, we may have to split the whole functionality into pieces and test each piece. Sometime it is sad that this may not be good enough and that's why we still need to test/verify the feature works end to end on a real cluster.

          However if you think that we dont have existing infra for that code path then we should create a follow up jira to add that infra and relevant tests. I have not followed the MR AM code changes for a while and so I cannot recall of the top of my head about any existing test cases. Maybe other committers may have some ideas.

          Just filed MAPREDUCE-6545 to track more test effort that comes later.

          With that caveat, the latest patch looks good to me. Thanks for your patience through the reviews.

          Thanks Bikas for your carefully review.

          Show
          djp Junping Du added a comment - About the overall test. The main overall change is to allow the retry AM to continue after seeing an in-progress commit from the previous AM. It seems incomplete to not have a test for that. I agree that it is better to add as many cases as possible in unit test. But due to limitations of our current unit test framework, we could miss many functional tests, especially related to MR AM failed/restart, like: in rolling upgrade story, we don't have tests to check AM failed over during NM/RM restart. Instead, we may have to split the whole functionality into pieces and test each piece. Sometime it is sad that this may not be good enough and that's why we still need to test/verify the feature works end to end on a real cluster. However if you think that we dont have existing infra for that code path then we should create a follow up jira to add that infra and relevant tests. I have not followed the MR AM code changes for a while and so I cannot recall of the top of my head about any existing test cases. Maybe other committers may have some ideas. Just filed MAPREDUCE-6545 to track more test effort that comes later. With that caveat, the latest patch looks good to me. Thanks for your patience through the reviews. Thanks Bikas for your carefully review.
          Hide
          jianhe Jian He added a comment -

          lgtm, committing

          Show
          jianhe Jian He added a comment - lgtm, committing
          Hide
          jianhe Jian He added a comment -

          Junping Du, there are some findbugs and ut failures, mind checking ?

          Show
          jianhe Jian He added a comment - Junping Du , there are some findbugs and ut failures, mind checking ?
          Hide
          djp Junping Du added a comment -

          Junping Du, there are some findbugs and ut failures, mind checking ?

          The unit test is unrelated and just fixed in MAPREDUCE-6533. The fingbugs warning belongs to wrong checking as org.apache.hadoop.mapred.FileOutputCommitter.isCommitJobRepeatable(Context) override the right method in parent abstract class (it wrongly recognize to override another abstract method). So I think we should ignore these warnings.

          Show
          djp Junping Du added a comment - Junping Du, there are some findbugs and ut failures, mind checking ? The unit test is unrelated and just fixed in MAPREDUCE-6533 . The fingbugs warning belongs to wrong checking as org.apache.hadoop.mapred.FileOutputCommitter.isCommitJobRepeatable(Context) override the right method in parent abstract class (it wrongly recognize to override another abstract method). So I think we should ignore these warnings.
          Hide
          djp Junping Du added a comment -

          Upload a patch for branch-2.7 in case we want to commit it for 2.7.3.

          Show
          djp Junping Du added a comment - Upload a patch for branch-2.7 in case we want to commit it for 2.7.3.
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #8809 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8809/)
          MAPREDUCE-5485. Allow repeating job commit by extending OutputCommitter (jianhe: rev 6502d59e73cd6f3f3a358fce58d398ca38a61fba)

          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/lib/output/TestFileOutputCommitter.java
          • hadoop-mapreduce-project/CHANGES.txt
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/FileOutputCommitter.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/OutputCommitter.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapred/TestFileOutputCommitter.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/commit/CommitterEventHandler.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/output/FileOutputCommitter.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/MRAppMaster.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/OutputCommitter.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #8809 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8809/ ) MAPREDUCE-5485 . Allow repeating job commit by extending OutputCommitter (jianhe: rev 6502d59e73cd6f3f3a358fce58d398ca38a61fba) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/lib/output/TestFileOutputCommitter.java hadoop-mapreduce-project/CHANGES.txt hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/FileOutputCommitter.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/OutputCommitter.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapred/TestFileOutputCommitter.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/commit/CommitterEventHandler.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/output/FileOutputCommitter.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/MRAppMaster.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/OutputCommitter.java
          Hide
          jianhe Jian He added a comment -

          Committed to trunk, branch-2, branch-2.7, thanks Junping !
          thanks Bikas for reviewing !

          Show
          jianhe Jian He added a comment - Committed to trunk, branch-2, branch-2.7, thanks Junping ! thanks Bikas for reviewing !
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #670 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/670/)
          MAPREDUCE-5485. Allow repeating job commit by extending OutputCommitter (jianhe: rev 6502d59e73cd6f3f3a358fce58d398ca38a61fba)

          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/commit/CommitterEventHandler.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/lib/output/TestFileOutputCommitter.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/OutputCommitter.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/output/FileOutputCommitter.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/OutputCommitter.java
          • hadoop-mapreduce-project/CHANGES.txt
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/FileOutputCommitter.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/MRAppMaster.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapred/TestFileOutputCommitter.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #670 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/670/ ) MAPREDUCE-5485 . Allow repeating job commit by extending OutputCommitter (jianhe: rev 6502d59e73cd6f3f3a358fce58d398ca38a61fba) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/commit/CommitterEventHandler.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/lib/output/TestFileOutputCommitter.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/OutputCommitter.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/output/FileOutputCommitter.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/OutputCommitter.java hadoop-mapreduce-project/CHANGES.txt hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/FileOutputCommitter.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/MRAppMaster.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapred/TestFileOutputCommitter.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk #2612 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2612/)
          MAPREDUCE-5485. Allow repeating job commit by extending OutputCommitter (jianhe: rev 6502d59e73cd6f3f3a358fce58d398ca38a61fba)

          • hadoop-mapreduce-project/CHANGES.txt
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/output/FileOutputCommitter.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/FileOutputCommitter.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/OutputCommitter.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/MRAppMaster.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/commit/CommitterEventHandler.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/OutputCommitter.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapred/TestFileOutputCommitter.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/lib/output/TestFileOutputCommitter.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #2612 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2612/ ) MAPREDUCE-5485 . Allow repeating job commit by extending OutputCommitter (jianhe: rev 6502d59e73cd6f3f3a358fce58d398ca38a61fba) hadoop-mapreduce-project/CHANGES.txt hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/output/FileOutputCommitter.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/FileOutputCommitter.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/OutputCommitter.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/MRAppMaster.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/commit/CommitterEventHandler.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/OutputCommitter.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapred/TestFileOutputCommitter.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/lib/output/TestFileOutputCommitter.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #683 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/683/)
          MAPREDUCE-5485. Allow repeating job commit by extending OutputCommitter (jianhe: rev 6502d59e73cd6f3f3a358fce58d398ca38a61fba)

          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/OutputCommitter.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/MRAppMaster.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/output/FileOutputCommitter.java
          • hadoop-mapreduce-project/CHANGES.txt
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/OutputCommitter.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapred/TestFileOutputCommitter.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/commit/CommitterEventHandler.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/lib/output/TestFileOutputCommitter.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/FileOutputCommitter.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #683 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/683/ ) MAPREDUCE-5485 . Allow repeating job commit by extending OutputCommitter (jianhe: rev 6502d59e73cd6f3f3a358fce58d398ca38a61fba) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/OutputCommitter.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/MRAppMaster.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/output/FileOutputCommitter.java hadoop-mapreduce-project/CHANGES.txt hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/OutputCommitter.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapred/TestFileOutputCommitter.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/commit/CommitterEventHandler.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/lib/output/TestFileOutputCommitter.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/FileOutputCommitter.java
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-Yarn-trunk #1407 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1407/)
          MAPREDUCE-5485. Allow repeating job commit by extending OutputCommitter (jianhe: rev 6502d59e73cd6f3f3a358fce58d398ca38a61fba)

          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapred/TestFileOutputCommitter.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/OutputCommitter.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/MRAppMaster.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/lib/output/TestFileOutputCommitter.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/output/FileOutputCommitter.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/OutputCommitter.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/FileOutputCommitter.java
          • hadoop-mapreduce-project/CHANGES.txt
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/commit/CommitterEventHandler.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Yarn-trunk #1407 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1407/ ) MAPREDUCE-5485 . Allow repeating job commit by extending OutputCommitter (jianhe: rev 6502d59e73cd6f3f3a358fce58d398ca38a61fba) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapred/TestFileOutputCommitter.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/OutputCommitter.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/MRAppMaster.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/lib/output/TestFileOutputCommitter.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/output/FileOutputCommitter.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/OutputCommitter.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/FileOutputCommitter.java hadoop-mapreduce-project/CHANGES.txt hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/commit/CommitterEventHandler.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk #2547 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2547/)
          MAPREDUCE-5485. Allow repeating job commit by extending OutputCommitter (jianhe: rev 6502d59e73cd6f3f3a358fce58d398ca38a61fba)

          • hadoop-mapreduce-project/CHANGES.txt
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/FileOutputCommitter.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/output/FileOutputCommitter.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/OutputCommitter.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/lib/output/TestFileOutputCommitter.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapred/TestFileOutputCommitter.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/OutputCommitter.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/commit/CommitterEventHandler.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/MRAppMaster.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2547 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2547/ ) MAPREDUCE-5485 . Allow repeating job commit by extending OutputCommitter (jianhe: rev 6502d59e73cd6f3f3a358fce58d398ca38a61fba) hadoop-mapreduce-project/CHANGES.txt hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/FileOutputCommitter.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/output/FileOutputCommitter.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/OutputCommitter.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/lib/output/TestFileOutputCommitter.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapred/TestFileOutputCommitter.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/OutputCommitter.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/commit/CommitterEventHandler.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/MRAppMaster.java
          Hide
          hudson Hudson added a comment -

          ABORTED: Integrated in Hadoop-Hdfs-trunk-Java8 #610 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/610/)
          MAPREDUCE-5485. Allow repeating job commit by extending OutputCommitter (jianhe: rev 6502d59e73cd6f3f3a358fce58d398ca38a61fba)

          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapred/TestFileOutputCommitter.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/output/FileOutputCommitter.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/commit/CommitterEventHandler.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/MRAppMaster.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/OutputCommitter.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/lib/output/TestFileOutputCommitter.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/OutputCommitter.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/FileOutputCommitter.java
          • hadoop-mapreduce-project/CHANGES.txt
          Show
          hudson Hudson added a comment - ABORTED: Integrated in Hadoop-Hdfs-trunk-Java8 #610 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/610/ ) MAPREDUCE-5485 . Allow repeating job commit by extending OutputCommitter (jianhe: rev 6502d59e73cd6f3f3a358fce58d398ca38a61fba) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapred/TestFileOutputCommitter.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/output/FileOutputCommitter.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/commit/CommitterEventHandler.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/MRAppMaster.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/OutputCommitter.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/lib/output/TestFileOutputCommitter.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/OutputCommitter.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/FileOutputCommitter.java hadoop-mapreduce-project/CHANGES.txt
          Hide
          ajisakaa Akira Ajisaka added a comment -

          TestMRAppMaster is failing after MAPREDUCE-5485. Hi Junping Du, would you take a look at MAPREDUCE-6555?

          Show
          ajisakaa Akira Ajisaka added a comment - TestMRAppMaster is failing after MAPREDUCE-5485 . Hi Junping Du , would you take a look at MAPREDUCE-6555 ?
          Hide
          djp Junping Du added a comment -

          Sure. Thanks Akira Ajisaka for reminding on this!

          Show
          djp Junping Du added a comment - Sure. Thanks Akira Ajisaka for reminding on this!
          Hide
          varun_saxena Varun Saxena added a comment -

          Junping Du, should we add exclusions for these findbugs warnings ? Because they unnecessarily show up in QA report then.

          Show
          varun_saxena Varun Saxena added a comment - Junping Du , should we add exclusions for these findbugs warnings ? Because they unnecessarily show up in QA report then.
          Hide
          ajisakaa Akira Ajisaka added a comment -

          Hi Junping Du, would you review MAPREDUCE-6595? The patch fixes findbugs warnings.

          Show
          ajisakaa Akira Ajisaka added a comment - Hi Junping Du , would you review MAPREDUCE-6595 ? The patch fixes findbugs warnings.
          Hide
          vinodkv Vinod Kumar Vavilapalli added a comment -

          Closing the JIRA as part of 2.7.3 release.

          Show
          vinodkv Vinod Kumar Vavilapalli added a comment - Closing the JIRA as part of 2.7.3 release.

            People

            • Assignee:
              djp Junping Du
              Reporter:
              nemon Nemon Lou
            • Votes:
              0 Vote for this issue
              Watchers:
              15 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development