Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-6361

NPE issue in shuffle caused by concurrent issue between copySucceeded() in one thread and copyFailed() in another thread on the same host

    Details

    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      The failure in log:
      2015-05-08 21:00:00,513 WARN [main] org.apache.hadoop.mapred.YarnChild: Exception running child : org.apache.hadoop.mapreduce.task.reduce.Shuffle$ShuffleError: error in shuffle in fetcher#25
      at org.apache.hadoop.mapreduce.task.reduce.Shuffle.run(Shuffle.java:134)
      at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:376)
      at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
      at java.security.AccessController.doPrivileged(Native Method)
      at javax.security.auth.Subject.doAs(Subject.java:415)
      at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
      at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
      Caused by: java.lang.NullPointerException
      at org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl.copyFailed(ShuffleSchedulerImpl.java:267)
      at org.apache.hadoop.mapreduce.task.reduce.Fetcher.copyFromHost(Fetcher.java:308)
      at org.apache.hadoop.mapreduce.task.reduce.Fetcher.run(Fetcher.java:193)

        Activity

        Hide
        djp Junping Du added a comment -

        NPE get throw in copyFailed() in ShuffleSchedulerImpl.java:267:

        "boolean hostFail = hostFailures.get(hostname).get() > getMaxHostFailures() ? true : false;"
        

        It means hostFailures doesn't include hostname that just failed, which is not expected because we call hostFailed() to put host into hostFailures before anytime to call copyFailed():

                scheduler.hostFailed(host.getHostName());
                for(TaskAttemptID left: failedTasks) {
                  scheduler.copyFailed(left, host, true, false);
                }
        

        Although hostFailed() and copyFailed() are both synchronized method (so as copySucceeded()), it is still possible (like the only reason) to cause this NPE for the other thread calls copySucceeded() on the same host (for other map output) between we call hostFailed() and copyFailed() in this thread when taking care of one map output failure.
        We need to fix this concurrent issue to get rid of NPE issue which failed map output copy directly without any retry.

        Show
        djp Junping Du added a comment - NPE get throw in copyFailed() in ShuffleSchedulerImpl.java:267: " boolean hostFail = hostFailures.get(hostname).get() > getMaxHostFailures() ? true : false ;" It means hostFailures doesn't include hostname that just failed, which is not expected because we call hostFailed() to put host into hostFailures before anytime to call copyFailed(): scheduler.hostFailed(host.getHostName()); for (TaskAttemptID left: failedTasks) { scheduler.copyFailed(left, host, true , false ); } Although hostFailed() and copyFailed() are both synchronized method (so as copySucceeded()), it is still possible (like the only reason) to cause this NPE for the other thread calls copySucceeded() on the same host (for other map output) between we call hostFailed() and copyFailed() in this thread when taking care of one map output failure. We need to fix this concurrent issue to get rid of NPE issue which failed map output copy directly without any retry.
        Hide
        djp Junping Du added a comment -

        There are basically two ways to fix the race condition here:
        1. abstract following code into a synchronized method, so copySucceeded() would get blocked until copyFailed() finished.

        scheduler.hostFailed(host.getHostName());
        for(TaskAttemptID left: failedTasks) {
            scheduler.copyFailed(left, host, true, false);
        }
        

        This sounds like more performance impact on shuffle as failure in fetching map output on one thread will block copySucceeded() for other threads with longer time.

        2. Update copyFailed() to have assumption that hostFailures could be cleanup in the other thread. In case of that, adding back host to hostFailed as the first time host failed.

        Prefer the 2nd option which sounds more lightweight. Will deliver a quick patch soon.

        Show
        djp Junping Du added a comment - There are basically two ways to fix the race condition here: 1. abstract following code into a synchronized method, so copySucceeded() would get blocked until copyFailed() finished. scheduler.hostFailed(host.getHostName()); for (TaskAttemptID left: failedTasks) { scheduler.copyFailed(left, host, true , false ); } This sounds like more performance impact on shuffle as failure in fetching map output on one thread will block copySucceeded() for other threads with longer time. 2. Update copyFailed() to have assumption that hostFailures could be cleanup in the other thread. In case of that, adding back host to hostFailed as the first time host failed. Prefer the 2nd option which sounds more lightweight. Will deliver a quick patch soon.
        Hide
        djp Junping Du added a comment -

        Upload the patch with the 2nd solution proposed above with unit test.

        Show
        djp Junping Du added a comment - Upload the patch with the 2nd solution proposed above with unit test.
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        0 pre-patch 14m 57s Pre-patch trunk compilation is healthy.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
        +1 javac 7m 37s There were no new javac warning messages.
        +1 javadoc 9m 41s There were no new javadoc warning messages.
        +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings.
        +1 checkstyle 0m 45s There were no new checkstyle issues.
        -1 whitespace 0m 0s The patch has 9 line(s) that end in whitespace. Use git apply --whitespace=fix.
        +1 install 1m 33s mvn install still works.
        +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse.
        +1 findbugs 1m 13s The patch does not introduce any new Findbugs (version 2.0.3) warnings.
        +1 mapreduce tests 1m 36s Tests passed in hadoop-mapreduce-client-core.
            38m 22s  



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12732215/MAPREDUCE-6361-v1.patch
        Optional Tests javadoc javac unit findbugs checkstyle
        git revision trunk / 8badd82
        whitespace https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5714/artifact/patchprocess/whitespace.txt
        hadoop-mapreduce-client-core test log https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5714/artifact/patchprocess/testrun_hadoop-mapreduce-client-core.txt
        Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5714/testReport/
        Java 1.7.0_55
        uname Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5714/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 14m 57s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 javac 7m 37s There were no new javac warning messages. +1 javadoc 9m 41s There were no new javadoc warning messages. +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings. +1 checkstyle 0m 45s There were no new checkstyle issues. -1 whitespace 0m 0s The patch has 9 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 install 1m 33s mvn install still works. +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse. +1 findbugs 1m 13s The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 mapreduce tests 1m 36s Tests passed in hadoop-mapreduce-client-core.     38m 22s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12732215/MAPREDUCE-6361-v1.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 8badd82 whitespace https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5714/artifact/patchprocess/whitespace.txt hadoop-mapreduce-client-core test log https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5714/artifact/patchprocess/testrun_hadoop-mapreduce-client-core.txt Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5714/testReport/ Java 1.7.0_55 uname Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5714/console This message was automatically generated.
        Hide
        ozawa Tsuyoshi Ozawa added a comment -

        +1, I confirmed that this is a bug about concurrency between threads of Fether#run as Junping mentioned. Committing this shortly.

        Show
        ozawa Tsuyoshi Ozawa added a comment - +1, I confirmed that this is a bug about concurrency between threads of Fether#run as Junping mentioned. Committing this shortly.
        Hide
        ozawa Tsuyoshi Ozawa added a comment -

        Committed this to trunk and branch-2. Thanks Junping Du for your report and contribution!

        Show
        ozawa Tsuyoshi Ozawa added a comment - Committed this to trunk and branch-2. Thanks Junping Du for your report and contribution!
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-trunk-Commit #7806 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7806/)
        MAPREDUCE-6361. NPE issue in shuffle caused by concurrent issue between copySucceeded() in one thread and copyFailed() in another thread on the same host. Contributed by Junping Du. (ozawa: rev f4e2b3cc0b1f4e49c306bc09a9dddd0495225bb2)

        • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/task/reduce/TestShuffleScheduler.java
        • hadoop-mapreduce-project/CHANGES.txt
        • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/task/reduce/ShuffleSchedulerImpl.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #7806 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7806/ ) MAPREDUCE-6361 . NPE issue in shuffle caused by concurrent issue between copySucceeded() in one thread and copyFailed() in another thread on the same host. Contributed by Junping Du. (ozawa: rev f4e2b3cc0b1f4e49c306bc09a9dddd0495225bb2) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/task/reduce/TestShuffleScheduler.java hadoop-mapreduce-project/CHANGES.txt hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/task/reduce/ShuffleSchedulerImpl.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #195 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/195/)
        MAPREDUCE-6361. NPE issue in shuffle caused by concurrent issue between copySucceeded() in one thread and copyFailed() in another thread on the same host. Contributed by Junping Du. (ozawa: rev f4e2b3cc0b1f4e49c306bc09a9dddd0495225bb2)

        • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/task/reduce/TestShuffleScheduler.java
        • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/task/reduce/ShuffleSchedulerImpl.java
        • hadoop-mapreduce-project/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #195 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/195/ ) MAPREDUCE-6361 . NPE issue in shuffle caused by concurrent issue between copySucceeded() in one thread and copyFailed() in another thread on the same host. Contributed by Junping Du. (ozawa: rev f4e2b3cc0b1f4e49c306bc09a9dddd0495225bb2) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/task/reduce/TestShuffleScheduler.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/task/reduce/ShuffleSchedulerImpl.java hadoop-mapreduce-project/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Hadoop-Yarn-trunk #926 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/926/)
        MAPREDUCE-6361. NPE issue in shuffle caused by concurrent issue between copySucceeded() in one thread and copyFailed() in another thread on the same host. Contributed by Junping Du. (ozawa: rev f4e2b3cc0b1f4e49c306bc09a9dddd0495225bb2)

        • hadoop-mapreduce-project/CHANGES.txt
        • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/task/reduce/TestShuffleScheduler.java
        • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/task/reduce/ShuffleSchedulerImpl.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Yarn-trunk #926 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/926/ ) MAPREDUCE-6361 . NPE issue in shuffle caused by concurrent issue between copySucceeded() in one thread and copyFailed() in another thread on the same host. Contributed by Junping Du. (ozawa: rev f4e2b3cc0b1f4e49c306bc09a9dddd0495225bb2) hadoop-mapreduce-project/CHANGES.txt hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/task/reduce/TestShuffleScheduler.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/task/reduce/ShuffleSchedulerImpl.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Hdfs-trunk #2124 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2124/)
        MAPREDUCE-6361. NPE issue in shuffle caused by concurrent issue between copySucceeded() in one thread and copyFailed() in another thread on the same host. Contributed by Junping Du. (ozawa: rev f4e2b3cc0b1f4e49c306bc09a9dddd0495225bb2)

        • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/task/reduce/TestShuffleScheduler.java
        • hadoop-mapreduce-project/CHANGES.txt
        • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/task/reduce/ShuffleSchedulerImpl.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2124 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2124/ ) MAPREDUCE-6361 . NPE issue in shuffle caused by concurrent issue between copySucceeded() in one thread and copyFailed() in another thread on the same host. Contributed by Junping Du. (ozawa: rev f4e2b3cc0b1f4e49c306bc09a9dddd0495225bb2) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/task/reduce/TestShuffleScheduler.java hadoop-mapreduce-project/CHANGES.txt hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/task/reduce/ShuffleSchedulerImpl.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #184 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/184/)
        MAPREDUCE-6361. NPE issue in shuffle caused by concurrent issue between copySucceeded() in one thread and copyFailed() in another thread on the same host. Contributed by Junping Du. (ozawa: rev f4e2b3cc0b1f4e49c306bc09a9dddd0495225bb2)

        • hadoop-mapreduce-project/CHANGES.txt
        • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/task/reduce/TestShuffleScheduler.java
        • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/task/reduce/ShuffleSchedulerImpl.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #184 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/184/ ) MAPREDUCE-6361 . NPE issue in shuffle caused by concurrent issue between copySucceeded() in one thread and copyFailed() in another thread on the same host. Contributed by Junping Du. (ozawa: rev f4e2b3cc0b1f4e49c306bc09a9dddd0495225bb2) hadoop-mapreduce-project/CHANGES.txt hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/task/reduce/TestShuffleScheduler.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/task/reduce/ShuffleSchedulerImpl.java
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Hadoop-Mapreduce-trunk-Java8 #194 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/194/)
        MAPREDUCE-6361. NPE issue in shuffle caused by concurrent issue between copySucceeded() in one thread and copyFailed() in another thread on the same host. Contributed by Junping Du. (ozawa: rev f4e2b3cc0b1f4e49c306bc09a9dddd0495225bb2)

        • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/task/reduce/ShuffleSchedulerImpl.java
        • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/task/reduce/TestShuffleScheduler.java
        • hadoop-mapreduce-project/CHANGES.txt
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Mapreduce-trunk-Java8 #194 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/194/ ) MAPREDUCE-6361 . NPE issue in shuffle caused by concurrent issue between copySucceeded() in one thread and copyFailed() in another thread on the same host. Contributed by Junping Du. (ozawa: rev f4e2b3cc0b1f4e49c306bc09a9dddd0495225bb2) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/task/reduce/ShuffleSchedulerImpl.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/task/reduce/TestShuffleScheduler.java hadoop-mapreduce-project/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Mapreduce-trunk #2142 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2142/)
        MAPREDUCE-6361. NPE issue in shuffle caused by concurrent issue between copySucceeded() in one thread and copyFailed() in another thread on the same host. Contributed by Junping Du. (ozawa: rev f4e2b3cc0b1f4e49c306bc09a9dddd0495225bb2)

        • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/task/reduce/TestShuffleScheduler.java
        • hadoop-mapreduce-project/CHANGES.txt
        • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/task/reduce/ShuffleSchedulerImpl.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #2142 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2142/ ) MAPREDUCE-6361 . NPE issue in shuffle caused by concurrent issue between copySucceeded() in one thread and copyFailed() in another thread on the same host. Contributed by Junping Du. (ozawa: rev f4e2b3cc0b1f4e49c306bc09a9dddd0495225bb2) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/task/reduce/TestShuffleScheduler.java hadoop-mapreduce-project/CHANGES.txt hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/task/reduce/ShuffleSchedulerImpl.java
        Hide
        djp Junping Du added a comment -

        Thanks Tsuyoshi Ozawa for review and commit the patch! Move the commit from 2.8 to 2.7.1 as we need this fix asap.

        Show
        djp Junping Du added a comment - Thanks Tsuyoshi Ozawa for review and commit the patch! Move the commit from 2.8 to 2.7.1 as we need this fix asap.
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Hadoop-trunk-Commit #7866 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7866/)
        Moving MAPREDUCE-6361 to 2.7.1 CHANGES.txt (junping_du: rev 8ca1dfeebb660741aa6e5b137cd1088815b614cf)

        • hadoop-mapreduce-project/CHANGES.txt
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-trunk-Commit #7866 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7866/ ) Moving MAPREDUCE-6361 to 2.7.1 CHANGES.txt (junping_du: rev 8ca1dfeebb660741aa6e5b137cd1088815b614cf) hadoop-mapreduce-project/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #202 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/202/)
        Moving MAPREDUCE-6361 to 2.7.1 CHANGES.txt (junping_du: rev 8ca1dfeebb660741aa6e5b137cd1088815b614cf)

        • hadoop-mapreduce-project/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #202 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/202/ ) Moving MAPREDUCE-6361 to 2.7.1 CHANGES.txt (junping_du: rev 8ca1dfeebb660741aa6e5b137cd1088815b614cf) hadoop-mapreduce-project/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Hadoop-Yarn-trunk #933 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/933/)
        Moving MAPREDUCE-6361 to 2.7.1 CHANGES.txt (junping_du: rev 8ca1dfeebb660741aa6e5b137cd1088815b614cf)

        • hadoop-mapreduce-project/CHANGES.txt
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Yarn-trunk #933 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/933/ ) Moving MAPREDUCE-6361 to 2.7.1 CHANGES.txt (junping_du: rev 8ca1dfeebb660741aa6e5b137cd1088815b614cf) hadoop-mapreduce-project/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Hdfs-trunk #2131 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2131/)
        Moving MAPREDUCE-6361 to 2.7.1 CHANGES.txt (junping_du: rev 8ca1dfeebb660741aa6e5b137cd1088815b614cf)

        • hadoop-mapreduce-project/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2131 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2131/ ) Moving MAPREDUCE-6361 to 2.7.1 CHANGES.txt (junping_du: rev 8ca1dfeebb660741aa6e5b137cd1088815b614cf) hadoop-mapreduce-project/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #191 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/191/)
        Moving MAPREDUCE-6361 to 2.7.1 CHANGES.txt (junping_du: rev 8ca1dfeebb660741aa6e5b137cd1088815b614cf)

        • hadoop-mapreduce-project/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #191 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/191/ ) Moving MAPREDUCE-6361 to 2.7.1 CHANGES.txt (junping_du: rev 8ca1dfeebb660741aa6e5b137cd1088815b614cf) hadoop-mapreduce-project/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Hadoop-Mapreduce-trunk-Java8 #201 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/201/)
        Moving MAPREDUCE-6361 to 2.7.1 CHANGES.txt (junping_du: rev 8ca1dfeebb660741aa6e5b137cd1088815b614cf)

        • hadoop-mapreduce-project/CHANGES.txt
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Mapreduce-trunk-Java8 #201 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/201/ ) Moving MAPREDUCE-6361 to 2.7.1 CHANGES.txt (junping_du: rev 8ca1dfeebb660741aa6e5b137cd1088815b614cf) hadoop-mapreduce-project/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Hadoop-Mapreduce-trunk #2149 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2149/)
        Moving MAPREDUCE-6361 to 2.7.1 CHANGES.txt (junping_du: rev 8ca1dfeebb660741aa6e5b137cd1088815b614cf)

        • hadoop-mapreduce-project/CHANGES.txt
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Mapreduce-trunk #2149 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2149/ ) Moving MAPREDUCE-6361 to 2.7.1 CHANGES.txt (junping_du: rev 8ca1dfeebb660741aa6e5b137cd1088815b614cf) hadoop-mapreduce-project/CHANGES.txt
        Hide
        sjlee0 Sangjin Lee added a comment -

        The patch applies to 2.6.0 cleanly.

        Show
        sjlee0 Sangjin Lee added a comment - The patch applies to 2.6.0 cleanly.
        Hide
        vinodkv Vinod Kumar Vavilapalli added a comment -

        Pulled this into 2.6.1, after fixing a minor merge conflict in TestShuffleScheduler.

        Ran compilation and TestShuffleScheduler before the push.

        Show
        vinodkv Vinod Kumar Vavilapalli added a comment - Pulled this into 2.6.1, after fixing a minor merge conflict in TestShuffleScheduler. Ran compilation and TestShuffleScheduler before the push.

          People

          • Assignee:
            djp Junping Du
            Reporter:
            djp Junping Du
          • Votes:
            0 Vote for this issue
            Watchers:
            10 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development