Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-5002

AM could potentially allocate a reduce container to a map attempt

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.0.3-alpha, 0.23.7, 2.7.0
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: mr-am
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      As discussed in MAPREDUCE-4982, after MAPREDUCE-4893 it is theoretically possible for the AM to accidentally assign a reducer container to a map attempt if the AM doesn't find a reduce attempt actively looking for the container (e.g.: the RM accidentally allocated too many reducer containers).

      1. MAPREDUCE-5002.1.patch
        6 kB
        Chang Li
      2. MAPREDUCE-5002.2.patch
        6 kB
        Chang Li
      3. MAPREDUCE-5002.2.patch
        6 kB
        Chang Li
      4. MAPREDUCE-5002.3.patch
        8 kB
        Chang Li
      5. MAPREDUCE-5002.4.patch
        8 kB
        Chang Li
      6. MAPREDUCE-5002.5.patch
        8 kB
        Chang Li
      7. MAPREDUCE-5002.6.patch
        8 kB
        Chang Li

        Issue Links

          Activity

          Hide
          bikassaha Bikas Saha added a comment -

          The fix might be to allow MAP to use FAILED_MAP priority but not REDUCE. For reduce, we could release excess REDUCE containers in the loop inside ScheduledRequests.assign() by checking against reduces.size() instead of reduces.isEmpty().

          Show
          bikassaha Bikas Saha added a comment - The fix might be to allow MAP to use FAILED_MAP priority but not REDUCE. For reduce, we could release excess REDUCE containers in the loop inside ScheduledRequests.assign() by checking against reduces.size() instead of reduces.isEmpty().
          Hide
          airbots Chen He added a comment -

          Is this JIRA fixed? If so, could we close it?

          Show
          airbots Chen He added a comment - Is this JIRA fixed? If so, could we close it?
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          -1 pre-patch 15m 43s Pre-patch trunk has 1 extant Findbugs (version 3.0.0) warnings.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
          +1 javac 7m 33s There were no new javac warning messages.
          +1 javadoc 9m 37s There were no new javadoc warning messages.
          +1 release audit 0m 24s The applied patch does not increase the total number of release audit warnings.
          -1 checkstyle 0m 32s The applied patch generated 1 new checkstyle issues (total was 115, now 116).
          -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix.
          +1 install 1m 33s mvn install still works.
          +1 eclipse:eclipse 0m 32s The patch built with eclipse:eclipse.
          +1 findbugs 1m 7s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          +1 mapreduce tests 9m 3s Tests passed in hadoop-mapreduce-client-app.
              46m 12s  



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12740268/MAPREDUCE-5002.1.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 295d678
          Pre-patch Findbugs warnings https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5812/artifact/patchprocess/trunkFindbugsWarningshadoop-mapreduce-client-app.html
          checkstyle https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5812/artifact/patchprocess/diffcheckstylehadoop-mapreduce-client-app.txt
          whitespace https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5812/artifact/patchprocess/whitespace.txt
          hadoop-mapreduce-client-app test log https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5812/artifact/patchprocess/testrun_hadoop-mapreduce-client-app.txt
          Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5812/testReport/
          Java 1.7.0_55
          uname Linux asf902.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5812/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment -1 pre-patch 15m 43s Pre-patch trunk has 1 extant Findbugs (version 3.0.0) warnings. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 javac 7m 33s There were no new javac warning messages. +1 javadoc 9m 37s There were no new javadoc warning messages. +1 release audit 0m 24s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 0m 32s The applied patch generated 1 new checkstyle issues (total was 115, now 116). -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 install 1m 33s mvn install still works. +1 eclipse:eclipse 0m 32s The patch built with eclipse:eclipse. +1 findbugs 1m 7s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 mapreduce tests 9m 3s Tests passed in hadoop-mapreduce-client-app.     46m 12s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12740268/MAPREDUCE-5002.1.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 295d678 Pre-patch Findbugs warnings https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5812/artifact/patchprocess/trunkFindbugsWarningshadoop-mapreduce-client-app.html checkstyle https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5812/artifact/patchprocess/diffcheckstylehadoop-mapreduce-client-app.txt whitespace https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5812/artifact/patchprocess/whitespace.txt hadoop-mapreduce-client-app test log https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5812/artifact/patchprocess/testrun_hadoop-mapreduce-client-app.txt Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5812/testReport/ Java 1.7.0_55 uname Linux asf902.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5812/console This message was automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          -1 pre-patch 17m 9s Pre-patch trunk has 1 extant Findbugs (version 3.0.0) warnings.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
          +1 javac 7m 48s There were no new javac warning messages.
          +1 javadoc 9m 39s There were no new javadoc warning messages.
          +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings.
          +1 checkstyle 0m 38s There were no new checkstyle issues.
          +1 whitespace 0m 0s The patch has no lines that end in whitespace.
          +1 install 1m 35s mvn install still works.
          +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse.
          +1 findbugs 1m 6s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          +1 mapreduce tests 9m 4s Tests passed in hadoop-mapreduce-client-app.
              48m 0s  



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12740565/MAPREDUCE-5002.2.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / c7d022b
          Pre-patch Findbugs warnings https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5821/artifact/patchprocess/trunkFindbugsWarningshadoop-mapreduce-client-app.html
          hadoop-mapreduce-client-app test log https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5821/artifact/patchprocess/testrun_hadoop-mapreduce-client-app.txt
          Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5821/testReport/
          Java 1.7.0_55
          uname Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5821/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment -1 pre-patch 17m 9s Pre-patch trunk has 1 extant Findbugs (version 3.0.0) warnings. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 javac 7m 48s There were no new javac warning messages. +1 javadoc 9m 39s There were no new javadoc warning messages. +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings. +1 checkstyle 0m 38s There were no new checkstyle issues. +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 35s mvn install still works. +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse. +1 findbugs 1m 6s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 mapreduce tests 9m 4s Tests passed in hadoop-mapreduce-client-app.     48m 0s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12740565/MAPREDUCE-5002.2.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / c7d022b Pre-patch Findbugs warnings https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5821/artifact/patchprocess/trunkFindbugsWarningshadoop-mapreduce-client-app.html hadoop-mapreduce-client-app test log https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5821/artifact/patchprocess/testrun_hadoop-mapreduce-client-app.txt Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5821/testReport/ Java 1.7.0_55 uname Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5821/console This message was automatically generated.
          Hide
          lichangleo Chang Li added a comment -

          Jason Lowe could you please help reveiw? Thanks!

          Show
          lichangleo Chang Li added a comment - Jason Lowe could you please help reveiw? Thanks!
          Hide
          jlowe Jason Lowe added a comment -

          Thanks for the patch, Chang!

          Main change of the patch looks good, just some issues with the test. I'm not a fan of having the test reach into the bowels of a private variable in the class and modify it directly. To me that's sort of an invalid setup. Instead the test should be able to accomplish the task via normal interfaces, otherwise the reported bug doesn't exist. In this case we should be able to send appropriate allocate responses to convince the original code to accidentally grant a reduce container to a map and see that the new code does not do this. It may be simpler to mock up the AM protocol directly rather than using a MockRM to get it to grant the excess containers required.

          Show
          jlowe Jason Lowe added a comment - Thanks for the patch, Chang! Main change of the patch looks good, just some issues with the test. I'm not a fan of having the test reach into the bowels of a private variable in the class and modify it directly. To me that's sort of an invalid setup. Instead the test should be able to accomplish the task via normal interfaces, otherwise the reported bug doesn't exist. In this case we should be able to send appropriate allocate responses to convince the original code to accidentally grant a reduce container to a map and see that the new code does not do this. It may be simpler to mock up the AM protocol directly rather than using a MockRM to get it to grant the excess containers required.
          Hide
          hadoopqa Hadoop QA added a comment -



          +1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 16m 40s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
          +1 javac 7m 46s There were no new javac warning messages.
          +1 javadoc 9m 58s There were no new javadoc warning messages.
          +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings.
          +1 checkstyle 0m 35s There were no new checkstyle issues.
          +1 whitespace 0m 0s The patch has no lines that end in whitespace.
          +1 install 1m 29s mvn install still works.
          +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse.
          +1 findbugs 1m 7s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          +1 mapreduce tests 9m 19s Tests passed in hadoop-mapreduce-client-app.
              47m 55s  



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12740565/MAPREDUCE-5002.2.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / d9c1fab
          hadoop-mapreduce-client-app test log https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5973/artifact/patchprocess/testrun_hadoop-mapreduce-client-app.txt
          Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5973/testReport/
          Java 1.7.0_55
          uname Linux asf901.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5973/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 pre-patch 16m 40s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 javac 7m 46s There were no new javac warning messages. +1 javadoc 9m 58s There were no new javadoc warning messages. +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings. +1 checkstyle 0m 35s There were no new checkstyle issues. +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 29s mvn install still works. +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse. +1 findbugs 1m 7s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 mapreduce tests 9m 19s Tests passed in hadoop-mapreduce-client-app.     47m 55s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12740565/MAPREDUCE-5002.2.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / d9c1fab hadoop-mapreduce-client-app test log https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5973/artifact/patchprocess/testrun_hadoop-mapreduce-client-app.txt Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5973/testReport/ Java 1.7.0_55 uname Linux asf901.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5973/console This message was automatically generated.
          Hide
          lichangleo Chang Li added a comment -

          there was a debugging print in .3 patch I forgot to delete. Upload .4 patch fix that

          Show
          lichangleo Chang Li added a comment - there was a debugging print in .3 patch I forgot to delete. Upload .4 patch fix that
          Hide
          lichangleo Chang Li added a comment -

          Thanks Jason Lowe for review! Have reworked my unit test. Please help review the updated patch. Thanks!

          Show
          lichangleo Chang Li added a comment - Thanks Jason Lowe for review! Have reworked my unit test. Please help review the updated patch. Thanks!
          Hide
          lichangleo Chang Li added a comment -

          .5 patch improve some naming and comment

          Show
          lichangleo Chang Li added a comment - .5 patch improve some naming and comment
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 16m 22s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
          +1 javac 7m 58s There were no new javac warning messages.
          +1 javadoc 10m 30s There were no new javadoc warning messages.
          +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings.
          +1 checkstyle 0m 33s There were no new checkstyle issues.
          -1 whitespace 0m 0s The patch has 3 line(s) that end in whitespace. Use git apply --whitespace=fix.
          +1 install 1m 27s mvn install still works.
          +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse.
          +1 findbugs 1m 6s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          +1 mapreduce tests 9m 20s Tests passed in hadoop-mapreduce-client-app.
              48m 18s  



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12755195/MAPREDUCE-5002.4.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 7766610
          whitespace https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5977/artifact/patchprocess/whitespace.txt
          hadoop-mapreduce-client-app test log https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5977/artifact/patchprocess/testrun_hadoop-mapreduce-client-app.txt
          Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5977/testReport/
          Java 1.7.0_55
          uname Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5977/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 16m 22s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 javac 7m 58s There were no new javac warning messages. +1 javadoc 10m 30s There were no new javadoc warning messages. +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings. +1 checkstyle 0m 33s There were no new checkstyle issues. -1 whitespace 0m 0s The patch has 3 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 install 1m 27s mvn install still works. +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse. +1 findbugs 1m 6s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 mapreduce tests 9m 20s Tests passed in hadoop-mapreduce-client-app.     48m 18s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12755195/MAPREDUCE-5002.4.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 7766610 whitespace https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5977/artifact/patchprocess/whitespace.txt hadoop-mapreduce-client-app test log https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5977/artifact/patchprocess/testrun_hadoop-mapreduce-client-app.txt Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5977/testReport/ Java 1.7.0_55 uname Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5977/console This message was automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 16m 20s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
          +1 javac 8m 0s There were no new javac warning messages.
          +1 javadoc 12m 9s There were no new javadoc warning messages.
          +1 release audit 0m 27s The applied patch does not increase the total number of release audit warnings.
          +1 checkstyle 0m 35s There were no new checkstyle issues.
          -1 whitespace 0m 0s The patch has 3 line(s) that end in whitespace. Use git apply --whitespace=fix.
          +1 install 1m 40s mvn install still works.
          +1 eclipse:eclipse 0m 36s The patch built with eclipse:eclipse.
          +1 findbugs 1m 22s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          +1 mapreduce tests 10m 46s Tests passed in hadoop-mapreduce-client-app.
              51m 58s  



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12755196/MAPREDUCE-5002.5.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 7766610
          whitespace https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5978/artifact/patchprocess/whitespace.txt
          hadoop-mapreduce-client-app test log https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5978/artifact/patchprocess/testrun_hadoop-mapreduce-client-app.txt
          Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5978/testReport/
          Java 1.7.0_55
          uname Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5978/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 16m 20s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 javac 8m 0s There were no new javac warning messages. +1 javadoc 12m 9s There were no new javadoc warning messages. +1 release audit 0m 27s The applied patch does not increase the total number of release audit warnings. +1 checkstyle 0m 35s There were no new checkstyle issues. -1 whitespace 0m 0s The patch has 3 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 install 1m 40s mvn install still works. +1 eclipse:eclipse 0m 36s The patch built with eclipse:eclipse. +1 findbugs 1m 22s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 mapreduce tests 10m 46s Tests passed in hadoop-mapreduce-client-app.     51m 58s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12755196/MAPREDUCE-5002.5.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 7766610 whitespace https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5978/artifact/patchprocess/whitespace.txt hadoop-mapreduce-client-app test log https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5978/artifact/patchprocess/testrun_hadoop-mapreduce-client-app.txt Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5978/testReport/ Java 1.7.0_55 uname Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5978/console This message was automatically generated.
          Hide
          lichangleo Chang Li added a comment -

          fix whitespace issue

          Show
          lichangleo Chang Li added a comment - fix whitespace issue
          Hide
          hadoopqa Hadoop QA added a comment -



          +1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 18m 26s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
          +1 javac 9m 2s There were no new javac warning messages.
          +1 javadoc 11m 12s There were no new javadoc warning messages.
          +1 release audit 0m 25s The applied patch does not increase the total number of release audit warnings.
          +1 checkstyle 0m 44s There were no new checkstyle issues.
          +1 whitespace 0m 0s The patch has no lines that end in whitespace.
          +1 install 1m 39s mvn install still works.
          +1 eclipse:eclipse 0m 38s The patch built with eclipse:eclipse.
          +1 findbugs 1m 16s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          +1 mapreduce tests 9m 54s Tests passed in hadoop-mapreduce-client-app.
              53m 21s  



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12755224/MAPREDUCE-5002.6.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / fbbb7ff
          hadoop-mapreduce-client-app test log https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5979/artifact/patchprocess/testrun_hadoop-mapreduce-client-app.txt
          Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5979/testReport/
          Java 1.7.0_55
          uname Linux asf902.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5979/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 pre-patch 18m 26s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 javac 9m 2s There were no new javac warning messages. +1 javadoc 11m 12s There were no new javadoc warning messages. +1 release audit 0m 25s The applied patch does not increase the total number of release audit warnings. +1 checkstyle 0m 44s There were no new checkstyle issues. +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 39s mvn install still works. +1 eclipse:eclipse 0m 38s The patch built with eclipse:eclipse. +1 findbugs 1m 16s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 mapreduce tests 9m 54s Tests passed in hadoop-mapreduce-client-app.     53m 21s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12755224/MAPREDUCE-5002.6.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / fbbb7ff hadoop-mapreduce-client-app test log https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5979/artifact/patchprocess/testrun_hadoop-mapreduce-client-app.txt Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5979/testReport/ Java 1.7.0_55 uname Linux asf902.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5979/console This message was automatically generated.
          Hide
          jlowe Jason Lowe added a comment -

          +1 latest patch lgtm. Committing this.

          Show
          jlowe Jason Lowe added a comment - +1 latest patch lgtm. Committing this.
          Hide
          jlowe Jason Lowe added a comment -

          Thanks, Chang! I committed this to trunk and branch-2.

          Show
          jlowe Jason Lowe added a comment - Thanks, Chang! I committed this to trunk and branch-2.
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #8471 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8471/)
          MAPREDUCE-5002. AM could potentially allocate a reduce container to a map attempt. Contributed by Chang Li (jlowe: rev 3f82f582e51c514cc0f052c828c2f58c6e8927ad)

          • hadoop-mapreduce-project/CHANGES.txt
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/rm/TestRMContainerAllocator.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/rm/RMContainerAllocator.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #8471 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8471/ ) MAPREDUCE-5002 . AM could potentially allocate a reduce container to a map attempt. Contributed by Chang Li (jlowe: rev 3f82f582e51c514cc0f052c828c2f58c6e8927ad) hadoop-mapreduce-project/CHANGES.txt hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/rm/TestRMContainerAllocator.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/rm/RMContainerAllocator.java
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-Yarn-trunk-Java8 #409 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/409/)
          MAPREDUCE-5002. AM could potentially allocate a reduce container to a map attempt. Contributed by Chang Li (jlowe: rev 3f82f582e51c514cc0f052c828c2f58c6e8927ad)

          • hadoop-mapreduce-project/CHANGES.txt
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/rm/TestRMContainerAllocator.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/rm/RMContainerAllocator.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Yarn-trunk-Java8 #409 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/409/ ) MAPREDUCE-5002 . AM could potentially allocate a reduce container to a map attempt. Contributed by Chang Li (jlowe: rev 3f82f582e51c514cc0f052c828c2f58c6e8927ad) hadoop-mapreduce-project/CHANGES.txt hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/rm/TestRMContainerAllocator.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/rm/RMContainerAllocator.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk #1143 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1143/)
          MAPREDUCE-5002. AM could potentially allocate a reduce container to a map attempt. Contributed by Chang Li (jlowe: rev 3f82f582e51c514cc0f052c828c2f58c6e8927ad)

          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/rm/RMContainerAllocator.java
          • hadoop-mapreduce-project/CHANGES.txt
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/rm/TestRMContainerAllocator.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk #1143 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1143/ ) MAPREDUCE-5002 . AM could potentially allocate a reduce container to a map attempt. Contributed by Chang Li (jlowe: rev 3f82f582e51c514cc0f052c828c2f58c6e8927ad) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/rm/RMContainerAllocator.java hadoop-mapreduce-project/CHANGES.txt hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/rm/TestRMContainerAllocator.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk #2349 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2349/)
          MAPREDUCE-5002. AM could potentially allocate a reduce container to a map attempt. Contributed by Chang Li (jlowe: rev 3f82f582e51c514cc0f052c828c2f58c6e8927ad)

          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/rm/RMContainerAllocator.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/rm/TestRMContainerAllocator.java
          • hadoop-mapreduce-project/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #2349 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2349/ ) MAPREDUCE-5002 . AM could potentially allocate a reduce container to a map attempt. Contributed by Chang Li (jlowe: rev 3f82f582e51c514cc0f052c828c2f58c6e8927ad) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/rm/RMContainerAllocator.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/rm/TestRMContainerAllocator.java hadoop-mapreduce-project/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #402 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/402/)
          MAPREDUCE-5002. AM could potentially allocate a reduce container to a map attempt. Contributed by Chang Li (jlowe: rev 3f82f582e51c514cc0f052c828c2f58c6e8927ad)

          • hadoop-mapreduce-project/CHANGES.txt
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/rm/RMContainerAllocator.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/rm/TestRMContainerAllocator.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #402 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/402/ ) MAPREDUCE-5002 . AM could potentially allocate a reduce container to a map attempt. Contributed by Chang Li (jlowe: rev 3f82f582e51c514cc0f052c828c2f58c6e8927ad) hadoop-mapreduce-project/CHANGES.txt hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/rm/RMContainerAllocator.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/rm/TestRMContainerAllocator.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #385 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/385/)
          MAPREDUCE-5002. AM could potentially allocate a reduce container to a map attempt. Contributed by Chang Li (jlowe: rev 3f82f582e51c514cc0f052c828c2f58c6e8927ad)

          • hadoop-mapreduce-project/CHANGES.txt
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/rm/TestRMContainerAllocator.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/rm/RMContainerAllocator.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #385 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/385/ ) MAPREDUCE-5002 . AM could potentially allocate a reduce container to a map attempt. Contributed by Chang Li (jlowe: rev 3f82f582e51c514cc0f052c828c2f58c6e8927ad) hadoop-mapreduce-project/CHANGES.txt hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/rm/TestRMContainerAllocator.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/rm/RMContainerAllocator.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk #2324 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2324/)
          MAPREDUCE-5002. AM could potentially allocate a reduce container to a map attempt. Contributed by Chang Li (jlowe: rev 3f82f582e51c514cc0f052c828c2f58c6e8927ad)

          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/rm/TestRMContainerAllocator.java
          • hadoop-mapreduce-project/CHANGES.txt
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/rm/RMContainerAllocator.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2324 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2324/ ) MAPREDUCE-5002 . AM could potentially allocate a reduce container to a map attempt. Contributed by Chang Li (jlowe: rev 3f82f582e51c514cc0f052c828c2f58c6e8927ad) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/rm/TestRMContainerAllocator.java hadoop-mapreduce-project/CHANGES.txt hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/rm/RMContainerAllocator.java

            People

            • Assignee:
              lichangleo Chang Li
              Reporter:
              jlowe Jason Lowe
            • Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development