Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-6697

Concurrent task limits should only be applied when necessary

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.7.0
    • Fix Version/s: 2.9.0, 2.7.4, 3.0.0-alpha4, 2.8.2
    • Component/s: mrv2
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      The concurrent task limit feature should only adjust the ANY portion of the AM heartbeat ask when a limit is truly necessary, otherwise extraneous containers could be allocated by the RM to the AM adding some overhead to both. Specifying a concurrent task limit that is beyond the total number of tasks in the job should be the same as asking for no limit.

      1. MAPREDUCE-6697-branch-2.7.001.patch
        7 kB
        Konstantin Shvachko
      2. MAPREDUCE-6697-branch-2.8.001.patch
        7 kB
        Nathan Roberts
      3. MAPREDUCE-6697-v1.patch
        7 kB
        Nathan Roberts

        Issue Links

          Activity

          Hide
          shv Konstantin Shvachko added a comment -

          I just committed this to branch 2.8 and branch-2.7. Thank you Nathan Roberts.

          Show
          shv Konstantin Shvachko added a comment - I just committed this to branch 2.8 and branch-2.7. Thank you Nathan Roberts .
          Hide
          shv Konstantin Shvachko added a comment -

          +1. The patch looks good for both branches.

          Show
          shv Konstantin Shvachko added a comment - +1. The patch looks good for both branches.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 11m 36s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 5m 53s branch-2.7 passed
          +1 compile 0m 18s branch-2.7 passed with JDK v1.8.0_131
          +1 compile 0m 22s branch-2.7 passed with JDK v1.7.0_131
          +1 checkstyle 0m 22s branch-2.7 passed
          +1 mvnsite 0m 27s branch-2.7 passed
          +1 mvneclipse 0m 14s branch-2.7 passed
          -1 findbugs 0m 42s hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app in branch-2.7 has 1 extant Findbugs warnings.
          +1 javadoc 0m 13s branch-2.7 passed with JDK v1.8.0_131
          +1 javadoc 0m 17s branch-2.7 passed with JDK v1.7.0_131
          +1 mvninstall 0m 20s the patch passed
          +1 compile 0m 16s the patch passed with JDK v1.8.0_131
          +1 javac 0m 16s the patch passed
          +1 compile 0m 20s the patch passed with JDK v1.7.0_131
          +1 javac 0m 20s the patch passed
          -1 checkstyle 0m 18s hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app: The patch generated 2 new + 390 unchanged - 0 fixed = 392 total (was 390)
          +1 mvnsite 0m 24s the patch passed
          +1 mvneclipse 0m 12s the patch passed
          -1 whitespace 0m 0s The patch has 2226 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
          -1 whitespace 1m 1s The patch 74 line(s) with tabs.
          +1 findbugs 0m 49s the patch passed
          +1 javadoc 0m 10s the patch passed with JDK v1.8.0_131
          +1 javadoc 0m 13s the patch passed with JDK v1.7.0_131
          +1 unit 8m 53s hadoop-mapreduce-client-app in the patch passed with JDK v1.7.0_131.
          -1 asflicense 0m 18s The patch generated 7 ASF License warnings.
          43m 12s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:67e87c9
          JIRA Issue MAPREDUCE-6697
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12876967/MAPREDUCE-6697-branch-2.7.001.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 0481734439dc 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12 13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision branch-2.7 / 187bb77
          Default Java 1.7.0_131
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_131 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_131
          findbugs v3.0.0
          findbugs https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/7026/artifact/patchprocess/branch-findbugs-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-warnings.html
          checkstyle https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/7026/artifact/patchprocess/diff-checkstyle-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app.txt
          whitespace https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/7026/artifact/patchprocess/whitespace-eol.txt
          whitespace https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/7026/artifact/patchprocess/whitespace-tabs.txt
          JDK v1.7.0_131 Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/7026/testReport/
          asflicense https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/7026/artifact/patchprocess/patch-asflicense-problems.txt
          modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app
          Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/7026/console
          Powered by Apache Yetus 0.4.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 11m 36s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 5m 53s branch-2.7 passed +1 compile 0m 18s branch-2.7 passed with JDK v1.8.0_131 +1 compile 0m 22s branch-2.7 passed with JDK v1.7.0_131 +1 checkstyle 0m 22s branch-2.7 passed +1 mvnsite 0m 27s branch-2.7 passed +1 mvneclipse 0m 14s branch-2.7 passed -1 findbugs 0m 42s hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app in branch-2.7 has 1 extant Findbugs warnings. +1 javadoc 0m 13s branch-2.7 passed with JDK v1.8.0_131 +1 javadoc 0m 17s branch-2.7 passed with JDK v1.7.0_131 +1 mvninstall 0m 20s the patch passed +1 compile 0m 16s the patch passed with JDK v1.8.0_131 +1 javac 0m 16s the patch passed +1 compile 0m 20s the patch passed with JDK v1.7.0_131 +1 javac 0m 20s the patch passed -1 checkstyle 0m 18s hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app: The patch generated 2 new + 390 unchanged - 0 fixed = 392 total (was 390) +1 mvnsite 0m 24s the patch passed +1 mvneclipse 0m 12s the patch passed -1 whitespace 0m 0s The patch has 2226 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply -1 whitespace 1m 1s The patch 74 line(s) with tabs. +1 findbugs 0m 49s the patch passed +1 javadoc 0m 10s the patch passed with JDK v1.8.0_131 +1 javadoc 0m 13s the patch passed with JDK v1.7.0_131 +1 unit 8m 53s hadoop-mapreduce-client-app in the patch passed with JDK v1.7.0_131. -1 asflicense 0m 18s The patch generated 7 ASF License warnings. 43m 12s Subsystem Report/Notes Docker Image:yetus/hadoop:67e87c9 JIRA Issue MAPREDUCE-6697 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12876967/MAPREDUCE-6697-branch-2.7.001.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 0481734439dc 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12 13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision branch-2.7 / 187bb77 Default Java 1.7.0_131 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_131 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_131 findbugs v3.0.0 findbugs https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/7026/artifact/patchprocess/branch-findbugs-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-warnings.html checkstyle https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/7026/artifact/patchprocess/diff-checkstyle-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app.txt whitespace https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/7026/artifact/patchprocess/whitespace-eol.txt whitespace https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/7026/artifact/patchprocess/whitespace-tabs.txt JDK v1.7.0_131 Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/7026/testReport/ asflicense https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/7026/artifact/patchprocess/patch-asflicense-problems.txt modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/7026/console Powered by Apache Yetus 0.4.0 http://yetus.apache.org This message was automatically generated.
          Hide
          shv Konstantin Shvachko added a comment -

          The patch applies cleanly to 2.7, and TestRMContainerAllocator passes.
          Let's just reattach the same patch for a full Jenkins run on branch-2.7.

          Show
          shv Konstantin Shvachko added a comment - The patch applies cleanly to 2.7, and TestRMContainerAllocator passes. Let's just reattach the same patch for a full Jenkins run on branch-2.7.
          Hide
          zhouyejoe Ye Zhou added a comment -

          Nathan RobertsThanks.
          Hi, Konstantin Shvachko. Please check the 2.7 patch.

          Show
          zhouyejoe Ye Zhou added a comment - Nathan Roberts Thanks. Hi, Konstantin Shvachko . Please check the 2.7 patch.
          Hide
          hadoopqa Hadoop QA added a comment -
          +1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 20s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 9m 6s branch-2.8 passed
          +1 compile 0m 20s branch-2.8 passed with JDK v1.8.0_131
          +1 compile 0m 26s branch-2.8 passed with JDK v1.7.0_131
          +1 checkstyle 0m 18s branch-2.8 passed
          +1 mvnsite 0m 30s branch-2.8 passed
          +1 mvneclipse 0m 18s branch-2.8 passed
          +1 findbugs 0m 48s branch-2.8 passed
          +1 javadoc 0m 16s branch-2.8 passed with JDK v1.8.0_131
          +1 javadoc 0m 18s branch-2.8 passed with JDK v1.7.0_131
          +1 mvninstall 0m 24s the patch passed
          +1 compile 0m 19s the patch passed with JDK v1.8.0_131
          +1 javac 0m 19s the patch passed
          +1 compile 0m 22s the patch passed with JDK v1.7.0_131
          +1 javac 0m 22s the patch passed
          +1 checkstyle 0m 15s the patch passed
          +1 mvnsite 0m 26s the patch passed
          +1 mvneclipse 0m 13s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 0m 57s the patch passed
          +1 javadoc 0m 12s the patch passed with JDK v1.8.0_131
          +1 javadoc 0m 16s the patch passed with JDK v1.7.0_131
          +1 unit 10m 2s hadoop-mapreduce-client-app in the patch passed with JDK v1.7.0_131.
          +1 asflicense 0m 17s The patch does not generate ASF License warnings.
          36m 43s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:d946387
          JIRA Issue MAPREDUCE-6697
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12876863/MAPREDUCE-6697-branch-2.8.001.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 09b34b6a11b8 3.13.0-117-generic #164-Ubuntu SMP Fri Apr 7 11:05:26 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision branch-2.8 / ebb66d2
          Default Java 1.7.0_131
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_131 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_131
          findbugs v3.0.0
          JDK v1.7.0_131 Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/7025/testReport/
          modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app
          Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/7025/console
          Powered by Apache Yetus 0.4.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 20s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 9m 6s branch-2.8 passed +1 compile 0m 20s branch-2.8 passed with JDK v1.8.0_131 +1 compile 0m 26s branch-2.8 passed with JDK v1.7.0_131 +1 checkstyle 0m 18s branch-2.8 passed +1 mvnsite 0m 30s branch-2.8 passed +1 mvneclipse 0m 18s branch-2.8 passed +1 findbugs 0m 48s branch-2.8 passed +1 javadoc 0m 16s branch-2.8 passed with JDK v1.8.0_131 +1 javadoc 0m 18s branch-2.8 passed with JDK v1.7.0_131 +1 mvninstall 0m 24s the patch passed +1 compile 0m 19s the patch passed with JDK v1.8.0_131 +1 javac 0m 19s the patch passed +1 compile 0m 22s the patch passed with JDK v1.7.0_131 +1 javac 0m 22s the patch passed +1 checkstyle 0m 15s the patch passed +1 mvnsite 0m 26s the patch passed +1 mvneclipse 0m 13s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 0m 57s the patch passed +1 javadoc 0m 12s the patch passed with JDK v1.8.0_131 +1 javadoc 0m 16s the patch passed with JDK v1.7.0_131 +1 unit 10m 2s hadoop-mapreduce-client-app in the patch passed with JDK v1.7.0_131. +1 asflicense 0m 17s The patch does not generate ASF License warnings. 36m 43s Subsystem Report/Notes Docker Image:yetus/hadoop:d946387 JIRA Issue MAPREDUCE-6697 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12876863/MAPREDUCE-6697-branch-2.8.001.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 09b34b6a11b8 3.13.0-117-generic #164-Ubuntu SMP Fri Apr 7 11:05:26 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision branch-2.8 / ebb66d2 Default Java 1.7.0_131 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_131 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_131 findbugs v3.0.0 JDK v1.7.0_131 Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/7025/testReport/ modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/7025/console Powered by Apache Yetus 0.4.0 http://yetus.apache.org This message was automatically generated.
          Hide
          nroberts Nathan Roberts added a comment -

          Applies to branch-2.8 and branch-2.7

          Show
          nroberts Nathan Roberts added a comment - Applies to branch-2.8 and branch-2.7
          Hide
          nroberts Nathan Roberts added a comment -

          Adding patch for 2.8/2.7

          Show
          nroberts Nathan Roberts added a comment - Adding patch for 2.8/2.7
          Hide
          zhouyejoe Ye Zhou added a comment -

          Hi, Nathan Roberts. Would you also create a patch for 2.7 branch? This should be helpful for the ongoing 2.7.4 release. Thanks.

          Show
          zhouyejoe Ye Zhou added a comment - Hi, Nathan Roberts . Would you also create a patch for 2.7 branch? This should be helpful for the ongoing 2.7.4 release. Thanks.
          Hide
          andrew.wang Andrew Wang added a comment -

          I'm going to close this so I can roll a release, please re-open if you need a Jenkins run after.

          Show
          andrew.wang Andrew Wang added a comment - I'm going to close this so I can roll a release, please re-open if you need a Jenkins run after.
          Hide
          ajisakaa Akira Ajisaka added a comment -

          Committed this to trunk and branch-2. Hi Nathan Roberts, would you create a patch for branch-2.8? This patch applies to branch-2.8 but compilation error occurs.

          [INFO] -------------------------------------------------------------
          [ERROR] COMPILATION ERROR : 
          [INFO] -------------------------------------------------------------
          [ERROR] /Users/ajisaka/git/hadoop/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/rm/TestRMContainerAllocator.java:[2759,24] cannot find symbol
            symbol:   method getInstance()
            location: class org.apache.hadoop.yarn.util.SystemClock
          [INFO] 1 error
          
          Show
          ajisakaa Akira Ajisaka added a comment - Committed this to trunk and branch-2. Hi Nathan Roberts , would you create a patch for branch-2.8? This patch applies to branch-2.8 but compilation error occurs. [INFO] ------------------------------------------------------------- [ERROR] COMPILATION ERROR : [INFO] ------------------------------------------------------------- [ERROR] /Users/ajisaka/git/hadoop/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/rm/TestRMContainerAllocator.java:[2759,24] cannot find symbol symbol: method getInstance() location: class org.apache.hadoop.yarn.util.SystemClock [INFO] 1 error
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11938 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11938/)
          MAPREDUCE-6697. Concurrent task limits should only be applied when (aajisaka: rev a5c0476a990ec1e7eb34ce2462a45aa52cc1350d)

          • (edit) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/rm/RMContainerAllocator.java
          • (edit) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/rm/TestRMContainerAllocator.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11938 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11938/ ) MAPREDUCE-6697 . Concurrent task limits should only be applied when (aajisaka: rev a5c0476a990ec1e7eb34ce2462a45aa52cc1350d) (edit) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/rm/RMContainerAllocator.java (edit) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/rm/TestRMContainerAllocator.java
          Hide
          ajisakaa Akira Ajisaka added a comment -

          Thank you, Nathan. It makes sense. I'm +1 for the patch.

          Show
          ajisakaa Akira Ajisaka added a comment - Thank you, Nathan. It makes sense. I'm +1 for the patch.
          Hide
          nroberts Nathan Roberts added a comment -

          @aajisaka Thanks for taking a look. I don't believe there was a specific reason for the 3 invocations. I think it was just to make sure a few heartbeats were simulated.

          Show
          nroberts Nathan Roberts added a comment - @aajisaka Thanks for taking a look. I don't believe there was a specific reason for the 3 invocations. I think it was just to make sure a few heartbeats were simulated.
          Hide
          hadoopqa Hadoop QA added a comment -
          +1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 15s Docker mode activated.
          +1 @author 0m 1s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 12m 35s trunk passed
          +1 compile 0m 22s trunk passed
          +1 checkstyle 0m 15s trunk passed
          +1 mvnsite 0m 27s trunk passed
          +1 mvneclipse 0m 18s trunk passed
          +1 findbugs 0m 34s trunk passed
          +1 javadoc 0m 13s trunk passed
          +1 mvninstall 0m 21s the patch passed
          +1 compile 0m 21s the patch passed
          +1 javac 0m 21s the patch passed
          +1 checkstyle 0m 12s the patch passed
          +1 mvnsite 0m 23s the patch passed
          +1 mvneclipse 0m 14s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 0m 39s the patch passed
          +1 javadoc 0m 10s the patch passed
          +1 unit 9m 7s hadoop-mapreduce-client-app in the patch passed.
          +1 asflicense 0m 18s The patch does not generate ASF License warnings.
          27m 19s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:14b5c93
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12810463/MAPREDUCE-6697-v1.patch
          JIRA Issue MAPREDUCE-6697
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 20048d93b059 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12 13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / e86eef9
          Default Java 1.8.0_131
          findbugs v3.1.0-RC1
          Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6992/testReport/
          modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app
          Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6992/console
          Powered by Apache Yetus 0.3.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 15s Docker mode activated. +1 @author 0m 1s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 12m 35s trunk passed +1 compile 0m 22s trunk passed +1 checkstyle 0m 15s trunk passed +1 mvnsite 0m 27s trunk passed +1 mvneclipse 0m 18s trunk passed +1 findbugs 0m 34s trunk passed +1 javadoc 0m 13s trunk passed +1 mvninstall 0m 21s the patch passed +1 compile 0m 21s the patch passed +1 javac 0m 21s the patch passed +1 checkstyle 0m 12s the patch passed +1 mvnsite 0m 23s the patch passed +1 mvneclipse 0m 14s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 0m 39s the patch passed +1 javadoc 0m 10s the patch passed +1 unit 9m 7s hadoop-mapreduce-client-app in the patch passed. +1 asflicense 0m 18s The patch does not generate ASF License warnings. 27m 19s Subsystem Report/Notes Docker Image:yetus/hadoop:14b5c93 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12810463/MAPREDUCE-6697-v1.patch JIRA Issue MAPREDUCE-6697 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 20048d93b059 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12 13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / e86eef9 Default Java 1.8.0_131 findbugs v3.1.0-RC1 Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6992/testReport/ modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6992/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -
          +1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 13s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 12m 31s trunk passed
          +1 compile 0m 22s trunk passed
          +1 checkstyle 0m 13s trunk passed
          +1 mvnsite 0m 23s trunk passed
          +1 mvneclipse 0m 15s trunk passed
          +1 findbugs 0m 34s trunk passed
          +1 javadoc 0m 13s trunk passed
          +1 mvninstall 0m 20s the patch passed
          +1 compile 0m 19s the patch passed
          +1 javac 0m 19s the patch passed
          +1 checkstyle 0m 11s the patch passed
          +1 mvnsite 0m 20s the patch passed
          +1 mvneclipse 0m 14s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 0m 39s the patch passed
          +1 javadoc 0m 10s the patch passed
          +1 unit 9m 0s hadoop-mapreduce-client-app in the patch passed.
          +1 asflicense 0m 17s The patch does not generate ASF License warnings.
          26m 49s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:14b5c93
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12810463/MAPREDUCE-6697-v1.patch
          JIRA Issue MAPREDUCE-6697
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux cd92358638a3 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12 13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / e86eef9
          Default Java 1.8.0_131
          findbugs v3.1.0-RC1
          Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6991/testReport/
          modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app
          Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6991/console
          Powered by Apache Yetus 0.3.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 13s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 12m 31s trunk passed +1 compile 0m 22s trunk passed +1 checkstyle 0m 13s trunk passed +1 mvnsite 0m 23s trunk passed +1 mvneclipse 0m 15s trunk passed +1 findbugs 0m 34s trunk passed +1 javadoc 0m 13s trunk passed +1 mvninstall 0m 20s the patch passed +1 compile 0m 19s the patch passed +1 javac 0m 19s the patch passed +1 checkstyle 0m 11s the patch passed +1 mvnsite 0m 20s the patch passed +1 mvneclipse 0m 14s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 0m 39s the patch passed +1 javadoc 0m 10s the patch passed +1 unit 9m 0s hadoop-mapreduce-client-app in the patch passed. +1 asflicense 0m 17s The patch does not generate ASF License warnings. 26m 49s Subsystem Report/Notes Docker Image:yetus/hadoop:14b5c93 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12810463/MAPREDUCE-6697-v1.patch JIRA Issue MAPREDUCE-6697 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux cd92358638a3 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12 13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / e86eef9 Default Java 1.8.0_131 findbugs v3.1.0-RC1 Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6991/testReport/ modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6991/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
          Hide
          ajisakaa Akira Ajisaka added a comment -

          Mostly looks good to me. I have one question in the test code:

              allocator.schedule();
              allocator.schedule();
              allocator.schedule();
          

          why the method is called 3 times?

          Show
          ajisakaa Akira Ajisaka added a comment - Mostly looks good to me. I have one question in the test code: allocator.schedule(); allocator.schedule(); allocator.schedule(); why the method is called 3 times?
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 18s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 6m 31s trunk passed
          +1 compile 0m 19s trunk passed
          +1 checkstyle 0m 18s trunk passed
          +1 mvnsite 0m 23s trunk passed
          +1 mvneclipse 0m 12s trunk passed
          +1 findbugs 0m 33s trunk passed
          +1 javadoc 0m 14s trunk passed
          +1 mvninstall 0m 19s the patch passed
          +1 compile 0m 16s the patch passed
          +1 javac 0m 16s the patch passed
          -1 checkstyle 0m 15s hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app: The patch generated 8 new + 225 unchanged - 0 fixed = 233 total (was 225)
          +1 mvnsite 0m 22s the patch passed
          +1 mvneclipse 0m 8s the patch passed
          +1 whitespace 0m 1s The patch has no whitespace issues.
          +1 findbugs 0m 39s the patch passed
          +1 javadoc 0m 12s the patch passed
          +1 unit 8m 50s hadoop-mapreduce-client-app in the patch passed.
          +1 asflicense 0m 18s The patch does not generate ASF License warnings.
          20m 44s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:2c91fd8
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12810463/MAPREDUCE-6697-v1.patch
          JIRA Issue MAPREDUCE-6697
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 8a837cb6fcc8 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 8e8cb4c
          Default Java 1.8.0_91
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6556/artifact/patchprocess/diff-checkstyle-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app.txt
          Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6556/testReport/
          modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app
          Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6556/console
          Powered by Apache Yetus 0.3.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 18s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 6m 31s trunk passed +1 compile 0m 19s trunk passed +1 checkstyle 0m 18s trunk passed +1 mvnsite 0m 23s trunk passed +1 mvneclipse 0m 12s trunk passed +1 findbugs 0m 33s trunk passed +1 javadoc 0m 14s trunk passed +1 mvninstall 0m 19s the patch passed +1 compile 0m 16s the patch passed +1 javac 0m 16s the patch passed -1 checkstyle 0m 15s hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app: The patch generated 8 new + 225 unchanged - 0 fixed = 233 total (was 225) +1 mvnsite 0m 22s the patch passed +1 mvneclipse 0m 8s the patch passed +1 whitespace 0m 1s The patch has no whitespace issues. +1 findbugs 0m 39s the patch passed +1 javadoc 0m 12s the patch passed +1 unit 8m 50s hadoop-mapreduce-client-app in the patch passed. +1 asflicense 0m 18s The patch does not generate ASF License warnings. 20m 44s Subsystem Report/Notes Docker Image:yetus/hadoop:2c91fd8 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12810463/MAPREDUCE-6697-v1.patch JIRA Issue MAPREDUCE-6697 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 8a837cb6fcc8 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 8e8cb4c Default Java 1.8.0_91 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6556/artifact/patchprocess/diff-checkstyle-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app.txt Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6556/testReport/ modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6556/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
          Hide
          nroberts Nathan Roberts added a comment -

          Attaching patch. Ideally it wouldn't call getTotalMaps/Reduces() on each heartbeat but it keeps the patch small and it looks to be a pretty cheap call.

          Show
          nroberts Nathan Roberts added a comment - Attaching patch. Ideally it wouldn't call getTotalMaps/Reduces() on each heartbeat but it keeps the patch small and it looks to be a pretty cheap call.

            People

            • Assignee:
              nroberts Nathan Roberts
              Reporter:
              jlowe Jason Lowe
            • Votes:
              0 Vote for this issue
              Watchers:
              14 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development