Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-4440

FSAppAttempt#getAllowedLocalityLevelByTime should init the lastScheduler time

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.7.1
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: fairscheduler
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      It seems there is a bug on FSAppAttempt#getAllowedLocalityLevelByTime method

      // default level is NODE_LOCAL
          if (! allowedLocalityLevel.containsKey(priority)) {
            allowedLocalityLevel.put(priority, NodeType.NODE_LOCAL);
            return NodeType.NODE_LOCAL;
          }
      

      If you first invoke this method, it doesn't init time in lastScheduledContainer and this will lead to execute these code for next invokation:

          // check waiting time
          long waitTime = currentTimeMs;
          if (lastScheduledContainer.containsKey(priority)) {
            waitTime -= lastScheduledContainer.get(priority);
          } else {
            waitTime -= getStartTime();
          }
      

      the waitTime will subtract to FsApp startTime, and this will be easily more than the delay time and allowedLocality degrade. Because FsApp startTime will be start earlier than currentTimeMs. So we should add the initial time of priority to prevent comparing with FsApp startTime and allowedLocalityLevel degrade. And this problem will have more negative influence for small-jobs. The YARN-4399 also discuss some problem in aspect of locality.

      1. YARN-4440.001.patch
        5 kB
        Yiqun Lin
      2. YARN-4440.002.patch
        5 kB
        Yiqun Lin
      3. YARN-4440.003.patch
        5 kB
        Yiqun Lin

        Issue Links

          Activity

          Hide
          linyiqun Yiqun Lin added a comment -

          I attach a initial patch and add the init time in lastScheduledContainer. Thanks reviewing!

          Show
          linyiqun Yiqun Lin added a comment - I attach a initial patch and add the init time in lastScheduledContainer . Thanks reviewing!
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 0s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 8m 1s trunk passed
          +1 compile 0m 29s trunk passed with JDK v1.8.0_66
          +1 compile 0m 32s trunk passed with JDK v1.7.0_91
          +1 checkstyle 0m 13s trunk passed
          +1 mvnsite 0m 39s trunk passed
          +1 mvneclipse 0m 16s trunk passed
          +1 findbugs 1m 14s trunk passed
          -1 javadoc 0m 22s hadoop-yarn-server-resourcemanager in trunk failed with JDK v1.8.0_66.
          +1 javadoc 0m 29s trunk passed with JDK v1.7.0_91
          -1 mvninstall 0m 31s hadoop-yarn-server-resourcemanager in the patch failed.
          -1 compile 0m 29s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_66.
          -1 javac 0m 29s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_66.
          -1 compile 0m 31s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_91.
          -1 javac 0m 31s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_91.
          +1 checkstyle 0m 14s the patch passed
          -1 mvnsite 0m 32s hadoop-yarn-server-resourcemanager in the patch failed.
          +1 mvneclipse 0m 15s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          -1 findbugs 0m 31s hadoop-yarn-server-resourcemanager in the patch failed.
          -1 javadoc 0m 24s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_66.
          +1 javadoc 0m 29s the patch passed with JDK v1.7.0_91
          -1 unit 0m 27s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_66.
          -1 unit 0m 31s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_91.
          +1 asflicense 0m 22s Patch does not generate ASF License warnings.
          18m 32s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:0ca8df7
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12776819/YARN-4440.001.patch
          JIRA Issue YARN-4440
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux e7e33fec9f9b 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / f910e4f
          findbugs v3.0.0
          javadoc https://builds.apache.org/job/PreCommit-YARN-Build/9926/artifact/patchprocess/branch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_66.txt
          mvninstall https://builds.apache.org/job/PreCommit-YARN-Build/9926/artifact/patchprocess/patch-mvninstall-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
          compile https://builds.apache.org/job/PreCommit-YARN-Build/9926/artifact/patchprocess/patch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_66.txt
          javac https://builds.apache.org/job/PreCommit-YARN-Build/9926/artifact/patchprocess/patch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_66.txt
          compile https://builds.apache.org/job/PreCommit-YARN-Build/9926/artifact/patchprocess/patch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_91.txt
          javac https://builds.apache.org/job/PreCommit-YARN-Build/9926/artifact/patchprocess/patch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_91.txt
          mvnsite https://builds.apache.org/job/PreCommit-YARN-Build/9926/artifact/patchprocess/patch-mvnsite-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
          findbugs https://builds.apache.org/job/PreCommit-YARN-Build/9926/artifact/patchprocess/patch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
          javadoc https://builds.apache.org/job/PreCommit-YARN-Build/9926/artifact/patchprocess/patch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_66.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/9926/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_66.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/9926/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_91.txt
          JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/9926/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
          Max memory used 75MB
          Powered by Apache Yetus http://yetus.apache.org
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/9926/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 8m 1s trunk passed +1 compile 0m 29s trunk passed with JDK v1.8.0_66 +1 compile 0m 32s trunk passed with JDK v1.7.0_91 +1 checkstyle 0m 13s trunk passed +1 mvnsite 0m 39s trunk passed +1 mvneclipse 0m 16s trunk passed +1 findbugs 1m 14s trunk passed -1 javadoc 0m 22s hadoop-yarn-server-resourcemanager in trunk failed with JDK v1.8.0_66. +1 javadoc 0m 29s trunk passed with JDK v1.7.0_91 -1 mvninstall 0m 31s hadoop-yarn-server-resourcemanager in the patch failed. -1 compile 0m 29s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_66. -1 javac 0m 29s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_66. -1 compile 0m 31s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_91. -1 javac 0m 31s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_91. +1 checkstyle 0m 14s the patch passed -1 mvnsite 0m 32s hadoop-yarn-server-resourcemanager in the patch failed. +1 mvneclipse 0m 15s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. -1 findbugs 0m 31s hadoop-yarn-server-resourcemanager in the patch failed. -1 javadoc 0m 24s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_66. +1 javadoc 0m 29s the patch passed with JDK v1.7.0_91 -1 unit 0m 27s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_66. -1 unit 0m 31s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_91. +1 asflicense 0m 22s Patch does not generate ASF License warnings. 18m 32s Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12776819/YARN-4440.001.patch JIRA Issue YARN-4440 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux e7e33fec9f9b 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / f910e4f findbugs v3.0.0 javadoc https://builds.apache.org/job/PreCommit-YARN-Build/9926/artifact/patchprocess/branch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_66.txt mvninstall https://builds.apache.org/job/PreCommit-YARN-Build/9926/artifact/patchprocess/patch-mvninstall-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt compile https://builds.apache.org/job/PreCommit-YARN-Build/9926/artifact/patchprocess/patch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_66.txt javac https://builds.apache.org/job/PreCommit-YARN-Build/9926/artifact/patchprocess/patch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_66.txt compile https://builds.apache.org/job/PreCommit-YARN-Build/9926/artifact/patchprocess/patch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_91.txt javac https://builds.apache.org/job/PreCommit-YARN-Build/9926/artifact/patchprocess/patch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_91.txt mvnsite https://builds.apache.org/job/PreCommit-YARN-Build/9926/artifact/patchprocess/patch-mvnsite-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt findbugs https://builds.apache.org/job/PreCommit-YARN-Build/9926/artifact/patchprocess/patch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt javadoc https://builds.apache.org/job/PreCommit-YARN-Build/9926/artifact/patchprocess/patch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_66.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/9926/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_66.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/9926/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_91.txt JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/9926/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager Max memory used 75MB Powered by Apache Yetus http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-YARN-Build/9926/console This message was automatically generated.
          Hide
          linyiqun Yiqun Lin added a comment -

          Resolve the remained error.

          Show
          linyiqun Yiqun Lin added a comment - Resolve the remained error.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 0s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 10m 17s trunk passed
          +1 compile 0m 48s trunk passed with JDK v1.8.0_66
          +1 compile 0m 41s trunk passed with JDK v1.7.0_91
          +1 checkstyle 0m 16s trunk passed
          +1 mvnsite 0m 47s trunk passed
          +1 mvneclipse 0m 19s trunk passed
          +1 findbugs 1m 35s trunk passed
          -1 javadoc 0m 35s hadoop-yarn-server-resourcemanager in trunk failed with JDK v1.8.0_66.
          +1 javadoc 0m 36s trunk passed with JDK v1.7.0_91
          -1 mvninstall 0m 26s hadoop-yarn-server-resourcemanager in the patch failed.
          -1 compile 0m 29s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_66.
          -1 javac 0m 29s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_66.
          -1 compile 0m 27s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_91.
          -1 javac 0m 27s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_91.
          +1 checkstyle 0m 17s the patch passed
          -1 mvnsite 0m 27s hadoop-yarn-server-resourcemanager in the patch failed.
          +1 mvneclipse 0m 18s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          -1 findbugs 0m 26s hadoop-yarn-server-resourcemanager in the patch failed.
          -1 javadoc 0m 36s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_66.
          +1 javadoc 0m 36s the patch passed with JDK v1.7.0_91
          -1 unit 0m 29s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_66.
          -1 unit 0m 25s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_91.
          +1 asflicense 0m 28s Patch does not generate ASF License warnings.
          22m 37s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:0ca8df7
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12776952/YARN-4440.002.patch
          JIRA Issue YARN-4440
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 159ea5fc3051 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / a483017
          findbugs v3.0.0
          javadoc https://builds.apache.org/job/PreCommit-YARN-Build/9936/artifact/patchprocess/branch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_66.txt
          mvninstall https://builds.apache.org/job/PreCommit-YARN-Build/9936/artifact/patchprocess/patch-mvninstall-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
          compile https://builds.apache.org/job/PreCommit-YARN-Build/9936/artifact/patchprocess/patch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_66.txt
          javac https://builds.apache.org/job/PreCommit-YARN-Build/9936/artifact/patchprocess/patch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_66.txt
          compile https://builds.apache.org/job/PreCommit-YARN-Build/9936/artifact/patchprocess/patch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_91.txt
          javac https://builds.apache.org/job/PreCommit-YARN-Build/9936/artifact/patchprocess/patch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_91.txt
          mvnsite https://builds.apache.org/job/PreCommit-YARN-Build/9936/artifact/patchprocess/patch-mvnsite-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
          findbugs https://builds.apache.org/job/PreCommit-YARN-Build/9936/artifact/patchprocess/patch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
          javadoc https://builds.apache.org/job/PreCommit-YARN-Build/9936/artifact/patchprocess/patch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_66.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/9936/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_66.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/9936/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_91.txt
          JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/9936/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
          Max memory used 76MB
          Powered by Apache Yetus 0.1.0-SNAPSHOT http://yetus.apache.org
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/9936/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 10m 17s trunk passed +1 compile 0m 48s trunk passed with JDK v1.8.0_66 +1 compile 0m 41s trunk passed with JDK v1.7.0_91 +1 checkstyle 0m 16s trunk passed +1 mvnsite 0m 47s trunk passed +1 mvneclipse 0m 19s trunk passed +1 findbugs 1m 35s trunk passed -1 javadoc 0m 35s hadoop-yarn-server-resourcemanager in trunk failed with JDK v1.8.0_66. +1 javadoc 0m 36s trunk passed with JDK v1.7.0_91 -1 mvninstall 0m 26s hadoop-yarn-server-resourcemanager in the patch failed. -1 compile 0m 29s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_66. -1 javac 0m 29s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_66. -1 compile 0m 27s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_91. -1 javac 0m 27s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_91. +1 checkstyle 0m 17s the patch passed -1 mvnsite 0m 27s hadoop-yarn-server-resourcemanager in the patch failed. +1 mvneclipse 0m 18s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. -1 findbugs 0m 26s hadoop-yarn-server-resourcemanager in the patch failed. -1 javadoc 0m 36s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_66. +1 javadoc 0m 36s the patch passed with JDK v1.7.0_91 -1 unit 0m 29s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_66. -1 unit 0m 25s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_91. +1 asflicense 0m 28s Patch does not generate ASF License warnings. 22m 37s Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12776952/YARN-4440.002.patch JIRA Issue YARN-4440 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 159ea5fc3051 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / a483017 findbugs v3.0.0 javadoc https://builds.apache.org/job/PreCommit-YARN-Build/9936/artifact/patchprocess/branch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_66.txt mvninstall https://builds.apache.org/job/PreCommit-YARN-Build/9936/artifact/patchprocess/patch-mvninstall-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt compile https://builds.apache.org/job/PreCommit-YARN-Build/9936/artifact/patchprocess/patch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_66.txt javac https://builds.apache.org/job/PreCommit-YARN-Build/9936/artifact/patchprocess/patch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_66.txt compile https://builds.apache.org/job/PreCommit-YARN-Build/9936/artifact/patchprocess/patch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_91.txt javac https://builds.apache.org/job/PreCommit-YARN-Build/9936/artifact/patchprocess/patch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_91.txt mvnsite https://builds.apache.org/job/PreCommit-YARN-Build/9936/artifact/patchprocess/patch-mvnsite-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt findbugs https://builds.apache.org/job/PreCommit-YARN-Build/9936/artifact/patchprocess/patch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt javadoc https://builds.apache.org/job/PreCommit-YARN-Build/9936/artifact/patchprocess/patch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_66.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/9936/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_66.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/9936/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_91.txt JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/9936/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager Max memory used 76MB Powered by Apache Yetus 0.1.0-SNAPSHOT http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-YARN-Build/9936/console This message was automatically generated.
          Hide
          linyiqun Yiqun Lin added a comment -

          Modify the error.

          Show
          linyiqun Yiqun Lin added a comment - Modify the error.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 0s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 8m 15s trunk passed
          +1 compile 0m 31s trunk passed with JDK v1.8.0_66
          +1 compile 0m 32s trunk passed with JDK v1.7.0_91
          +1 checkstyle 0m 13s trunk passed
          +1 mvnsite 0m 39s trunk passed
          +1 mvneclipse 0m 16s trunk passed
          +1 findbugs 1m 16s trunk passed
          -1 javadoc 0m 23s hadoop-yarn-server-resourcemanager in trunk failed with JDK v1.8.0_66.
          +1 javadoc 0m 29s trunk passed with JDK v1.7.0_91
          +1 mvninstall 0m 35s the patch passed
          +1 compile 0m 31s the patch passed with JDK v1.8.0_66
          +1 javac 0m 31s the patch passed
          +1 compile 0m 32s the patch passed with JDK v1.7.0_91
          +1 javac 0m 32s the patch passed
          +1 checkstyle 0m 14s the patch passed
          +1 mvnsite 0m 38s the patch passed
          +1 mvneclipse 0m 16s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 findbugs 1m 29s the patch passed
          -1 javadoc 0m 24s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_66.
          +1 javadoc 0m 29s the patch passed with JDK v1.7.0_91
          -1 unit 65m 6s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_66.
          -1 unit 67m 10s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_91.
          +1 asflicense 0m 24s Patch does not generate ASF License warnings.
          151m 27s



          Reason Tests
          JDK v1.8.0_66 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens
            hadoop.yarn.server.resourcemanager.TestAMAuthorization
          JDK v1.7.0_91 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens
            hadoop.yarn.server.resourcemanager.TestAMAuthorization



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:0ca8df7
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12777261/YARN-4440.003.patch
          JIRA Issue YARN-4440
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux ee3e30ecf1b8 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / bf5295b
          findbugs v3.0.0
          javadoc https://builds.apache.org/job/PreCommit-YARN-Build/9944/artifact/patchprocess/branch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_66.txt
          javadoc https://builds.apache.org/job/PreCommit-YARN-Build/9944/artifact/patchprocess/patch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_66.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/9944/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_66.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/9944/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_91.txt
          unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/9944/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-YARN-Build/9944/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_91.txt
          JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/9944/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
          Max memory used 76MB
          Powered by Apache Yetus 0.1.0-SNAPSHOT http://yetus.apache.org
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/9944/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 8m 15s trunk passed +1 compile 0m 31s trunk passed with JDK v1.8.0_66 +1 compile 0m 32s trunk passed with JDK v1.7.0_91 +1 checkstyle 0m 13s trunk passed +1 mvnsite 0m 39s trunk passed +1 mvneclipse 0m 16s trunk passed +1 findbugs 1m 16s trunk passed -1 javadoc 0m 23s hadoop-yarn-server-resourcemanager in trunk failed with JDK v1.8.0_66. +1 javadoc 0m 29s trunk passed with JDK v1.7.0_91 +1 mvninstall 0m 35s the patch passed +1 compile 0m 31s the patch passed with JDK v1.8.0_66 +1 javac 0m 31s the patch passed +1 compile 0m 32s the patch passed with JDK v1.7.0_91 +1 javac 0m 32s the patch passed +1 checkstyle 0m 14s the patch passed +1 mvnsite 0m 38s the patch passed +1 mvneclipse 0m 16s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 1m 29s the patch passed -1 javadoc 0m 24s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_66. +1 javadoc 0m 29s the patch passed with JDK v1.7.0_91 -1 unit 65m 6s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_66. -1 unit 67m 10s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_91. +1 asflicense 0m 24s Patch does not generate ASF License warnings. 151m 27s Reason Tests JDK v1.8.0_66 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens   hadoop.yarn.server.resourcemanager.TestAMAuthorization JDK v1.7.0_91 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens   hadoop.yarn.server.resourcemanager.TestAMAuthorization Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12777261/YARN-4440.003.patch JIRA Issue YARN-4440 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux ee3e30ecf1b8 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / bf5295b findbugs v3.0.0 javadoc https://builds.apache.org/job/PreCommit-YARN-Build/9944/artifact/patchprocess/branch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_66.txt javadoc https://builds.apache.org/job/PreCommit-YARN-Build/9944/artifact/patchprocess/patch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_66.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/9944/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_66.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/9944/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_91.txt unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/9944/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-YARN-Build/9944/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_91.txt JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/9944/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager Max memory used 76MB Powered by Apache Yetus 0.1.0-SNAPSHOT http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-YARN-Build/9944/console This message was automatically generated.
          Hide
          zxu zhihai xu added a comment -

          Good catch! thanks for working on this issue Yiqun Lin!
          +1 for the latest patch, The test failures are not related to the patch, These failures were already reported at YARN-4318 and YARN-4306.
          Will commit it tomorrow if no one objects.

          Show
          zxu zhihai xu added a comment - Good catch! thanks for working on this issue Yiqun Lin ! +1 for the latest patch, The test failures are not related to the patch, These failures were already reported at YARN-4318 and YARN-4306 . Will commit it tomorrow if no one objects.
          Hide
          zxu zhihai xu added a comment -

          Committed it to trunk and branch-2. thanks Yiqun Lin for the contributions!

          Show
          zxu zhihai xu added a comment - Committed it to trunk and branch-2. thanks Yiqun Lin for the contributions!
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #8968 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8968/)
          YARN-4440. FSAppAttempt#getAllowedLocalityLevelByTime should init the (zxu: rev 2aaed1032719e978660d1f14a8ed299e83b5f65e)

          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/TestFairScheduler.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FSAppAttempt.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #8968 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8968/ ) YARN-4440 . FSAppAttempt#getAllowedLocalityLevelByTime should init the (zxu: rev 2aaed1032719e978660d1f14a8ed299e83b5f65e) hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/TestFairScheduler.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FSAppAttempt.java
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-Hdfs-trunk-Java8 #694 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/694/)
          YARN-4440. FSAppAttempt#getAllowedLocalityLevelByTime should init the (zxu: rev 2aaed1032719e978660d1f14a8ed299e83b5f65e)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/TestFairScheduler.java
          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FSAppAttempt.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Hdfs-trunk-Java8 #694 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/694/ ) YARN-4440 . FSAppAttempt#getAllowedLocalityLevelByTime should init the (zxu: rev 2aaed1032719e978660d1f14a8ed299e83b5f65e) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/TestFairScheduler.java hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FSAppAttempt.java
          Hide
          linyiqun Yiqun Lin added a comment -

          Thanks zhihai xu for review and commit.

          Show
          linyiqun Yiqun Lin added a comment - Thanks zhihai xu for review and commit.
          Hide
          leftnoteasy Wangda Tan added a comment -

          Committed to branch-2.8.

          Show
          leftnoteasy Wangda Tan added a comment - Committed to branch-2.8.
          Hide
          zxu zhihai xu added a comment -

          yes, thanks Wangda Tan for committing it to branch-2.8!

          Show
          zxu zhihai xu added a comment - yes, thanks Wangda Tan for committing it to branch-2.8!

            People

            • Assignee:
              linyiqun Yiqun Lin
              Reporter:
              linyiqun Yiqun Lin
            • Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development