Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-4403

(AM/NM/Container)LivelinessMonitor should use monotonic time when calculating period

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: None
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      Currently, (AM/NM/Container)LivelinessMonitor use current system time to calculate a duration of expire which could be broken by settimeofday. We should use Time.monotonicNow() instead.

      1. YARN-4403.patch
        6 kB
        Junping Du
      2. YARN-4403-v2.patch
        8 kB
        Junping Du

        Issue Links

          Activity

          Hide
          djp Junping Du added a comment -

          Update a patch to fix this issue in (AM/NM/Container)LivelinessMonitor. I think there are also similar issues in other YARN components. Will file other JIRAs to follow up when we have MonotonicClock there.

          Show
          djp Junping Du added a comment - Update a patch to fix this issue in (AM/NM/Container)LivelinessMonitor. I think there are also similar issues in other YARN components. Will file other JIRAs to follow up when we have MonotonicClock there.
          Hide
          sunilg Sunil G added a comment -

          Hi Junping Du
          Thanks for the patch. Yes, its better to use MonotonicClock generally.
          We also use SystemClock in proportional preemption policy, I feel as you mentioned a general YARN ticket can handle all this together.

          Show
          sunilg Sunil G added a comment - Hi Junping Du Thanks for the patch. Yes, its better to use MonotonicClock generally. We also use SystemClock in proportional preemption policy, I feel as you mentioned a general YARN ticket can handle all this together.
          Hide
          djp Junping Du added a comment -

          Agree. LivelinessMonitor is more critical as it affects all YARN daemons/containers lifecycle, so I prefer we get in this first.
          Later, we can file two separate JIRAs: one for YARN and the other for MapReduce to address other places. I am sure there are many places to change as all timeout could be affected and we should be carefully. Hadoop/HDFS projects should already adopt this early.

          Show
          djp Junping Du added a comment - Agree. LivelinessMonitor is more critical as it affects all YARN daemons/containers lifecycle, so I prefer we get in this first. Later, we can file two separate JIRAs: one for YARN and the other for MapReduce to address other places. I am sure there are many places to change as all timeout could be affected and we should be carefully. Hadoop/HDFS projects should already adopt this early.
          Hide
          sunilg Sunil G added a comment -

          Yes. That sounds good. +1 for the patch.
          For YARN/MR, I could also definitely help in getting it in shape once this is in.

          Show
          sunilg Sunil G added a comment - Yes. That sounds good. +1 for the patch. For YARN/MR, I could also definitely help in getting it in shape once this is in.
          Hide
          djp Junping Du added a comment -

          For YARN/MR, I could also definitely help in getting it in shape once this is in.

          Sure. Feel free to create/assign JIRA and work on it. I will help to review. Thanks!

          Show
          djp Junping Du added a comment - For YARN/MR, I could also definitely help in getting it in shape once this is in. Sure. Feel free to create/assign JIRA and work on it. I will help to review. Thanks!
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 0s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          +1 mvninstall 7m 55s trunk passed
          +1 compile 1m 59s trunk passed with JDK v1.8.0_66
          +1 compile 2m 23s trunk passed with JDK v1.7.0_85
          +1 checkstyle 0m 31s trunk passed
          +1 mvnsite 1m 11s trunk passed
          +1 mvneclipse 0m 28s trunk passed
          +1 findbugs 2m 41s trunk passed
          +1 javadoc 0m 56s trunk passed with JDK v1.8.0_66
          +1 javadoc 1m 3s trunk passed with JDK v1.7.0_85
          +1 mvninstall 1m 7s the patch passed
          +1 compile 2m 5s the patch passed with JDK v1.8.0_66
          +1 javac 2m 5s the patch passed
          +1 compile 2m 17s the patch passed with JDK v1.7.0_85
          +1 javac 2m 17s the patch passed
          +1 checkstyle 0m 29s the patch passed
          +1 mvnsite 1m 11s the patch passed
          +1 mvneclipse 0m 28s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 findbugs 2m 53s the patch passed
          +1 javadoc 0m 53s the patch passed with JDK v1.8.0_66
          +1 javadoc 1m 1s the patch passed with JDK v1.7.0_85
          +1 unit 1m 59s hadoop-yarn-common in the patch passed with JDK v1.8.0_66.
          -1 unit 64m 3s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_66.
          +1 unit 2m 14s hadoop-yarn-common in the patch passed with JDK v1.7.0_85.
          -1 unit 65m 42s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_85.
          +1 asflicense 0m 22s Patch does not generate ASF License warnings.
          168m 35s



          Reason Tests
          JDK v1.8.0_66 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens
            hadoop.yarn.server.resourcemanager.TestAMAuthorization
          JDK v1.7.0_85 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens
            hadoop.yarn.server.resourcemanager.TestAMAuthorization



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:0ca8df7
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12775279/YARN-4403.patch
          JIRA Issue YARN-4403
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux c78761e0951a 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 53e3bf7
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-YARN-Build/9837/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_66.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/9837/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_85.txt
          unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/9837/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-YARN-Build/9837/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_85.txt
          JDK v1.7.0_85 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/9837/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn
          Max memory used 75MB
          Powered by Apache Yetus http://yetus.apache.org
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/9837/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 7m 55s trunk passed +1 compile 1m 59s trunk passed with JDK v1.8.0_66 +1 compile 2m 23s trunk passed with JDK v1.7.0_85 +1 checkstyle 0m 31s trunk passed +1 mvnsite 1m 11s trunk passed +1 mvneclipse 0m 28s trunk passed +1 findbugs 2m 41s trunk passed +1 javadoc 0m 56s trunk passed with JDK v1.8.0_66 +1 javadoc 1m 3s trunk passed with JDK v1.7.0_85 +1 mvninstall 1m 7s the patch passed +1 compile 2m 5s the patch passed with JDK v1.8.0_66 +1 javac 2m 5s the patch passed +1 compile 2m 17s the patch passed with JDK v1.7.0_85 +1 javac 2m 17s the patch passed +1 checkstyle 0m 29s the patch passed +1 mvnsite 1m 11s the patch passed +1 mvneclipse 0m 28s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 2m 53s the patch passed +1 javadoc 0m 53s the patch passed with JDK v1.8.0_66 +1 javadoc 1m 1s the patch passed with JDK v1.7.0_85 +1 unit 1m 59s hadoop-yarn-common in the patch passed with JDK v1.8.0_66. -1 unit 64m 3s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_66. +1 unit 2m 14s hadoop-yarn-common in the patch passed with JDK v1.7.0_85. -1 unit 65m 42s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_85. +1 asflicense 0m 22s Patch does not generate ASF License warnings. 168m 35s Reason Tests JDK v1.8.0_66 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens   hadoop.yarn.server.resourcemanager.TestAMAuthorization JDK v1.7.0_85 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens   hadoop.yarn.server.resourcemanager.TestAMAuthorization Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12775279/YARN-4403.patch JIRA Issue YARN-4403 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux c78761e0951a 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 53e3bf7 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-YARN-Build/9837/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_66.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/9837/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_85.txt unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/9837/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-YARN-Build/9837/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_85.txt JDK v1.7.0_85 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/9837/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn Max memory used 75MB Powered by Apache Yetus http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-YARN-Build/9837/console This message was automatically generated.
          Hide
          xinxianyin Xianyin Xin added a comment -

          hi Junping Du, this is a good suggestion, and YARN-4177 provides some discussion on this, so link it.

          Show
          xinxianyin Xianyin Xin added a comment - hi Junping Du , this is a good suggestion, and YARN-4177 provides some discussion on this, so link it.
          Hide
          xinxianyin Xianyin Xin added a comment -

          And will provide new patch of YARN-4177 once this is in.

          Show
          xinxianyin Xianyin Xin added a comment - And will provide new patch of YARN-4177 once this is in.
          Hide
          sunilg Sunil G added a comment -

          Thanks Xianyin Xin for updating this, I missed this somehow.
          It seems like you are handling Monotonic Clock in various YARN code which uses clock.
          So it can be made as general ticket in YARN.
          Meantime I will raise an MR ticket to handle this for MapReduce. MAPREDUCE-6562 is linked for the same.

          Show
          sunilg Sunil G added a comment - Thanks Xianyin Xin for updating this, I missed this somehow. It seems like you are handling Monotonic Clock in various YARN code which uses clock. So it can be made as general ticket in YARN. Meantime I will raise an MR ticket to handle this for MapReduce. MAPREDUCE-6562 is linked for the same.
          Hide
          xinxianyin Xianyin Xin added a comment -

          Thanks, Sunil G.

          Show
          xinxianyin Xianyin Xin added a comment - Thanks, Sunil G .
          Hide
          djp Junping Du added a comment -

          Thanks Xianyin Xin to link YARN-4177. I missed that discussion before. Make YARN-4177 as a general ticket for YARN sounds good to me.

          Show
          djp Junping Du added a comment - Thanks Xianyin Xin to link YARN-4177 . I missed that discussion before. Make YARN-4177 as a general ticket for YARN sounds good to me.
          Hide
          jianhe Jian He added a comment -

          patch looks good to me.
          one minor suggestion, do you think we can change the base AbstractLivelinessMonitor to have a default constructor with MonotonicClock and callers can use this constructor instead ?

          Show
          jianhe Jian He added a comment - patch looks good to me. one minor suggestion, do you think we can change the base AbstractLivelinessMonitor to have a default constructor with MonotonicClock and callers can use this constructor instead ?
          Hide
          djp Junping Du added a comment -

          Thanks Jian for review and comments. My first thought before is to replace AbstractLivelinessMonitor(String name, Clock clock) with AbstractLivelinessMonitor(String name) with MonotonicClock as default. However, we have TestAMLivelinessMonitor#testResetTimer need to set ControlledClock instead of SystemClock/MonotonicClock for test logic that means caller could use clock parameter in future for test purpose. May be we should keep current constructor there for more flexibility from caller?

          Show
          djp Junping Du added a comment - Thanks Jian for review and comments. My first thought before is to replace AbstractLivelinessMonitor(String name, Clock clock) with AbstractLivelinessMonitor(String name) with MonotonicClock as default. However, we have TestAMLivelinessMonitor#testResetTimer need to set ControlledClock instead of SystemClock/MonotonicClock for test logic that means caller could use clock parameter in future for test purpose. May be we should keep current constructor there for more flexibility from caller?
          Hide
          jianhe Jian He added a comment -

          we can actually add a separate one ?

          Show
          jianhe Jian He added a comment - we can actually add a separate one ?
          Hide
          djp Junping Du added a comment -

          Ok. That sounds good. Will update the patch soon.

          Show
          djp Junping Du added a comment - Ok. That sounds good. Will update the patch soon.
          Hide
          djp Junping Du added a comment -

          Update patch with incorporate Jian's comments.

          Show
          djp Junping Du added a comment - Update patch with incorporate Jian's comments.
          Hide
          sunilg Sunil G added a comment -

          Hi Junping Du
          Thanks for the updated patch. I have one doubt here, we could see tat SystemClock#getTime is Public and Stable. Now there is a note saying that its advisable to use MonotonicClock, so any annotation change is needed for SystemClock?

          Show
          sunilg Sunil G added a comment - Hi Junping Du Thanks for the updated patch. I have one doubt here, we could see tat SystemClock#getTime is Public and Stable . Now there is a note saying that its advisable to use MonotonicClock , so any annotation change is needed for SystemClock ?
          Hide
          djp Junping Du added a comment -

          It depends on if any consumer of SystemClock are using it to track absolute time but not for duration or interval. I didn't check other calling places in YARN/MR, also theoretically, it could be consumers outside of YARN given this a public API.
          We may consider to mark this API as deprecated later if we check all known calling places are for duration or interval only. But for now, it could be better to keep annotation no changed but with NOTE.

          Show
          djp Junping Du added a comment - It depends on if any consumer of SystemClock are using it to track absolute time but not for duration or interval. I didn't check other calling places in YARN/MR, also theoretically, it could be consumers outside of YARN given this a public API. We may consider to mark this API as deprecated later if we check all known calling places are for duration or interval only. But for now, it could be better to keep annotation no changed but with NOTE.
          Hide
          sunilg Sunil G added a comment -

          Yes. That's definitely a valid reason as per the current usage. Thank you very much for clarifying.

          Show
          sunilg Sunil G added a comment - Yes. That's definitely a valid reason as per the current usage. Thank you very much for clarifying.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 0s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          +1 mvninstall 8m 4s trunk passed
          +1 compile 2m 6s trunk passed with JDK v1.8.0_66
          +1 compile 2m 20s trunk passed with JDK v1.7.0_85
          +1 checkstyle 0m 30s trunk passed
          +1 mvnsite 1m 11s trunk passed
          +1 mvneclipse 0m 27s trunk passed
          +1 findbugs 2m 36s trunk passed
          -1 javadoc 0m 24s hadoop-yarn-server-resourcemanager in trunk failed with JDK v1.8.0_66.
          +1 javadoc 1m 4s trunk passed with JDK v1.7.0_85
          +1 mvninstall 1m 7s the patch passed
          +1 compile 2m 4s the patch passed with JDK v1.8.0_66
          -1 javac 6m 26s hadoop-yarn-project_hadoop-yarn-jdk1.8.0_66 with JDK v1.8.0_66 generated 1 new issues (was 14, now 14).
          +1 javac 2m 4s the patch passed
          +1 compile 2m 19s the patch passed with JDK v1.7.0_85
          +1 javac 2m 19s the patch passed
          +1 checkstyle 0m 28s the patch passed
          +1 mvnsite 1m 13s the patch passed
          +1 mvneclipse 0m 28s the patch passed
          -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix.
          +1 findbugs 3m 0s the patch passed
          -1 javadoc 0m 24s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_66.
          +1 javadoc 1m 3s the patch passed with JDK v1.7.0_85
          +1 unit 2m 0s hadoop-yarn-common in the patch passed with JDK v1.8.0_66.
          -1 unit 66m 0s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_66.
          +1 unit 2m 20s hadoop-yarn-common in the patch passed with JDK v1.7.0_85.
          -1 unit 66m 26s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_85.
          -1 asflicense 0m 23s Patch generated 3 ASF License warnings.
          170m 20s



          Reason Tests
          JDK v1.8.0_66 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens
            hadoop.yarn.server.resourcemanager.TestAMAuthorization
            hadoop.yarn.server.resourcemanager.scheduler.capacity.TestNodeLabelContainerAllocation
          JDK v1.7.0_85 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens
            hadoop.yarn.server.resourcemanager.TestAMAuthorization



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:0ca8df7
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12776330/YARN-4403-v2.patch
          JIRA Issue YARN-4403
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 709e55ffbb56 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / fc47084
          findbugs v3.0.0
          javadoc https://builds.apache.org/job/PreCommit-YARN-Build/9900/artifact/patchprocess/branch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_66.txt
          javac hadoop-yarn-project_hadoop-yarn-jdk1.8.0_66: https://builds.apache.org/job/PreCommit-YARN-Build/9900/artifact/patchprocess/diff-compile-javac-hadoop-yarn-project_hadoop-yarn-jdk1.8.0_66.txt
          whitespace https://builds.apache.org/job/PreCommit-YARN-Build/9900/artifact/patchprocess/whitespace-eol.txt
          javadoc https://builds.apache.org/job/PreCommit-YARN-Build/9900/artifact/patchprocess/patch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_66.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/9900/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_66.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/9900/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_85.txt
          unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/9900/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-YARN-Build/9900/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_85.txt
          JDK v1.7.0_85 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/9900/testReport/
          asflicense https://builds.apache.org/job/PreCommit-YARN-Build/9900/artifact/patchprocess/patch-asflicense-problems.txt
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn
          Max memory used 76MB
          Powered by Apache Yetus http://yetus.apache.org
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/9900/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 8m 4s trunk passed +1 compile 2m 6s trunk passed with JDK v1.8.0_66 +1 compile 2m 20s trunk passed with JDK v1.7.0_85 +1 checkstyle 0m 30s trunk passed +1 mvnsite 1m 11s trunk passed +1 mvneclipse 0m 27s trunk passed +1 findbugs 2m 36s trunk passed -1 javadoc 0m 24s hadoop-yarn-server-resourcemanager in trunk failed with JDK v1.8.0_66. +1 javadoc 1m 4s trunk passed with JDK v1.7.0_85 +1 mvninstall 1m 7s the patch passed +1 compile 2m 4s the patch passed with JDK v1.8.0_66 -1 javac 6m 26s hadoop-yarn-project_hadoop-yarn-jdk1.8.0_66 with JDK v1.8.0_66 generated 1 new issues (was 14, now 14). +1 javac 2m 4s the patch passed +1 compile 2m 19s the patch passed with JDK v1.7.0_85 +1 javac 2m 19s the patch passed +1 checkstyle 0m 28s the patch passed +1 mvnsite 1m 13s the patch passed +1 mvneclipse 0m 28s the patch passed -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 findbugs 3m 0s the patch passed -1 javadoc 0m 24s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_66. +1 javadoc 1m 3s the patch passed with JDK v1.7.0_85 +1 unit 2m 0s hadoop-yarn-common in the patch passed with JDK v1.8.0_66. -1 unit 66m 0s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.8.0_66. +1 unit 2m 20s hadoop-yarn-common in the patch passed with JDK v1.7.0_85. -1 unit 66m 26s hadoop-yarn-server-resourcemanager in the patch failed with JDK v1.7.0_85. -1 asflicense 0m 23s Patch generated 3 ASF License warnings. 170m 20s Reason Tests JDK v1.8.0_66 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens   hadoop.yarn.server.resourcemanager.TestAMAuthorization   hadoop.yarn.server.resourcemanager.scheduler.capacity.TestNodeLabelContainerAllocation JDK v1.7.0_85 Failed junit tests hadoop.yarn.server.resourcemanager.TestClientRMTokens   hadoop.yarn.server.resourcemanager.TestAMAuthorization Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12776330/YARN-4403-v2.patch JIRA Issue YARN-4403 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 709e55ffbb56 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / fc47084 findbugs v3.0.0 javadoc https://builds.apache.org/job/PreCommit-YARN-Build/9900/artifact/patchprocess/branch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_66.txt javac hadoop-yarn-project_hadoop-yarn-jdk1.8.0_66: https://builds.apache.org/job/PreCommit-YARN-Build/9900/artifact/patchprocess/diff-compile-javac-hadoop-yarn-project_hadoop-yarn-jdk1.8.0_66.txt whitespace https://builds.apache.org/job/PreCommit-YARN-Build/9900/artifact/patchprocess/whitespace-eol.txt javadoc https://builds.apache.org/job/PreCommit-YARN-Build/9900/artifact/patchprocess/patch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_66.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/9900/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_66.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/9900/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_85.txt unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/9900/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-YARN-Build/9900/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdk1.7.0_85.txt JDK v1.7.0_85 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/9900/testReport/ asflicense https://builds.apache.org/job/PreCommit-YARN-Build/9900/artifact/patchprocess/patch-asflicense-problems.txt modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn Max memory used 76MB Powered by Apache Yetus http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-YARN-Build/9900/console This message was automatically generated.
          Hide
          jianhe Jian He added a comment -

          Committed to trunk, branch-2, branch-2.8 thanks Junping Du !
          thanks Sunil G for reviewing the patch !

          Show
          jianhe Jian He added a comment - Committed to trunk, branch-2, branch-2.8 thanks Junping Du ! thanks Sunil G for reviewing the patch !
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #8965 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8965/)
          YARN-4403. (AM/NM/Container)LivelinessMonitor should use monotonic time (jianhe: rev 1cb3299b48a06a842aa3f6cf37ccf44a49af43b5)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/SystemClock.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/MonotonicClock.java
          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/NMLivelinessMonitor.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/attempt/AMLivelinessMonitor.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/AbstractLivelinessMonitor.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmcontainer/ContainerAllocationExpirer.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #8965 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8965/ ) YARN-4403 . (AM/NM/Container)LivelinessMonitor should use monotonic time (jianhe: rev 1cb3299b48a06a842aa3f6cf37ccf44a49af43b5) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/SystemClock.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/MonotonicClock.java hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/NMLivelinessMonitor.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/attempt/AMLivelinessMonitor.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/AbstractLivelinessMonitor.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmcontainer/ContainerAllocationExpirer.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #693 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/693/)
          YARN-4403. (AM/NM/Container)LivelinessMonitor should use monotonic time (jianhe: rev 1cb3299b48a06a842aa3f6cf37ccf44a49af43b5)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/SystemClock.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/MonotonicClock.java
          • hadoop-yarn-project/CHANGES.txt
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/NMLivelinessMonitor.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmcontainer/ContainerAllocationExpirer.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/AbstractLivelinessMonitor.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/attempt/AMLivelinessMonitor.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #693 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/693/ ) YARN-4403 . (AM/NM/Container)LivelinessMonitor should use monotonic time (jianhe: rev 1cb3299b48a06a842aa3f6cf37ccf44a49af43b5) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/SystemClock.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/MonotonicClock.java hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/NMLivelinessMonitor.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmcontainer/ContainerAllocationExpirer.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/AbstractLivelinessMonitor.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/attempt/AMLivelinessMonitor.java

            People

            • Assignee:
              djp Junping Du
              Reporter:
              djp Junping Du
            • Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development