Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-4682

AMRM client to log when AMRM token updated

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.7.2
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: client
    • Labels:
      None
    • Target Version/s:

      Description

      There's no information right now as to when the AMRM token gets updated; if something has gone wrong with the update, you can't tell when it last when through.

      fix: add a log statement.

      1. YARN-4682.patch
        0.8 kB
        Prabhu Joseph
      2. YARN-4682.patch.1
        0.8 kB
        Prabhu Joseph
      3. YARN-4682-002.patch
        1.0 kB
        Steve Loughran

        Issue Links

          Activity

          Hide
          Prabhu Joseph Prabhu Joseph added a comment -

          Steve Loughran Added a info log.

          Show
          Prabhu Joseph Prabhu Joseph added a comment - Steve Loughran Added a info log.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 0s Docker mode activated.
          -1 patch 0m 3s YARN-4682 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help.



          Subsystem Report/Notes
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12787249/YARN-4682.patch
          JIRA Issue YARN-4682
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/10544/console
          Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. -1 patch 0m 3s YARN-4682 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. Subsystem Report/Notes JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12787249/YARN-4682.patch JIRA Issue YARN-4682 Console output https://builds.apache.org/job/PreCommit-YARN-Build/10544/console Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          stevel@apache.org Steve Loughran added a comment -

          Prabhu, the patch isn't applying as it's against an old version of Hadoop. 2.6.0 probably

          1. can you check out trunk/branch-2 and create the patch there
          2. the version of code you are running doesn't have YARN-3103 applied (which is is why the patch is failing). That's probably the root cause of the token renewal problems discussed on the mailing list.

          If you are using hadoop 2.6.0, you need to upgrade to 2.6.1. If you are using something else (EMR, CDH, HDP, ...) , can you add it and its version to the environment field, so we can track how those releases are in relation to this issue.

          Show
          stevel@apache.org Steve Loughran added a comment - Prabhu, the patch isn't applying as it's against an old version of Hadoop. 2.6.0 probably can you check out trunk/branch-2 and create the patch there the version of code you are running doesn't have YARN-3103 applied (which is is why the patch is failing). That's probably the root cause of the token renewal problems discussed on the mailing list. If you are using hadoop 2.6.0, you need to upgrade to 2.6.1. If you are using something else (EMR, CDH, HDP, ...) , can you add it and its version to the environment field, so we can track how those releases are in relation to this issue.
          Hide
          Prabhu Joseph Prabhu Joseph added a comment -

          Steve Loughran

          Steve, do i need to checkout branch-2.

          The issue "No AMRMToken" happened on hadoop-2.4.1. So like you mentioned, the fix of YARN-3103 and YARN-2212 is missing there. I am doing a testing with the YARN-3103 fix, for every yarn.resourcemanager.am-rm-tokens.master-key-rolling-interval-secs, the AMRMToken gets updated. How to decrease the life time of a token, trying to simulate the issue again.

          Show
          Prabhu Joseph Prabhu Joseph added a comment - Steve Loughran Steve, do i need to checkout branch-2. The issue "No AMRMToken" happened on hadoop-2.4.1. So like you mentioned, the fix of YARN-3103 and YARN-2212 is missing there. I am doing a testing with the YARN-3103 fix, for every yarn.resourcemanager.am-rm-tokens.master-key-rolling-interval-secs, the AMRMToken gets updated. How to decrease the life time of a token, trying to simulate the issue again.
          Hide
          stevel@apache.org Steve Loughran added a comment -

          to get a patch we can put into Hadoop, yes, you need to check out trunk or branch-2; that's where all patches go in, with some being cherry-picked earlier.

          Moving off hadoop 2.4 to 2.6.1 should fix your problem, but having a log entry should help anyway.

          how to decrease the expiry time? that should be the key. Looking at the code, it needs to be at least 3 times the value of yarn.am.liveness-monitor.expiry-interval-ms. Make that nice and short

          Show
          stevel@apache.org Steve Loughran added a comment - to get a patch we can put into Hadoop, yes, you need to check out trunk or branch-2; that's where all patches go in, with some being cherry-picked earlier. Moving off hadoop 2.4 to 2.6.1 should fix your problem, but having a log entry should help anyway. how to decrease the expiry time? that should be the key. Looking at the code, it needs to be at least 3 times the value of yarn.am.liveness-monitor.expiry-interval-ms . Make that nice and short
          Hide
          Prabhu Joseph Prabhu Joseph added a comment -

          git checkout branch-2
          git diff > YARN-4682.patch.1
          But I am not seeing any difference with the previous patch.

          Show
          Prabhu Joseph Prabhu Joseph added a comment - git checkout branch-2 git diff > YARN-4682 .patch.1 But I am not seeing any difference with the previous patch.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 0s Docker mode activated.
          0 patch 0m 1s The patch file was not named according to hadoop's naming conventions. Please see https://wiki.apache.org/hadoop/HowToContribute for instructions.
          -1 patch 0m 4s YARN-4682 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help.



          Subsystem Report/Notes
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12787425/YARN-4682.patch.1
          JIRA Issue YARN-4682
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/10551/console
          Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. 0 patch 0m 1s The patch file was not named according to hadoop's naming conventions. Please see https://wiki.apache.org/hadoop/HowToContribute for instructions. -1 patch 0m 4s YARN-4682 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. Subsystem Report/Notes JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12787425/YARN-4682.patch.1 JIRA Issue YARN-4682 Console output https://builds.apache.org/job/PreCommit-YARN-Build/10551/console Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          stevel@apache.org Steve Loughran added a comment -

          the issue is just that the lines of code above your patch had changed, so the diff wouldn't take. This is the patch with that changed; if jenkins/yetus is happy, i'll commit it

          Show
          stevel@apache.org Steve Loughran added a comment - the issue is just that the lines of code above your patch had changed, so the diff wouldn't take. This is the patch with that changed; if jenkins/yetus is happy, i'll commit it
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 14s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          +1 mvninstall 6m 27s trunk passed
          +1 compile 0m 16s trunk passed with JDK v1.8.0_72
          +1 compile 0m 18s trunk passed with JDK v1.7.0_95
          +1 checkstyle 0m 14s trunk passed
          +1 mvnsite 0m 23s trunk passed
          +1 mvneclipse 0m 13s trunk passed
          +1 findbugs 0m 33s trunk passed
          +1 javadoc 0m 14s trunk passed with JDK v1.8.0_72
          +1 javadoc 0m 17s trunk passed with JDK v1.7.0_95
          +1 mvninstall 0m 18s the patch passed
          +1 compile 0m 13s the patch passed with JDK v1.8.0_72
          +1 javac 0m 13s the patch passed
          +1 compile 0m 16s the patch passed with JDK v1.7.0_95
          +1 javac 0m 16s the patch passed
          +1 checkstyle 0m 11s the patch passed
          +1 mvnsite 0m 20s the patch passed
          +1 mvneclipse 0m 10s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 findbugs 0m 44s the patch passed
          +1 javadoc 0m 11s the patch passed with JDK v1.8.0_72
          +1 javadoc 0m 15s the patch passed with JDK v1.7.0_95
          -1 unit 64m 28s hadoop-yarn-client in the patch failed with JDK v1.8.0_72.
          -1 unit 64m 43s hadoop-yarn-client in the patch failed with JDK v1.7.0_95.
          +1 asflicense 0m 17s Patch does not generate ASF License warnings.
          142m 14s



          Reason Tests
          JDK v1.8.0_72 Failed junit tests hadoop.yarn.client.TestGetGroups
          JDK v1.8.0_72 Timed out junit tests org.apache.hadoop.yarn.client.cli.TestYarnCLI
            org.apache.hadoop.yarn.client.api.impl.TestYarnClient
            org.apache.hadoop.yarn.client.api.impl.TestAMRMClient
            org.apache.hadoop.yarn.client.api.impl.TestNMClient
          JDK v1.7.0_95 Failed junit tests hadoop.yarn.client.TestGetGroups
          JDK v1.7.0_95 Timed out junit tests org.apache.hadoop.yarn.client.cli.TestYarnCLI
            org.apache.hadoop.yarn.client.api.impl.TestYarnClient
            org.apache.hadoop.yarn.client.api.impl.TestAMRMClient
            org.apache.hadoop.yarn.client.api.impl.TestNMClient



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:0ca8df7
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12787454/YARN-4682-002.patch
          JIRA Issue YARN-4682
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 906df2347bb0 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / fa00d3e
          Default Java 1.7.0_95
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_72 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-YARN-Build/10555/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client-jdk1.8.0_72.txt
          unit https://builds.apache.org/job/PreCommit-YARN-Build/10555/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client-jdk1.7.0_95.txt
          unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/10555/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client-jdk1.8.0_72.txt https://builds.apache.org/job/PreCommit-YARN-Build/10555/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client-jdk1.7.0_95.txt
          JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/10555/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client
          Max memory used 77MB
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/10555/console
          Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 14s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 6m 27s trunk passed +1 compile 0m 16s trunk passed with JDK v1.8.0_72 +1 compile 0m 18s trunk passed with JDK v1.7.0_95 +1 checkstyle 0m 14s trunk passed +1 mvnsite 0m 23s trunk passed +1 mvneclipse 0m 13s trunk passed +1 findbugs 0m 33s trunk passed +1 javadoc 0m 14s trunk passed with JDK v1.8.0_72 +1 javadoc 0m 17s trunk passed with JDK v1.7.0_95 +1 mvninstall 0m 18s the patch passed +1 compile 0m 13s the patch passed with JDK v1.8.0_72 +1 javac 0m 13s the patch passed +1 compile 0m 16s the patch passed with JDK v1.7.0_95 +1 javac 0m 16s the patch passed +1 checkstyle 0m 11s the patch passed +1 mvnsite 0m 20s the patch passed +1 mvneclipse 0m 10s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 0m 44s the patch passed +1 javadoc 0m 11s the patch passed with JDK v1.8.0_72 +1 javadoc 0m 15s the patch passed with JDK v1.7.0_95 -1 unit 64m 28s hadoop-yarn-client in the patch failed with JDK v1.8.0_72. -1 unit 64m 43s hadoop-yarn-client in the patch failed with JDK v1.7.0_95. +1 asflicense 0m 17s Patch does not generate ASF License warnings. 142m 14s Reason Tests JDK v1.8.0_72 Failed junit tests hadoop.yarn.client.TestGetGroups JDK v1.8.0_72 Timed out junit tests org.apache.hadoop.yarn.client.cli.TestYarnCLI   org.apache.hadoop.yarn.client.api.impl.TestYarnClient   org.apache.hadoop.yarn.client.api.impl.TestAMRMClient   org.apache.hadoop.yarn.client.api.impl.TestNMClient JDK v1.7.0_95 Failed junit tests hadoop.yarn.client.TestGetGroups JDK v1.7.0_95 Timed out junit tests org.apache.hadoop.yarn.client.cli.TestYarnCLI   org.apache.hadoop.yarn.client.api.impl.TestYarnClient   org.apache.hadoop.yarn.client.api.impl.TestAMRMClient   org.apache.hadoop.yarn.client.api.impl.TestNMClient Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12787454/YARN-4682-002.patch JIRA Issue YARN-4682 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 906df2347bb0 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / fa00d3e Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_72 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-YARN-Build/10555/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client-jdk1.8.0_72.txt unit https://builds.apache.org/job/PreCommit-YARN-Build/10555/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client-jdk1.7.0_95.txt unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/10555/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client-jdk1.8.0_72.txt https://builds.apache.org/job/PreCommit-YARN-Build/10555/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client-jdk1.7.0_95.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/10555/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client Max memory used 77MB Console output https://builds.apache.org/job/PreCommit-YARN-Build/10555/console Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          stevel@apache.org Steve Loughran added a comment -

          +1, committed to branch 2.8. It may seem a minor patch, but people will appreciate it.

          Also, I'm trying to get the permissions to list you as the assignee for this issue: your code, you get the credit

          Show
          stevel@apache.org Steve Loughran added a comment - +1, committed to branch 2.8. It may seem a minor patch, but people will appreciate it. Also, I'm trying to get the permissions to list you as the assignee for this issue: your code, you get the credit
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #9292 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9292/)
          YARN-4682. AMRM client to log when AMRM token updated. (Prabhu Joseph (stevel: rev 51fc7f542747bf87b48580747f51610e52bdc98b)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/main/java/org/apache/hadoop/yarn/client/api/impl/AMRMClientImpl.java
          • hadoop-yarn-project/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #9292 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9292/ ) YARN-4682 . AMRM client to log when AMRM token updated. (Prabhu Joseph (stevel: rev 51fc7f542747bf87b48580747f51610e52bdc98b) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/main/java/org/apache/hadoop/yarn/client/api/impl/AMRMClientImpl.java hadoop-yarn-project/CHANGES.txt
          Hide
          Prabhu Joseph Prabhu Joseph added a comment -

          Thanks Steve

          Show
          Prabhu Joseph Prabhu Joseph added a comment - Thanks Steve
          Hide
          cnauroth Chris Nauroth added a comment -

          Prabhu Joseph, I added you as a contributor on the YARN project and assigned this issue to you. Thanks for the patch!

          Steve Loughran, I added you to the Committers role in the YARN project, so you should have the rights to do this in the future.

          Show
          cnauroth Chris Nauroth added a comment - Prabhu Joseph , I added you as a contributor on the YARN project and assigned this issue to you. Thanks for the patch! Steve Loughran , I added you to the Committers role in the YARN project, so you should have the rights to do this in the future.

            People

            • Assignee:
              Prabhu Joseph Prabhu Joseph
              Reporter:
              stevel@apache.org Steve Loughran
            • Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Time Tracking

                Estimated:
                Original Estimate - 0.25h
                0.25h
                Remaining:
                Remaining Estimate - 0.25h
                0.25h
                Logged:
                Time Spent - Not Specified
                Not Specified

                  Development