Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-4464

Lower the default max applications stored in the RM and store

    Details

    • Type: Wish
    • Status: Resolved
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.0.0-alpha2
    • Component/s: resourcemanager
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Incompatible change
    • Release Note:
      The maximum applications the RM stores in memory and in the state-store by default has been lowered from 10,000 to 1,000. This should ease the pressure on the state-store. However, installations relying on the default to be 10,000 are affected.

      Description

      my cluster has 120 nodes.
      I configured RM Restart feature.

      yarn.resourcemanager.recovery.enabled=true
      yarn.resourcemanager.store.class=org.apache.hadoop.yarn.server.resourcemanager.recovery.FileSystemRMStateStore
      yarn.resourcemanager.fs.state-store.uri=/system/yarn/rmstore
      

      unfortunately I did not configure yarn.resourcemanager.state-store.max-completed-applications.
      so that property configured default value 10,000.

      I have restarted RM due to changing another configuartion.
      I expected that RM restart immediately.

      recovery process was very slow. I have waited about 20min.

      realize missing yarn.resourcemanager.state-store.max-completed-applications.

      its default value is very huge.
      need to change lower value or document notice on RM Restart page.

      1. YARN-4464.001.patch
        1.0 kB
        Daniel Templeton
      2. YARN-4464.002.patch
        2 kB
        Daniel Templeton
      3. YARN-4464.003.patch
        2 kB
        Daniel Templeton
      4. YARN-4464.004.patch
        3 kB
        Daniel Templeton
      5. YARN-4464.005.patch
        2 kB
        Daniel Templeton
      6. YARN-4464.006.patch
        3 kB
        Daniel Templeton
      7. YARN-4464.007.patch
        4 kB
        Daniel Templeton
      8. YARN-4464.008.patch
        4 kB
        Daniel Templeton

        Activity

        Hide
        kasha Karthik Kambatla added a comment -

        Agree that the value is too high in branch-2. We advise our customers to lower this all the time. However, changing it in branch-2 would be incompatible. I think we should do this only on trunk; making it a blocker so we don't miss it.

        How about setting the default to 0?

        Show
        kasha Karthik Kambatla added a comment - Agree that the value is too high in branch-2. We advise our customers to lower this all the time. However, changing it in branch-2 would be incompatible. I think we should do this only on trunk; making it a blocker so we don't miss it. How about setting the default to 0?
        Hide
        magnum KWON BYUNGCHANG added a comment -

        0 is good

        Show
        magnum KWON BYUNGCHANG added a comment - 0 is good
        Hide
        magnum KWON BYUNGCHANG added a comment -

        I have a question.
        my cluster is not secure mode.
        RMAppRoot has 6 files. however RMDTSecretManagerRoot has 4020 files.

        magnum-mbp:~/c3$ hdfs dfs -ls /system/yarn/rmstore/FSRMStateRoot/RMAppRoot | wc -l
               6
        magnum-mbp:~/c3$ hdfs dfs -ls /system/yarn/rmstore/FSRMStateRoot/RMDTSecretManagerRoot/ | wc -l
            4020
        

        I think files count of RMDTSecretManagerRoot will influence RM recovery process.
        How do I reduce files count?

        I do not configure below properties.

        yarn.resourcemanager.delegation.key.update-interval
        yarn.resourcemanager.delegation.token.renew-interval
        yarn.resourcemanager.delegation.token.max-lifetime
        
        Show
        magnum KWON BYUNGCHANG added a comment - I have a question. my cluster is not secure mode. RMAppRoot has 6 files. however RMDTSecretManagerRoot has 4020 files. magnum-mbp:~/c3$ hdfs dfs -ls /system/yarn/rmstore/FSRMStateRoot/RMAppRoot | wc -l 6 magnum-mbp:~/c3$ hdfs dfs -ls /system/yarn/rmstore/FSRMStateRoot/RMDTSecretManagerRoot/ | wc -l 4020 I think files count of RMDTSecretManagerRoot will influence RM recovery process. How do I reduce files count? I do not configure below properties. yarn.resourcemanager.delegation.key.update-interval yarn.resourcemanager.delegation.token.renew-interval yarn.resourcemanager.delegation.token.max-lifetime
        Hide
        andrew.wang Andrew Wang added a comment -

        If it's just changing the config value, this one sounds pretty easy. Daniel Templeton do you think we can get it in soon-ish?

        Show
        andrew.wang Andrew Wang added a comment - If it's just changing the config value, this one sounds pretty easy. Daniel Templeton do you think we can get it in soon-ish?
        Hide
        kasha Karthik Kambatla added a comment -

        KWON BYUNGCHANG - any chance you are running into YARN-3104?

        Show
        kasha Karthik Kambatla added a comment - KWON BYUNGCHANG - any chance you are running into YARN-3104 ?
        Hide
        templedf Daniel Templeton added a comment -

        Let's see if this breaks any tests...

        Show
        templedf Daniel Templeton added a comment - Let's see if this breaks any tests...
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 11s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 mvninstall 7m 4s trunk passed
        +1 compile 0m 22s trunk passed with JDK v1.8.0_91
        +1 compile 0m 23s trunk passed with JDK v1.7.0_95
        +1 checkstyle 0m 16s trunk passed
        +1 mvnsite 0m 27s trunk passed
        +1 mvneclipse 0m 11s trunk passed
        +1 findbugs 1m 10s trunk passed
        +1 javadoc 0m 39s trunk passed with JDK v1.8.0_91
        +1 javadoc 3m 0s trunk passed with JDK v1.7.0_95
        +1 mvninstall 0m 23s the patch passed
        +1 compile 0m 19s the patch passed with JDK v1.8.0_91
        +1 javac 0m 19s the patch passed
        +1 compile 0m 20s the patch passed with JDK v1.7.0_95
        +1 javac 0m 20s the patch passed
        +1 checkstyle 0m 14s the patch passed
        +1 mvnsite 0m 25s the patch passed
        +1 mvneclipse 0m 10s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        +1 findbugs 1m 23s the patch passed
        +1 javadoc 0m 36s the patch passed with JDK v1.8.0_91
        +1 javadoc 2m 56s the patch passed with JDK v1.7.0_95
        +1 unit 0m 21s hadoop-yarn-api in the patch passed with JDK v1.8.0_91.
        +1 unit 0m 23s hadoop-yarn-api in the patch passed with JDK v1.7.0_95.
        +1 asflicense 0m 17s Patch does not generate ASF License warnings.
        22m 28s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:cf2ee45
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12803311/YARN-4464.001.patch
        JIRA Issue YARN-4464
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 29bbe167a134 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 27242f2
        Default Java 1.7.0_95
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_91 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
        findbugs v3.0.0
        JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/11402/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/11402/console
        Powered by Apache Yetus 0.2.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 11s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 7m 4s trunk passed +1 compile 0m 22s trunk passed with JDK v1.8.0_91 +1 compile 0m 23s trunk passed with JDK v1.7.0_95 +1 checkstyle 0m 16s trunk passed +1 mvnsite 0m 27s trunk passed +1 mvneclipse 0m 11s trunk passed +1 findbugs 1m 10s trunk passed +1 javadoc 0m 39s trunk passed with JDK v1.8.0_91 +1 javadoc 3m 0s trunk passed with JDK v1.7.0_95 +1 mvninstall 0m 23s the patch passed +1 compile 0m 19s the patch passed with JDK v1.8.0_91 +1 javac 0m 19s the patch passed +1 compile 0m 20s the patch passed with JDK v1.7.0_95 +1 javac 0m 20s the patch passed +1 checkstyle 0m 14s the patch passed +1 mvnsite 0m 25s the patch passed +1 mvneclipse 0m 10s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 1m 23s the patch passed +1 javadoc 0m 36s the patch passed with JDK v1.8.0_91 +1 javadoc 2m 56s the patch passed with JDK v1.7.0_95 +1 unit 0m 21s hadoop-yarn-api in the patch passed with JDK v1.8.0_91. +1 unit 0m 23s hadoop-yarn-api in the patch passed with JDK v1.7.0_95. +1 asflicense 0m 17s Patch does not generate ASF License warnings. 22m 28s Subsystem Report/Notes Docker Image:yetus/hadoop:cf2ee45 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12803311/YARN-4464.001.patch JIRA Issue YARN-4464 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 29bbe167a134 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 27242f2 Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_91 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/11402/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api Console output https://builds.apache.org/job/PreCommit-YARN-Build/11402/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
        Hide
        hex108 Jun Gong added a comment -

        Thanks Daniel Templeton for the patch.

        From KWON BYUNGCHANG's comment https://issues.apache.org/jira/browse/YARN-4464?focusedCommentId=15061968&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15061968, there were just 6 RM apps.

        Another thought: If the slow recovery process is caused by completed applications, YARN-4494 might be another solution.

        Show
        hex108 Jun Gong added a comment - Thanks Daniel Templeton for the patch. From KWON BYUNGCHANG 's comment https://issues.apache.org/jira/browse/YARN-4464?focusedCommentId=15061968&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15061968 , there were just 6 RM apps. Another thought: If the slow recovery process is caused by completed applications, YARN-4494 might be another solution.
        Hide
        kasha Karthik Kambatla added a comment -

        Jian He - do you think it is okay to lower this all the way to 0? Personally, I think setting it to 0 might be better so users immediately realize they don't see any completed apps after a restart.

        Can we update the default value in yarn-default.xml to match this? I am also pleasantly surprised that none of our recovery tests assumed this is not zero

        Show
        kasha Karthik Kambatla added a comment - Jian He - do you think it is okay to lower this all the way to 0? Personally, I think setting it to 0 might be better so users immediately realize they don't see any completed apps after a restart. Can we update the default value in yarn-default.xml to match this? I am also pleasantly surprised that none of our recovery tests assumed this is not zero
        Hide
        rchiang Ray Chiang added a comment -

        Agree about making this trunk-only.

        One thought. If we change the default value to 0, it would be good to verify that we never actually create a local state store file or open a ZK connection. It would be wasteful to actually do any file/network open if we're going to send nothing.

        Show
        rchiang Ray Chiang added a comment - Agree about making this trunk-only. One thought. If we change the default value to 0, it would be good to verify that we never actually create a local state store file or open a ZK connection. It would be wasteful to actually do any file/network open if we're going to send nothing.
        Hide
        kasha Karthik Kambatla added a comment -

        If we change the default value to 0

        The zero is only for completed applications. We need the store to store all running applications so they can be recovered. So, the ZK connection is still required. Am I missing something?

        Show
        kasha Karthik Kambatla added a comment - If we change the default value to 0 The zero is only for completed applications. We need the store to store all running applications so they can be recovered. So, the ZK connection is still required. Am I missing something?
        Hide
        rchiang Ray Chiang added a comment -

        Nope. Thanks for the explanation.

        Show
        rchiang Ray Chiang added a comment - Nope. Thanks for the explanation.
        Hide
        templedf Daniel Templeton added a comment -

        Added the yarn-defaults.xml change.

        Show
        templedf Daniel Templeton added a comment - Added the yarn-defaults.xml change.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 0s Docker mode activated.
        -1 patch 0m 4s YARN-4464 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help.



        Subsystem Report/Notes
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12804833/YARN-4664.002.patch
        JIRA Issue YARN-4464
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/11543/console
        Powered by Apache Yetus 0.2.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. -1 patch 0m 4s YARN-4464 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. Subsystem Report/Notes JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12804833/YARN-4664.002.patch JIRA Issue YARN-4464 Console output https://builds.apache.org/job/PreCommit-YARN-Build/11543/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
        Hide
        Naganarasimha Naganarasimha G R added a comment -

        Hi Daniel Templeton,
        Seems like latest patch uploaded by you is not for this jira,
        Given that we have timelineserver to store the finished app information its not required to store 10000 apps, IMO it would be ideal to have this in for branch-2 also but may be with a better lower value like 500 or so( any val < 1000), so that its does completely break the compatability. Thoughts ?

        Show
        Naganarasimha Naganarasimha G R added a comment - Hi Daniel Templeton , Seems like latest patch uploaded by you is not for this jira, Given that we have timelineserver to store the finished app information its not required to store 10000 apps, IMO it would be ideal to have this in for branch-2 also but may be with a better lower value like 500 or so( any val < 1000), so that its does completely break the compatability. Thoughts ?
        Hide
        templedf Daniel Templeton added a comment -

        Whoops. The patch got merged with another. I'll tease them apart and repost.

        I'm not sure it's a good idea to change defaults in branch-2. We don't want to create incompatibilities during upgrades.

        Show
        templedf Daniel Templeton added a comment - Whoops. The patch got merged with another. I'll tease them apart and repost. I'm not sure it's a good idea to change defaults in branch-2. We don't want to create incompatibilities during upgrades.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 0s Docker mode activated.
        -1 patch 0m 3s YARN-4464 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help.



        Subsystem Report/Notes
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12804833/YARN-4664.002.patch
        JIRA Issue YARN-4464
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/11552/console
        Powered by Apache Yetus 0.2.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. -1 patch 0m 3s YARN-4464 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. Subsystem Report/Notes JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12804833/YARN-4664.002.patch JIRA Issue YARN-4464 Console output https://builds.apache.org/job/PreCommit-YARN-Build/11552/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 0s Docker mode activated.
        -1 patch 0m 3s YARN-4464 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help.



        Subsystem Report/Notes
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12804984/YARN-4664.002.patch
        JIRA Issue YARN-4464
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/11561/console
        Powered by Apache Yetus 0.2.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. -1 patch 0m 3s YARN-4464 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. Subsystem Report/Notes JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12804984/YARN-4664.002.patch JIRA Issue YARN-4464 Console output https://builds.apache.org/job/PreCommit-YARN-Build/11561/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 20s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        0 mvndep 0m 12s Maven dependency ordering for branch
        +1 mvninstall 7m 46s trunk passed
        +1 compile 2m 27s trunk passed
        +1 checkstyle 0m 48s trunk passed
        +1 mvnsite 1m 7s trunk passed
        +1 mvneclipse 0m 23s trunk passed
        +1 findbugs 2m 22s trunk passed
        +1 javadoc 1m 17s trunk passed
        0 mvndep 0m 9s Maven dependency ordering for patch
        +1 mvninstall 0m 52s the patch passed
        +1 compile 2m 15s the patch passed
        +1 javac 2m 15s the patch passed
        +1 checkstyle 0m 40s the patch passed
        +1 mvnsite 1m 2s the patch passed
        +1 mvneclipse 0m 22s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        +1 xml 0m 1s The patch has no ill-formed XML file.
        +1 findbugs 2m 28s the patch passed
        +1 javadoc 1m 12s the patch passed
        +1 unit 0m 24s hadoop-yarn-api in the patch passed.
        +1 unit 2m 22s hadoop-yarn-common in the patch passed.
        +1 asflicense 0m 18s Patch does not generate ASF License warnings.
        29m 38s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:2c91fd8
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12805011/YARN-4464.002.patch
        JIRA Issue YARN-4464
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml
        uname Linux 258d91ec46bc 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 141873c
        Default Java 1.8.0_91
        findbugs v3.0.0
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/11564/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common U: hadoop-yarn-project/hadoop-yarn
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/11564/console
        Powered by Apache Yetus 0.2.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 20s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. 0 mvndep 0m 12s Maven dependency ordering for branch +1 mvninstall 7m 46s trunk passed +1 compile 2m 27s trunk passed +1 checkstyle 0m 48s trunk passed +1 mvnsite 1m 7s trunk passed +1 mvneclipse 0m 23s trunk passed +1 findbugs 2m 22s trunk passed +1 javadoc 1m 17s trunk passed 0 mvndep 0m 9s Maven dependency ordering for patch +1 mvninstall 0m 52s the patch passed +1 compile 2m 15s the patch passed +1 javac 2m 15s the patch passed +1 checkstyle 0m 40s the patch passed +1 mvnsite 1m 2s the patch passed +1 mvneclipse 0m 22s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 xml 0m 1s The patch has no ill-formed XML file. +1 findbugs 2m 28s the patch passed +1 javadoc 1m 12s the patch passed +1 unit 0m 24s hadoop-yarn-api in the patch passed. +1 unit 2m 22s hadoop-yarn-common in the patch passed. +1 asflicense 0m 18s Patch does not generate ASF License warnings. 29m 38s Subsystem Report/Notes Docker Image:yetus/hadoop:2c91fd8 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12805011/YARN-4464.002.patch JIRA Issue YARN-4464 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml uname Linux 258d91ec46bc 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 141873c Default Java 1.8.0_91 findbugs v3.0.0 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/11564/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common U: hadoop-yarn-project/hadoop-yarn Console output https://builds.apache.org/job/PreCommit-YARN-Build/11564/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
        Hide
        rchiang Ray Chiang added a comment -

        This patch looks like it modifies the property yarn.timeline-service.leveldb-timeline-store.start-time-write-cache-size in yarn-default.xml, not yarn.resourcemanager.state-store.max-completed-applications. Or am I missing something?

        Show
        rchiang Ray Chiang added a comment - This patch looks like it modifies the property yarn.timeline-service.leveldb-timeline-store.start-time-write-cache-size in yarn-default.xml, not yarn.resourcemanager.state-store.max-completed-applications. Or am I missing something?
        Hide
        templedf Daniel Templeton added a comment -

        Oy vey. I clearly need sleep. Here's the correct correct patch.

        Show
        templedf Daniel Templeton added a comment - Oy vey. I clearly need sleep. Here's the correct correct patch.
        Hide
        rchiang Ray Chiang added a comment -

        +1 (nonbinding), pending Jenkins. Passed TestYarnConfigurationFields unit test + visual inspection in my tree.

        Show
        rchiang Ray Chiang added a comment - +1 (nonbinding), pending Jenkins. Passed TestYarnConfigurationFields unit test + visual inspection in my tree.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 17s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        0 mvndep 0m 11s Maven dependency ordering for branch
        +1 mvninstall 6m 56s trunk passed
        +1 compile 1m 57s trunk passed
        +1 checkstyle 0m 37s trunk passed
        +1 mvnsite 0m 52s trunk passed
        +1 mvneclipse 0m 21s trunk passed
        +1 findbugs 1m 49s trunk passed
        +1 javadoc 1m 3s trunk passed
        0 mvndep 0m 7s Maven dependency ordering for patch
        +1 mvninstall 0m 44s the patch passed
        +1 compile 1m 49s the patch passed
        +1 javac 1m 50s the patch passed
        +1 checkstyle 0m 35s the patch passed
        +1 mvnsite 0m 48s the patch passed
        +1 mvneclipse 0m 18s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        +1 xml 0m 1s The patch has no ill-formed XML file.
        +1 findbugs 2m 1s the patch passed
        +1 javadoc 0m 59s the patch passed
        +1 unit 0m 21s hadoop-yarn-api in the patch passed.
        +1 unit 2m 4s hadoop-yarn-common in the patch passed.
        +1 asflicense 0m 16s Patch does not generate ASF License warnings.
        24m 52s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:2c91fd8
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12805027/YARN-4464.003.patch
        JIRA Issue YARN-4464
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml
        uname Linux d299ee3e3c97 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 141873c
        Default Java 1.8.0_91
        findbugs v3.0.0
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/11566/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common U: hadoop-yarn-project/hadoop-yarn
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/11566/console
        Powered by Apache Yetus 0.2.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 17s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. 0 mvndep 0m 11s Maven dependency ordering for branch +1 mvninstall 6m 56s trunk passed +1 compile 1m 57s trunk passed +1 checkstyle 0m 37s trunk passed +1 mvnsite 0m 52s trunk passed +1 mvneclipse 0m 21s trunk passed +1 findbugs 1m 49s trunk passed +1 javadoc 1m 3s trunk passed 0 mvndep 0m 7s Maven dependency ordering for patch +1 mvninstall 0m 44s the patch passed +1 compile 1m 49s the patch passed +1 javac 1m 50s the patch passed +1 checkstyle 0m 35s the patch passed +1 mvnsite 0m 48s the patch passed +1 mvneclipse 0m 18s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 xml 0m 1s The patch has no ill-formed XML file. +1 findbugs 2m 1s the patch passed +1 javadoc 0m 59s the patch passed +1 unit 0m 21s hadoop-yarn-api in the patch passed. +1 unit 2m 4s hadoop-yarn-common in the patch passed. +1 asflicense 0m 16s Patch does not generate ASF License warnings. 24m 52s Subsystem Report/Notes Docker Image:yetus/hadoop:2c91fd8 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12805027/YARN-4464.003.patch JIRA Issue YARN-4464 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml uname Linux d299ee3e3c97 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 141873c Default Java 1.8.0_91 findbugs v3.0.0 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/11566/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common U: hadoop-yarn-project/hadoop-yarn Console output https://builds.apache.org/job/PreCommit-YARN-Build/11566/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
        Hide
        kasha Karthik Kambatla added a comment -

        +1. Will commit this to trunk tomorrow if no one objects. /cc Jian He

        Don't think we should do this in branch-2 though.

        Show
        kasha Karthik Kambatla added a comment - +1. Will commit this to trunk tomorrow if no one objects. /cc Jian He Don't think we should do this in branch-2 though.
        Hide
        jianhe Jian He added a comment -

        The patch is changing the config for max number of completed apps RM keeps in memory, NOT in store. Am I missing something ?

        Also, is the intention to not keep any completed apps in store ? IIRC, recovering 10k apps from ZK/HDFS is taking about less than 30 seconds. KWON BYUNGCHANG, are you sure the 20 min delay is caused by recovering these 10k competed apps ?

        I think setting it to 0 might be better so users immediately realize they don't see any completed apps after a restart.

        Karthik Kambatla, why do you think it's better to not let users see the historical apps ? I feel it is sometimes convenient to see the finished apps after restart.

        Show
        jianhe Jian He added a comment - The patch is changing the config for max number of completed apps RM keeps in memory, NOT in store. Am I missing something ? Also, is the intention to not keep any completed apps in store ? IIRC, recovering 10k apps from ZK/HDFS is taking about less than 30 seconds. KWON BYUNGCHANG , are you sure the 20 min delay is caused by recovering these 10k competed apps ? I think setting it to 0 might be better so users immediately realize they don't see any completed apps after a restart. Karthik Kambatla , why do you think it's better to not let users see the historical apps ? I feel it is sometimes convenient to see the finished apps after restart.
        Hide
        kasha Karthik Kambatla added a comment -

        The patch is changing the config for max number of completed apps RM keeps in memory, NOT in store. Am I missing something ?

        Thanks for catching that, Jian He. My bad, completely missed that.

        why do you think it's better to not let users see the historical apps ? I feel it is sometimes convenient to see the finished apps after restart.

        I agree that it is convenient to see finished apps. The current default of 10,000 is too large and we have seen issues with recovery due to the heavy load on ZK. If we pick a smaller value that is not zero, it is true that users will continue to see some finished apps. After upgrade, it is also quite possible that users don't realize the number of jobs stored on restart has been lowered; if they rely on the RM storing these, they might be in for a surprise later on. Picking zero should expose this change in behavior in any test cluster as well, and they could be a number appropriate for them. Long story short, I agree that a number like 1000 might have been a good default to begin with. Now that we are lowering, zero will be more in the face and avoid delayed surprises.

        I am not wedded to this and am open to persuaded to a different default value.

        Show
        kasha Karthik Kambatla added a comment - The patch is changing the config for max number of completed apps RM keeps in memory, NOT in store. Am I missing something ? Thanks for catching that, Jian He . My bad, completely missed that. why do you think it's better to not let users see the historical apps ? I feel it is sometimes convenient to see the finished apps after restart. I agree that it is convenient to see finished apps. The current default of 10,000 is too large and we have seen issues with recovery due to the heavy load on ZK. If we pick a smaller value that is not zero, it is true that users will continue to see some finished apps. After upgrade, it is also quite possible that users don't realize the number of jobs stored on restart has been lowered; if they rely on the RM storing these, they might be in for a surprise later on. Picking zero should expose this change in behavior in any test cluster as well, and they could be a number appropriate for them. Long story short, I agree that a number like 1000 might have been a good default to begin with. Now that we are lowering, zero will be more in the face and avoid delayed surprises. I am not wedded to this and am open to persuaded to a different default value.
        Hide
        templedf Daniel Templeton added a comment - - edited

        *face palm* Thanks for catching that. Let's try that one more time.

        Show
        templedf Daniel Templeton added a comment - - edited *face palm* Thanks for catching that. Let's try that one more time.
        Hide
        jianhe Jian He added a comment -

        ok, may be put it 1000 or a proportion of the number of completed apps in memory.

        Show
        jianhe Jian He added a comment - ok, may be put it 1000 or a proportion of the number of completed apps in memory.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 14s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        0 mvndep 0m 12s Maven dependency ordering for branch
        +1 mvninstall 7m 4s trunk passed
        +1 compile 2m 13s trunk passed
        +1 checkstyle 0m 38s trunk passed
        +1 mvnsite 1m 0s trunk passed
        +1 mvneclipse 0m 28s trunk passed
        +1 findbugs 2m 35s trunk passed
        +1 javadoc 1m 27s trunk passed
        0 mvndep 0m 11s Maven dependency ordering for patch
        +1 mvninstall 1m 4s the patch passed
        +1 compile 2m 45s the patch passed
        +1 javac 2m 45s the patch passed
        +1 checkstyle 0m 42s hadoop-yarn-project/hadoop-yarn: patch generated 0 new + 209 unchanged - 1 fixed = 209 total (was 210)
        +1 mvnsite 1m 6s the patch passed
        +1 mvneclipse 0m 24s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        +1 xml 0m 3s The patch has no ill-formed XML file.
        +1 findbugs 2m 46s the patch passed
        +1 javadoc 1m 21s the patch passed
        +1 unit 0m 30s hadoop-yarn-api in the patch passed.
        +1 unit 2m 28s hadoop-yarn-common in the patch passed.
        +1 asflicense 0m 22s Patch does not generate ASF License warnings.
        30m 30s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:2c91fd8
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12805251/YARN-4464.004.patch
        JIRA Issue YARN-4464
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml
        uname Linux f9f0284580c2 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 757050f
        Default Java 1.8.0_91
        findbugs v3.0.0
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/11590/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common U: hadoop-yarn-project/hadoop-yarn
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/11590/console
        Powered by Apache Yetus 0.2.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 14s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. 0 mvndep 0m 12s Maven dependency ordering for branch +1 mvninstall 7m 4s trunk passed +1 compile 2m 13s trunk passed +1 checkstyle 0m 38s trunk passed +1 mvnsite 1m 0s trunk passed +1 mvneclipse 0m 28s trunk passed +1 findbugs 2m 35s trunk passed +1 javadoc 1m 27s trunk passed 0 mvndep 0m 11s Maven dependency ordering for patch +1 mvninstall 1m 4s the patch passed +1 compile 2m 45s the patch passed +1 javac 2m 45s the patch passed +1 checkstyle 0m 42s hadoop-yarn-project/hadoop-yarn: patch generated 0 new + 209 unchanged - 1 fixed = 209 total (was 210) +1 mvnsite 1m 6s the patch passed +1 mvneclipse 0m 24s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 xml 0m 3s The patch has no ill-formed XML file. +1 findbugs 2m 46s the patch passed +1 javadoc 1m 21s the patch passed +1 unit 0m 30s hadoop-yarn-api in the patch passed. +1 unit 2m 28s hadoop-yarn-common in the patch passed. +1 asflicense 0m 22s Patch does not generate ASF License warnings. 30m 30s Subsystem Report/Notes Docker Image:yetus/hadoop:2c91fd8 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12805251/YARN-4464.004.patch JIRA Issue YARN-4464 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml uname Linux f9f0284580c2 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 757050f Default Java 1.8.0_91 findbugs v3.0.0 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/11590/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common U: hadoop-yarn-project/hadoop-yarn Console output https://builds.apache.org/job/PreCommit-YARN-Build/11590/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
        Hide
        Naganarasimha Naganarasimha G R added a comment -

        Hi Daniel Templeton,

        I'm not sure it's a good idea to change defaults in branch-2. We don't want to create incompatibilities during upgrades.

        Based on the latest comments, IIUC we are not planning to set the default value as 0 and planning to set it as 1000, so would presume it would be good enough for earlier versions too. Would like to understand whether actually reducing this value is really a incompatible change ? IMHO Its not like some feature is not working after upgrade right ? so i would vouch for having it across all versions, would like to know the opinion of Jian He and Karthik Kambatla

        Show
        Naganarasimha Naganarasimha G R added a comment - Hi Daniel Templeton , I'm not sure it's a good idea to change defaults in branch-2. We don't want to create incompatibilities during upgrades. Based on the latest comments, IIUC we are not planning to set the default value as 0 and planning to set it as 1000, so would presume it would be good enough for earlier versions too. Would like to understand whether actually reducing this value is really a incompatible change ? IMHO Its not like some feature is not working after upgrade right ? so i would vouch for having it across all versions, would like to know the opinion of Jian He and Karthik Kambatla
        Hide
        kasha Karthik Kambatla added a comment -

        My preference for changing defaults is of least astonishment - surprise them when they are not expecting.

        Users expect slight changes in behavior on major version upgrades. Changing in branch-2 has the potential to surprise users, and users could just set it to a lower value themselves if they want to. Setting to 1000 in trunk can also surprise users not when they upgrade but later when they try to fetch jobs older than 1000.

        My vote would still be for 0 and doing it in Hadoop 3. I would be -1 to doing it in Hadoop 2.x, and -0 to 1000 in Hadoop 3.

        Show
        kasha Karthik Kambatla added a comment - My preference for changing defaults is of least astonishment - surprise them when they are not expecting. Users expect slight changes in behavior on major version upgrades. Changing in branch-2 has the potential to surprise users, and users could just set it to a lower value themselves if they want to. Setting to 1000 in trunk can also surprise users not when they upgrade but later when they try to fetch jobs older than 1000. My vote would still be for 0 and doing it in Hadoop 3. I would be -1 to doing it in Hadoop 2.x, and -0 to 1000 in Hadoop 3.
        Hide
        kasha Karthik Kambatla added a comment -

        NGarla_Unused, Jian He - we should converge on one thing here. Should we toss a coin or gather more feedback from others?

        Vinod Kumar Vavilapalli: mind being our coin?

        Show
        kasha Karthik Kambatla added a comment - NGarla_Unused , Jian He - we should converge on one thing here. Should we toss a coin or gather more feedback from others? Vinod Kumar Vavilapalli : mind being our coin?
        Hide
        templedf Daniel Templeton added a comment -

        Ping, NGarla_Unused, Jian He, Vinod Kumar Vavilapalli. Can we get consensus so we can get this change in for 3.0?

        Show
        templedf Daniel Templeton added a comment - Ping, NGarla_Unused , Jian He , Vinod Kumar Vavilapalli . Can we get consensus so we can get this change in for 3.0?
        Hide
        vinodkv Vinod Kumar Vavilapalli added a comment -

        Tx for the ping Daniel Templeton.

        I haven't paid attention to this before. Apologies for pitching in very late.

        recovery process was very slow. I have waited about 20min.

        Did we ever find out why this takes 20mins? As part of original recovery feature, I remember that Jian He did some benchmarking to demonstrate that recovery of 10K apps takes only 10 seconds. We need to understand the root-cause here.

        Irrespective of that, even in trunk, if we prove that recovery takes much longer than 10 seconds in some unavoidable cases, the right solution is to make the recovery of completed applications alone to be in the background.

        Show
        vinodkv Vinod Kumar Vavilapalli added a comment - Tx for the ping Daniel Templeton . I haven't paid attention to this before. Apologies for pitching in very late. recovery process was very slow. I have waited about 20min. Did we ever find out why this takes 20mins? As part of original recovery feature, I remember that Jian He did some benchmarking to demonstrate that recovery of 10K apps takes only 10 seconds. We need to understand the root-cause here. Irrespective of that, even in trunk, if we prove that recovery takes much longer than 10 seconds in some unavoidable cases, the right solution is to make the recovery of completed applications alone to be in the background.
        Hide
        Naganarasimha Naganarasimha G R added a comment -

        Thanks for the comments Vinod Kumar Vavilapalli,

        I remember that Jian He did some benchmarking to demonstrate that recovery of 10K apps takes only 10 seconds. We need to understand the root-cause here.

        You are right though initially there were some discussions on the propable cause for the delay later on it just went on modifying the default value. Initially i thought it might be because of YARN-3104 (as mentioned by Karthik Kambatla) or YARN-4041, but not quite sure about it.

        But having said that, i was thinking more in the lines whether its required to store so many finished apps when we are already supporting ATS. Apart from adding to the startup time (though nominal but unnecessary when we have many running apps in large cluster) it was also adding lot of unnecessary logs and publish of ATS events. Hence was more inclined to reducing the default value.

        Show
        Naganarasimha Naganarasimha G R added a comment - Thanks for the comments Vinod Kumar Vavilapalli , I remember that Jian He did some benchmarking to demonstrate that recovery of 10K apps takes only 10 seconds. We need to understand the root-cause here. You are right though initially there were some discussions on the propable cause for the delay later on it just went on modifying the default value. Initially i thought it might be because of YARN-3104 (as mentioned by Karthik Kambatla ) or YARN-4041 , but not quite sure about it. But having said that, i was thinking more in the lines whether its required to store so many finished apps when we are already supporting ATS. Apart from adding to the startup time (though nominal but unnecessary when we have many running apps in large cluster) it was also adding lot of unnecessary logs and publish of ATS events. Hence was more inclined to reducing the default value.
        Hide
        templedf Daniel Templeton added a comment -

        With ATS, I don't see a lot of need to keep 10k completed apps lying about. Not only is it a startup burden, but it also is a ZK burden. We regularly tell customers to set it lower because of ZK cache load. Improving the recovery logic is something we should also do, but the best doesn't need to be the enemy of the good. Vinod Kumar Vavilapalli, Naganarasimha G R, Karthik Kambatla, can we come to a conclusion?

        Show
        templedf Daniel Templeton added a comment - With ATS, I don't see a lot of need to keep 10k completed apps lying about. Not only is it a startup burden, but it also is a ZK burden. We regularly tell customers to set it lower because of ZK cache load. Improving the recovery logic is something we should also do, but the best doesn't need to be the enemy of the good. Vinod Kumar Vavilapalli , Naganarasimha G R , Karthik Kambatla , can we come to a conclusion?
        Hide
        vinodkv Vinod Kumar Vavilapalli added a comment -

        We need ATS in production - aka ATS V2. With that in the picture, I agree that we don't need to keep any completed applications in RM memory at all.

        Show
        vinodkv Vinod Kumar Vavilapalli added a comment - We need ATS in production - aka ATS V2. With that in the picture, I agree that we don't need to keep any completed applications in RM memory at all.
        Hide
        Naganarasimha Naganarasimha G R added a comment -

        Thanks Vinod Kumar Vavilapalli, it looks ideal to have default value as zero but not sure all production cluster will adopt ATS immediately, in that per se i thought of having around last 500 ~ 1000 completed apps in RM.
        If all are ok with no completed apps in RM Memory as default then i am fine with it, its like -0 from my side. And i am ok with no change in Hadoop 2.x.

        Show
        Naganarasimha Naganarasimha G R added a comment - Thanks Vinod Kumar Vavilapalli , it looks ideal to have default value as zero but not sure all production cluster will adopt ATS immediately, in that per se i thought of having around last 500 ~ 1000 completed apps in RM. If all are ok with no completed apps in RM Memory as default then i am fine with it, its like -0 from my side. And i am ok with no change in Hadoop 2.x.
        Hide
        kasha Karthik Kambatla added a comment -

        I know there is no right answer here. We should have picked a better default to begin with.

        IAC, my preference would be whatever least astonishes the admins/users. Options sorted by least astonishment:

        1. Don't change anything. Keep it at 10,000 and deal with recovery slowness etc.
        2. Change it to 0. When people try out Hadoop 3 and failover, they immediately realize they don't see any completed applications. However, they all will likely have to change it
        3. Change it to 1000. People will realize it late, but most users might not necessarily run into any issues ever.

        By the way, one other change we should make is to limit rm.store.max-completed-apps to rm.max-completed-apps.

        Show
        kasha Karthik Kambatla added a comment - I know there is no right answer here. We should have picked a better default to begin with. IAC, my preference would be whatever least astonishes the admins/users. Options sorted by least astonishment: Don't change anything. Keep it at 10,000 and deal with recovery slowness etc. Change it to 0. When people try out Hadoop 3 and failover, they immediately realize they don't see any completed applications. However, they all will likely have to change it Change it to 1000. People will realize it late, but most users might not necessarily run into any issues ever. By the way, one other change we should make is to limit rm.store.max-completed-apps to rm.max-completed-apps .
        Hide
        jianhe Jian He added a comment -

        I vote for 3) which can solve the slowness problem and preserves the behavior to some extend

        Show
        jianhe Jian He added a comment - I vote for 3) which can solve the slowness problem and preserves the behavior to some extend
        Hide
        jianhe Jian He added a comment -

        I vote for 3) which can solve the slowness problem and preserves the behavior to some extend

        Show
        jianhe Jian He added a comment - I vote for 3) which can solve the slowness problem and preserves the behavior to some extend
        Hide
        Naganarasimha Naganarasimha G R added a comment -

        +1 for Option 3

        Show
        Naganarasimha Naganarasimha G R added a comment - +1 for Option 3
        Hide
        kasha Karthik Kambatla added a comment -

        I am okay with option 3.

        What should be the default for rm.max-completed-apps? Should that also be lowered to 1000? If yes, should the state-store config not have a value and fallback to rm.max-completed-apps if not set.

        Show
        kasha Karthik Kambatla added a comment - I am okay with option 3. What should be the default for rm.max-completed-apps ? Should that also be lowered to 1000? If yes, should the state-store config not have a value and fallback to rm.max-completed-apps if not set.
        Hide
        templedf Daniel Templeton added a comment - - edited

        I would vote for lowering the limit on the state store to 1000 and having an unset rm.max-completed-apps that defaults to the state store value. I'd also say that the rm.store.max-completed-apps should be capped at the rm.store.max-completed-apps value.

        Show
        templedf Daniel Templeton added a comment - - edited I would vote for lowering the limit on the state store to 1000 and having an unset rm.max-completed-apps that defaults to the state store value. I'd also say that the rm.store.max-completed-apps should be capped at the rm.store.max-completed-apps value.
        Hide
        jianhe Jian He added a comment -

        IIUC, the rm.max-completed-apps controls the number of apps in memory only, which doesn't matter much in this case ? that could stay the same ?

        Show
        jianhe Jian He added a comment - IIUC, the rm.max-completed-apps controls the number of apps in memory only, which doesn't matter much in this case ? that could stay the same ?
        Hide
        templedf Daniel Templeton added a comment -

        I would rather keep them in sync. There's no point to having the store hold more apps than the RM. The issue with having the RM hold more apps than the store is that on failover the RM will suddenly forget a bunch of apps. I think it would be less confusing to keep the two settings in sync unless the user explicitly changes it.

        Show
        templedf Daniel Templeton added a comment - I would rather keep them in sync. There's no point to having the store hold more apps than the RM. The issue with having the RM hold more apps than the store is that on failover the RM will suddenly forget a bunch of apps. I think it would be less confusing to keep the two settings in sync unless the user explicitly changes it.
        Hide
        jianhe Jian He added a comment -

        My opinion was that restart scenario occurs so rarely that having 1000 vs 10000 apps in store doesn't affect so much. Normal runs have no impact. Most users may not even notice the change. Rather, changing the way how much apps kept in memory will immediately have impact. Users will likely notice the change. I can't speak for our users/customers what number is enough usually. Is this intended for branch-2 too ? if there's no performance impact, I would rather keep the number unchanged for branch-2. If it's only for 3.x, maybe it's fine to keep in sync.

        Show
        jianhe Jian He added a comment - My opinion was that restart scenario occurs so rarely that having 1000 vs 10000 apps in store doesn't affect so much. Normal runs have no impact. Most users may not even notice the change. Rather, changing the way how much apps kept in memory will immediately have impact. Users will likely notice the change. I can't speak for our users/customers what number is enough usually. Is this intended for branch-2 too ? if there's no performance impact, I would rather keep the number unchanged for branch-2. If it's only for 3.x, maybe it's fine to keep in sync.
        Hide
        Naganarasimha Naganarasimha G R added a comment -

        I think Vinod Kumar Vavilapalli, Karthik Kambatla and Daniel Templeton were of the opinion earlier not to include this change in branch-2, which made sense too. Hence discussion of approaches were happening only for the trunk, if so i feel i am of the same opinion as Daniel i.e, setting both the configurations to same default values but flexibility to configure should be there

        Show
        Naganarasimha Naganarasimha G R added a comment - I think Vinod Kumar Vavilapalli , Karthik Kambatla and Daniel Templeton were of the opinion earlier not to include this change in branch-2, which made sense too. Hence discussion of approaches were happening only for the trunk, if so i feel i am of the same opinion as Daniel i.e, setting both the configurations to same default values but flexibility to configure should be there
        Hide
        jianhe Jian He added a comment -

        sure, sounds good.

        Show
        jianhe Jian He added a comment - sure, sounds good.
        Hide
        kasha Karthik Kambatla added a comment -

        Sure. Just to confirm, here is my understanding of the scope here:

        1. Trunk only changes
        2. Set rm.completed-apps to 1000
        3. Don't have a default value for rm.store.completed-apps. If not specified fall back to rm.completed-apps.
        Show
        kasha Karthik Kambatla added a comment - Sure. Just to confirm, here is my understanding of the scope here: Trunk only changes Set rm.completed-apps to 1000 Don't have a default value for rm.store.completed-apps. If not specified fall back to rm.completed-apps.
        Hide
        templedf Daniel Templeton added a comment -

        Current behavior is the other way around for #3: the store defaults to the RM value. Gonna say we probably shouldn't change it arbitrarily.

        Show
        templedf Daniel Templeton added a comment - Current behavior is the other way around for #3: the store defaults to the RM value. Gonna say we probably shouldn't change it arbitrarily.
        Hide
        templedf Daniel Templeton added a comment -

        Here's a patch to set the default to 1000 for the RM, leaving the store defaulting to the RM value.

        Show
        templedf Daniel Templeton added a comment - Here's a patch to set the default to 1000 for the RM, leaving the store defaulting to the RM value.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 19s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        0 mvndep 0m 10s Maven dependency ordering for branch
        +1 mvninstall 6m 39s trunk passed
        +1 compile 2m 15s trunk passed
        +1 checkstyle 0m 40s trunk passed
        +1 mvnsite 0m 56s trunk passed
        +1 mvneclipse 0m 26s trunk passed
        +1 findbugs 1m 54s trunk passed
        +1 javadoc 0m 46s trunk passed
        0 mvndep 0m 9s Maven dependency ordering for patch
        +1 mvninstall 0m 47s the patch passed
        +1 compile 2m 16s the patch passed
        +1 javac 2m 16s the patch passed
        +1 checkstyle 0m 37s the patch passed
        +1 mvnsite 0m 53s the patch passed
        +1 mvneclipse 0m 21s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 xml 0m 1s The patch has no ill-formed XML file.
        +1 findbugs 2m 5s the patch passed
        +1 javadoc 0m 43s the patch passed
        +1 unit 0m 23s hadoop-yarn-api in the patch passed.
        +1 unit 2m 19s hadoop-yarn-common in the patch passed.
        +1 asflicense 0m 17s The patch does not generate ASF License warnings.
        25m 43s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12824630/YARN-4464.005.patch
        JIRA Issue YARN-4464
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml
        uname Linux 3ee4825abc07 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 763f049
        Default Java 1.8.0_101
        findbugs v3.0.0
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12836/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common U: hadoop-yarn-project/hadoop-yarn
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/12836/console
        Powered by Apache Yetus 0.3.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 19s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. 0 mvndep 0m 10s Maven dependency ordering for branch +1 mvninstall 6m 39s trunk passed +1 compile 2m 15s trunk passed +1 checkstyle 0m 40s trunk passed +1 mvnsite 0m 56s trunk passed +1 mvneclipse 0m 26s trunk passed +1 findbugs 1m 54s trunk passed +1 javadoc 0m 46s trunk passed 0 mvndep 0m 9s Maven dependency ordering for patch +1 mvninstall 0m 47s the patch passed +1 compile 2m 16s the patch passed +1 javac 2m 16s the patch passed +1 checkstyle 0m 37s the patch passed +1 mvnsite 0m 53s the patch passed +1 mvneclipse 0m 21s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 1s The patch has no ill-formed XML file. +1 findbugs 2m 5s the patch passed +1 javadoc 0m 43s the patch passed +1 unit 0m 23s hadoop-yarn-api in the patch passed. +1 unit 2m 19s hadoop-yarn-common in the patch passed. +1 asflicense 0m 17s The patch does not generate ASF License warnings. 25m 43s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12824630/YARN-4464.005.patch JIRA Issue YARN-4464 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml uname Linux 3ee4825abc07 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 763f049 Default Java 1.8.0_101 findbugs v3.0.0 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/12836/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common U: hadoop-yarn-project/hadoop-yarn Console output https://builds.apache.org/job/PreCommit-YARN-Build/12836/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
        Hide
        kasha Karthik Kambatla added a comment -

        Daniel Templeton - if the user sets the max-applications-in-memory to a value different from the default and does not update the value for max-applications-in-store, the latter should use the value set for the former.

        In the following piece of code from RMAppManager, the default value for maxCompletedAppsInStore should be maxCompletedAppsInMemory instead of YarnConfiguration.DEFAULT_RM_STATE_STORE_MAX_COMPLETED_APPLICATIONS.

            this.maxCompletedAppsInMemory = conf.getInt(
                YarnConfiguration.RM_MAX_COMPLETED_APPLICATIONS,
                YarnConfiguration.DEFAULT_RM_MAX_COMPLETED_APPLICATIONS);
            this.maxCompletedAppsInStateStore =
                conf.getInt(
                  YarnConfiguration.RM_STATE_STORE_MAX_COMPLETED_APPLICATIONS,
                  YarnConfiguration.DEFAULT_RM_STATE_STORE_MAX_COMPLETED_APPLICATIONS);
        

        yarn-site.xml needs to be updated accordingly to not have a default value.

        Show
        kasha Karthik Kambatla added a comment - Daniel Templeton - if the user sets the max-applications-in-memory to a value different from the default and does not update the value for max-applications-in-store, the latter should use the value set for the former. In the following piece of code from RMAppManager, the default value for maxCompletedAppsInStore should be maxCompletedAppsInMemory instead of YarnConfiguration.DEFAULT_RM_STATE_STORE_MAX_COMPLETED_APPLICATIONS . this .maxCompletedAppsInMemory = conf.getInt( YarnConfiguration.RM_MAX_COMPLETED_APPLICATIONS, YarnConfiguration.DEFAULT_RM_MAX_COMPLETED_APPLICATIONS); this .maxCompletedAppsInStateStore = conf.getInt( YarnConfiguration.RM_STATE_STORE_MAX_COMPLETED_APPLICATIONS, YarnConfiguration.DEFAULT_RM_STATE_STORE_MAX_COMPLETED_APPLICATIONS); yarn-site.xml needs to be updated accordingly to not have a default value.
        Hide
        templedf Daniel Templeton added a comment -

        The docs are already correct. I foolishly believed them that the code was correct.

        Show
        templedf Daniel Templeton added a comment - The docs are already correct. I foolishly believed them that the code was correct.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 15s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        0 mvndep 0m 10s Maven dependency ordering for branch
        +1 mvninstall 7m 12s trunk passed
        +1 compile 2m 27s trunk passed
        +1 checkstyle 0m 41s trunk passed
        +1 mvnsite 1m 37s trunk passed
        +1 mvneclipse 0m 42s trunk passed
        +1 findbugs 3m 2s trunk passed
        +1 javadoc 1m 5s trunk passed
        0 mvndep 0m 9s Maven dependency ordering for patch
        +1 mvninstall 1m 22s the patch passed
        +1 compile 2m 23s the patch passed
        +1 javac 2m 23s the patch passed
        +1 checkstyle 0m 40s the patch passed
        +1 mvnsite 1m 32s the patch passed
        +1 mvneclipse 0m 36s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 xml 0m 2s The patch has no ill-formed XML file.
        +1 findbugs 3m 22s the patch passed
        +1 javadoc 1m 0s the patch passed
        +1 unit 0m 25s hadoop-yarn-api in the patch passed.
        +1 unit 2m 20s hadoop-yarn-common in the patch passed.
        +1 unit 33m 38s hadoop-yarn-server-resourcemanager in the patch passed.
        +1 asflicense 0m 19s The patch does not generate ASF License warnings.
        65m 52s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12829912/YARN-4464.006.patch
        JIRA Issue YARN-4464
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml
        uname Linux f0ae1da3118f 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18 21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 40acace
        Default Java 1.8.0_101
        findbugs v3.0.0
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13189/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/13189/console
        Powered by Apache Yetus 0.3.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 15s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. 0 mvndep 0m 10s Maven dependency ordering for branch +1 mvninstall 7m 12s trunk passed +1 compile 2m 27s trunk passed +1 checkstyle 0m 41s trunk passed +1 mvnsite 1m 37s trunk passed +1 mvneclipse 0m 42s trunk passed +1 findbugs 3m 2s trunk passed +1 javadoc 1m 5s trunk passed 0 mvndep 0m 9s Maven dependency ordering for patch +1 mvninstall 1m 22s the patch passed +1 compile 2m 23s the patch passed +1 javac 2m 23s the patch passed +1 checkstyle 0m 40s the patch passed +1 mvnsite 1m 32s the patch passed +1 mvneclipse 0m 36s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 2s The patch has no ill-formed XML file. +1 findbugs 3m 22s the patch passed +1 javadoc 1m 0s the patch passed +1 unit 0m 25s hadoop-yarn-api in the patch passed. +1 unit 2m 20s hadoop-yarn-common in the patch passed. +1 unit 33m 38s hadoop-yarn-server-resourcemanager in the patch passed. +1 asflicense 0m 19s The patch does not generate ASF License warnings. 65m 52s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12829912/YARN-4464.006.patch JIRA Issue YARN-4464 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml uname Linux f0ae1da3118f 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18 21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 40acace Default Java 1.8.0_101 findbugs v3.0.0 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13189/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn Console output https://builds.apache.org/job/PreCommit-YARN-Build/13189/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
        Hide
        Naganarasimha Naganarasimha G R added a comment -

        Thanks Daniel Templeton,
        +1, Latest patch LGTM.

        Show
        Naganarasimha Naganarasimha G R added a comment - Thanks Daniel Templeton , +1, Latest patch LGTM.
        Hide
        kasha Karthik Kambatla added a comment -

        Latest patch looks good, but for one issue: dropping DEFAULT_RM_STATE_STORE_MAX_COMPLETED_APPLICATIONS is incompatible and will break any code that is accessing that constant. Can we deprecate it instead?

        Show
        kasha Karthik Kambatla added a comment - Latest patch looks good, but for one issue: dropping DEFAULT_RM_STATE_STORE_MAX_COMPLETED_APPLICATIONS is incompatible and will break any code that is accessing that constant. Can we deprecate it instead?
        Hide
        templedf Daniel Templeton added a comment -

        Changing the defaults is already incompatible, so this is only going into trunk/3.0. Do we also need to worry about removing the constant in that case?

        Show
        templedf Daniel Templeton added a comment - Changing the defaults is already incompatible, so this is only going into trunk/3.0. Do we also need to worry about removing the constant in that case?
        Hide
        kasha Karthik Kambatla added a comment -

        I find it silly that anyone would be using these constants. That said, if people do, their code will need changing.

        Our compatibility guidelines say we can't remove @Public @Stable APIs without deprecating them for a major release.

        Show
        kasha Karthik Kambatla added a comment - I find it silly that anyone would be using these constants. That said, if people do, their code will need changing. Our compatibility guidelines say we can't remove @Public @Stable APIs without deprecating them for a major release.
        Hide
        templedf Daniel Templeton added a comment -

        How about we deprecate in branch-2? It's already unused there.

        Show
        templedf Daniel Templeton added a comment - How about we deprecate in branch-2? It's already unused there.
        Hide
        kasha Karthik Kambatla added a comment -

        We should deprecate in branch-2 and not use it, but that does not absolve us of deprecating in trunk also. BTW, mind posting a patch for branch-2 as well for the change outside of changing default value?

        Show
        kasha Karthik Kambatla added a comment - We should deprecate in branch-2 and not use it, but that does not absolve us of deprecating in trunk also. BTW, mind posting a patch for branch-2 as well for the change outside of changing default value?
        Hide
        kasha Karthik Kambatla added a comment -

        Canceling patch to address review comments.

        Show
        kasha Karthik Kambatla added a comment - Canceling patch to address review comments.
        Hide
        templedf Daniel Templeton added a comment -

        Addressed comments.

        Show
        templedf Daniel Templeton added a comment - Addressed comments.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 11s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        0 mvndep 0m 55s Maven dependency ordering for branch
        +1 mvninstall 7m 1s trunk passed
        +1 compile 2m 28s trunk passed
        +1 checkstyle 0m 42s trunk passed
        +1 mvnsite 1m 34s trunk passed
        +1 mvneclipse 0m 41s trunk passed
        +1 findbugs 3m 2s trunk passed
        +1 javadoc 1m 6s trunk passed
        0 mvndep 0m 10s Maven dependency ordering for patch
        +1 mvninstall 1m 21s the patch passed
        +1 compile 2m 19s the patch passed
        +1 javac 2m 19s the patch passed
        +1 checkstyle 0m 39s hadoop-yarn-project/hadoop-yarn: The patch generated 0 new + 240 unchanged - 1 fixed = 240 total (was 241)
        +1 mvnsite 1m 28s the patch passed
        +1 mvneclipse 0m 36s the patch passed
        -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix.
        +1 xml 0m 2s The patch has no ill-formed XML file.
        +1 findbugs 3m 10s the patch passed
        +1 javadoc 0m 59s the patch passed
        +1 unit 0m 24s hadoop-yarn-api in the patch passed.
        +1 unit 2m 17s hadoop-yarn-common in the patch passed.
        +1 unit 35m 22s hadoop-yarn-server-resourcemanager in the patch passed.
        +1 asflicense 0m 18s The patch does not generate ASF License warnings.
        67m 40s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12832214/YARN-4464.007.patch
        JIRA Issue YARN-4464
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml
        uname Linux 44afbc415b20 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 6a38d11
        Default Java 1.8.0_101
        findbugs v3.0.0
        whitespace https://builds.apache.org/job/PreCommit-YARN-Build/13320/artifact/patchprocess/whitespace-eol.txt
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13320/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/13320/console
        Powered by Apache Yetus 0.3.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 11s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. 0 mvndep 0m 55s Maven dependency ordering for branch +1 mvninstall 7m 1s trunk passed +1 compile 2m 28s trunk passed +1 checkstyle 0m 42s trunk passed +1 mvnsite 1m 34s trunk passed +1 mvneclipse 0m 41s trunk passed +1 findbugs 3m 2s trunk passed +1 javadoc 1m 6s trunk passed 0 mvndep 0m 10s Maven dependency ordering for patch +1 mvninstall 1m 21s the patch passed +1 compile 2m 19s the patch passed +1 javac 2m 19s the patch passed +1 checkstyle 0m 39s hadoop-yarn-project/hadoop-yarn: The patch generated 0 new + 240 unchanged - 1 fixed = 240 total (was 241) +1 mvnsite 1m 28s the patch passed +1 mvneclipse 0m 36s the patch passed -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 xml 0m 2s The patch has no ill-formed XML file. +1 findbugs 3m 10s the patch passed +1 javadoc 0m 59s the patch passed +1 unit 0m 24s hadoop-yarn-api in the patch passed. +1 unit 2m 17s hadoop-yarn-common in the patch passed. +1 unit 35m 22s hadoop-yarn-server-resourcemanager in the patch passed. +1 asflicense 0m 18s The patch does not generate ASF License warnings. 67m 40s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12832214/YARN-4464.007.patch JIRA Issue YARN-4464 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml uname Linux 44afbc415b20 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 6a38d11 Default Java 1.8.0_101 findbugs v3.0.0 whitespace https://builds.apache.org/job/PreCommit-YARN-Build/13320/artifact/patchprocess/whitespace-eol.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13320/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn Console output https://builds.apache.org/job/PreCommit-YARN-Build/13320/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
        Hide
        templedf Daniel Templeton added a comment -

        Another patch to remove the space that sneaked in.

        Show
        templedf Daniel Templeton added a comment - Another patch to remove the space that sneaked in.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 17s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        0 mvndep 0m 57s Maven dependency ordering for branch
        +1 mvninstall 8m 17s trunk passed
        +1 compile 3m 1s trunk passed
        +1 checkstyle 0m 48s trunk passed
        +1 mvnsite 1m 52s trunk passed
        +1 mvneclipse 0m 47s trunk passed
        +1 findbugs 3m 22s trunk passed
        +1 javadoc 1m 16s trunk passed
        0 mvndep 0m 9s Maven dependency ordering for patch
        +1 mvninstall 1m 30s the patch passed
        +1 compile 2m 38s the patch passed
        +1 javac 2m 38s the patch passed
        +1 checkstyle 0m 45s hadoop-yarn-project/hadoop-yarn: The patch generated 0 new + 240 unchanged - 1 fixed = 240 total (was 241)
        +1 mvnsite 1m 41s the patch passed
        +1 mvneclipse 0m 42s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 xml 0m 2s The patch has no ill-formed XML file.
        +1 findbugs 3m 34s the patch passed
        +1 javadoc 0m 58s the patch passed
        +1 unit 0m 23s hadoop-yarn-api in the patch passed.
        +1 unit 2m 15s hadoop-yarn-common in the patch passed.
        -1 unit 35m 4s hadoop-yarn-server-resourcemanager in the patch failed.
        +1 asflicense 0m 18s The patch does not generate ASF License warnings.
        71m 30s



        Reason Tests
        Failed junit tests hadoop.yarn.server.resourcemanager.TestRMRestart



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12832271/YARN-4464.008.patch
        JIRA Issue YARN-4464
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml
        uname Linux be142f178059 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 4d10621
        Default Java 1.8.0_101
        findbugs v3.0.0
        unit https://builds.apache.org/job/PreCommit-YARN-Build/13323/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
        unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/13323/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
        Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13323/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/13323/console
        Powered by Apache Yetus 0.3.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 17s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. 0 mvndep 0m 57s Maven dependency ordering for branch +1 mvninstall 8m 17s trunk passed +1 compile 3m 1s trunk passed +1 checkstyle 0m 48s trunk passed +1 mvnsite 1m 52s trunk passed +1 mvneclipse 0m 47s trunk passed +1 findbugs 3m 22s trunk passed +1 javadoc 1m 16s trunk passed 0 mvndep 0m 9s Maven dependency ordering for patch +1 mvninstall 1m 30s the patch passed +1 compile 2m 38s the patch passed +1 javac 2m 38s the patch passed +1 checkstyle 0m 45s hadoop-yarn-project/hadoop-yarn: The patch generated 0 new + 240 unchanged - 1 fixed = 240 total (was 241) +1 mvnsite 1m 41s the patch passed +1 mvneclipse 0m 42s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 2s The patch has no ill-formed XML file. +1 findbugs 3m 34s the patch passed +1 javadoc 0m 58s the patch passed +1 unit 0m 23s hadoop-yarn-api in the patch passed. +1 unit 2m 15s hadoop-yarn-common in the patch passed. -1 unit 35m 4s hadoop-yarn-server-resourcemanager in the patch failed. +1 asflicense 0m 18s The patch does not generate ASF License warnings. 71m 30s Reason Tests Failed junit tests hadoop.yarn.server.resourcemanager.TestRMRestart Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12832271/YARN-4464.008.patch JIRA Issue YARN-4464 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml uname Linux be142f178059 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 4d10621 Default Java 1.8.0_101 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-YARN-Build/13323/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt unit test logs https://builds.apache.org/job/PreCommit-YARN-Build/13323/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt Test Results https://builds.apache.org/job/PreCommit-YARN-Build/13323/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn Console output https://builds.apache.org/job/PreCommit-YARN-Build/13323/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
        Hide
        kasha Karthik Kambatla added a comment -

        The test failure appears related. Daniel Templeton - mind looking into it?

        Show
        kasha Karthik Kambatla added a comment - The test failure appears related. Daniel Templeton - mind looking into it?
        Hide
        kasha Karthik Kambatla added a comment -

        Actually, the test passes for me locally with the patch applied.

        +1. Checking this in..

        Show
        kasha Karthik Kambatla added a comment - Actually, the test passes for me locally with the patch applied. +1. Checking this in..
        Hide
        kasha Karthik Kambatla added a comment -

        Thanks Daniel Templeton for working on this, and Naganarasimha Garla, Jian He, Vinod Kumar Vavilapalli for your inputs.

        Just committed this to trunk.

        Show
        kasha Karthik Kambatla added a comment - Thanks Daniel Templeton for working on this, and Naganarasimha Garla , Jian He , Vinod Kumar Vavilapalli for your inputs. Just committed this to trunk.
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10595 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10595/)
        YARN-4464. Lower the default max applications stored in the RM and (kasha: rev 6378845f9ef789c3fda862c43bcd498aa3f35068)

        • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
        • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMAppManager.java
        • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10595 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10595/ ) YARN-4464 . Lower the default max applications stored in the RM and (kasha: rev 6378845f9ef789c3fda862c43bcd498aa3f35068) (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMAppManager.java (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml

          People

          • Assignee:
            templedf Daniel Templeton
            Reporter:
            magnum KWON BYUNGCHANG
          • Votes:
            0 Vote for this issue
            Watchers:
            13 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development