Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-3230

Clarify application states on the web UI

    Details

    • Hadoop Flags:
      Reviewed

      Description

      Today, application state are simply surfaced as a single word on the web UI. Not everyone understands the meaning of "NEW_SAVING, SUBMITTED, ACCEPTED". This jira is to clarify the meaning of these states, things like what the application is waiting for at this state.

      In addition,the difference between application state and FinalStatus are fairly confusing to users, especially when state=FINISHED, but FinalStatus=FAILED

      1. application page.png
        109 kB
        Jian He
      2. YARN-3230.1.patch
        4 kB
        Jian He
      3. YARN-3230.2.patch
        6 kB
        Jian He
      4. YARN-3230.3.patch
        6 kB
        Jian He
      5. YARN-3230.3.patch
        6 kB
        Jian He

        Activity

        Hide
        jianhe Jian He added a comment -

        Uploaded a patch to add more text to clarify the application state

        Show
        jianhe Jian He added a comment - Uploaded a patch to add more text to clarify the application state
        Hide
        jianhe Jian He added a comment -

        uploaded an application page screen shot

        Show
        jianhe Jian He added a comment - uploaded an application page screen shot
        Hide
        hadoopqa Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12699760/application%20page.png
        against trunk revision d49ae72.

        -1 patch. The patch command could not apply the patch.

        Console output: https://builds.apache.org/job/PreCommit-YARN-Build/6672//console

        This message is automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12699760/application%20page.png against trunk revision d49ae72. -1 patch . The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-YARN-Build/6672//console This message is automatically generated.
        Hide
        leftnoteasy Wangda Tan added a comment -

        Jian He, thanks for working on this, generally looks good to me, some minor comments:
        1) "FinalStatus from Application's POV:" to "Final State Reported by Application Master"?
        2) NEW_SAVING: is not necessary to be seen by client?
        3) RUNNING: AM container has registered to RM and started running.

        Wangda

        Show
        leftnoteasy Wangda Tan added a comment - Jian He , thanks for working on this, generally looks good to me, some minor comments: 1) "FinalStatus from Application's POV:" to "Final State Reported by Application Master"? 2) NEW_SAVING: is not necessary to be seen by client? 3) RUNNING: AM container has registered to RM and started running. Wangda
        Hide
        jianhe Jian He added a comment -

        thanks for reviewing the patch, Wangda !

        2) NEW_SAVING: is not necessary to be seen by client?

        Agree, I think both NEW_SAVING and SUBMITTED are not necessary to be exposed as they are mostly transient and are internal to the system. I can open a jira for this.
        Addressed other comments.

        Show
        jianhe Jian He added a comment - thanks for reviewing the patch, Wangda ! 2) NEW_SAVING: is not necessary to be seen by client? Agree, I think both NEW_SAVING and SUBMITTED are not necessary to be exposed as they are mostly transient and are internal to the system. I can open a jira for this. Addressed other comments.
        Hide
        jlowe Jason Lowe added a comment -

        NEW_SAVING: is not necessary to be seen by client?

        I agree, it's not a state that seems relevant to a client. We're not even consistent about it, since FINAL_SAVING is not visible to clients and is silently translated to the previous state. However might be kind of hard to remove completely at this point since it's been published.

        Show
        jlowe Jason Lowe added a comment - NEW_SAVING: is not necessary to be seen by client? I agree, it's not a state that seems relevant to a client. We're not even consistent about it, since FINAL_SAVING is not visible to clients and is silently translated to the previous state. However might be kind of hard to remove completely at this point since it's been published.
        Hide
        hadoopqa Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12699766/YARN-3230.3.patch
        against trunk revision d49ae72.

        -1 patch. Trunk compilation may be broken.

        Console output: https://builds.apache.org/job/PreCommit-YARN-Build/6673//console

        This message is automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12699766/YARN-3230.3.patch against trunk revision d49ae72. -1 patch . Trunk compilation may be broken. Console output: https://builds.apache.org/job/PreCommit-YARN-Build/6673//console This message is automatically generated.
        Hide
        hadoopqa Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12699770/YARN-3230.3.patch
        against trunk revision d49ae72.

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. There were no new javadoc warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        -1 findbugs. The patch appears to introduce 5 new Findbugs (version 2.0.3) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.

        Test results: https://builds.apache.org/job/PreCommit-YARN-Build/6677//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-YARN-Build/6677//artifact/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-resourcemanager.html
        Console output: https://builds.apache.org/job/PreCommit-YARN-Build/6677//console

        This message is automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12699770/YARN-3230.3.patch against trunk revision d49ae72. +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. -1 findbugs . The patch appears to introduce 5 new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager. Test results: https://builds.apache.org/job/PreCommit-YARN-Build/6677//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-YARN-Build/6677//artifact/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-resourcemanager.html Console output: https://builds.apache.org/job/PreCommit-YARN-Build/6677//console This message is automatically generated.
        Hide
        leftnoteasy Wangda Tan added a comment -

        Since the new_saving issue seems hard to fit in this ticket, I suggest to file a separated one to tracking it.

        Patch looks good to me, findbugs warning not related to this patch. I will commit it today.

        Show
        leftnoteasy Wangda Tan added a comment - Since the new_saving issue seems hard to fit in this ticket, I suggest to file a separated one to tracking it. Patch looks good to me, findbugs warning not related to this patch. I will commit it today.
        Hide
        leftnoteasy Wangda Tan added a comment -

        Committing..

        Show
        leftnoteasy Wangda Tan added a comment - Committing..
        Hide
        leftnoteasy Wangda Tan added a comment -

        Committed to trunk and branch-2, thanks Jian!

        Show
        leftnoteasy Wangda Tan added a comment - Committed to trunk and branch-2, thanks Jian!
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Hadoop-trunk-Commit #7164 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7164/)
        YARN-3230. Clarify application states on the web UI. (Jian He via wangda) (wangda: rev ce5bf927c3d9f212798de1bf8706e5e9def235a1)

        • hadoop-yarn-project/CHANGES.txt
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/AppsBlock.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/AppBlock.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/AppInfo.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-trunk-Commit #7164 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7164/ ) YARN-3230 . Clarify application states on the web UI. (Jian He via wangda) (wangda: rev ce5bf927c3d9f212798de1bf8706e5e9def235a1) hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/AppsBlock.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/AppBlock.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/AppInfo.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #111 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/111/)
        YARN-3230. Clarify application states on the web UI. (Jian He via wangda) (wangda: rev ce5bf927c3d9f212798de1bf8706e5e9def235a1)

        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/AppBlock.java
        • hadoop-yarn-project/CHANGES.txt
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/AppInfo.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/AppsBlock.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #111 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/111/ ) YARN-3230 . Clarify application states on the web UI. (Jian He via wangda) (wangda: rev ce5bf927c3d9f212798de1bf8706e5e9def235a1) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/AppBlock.java hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/AppInfo.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/AppsBlock.java
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Hadoop-Yarn-trunk #845 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/845/)
        YARN-3230. Clarify application states on the web UI. (Jian He via wangda) (wangda: rev ce5bf927c3d9f212798de1bf8706e5e9def235a1)

        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/AppsBlock.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/AppBlock.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/AppInfo.java
        • hadoop-yarn-project/CHANGES.txt
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Yarn-trunk #845 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/845/ ) YARN-3230 . Clarify application states on the web UI. (Jian He via wangda) (wangda: rev ce5bf927c3d9f212798de1bf8706e5e9def235a1) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/AppsBlock.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/AppBlock.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/AppInfo.java hadoop-yarn-project/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Hdfs-trunk #2043 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2043/)
        YARN-3230. Clarify application states on the web UI. (Jian He via wangda) (wangda: rev ce5bf927c3d9f212798de1bf8706e5e9def235a1)

        • hadoop-yarn-project/CHANGES.txt
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/AppsBlock.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/AppBlock.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/AppInfo.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2043 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2043/ ) YARN-3230 . Clarify application states on the web UI. (Jian He via wangda) (wangda: rev ce5bf927c3d9f212798de1bf8706e5e9def235a1) hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/AppsBlock.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/AppBlock.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/AppInfo.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #102 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/102/)
        YARN-3230. Clarify application states on the web UI. (Jian He via wangda) (wangda: rev ce5bf927c3d9f212798de1bf8706e5e9def235a1)

        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/AppsBlock.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/AppInfo.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/AppBlock.java
        • hadoop-yarn-project/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #102 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/102/ ) YARN-3230 . Clarify application states on the web UI. (Jian He via wangda) (wangda: rev ce5bf927c3d9f212798de1bf8706e5e9def235a1) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/AppsBlock.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/AppInfo.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/AppBlock.java hadoop-yarn-project/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #112 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/112/)
        YARN-3230. Clarify application states on the web UI. (Jian He via wangda) (wangda: rev ce5bf927c3d9f212798de1bf8706e5e9def235a1)

        • hadoop-yarn-project/CHANGES.txt
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/AppsBlock.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/AppInfo.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/AppBlock.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #112 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/112/ ) YARN-3230 . Clarify application states on the web UI. (Jian He via wangda) (wangda: rev ce5bf927c3d9f212798de1bf8706e5e9def235a1) hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/AppsBlock.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/AppInfo.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/AppBlock.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Mapreduce-trunk #2062 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2062/)
        YARN-3230. Clarify application states on the web UI. (Jian He via wangda) (wangda: rev ce5bf927c3d9f212798de1bf8706e5e9def235a1)

        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/AppInfo.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/AppsBlock.java
        • hadoop-yarn-project/CHANGES.txt
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/AppBlock.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #2062 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2062/ ) YARN-3230 . Clarify application states on the web UI. (Jian He via wangda) (wangda: rev ce5bf927c3d9f212798de1bf8706e5e9def235a1) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/AppInfo.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/AppsBlock.java hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/AppBlock.java
        Hide
        vinodkv Vinod Kumar Vavilapalli added a comment -

        Pulled this into 2.6.1 as a dependency for YARN-1809. Ran compilation before the push. Patch applied cleanly.

        Show
        vinodkv Vinod Kumar Vavilapalli added a comment - Pulled this into 2.6.1 as a dependency for YARN-1809 . Ran compilation before the push. Patch applied cleanly.

          People

          • Assignee:
            jianhe Jian He
            Reporter:
            jianhe Jian He
          • Votes:
            0 Vote for this issue
            Watchers:
            8 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development