Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-4207

Add a non-judgemental YARN app completion status

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: None
    • Labels:
    • Target Version/s:
    • Release Note:
      Adds the ENDED attribute to o.a.h.yarn.api.records.FinalApplicationStatus

      Description

      For certain applications, it doesn't make sense to have SUCCEEDED or FAILED end state. For example, Tez sessions may include multiple DAGs, some of which have succeeded and some have failed; there's no clear status for the session both logically and from user perspective (users are confused either way).
      There needs to be a status not implying success or failure, such as "done"/"ended"/"finished".

        Issue Links

          Activity

          Hide
          sershe Sergey Shelukhin added a comment -
          Show
          sershe Sergey Shelukhin added a comment - Siddharth Seth
          Hide
          sseth Siddharth Seth added a comment -
          Show
          sseth Siddharth Seth added a comment - cc Vinod Kumar Vavilapalli
          Hide
          rhaase Rich Haase added a comment -

          This looks like a pretty trivial change. Adding an additional value to the o.a.h.yarn.records.FinalApplicationStatus enum. In a quick search I didn't see anything downstream within Hadoop that would be impacted by such a patch. If no one else is working on this JIRA and the approach I've described is acceptable I will put together a patch.

          Show
          rhaase Rich Haase added a comment - This looks like a pretty trivial change. Adding an additional value to the o.a.h.yarn.records.FinalApplicationStatus enum. In a quick search I didn't see anything downstream within Hadoop that would be impacted by such a patch. If no one else is working on this JIRA and the approach I've described is acceptable I will put together a patch.
          Hide
          sershe Sergey Shelukhin added a comment -

          It's unassigned, so I gather noone is working on it. This plan sounds good to me (non-binding )

          Show
          sershe Sergey Shelukhin added a comment - It's unassigned, so I gather noone is working on it. This plan sounds good to me (non-binding )
          Hide
          rhaase Rich Haase added a comment -

          Trivial patch that adds ENDED as a YARN application status.

          Show
          rhaase Rich Haase added a comment - Trivial patch that adds ENDED as a YARN application status.
          Hide
          sseth Siddharth Seth added a comment -

          Rich Haase - thanks for taking this up.
          Along with the change to FinalApplicationStatus, a change is also required to the proto definition (yarn_protos.proto). There'll be a set of converter methods which translate between the proto and FinalApplicationStatus which will also need to be changed.

          Other than that, I believe adding this additional value is a safe change.

          Show
          sseth Siddharth Seth added a comment - Rich Haase - thanks for taking this up. Along with the change to FinalApplicationStatus, a change is also required to the proto definition (yarn_protos.proto). There'll be a set of converter methods which translate between the proto and FinalApplicationStatus which will also need to be changed. Other than that, I believe adding this additional value is a safe change.
          Hide
          rhaase Rich Haase added a comment -

          Sorry for the slow response all. I've update the patch to include the new FinalApplicationStatus in yarn_protos.proto.

          Show
          rhaase Rich Haase added a comment - Sorry for the slow response all. I've update the patch to include the new FinalApplicationStatus in yarn_protos.proto.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 0s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          +1 mvninstall 7m 48s trunk passed
          +1 compile 0m 21s trunk passed with JDK v1.8.0_66
          +1 compile 0m 24s trunk passed with JDK v1.7.0_91
          +1 checkstyle 0m 9s trunk passed
          +1 mvnsite 0m 28s trunk passed
          +1 mvneclipse 0m 12s trunk passed
          +1 findbugs 1m 19s trunk passed
          +1 javadoc 0m 38s trunk passed with JDK v1.8.0_66
          +1 javadoc 2m 46s trunk passed with JDK v1.7.0_91
          +1 mvninstall 0m 27s the patch passed
          +1 compile 0m 21s the patch passed with JDK v1.8.0_66
          +1 cc 0m 21s the patch passed
          +1 javac 0m 21s the patch passed
          +1 compile 0m 24s the patch passed with JDK v1.7.0_91
          +1 cc 0m 24s the patch passed
          +1 javac 0m 24s the patch passed
          +1 checkstyle 0m 9s the patch passed
          +1 mvnsite 0m 29s the patch passed
          +1 mvneclipse 0m 13s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 findbugs 1m 27s the patch passed
          +1 javadoc 0m 36s the patch passed with JDK v1.8.0_66
          +1 javadoc 2m 43s the patch passed with JDK v1.7.0_91
          +1 unit 0m 21s hadoop-yarn-api in the patch passed with JDK v1.8.0_66.
          +1 unit 0m 25s hadoop-yarn-api in the patch passed with JDK v1.7.0_91.
          +1 asflicense 0m 24s Patch does not generate ASF License warnings.
          23m 14s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:0ca8df7
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12777907/YARN-4207.patch
          JIRA Issue YARN-4207
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc
          uname Linux c411f1e9f39a 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / ae67c84
          findbugs v3.0.0
          JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/10002/testReport/
          modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api
          Max memory used 75MB
          Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org
          Console output https://builds.apache.org/job/PreCommit-YARN-Build/10002/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 7m 48s trunk passed +1 compile 0m 21s trunk passed with JDK v1.8.0_66 +1 compile 0m 24s trunk passed with JDK v1.7.0_91 +1 checkstyle 0m 9s trunk passed +1 mvnsite 0m 28s trunk passed +1 mvneclipse 0m 12s trunk passed +1 findbugs 1m 19s trunk passed +1 javadoc 0m 38s trunk passed with JDK v1.8.0_66 +1 javadoc 2m 46s trunk passed with JDK v1.7.0_91 +1 mvninstall 0m 27s the patch passed +1 compile 0m 21s the patch passed with JDK v1.8.0_66 +1 cc 0m 21s the patch passed +1 javac 0m 21s the patch passed +1 compile 0m 24s the patch passed with JDK v1.7.0_91 +1 cc 0m 24s the patch passed +1 javac 0m 24s the patch passed +1 checkstyle 0m 9s the patch passed +1 mvnsite 0m 29s the patch passed +1 mvneclipse 0m 13s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 1m 27s the patch passed +1 javadoc 0m 36s the patch passed with JDK v1.8.0_66 +1 javadoc 2m 43s the patch passed with JDK v1.7.0_91 +1 unit 0m 21s hadoop-yarn-api in the patch passed with JDK v1.8.0_66. +1 unit 0m 25s hadoop-yarn-api in the patch passed with JDK v1.7.0_91. +1 asflicense 0m 24s Patch does not generate ASF License warnings. 23m 14s Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12777907/YARN-4207.patch JIRA Issue YARN-4207 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc uname Linux c411f1e9f39a 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / ae67c84 findbugs v3.0.0 JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/10002/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api Max memory used 75MB Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-YARN-Build/10002/console This message was automatically generated.
          Hide
          rhaase Rich Haase added a comment -

          Justification for not adding a test: adding a value to an enum is to trivial to bother testing.

          Show
          rhaase Rich Haase added a comment - Justification for not adding a test: adding a value to an enum is to trivial to bother testing.
          Hide
          sseth Siddharth Seth added a comment -

          +1. This looks good. Thanks Rich Haase

          Show
          sseth Siddharth Seth added a comment - +1. This looks good. Thanks Rich Haase
          Hide
          sseth Siddharth Seth added a comment -

          Committed to trunk, branch-2 and branch-2.8

          Show
          sseth Siddharth Seth added a comment - Committed to trunk, branch-2 and branch-2.8
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #8977 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8977/)
          YARN-4207. Add a non-judgemental YARN app completion status. Contributed (sseth: rev 0f708d465fbc4260f2c36e8067e27cd8b285fde7)

          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_protos.proto
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/FinalApplicationStatus.java
          • hadoop-yarn-project/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #8977 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8977/ ) YARN-4207 . Add a non-judgemental YARN app completion status. Contributed (sseth: rev 0f708d465fbc4260f2c36e8067e27cd8b285fde7) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_protos.proto hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/FinalApplicationStatus.java hadoop-yarn-project/CHANGES.txt

            People

            • Assignee:
              rhaase Rich Haase
              Reporter:
              sershe Sergey Shelukhin
            • Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development