Hive
  1. Hive
  2. HIVE-3235

Support jobId notification for MaredTasks

    Details

    • Type: Improvement Improvement
    • Status: Patch Available
    • Priority: Trivial Trivial
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Diagnosability
    • Labels:
      None

      Description

      JobId for each MapredTask is requested for monitoring purpose.

        Activity

        Hide
        Navis added a comment -
        Show
        Navis added a comment - https://reviews.facebook.net/D4017
        Hide
        Edward Capriolo added a comment -

        Q: How is this different then hadoop's job notification? Can we leverage that code?

        Show
        Edward Capriolo added a comment - Q: How is this different then hadoop's job notification? Can we leverage that code?
        Hide
        Navis added a comment -

        End notification is delegated to hadoop, but I couldn't find start notification in that.
        Modifying hadoop could be better but I simply added job start notification in ExecDriver (notifying code is also copied from hadoop except retrying policy).

        Show
        Navis added a comment - End notification is delegated to hadoop, but I couldn't find start notification in that. Modifying hadoop could be better but I simply added job start notification in ExecDriver (notifying code is also copied from hadoop except retrying policy).
        Hide
        Zhenxiao Luo added a comment -

        How about file a Hadoop Jira to get JobStartNotification, and make this Jira depends on that one?

        Show
        Zhenxiao Luo added a comment - How about file a Hadoop Jira to get JobStartNotification, and make this Jira depends on that one?
        Hide
        Zhenxiao Luo added a comment -

        comments on Phabricator

        Show
        Zhenxiao Luo added a comment - comments on Phabricator
        Hide
        Phabricator added a comment -

        zhenxiao has commented on the revision "HIVE-3235 [jira] Support jobId notification for MaredTasks".

        INLINE COMMENTS
        ql/src/java/org/apache/hadoop/hive/ql/exec/Task.java:156 log error instead of warning?
        ql/src/java/org/apache/hadoop/hive/ql/exec/Task.java:152 log error instead of warning?

        REVISION DETAIL
        https://reviews.facebook.net/D4017

        To: JIRA, navis
        Cc: zhenxiao

        Show
        Phabricator added a comment - zhenxiao has commented on the revision " HIVE-3235 [jira] Support jobId notification for MaredTasks". INLINE COMMENTS ql/src/java/org/apache/hadoop/hive/ql/exec/Task.java:156 log error instead of warning? ql/src/java/org/apache/hadoop/hive/ql/exec/Task.java:152 log error instead of warning? REVISION DETAIL https://reviews.facebook.net/D4017 To: JIRA, navis Cc: zhenxiao
        Hide
        Phabricator added a comment -

        navis updated the revision "HIVE-3235 [jira] Support jobId notification for MaredTasks".

        Addressed comments & rebased to trunk

        Reviewers: JIRA

        REVISION DETAIL
        https://reviews.facebook.net/D4017

        CHANGE SINCE LAST DIFF
        https://reviews.facebook.net/D4017?vs=12885&id=28605#toc

        AFFECTED FILES
        common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
        ql/src/java/org/apache/hadoop/hive/ql/exec/ExecDriver.java
        ql/src/java/org/apache/hadoop/hive/ql/exec/Task.java

        To: JIRA, navis
        Cc: zhenxiao

        Show
        Phabricator added a comment - navis updated the revision " HIVE-3235 [jira] Support jobId notification for MaredTasks". Addressed comments & rebased to trunk Reviewers: JIRA REVISION DETAIL https://reviews.facebook.net/D4017 CHANGE SINCE LAST DIFF https://reviews.facebook.net/D4017?vs=12885&id=28605#toc AFFECTED FILES common/src/java/org/apache/hadoop/hive/conf/HiveConf.java ql/src/java/org/apache/hadoop/hive/ql/exec/ExecDriver.java ql/src/java/org/apache/hadoop/hive/ql/exec/Task.java To: JIRA, navis Cc: zhenxiao

          People

          • Assignee:
            Navis
            Reporter:
            Navis
          • Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:

              Development