Hive
  1. Hive
  2. HIVE-2969

Log Time To Submit metric with PerfLogger

    Details

    • Type: Wish Wish
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.10.0
    • Component/s: Logging
    • Labels:
      None

      Description

      Logging the time from when Driver.run starts to when we begin submitting jobs to map reduce would be helpful in determining how much of the lag in starting a query is due to Hive vs. Hadoop.

        Activity

        Transition Time In Source Status Execution Times Last Executer Last Execution Date
        Open Open Patch Available Patch Available
        3m 17s 1 Kevin Wilfong 21/Apr/12 01:58
        Patch Available Patch Available Resolved Resolved
        51d 17h 23m 1 Kevin Wilfong 11/Jun/12 19:21
        Resolved Resolved Closed Closed
        213d 1h 31m 1 Ashutosh Chauhan 10/Jan/13 19:53
        Ashutosh Chauhan made changes -
        Status Resolved [ 5 ] Closed [ 6 ]
        Hide
        Ashutosh Chauhan added a comment -

        This issue is fixed and released as part of 0.10.0 release. If you find an issue which seems to be related to this one, please create a new jira and link this one with new jira.

        Show
        Ashutosh Chauhan added a comment - This issue is fixed and released as part of 0.10.0 release. If you find an issue which seems to be related to this one, please create a new jira and link this one with new jira.
        Hide
        Hudson added a comment -

        Integrated in Hive-trunk-hadoop2 #54 (See https://builds.apache.org/job/Hive-trunk-hadoop2/54/)
        HIVE-2969. Log Time To Submit metric with PerfLogger. (kevinwilfong reviewed by ashutoshc) (Revision 1347870)

        Result = ABORTED
        kevinwilfong : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1347870
        Files :

        • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/Driver.java
        • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/log/PerfLogger.java
        Show
        Hudson added a comment - Integrated in Hive-trunk-hadoop2 #54 (See https://builds.apache.org/job/Hive-trunk-hadoop2/54/ ) HIVE-2969 . Log Time To Submit metric with PerfLogger. (kevinwilfong reviewed by ashutoshc) (Revision 1347870) Result = ABORTED kevinwilfong : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1347870 Files : /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/Driver.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/log/PerfLogger.java
        Ashutosh Chauhan made changes -
        Fix Version/s 0.10.0 [ 12320745 ]
        Affects Version/s 0.10.0 [ 12320745 ]
        Kevin Wilfong made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Resolution Fixed [ 1 ]
        Hide
        Kevin Wilfong added a comment -

        Committed

        Show
        Kevin Wilfong added a comment - Committed
        Hide
        Hudson added a comment -

        Integrated in Hive-trunk-h0.21 #1474 (See https://builds.apache.org/job/Hive-trunk-h0.21/1474/)
        HIVE-2969. Log Time To Submit metric with PerfLogger. (kevinwilfong reviewed by ashutoshc) (Revision 1347870)

        Result = FAILURE
        kevinwilfong : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1347870
        Files :

        • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/Driver.java
        • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/log/PerfLogger.java
        Show
        Hudson added a comment - Integrated in Hive-trunk-h0.21 #1474 (See https://builds.apache.org/job/Hive-trunk-h0.21/1474/ ) HIVE-2969 . Log Time To Submit metric with PerfLogger. (kevinwilfong reviewed by ashutoshc) (Revision 1347870) Result = FAILURE kevinwilfong : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1347870 Files : /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/Driver.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/log/PerfLogger.java
        Kevin Wilfong made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Phabricator made changes -
        Field Original Value New Value
        Attachment HIVE-2969.D2919.1.patch [ 12523602 ]
        Hide
        Phabricator added a comment -

        kevinwilfong requested code review of "HIVE-2969 [jira] Log Time To Submit metric with PerfLogger".
        Reviewers: JIRA

        https://issues.apache.org/jira/browse/HIVE-2969

        Added Time To Submit to the PerfLogger, beginning just after the PerfLogger is first initialized, and ending just before the loop which starts submitting jobs to map reduce.

        Logging the time from when Driver.run starts to when we begin submitting jobs to map reduce would be helpful in determining how much of the lag in starting a query is due to Hive vs. Hadoop.

        TEST PLAN
        EMPTY

        REVISION DETAIL
        https://reviews.facebook.net/D2919

        AFFECTED FILES
        ql/src/java/org/apache/hadoop/hive/ql/log/PerfLogger.java
        ql/src/java/org/apache/hadoop/hive/ql/Driver.java

        MANAGE HERALD DIFFERENTIAL RULES
        https://reviews.facebook.net/herald/view/differential/

        WHY DID I GET THIS EMAIL?
        https://reviews.facebook.net/herald/transcript/6651/

        Tip: use the X-Herald-Rules header to filter Herald messages in your client.

        Show
        Phabricator added a comment - kevinwilfong requested code review of " HIVE-2969 [jira] Log Time To Submit metric with PerfLogger". Reviewers: JIRA https://issues.apache.org/jira/browse/HIVE-2969 Added Time To Submit to the PerfLogger, beginning just after the PerfLogger is first initialized, and ending just before the loop which starts submitting jobs to map reduce. Logging the time from when Driver.run starts to when we begin submitting jobs to map reduce would be helpful in determining how much of the lag in starting a query is due to Hive vs. Hadoop. TEST PLAN EMPTY REVISION DETAIL https://reviews.facebook.net/D2919 AFFECTED FILES ql/src/java/org/apache/hadoop/hive/ql/log/PerfLogger.java ql/src/java/org/apache/hadoop/hive/ql/Driver.java MANAGE HERALD DIFFERENTIAL RULES https://reviews.facebook.net/herald/view/differential/ WHY DID I GET THIS EMAIL? https://reviews.facebook.net/herald/transcript/6651/ Tip: use the X-Herald-Rules header to filter Herald messages in your client.
        Kevin Wilfong created issue -

          People

          • Assignee:
            Kevin Wilfong
            Reporter:
            Kevin Wilfong
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development