Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-9673

Set operationhandle in ATS entities for lookups

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.2.0
    • Component/s: None
    • Labels:
      None

      Description

      Yarn App Timeline Server (ATS) users can find their query using hive query-id.
      However, query id is available only through the logs at the moment.
      Thrift api users such as Hue have another unique id for queries, which the operation handle contains (TExecuteStatementResp.TOperationHandle.THandleIdentifier.guid) . Adding the operationhandle guid to ATS will enable such thrift users to get information from ATS for the queries that they have spawned.

      1. HIVE-9673.1.patch
        10 kB
        Thejas M Nair
      2. HIVE-9673.2.patch
        10 kB
        Thejas M Nair

        Activity

        Hide
        thejas Thejas M Nair added a comment -

        HIVE-9673.2.patch - using url safe encoding of guid

        Show
        thejas Thejas M Nair added a comment - HIVE-9673 .2.patch - using url safe encoding of guid
        Hide
        hagleitn Gunther Hagleitner added a comment -

        Looks good - but how do you know that none of the following objs are going to be null (handle/id/...)?

        +      String guid64 = Base64.encodeBase64URLSafeString(getHandle().getHandleIdentifier()
        +          .toTHandleIdentifier().getGuid()).trim();
        
        Show
        hagleitn Gunther Hagleitner added a comment - Looks good - but how do you know that none of the following objs are going to be null (handle/id/...)? + String guid64 = Base64.encodeBase64URLSafeString(getHandle().getHandleIdentifier() + .toTHandleIdentifier().getGuid()).trim();
        Hide
        thejas Thejas M Nair added a comment -

        Operations always have handle, it is set in the constructor. HandleIdentifier is also set in Handle constructor. So it will not be null.

        Show
        thejas Thejas M Nair added a comment - Operations always have handle, it is set in the constructor. HandleIdentifier is also set in Handle constructor. So it will not be null.
        Hide
        hagleitn Gunther Hagleitner added a comment -

        +1

        Show
        hagleitn Gunther Hagleitner added a comment - +1
        Hide
        hiveqa Hive QA added a comment -

        Overall: -1 at least one tests failed

        Here are the results of testing the latest attachment:
        https://issues.apache.org/jira/secure/attachment/12698578/HIVE-9673.2.patch

        ERROR: -1 due to 1 failed/errored test(s), 7542 tests executed
        Failed tests:

        org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_udaf_percentile_approx_23
        

        Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2785/testReport
        Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2785/console
        Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-2785/

        Messages:

        Executing org.apache.hive.ptest.execution.PrepPhase
        Executing org.apache.hive.ptest.execution.ExecutionPhase
        Executing org.apache.hive.ptest.execution.ReportingPhase
        Tests exited with: TestsFailedException: 1 tests failed
        

        This message is automatically generated.

        ATTACHMENT ID: 12698578 - PreCommit-HIVE-TRUNK-Build

        Show
        hiveqa Hive QA added a comment - Overall : -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12698578/HIVE-9673.2.patch ERROR: -1 due to 1 failed/errored test(s), 7542 tests executed Failed tests: org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_udaf_percentile_approx_23 Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2785/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2785/console Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-2785/ Messages: Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 1 tests failed This message is automatically generated. ATTACHMENT ID: 12698578 - PreCommit-HIVE-TRUNK-Build
        Hide
        thejas Thejas M Nair added a comment -

        testCliDriver_udaf_percentile_approx_23 failure is not related to this one, it has been failing in many other builds.

        Show
        thejas Thejas M Nair added a comment - testCliDriver_udaf_percentile_approx_23 failure is not related to this one, it has been failing in many other builds.
        Hide
        hagleitn Gunther Hagleitner added a comment -

        committed to trunk. thanks Thejas M Nair!

        Show
        hagleitn Gunther Hagleitner added a comment - committed to trunk. thanks Thejas M Nair !
        Hide
        leftylev Lefty Leverenz added a comment -

        Does this need Hive documentation, or maybe YARN documentation?

        Show
        leftylev Lefty Leverenz added a comment - Does this need Hive documentation, or maybe YARN documentation?
        Hide
        sushanth Sushanth Sowmyan added a comment -

        This issue has been fixed and released as part of the 1.2.0 release. If you find an issue which seems to be related to this one, please create a new jira and link this one with new jira.

        Show
        sushanth Sushanth Sowmyan added a comment - This issue has been fixed and released as part of the 1.2.0 release. If you find an issue which seems to be related to this one, please create a new jira and link this one with new jira.

          People

          • Assignee:
            thejas Thejas M Nair
            Reporter:
            thejas Thejas M Nair
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development