Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-15572

Improve the response time for query canceling when it happens during acquiring locks

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None
    • Target Version/s:

      Description

      When query canceling command sent during Hive Acquire locks (from zookeeper), hive will finish acquiring all the locks and release them. As it is shown in the following log:
      It took 165 s to finish acquire the lock,then spend 81s to release them.
      We can improve the performance by not acquiring any more locks and releasing held locks when the query canceling command is received.

      Background-Pool: Thread-224]: <PERFLOG method=TimeToSubmit from=org.apache.hadoop.hive.ql.Driver>
      2017-01-03 10:50:35,413 INFO  org.apache.hadoop.hive.ql.log.PerfLogger: [HiveServer2-Background-Pool: Thread-224]: <PERFLOG method=acquireReadWriteLocks from=org.apache.hadoop.hive.ql.Driver>
      2017-01-03 10:51:00,671 INFO  org.apache.hadoop.hive.ql.log.PerfLogger: [HiveServer2-Background-Pool: Thread-218]: </PERFLOG method=acquireReadWriteLocks start=1483469295080 end=1483469460671 duration=165591 from=org.apache.hadoop.hive.ql.Driver>
      2017-01-03 10:51:00,672 INFO  org.apache.hadoop.hive.ql.log.PerfLogger: [HiveServer2-Background-Pool: Thread-218]: <PERFLOG method=Driver.execute from=org.apache.hadoop.hive.ql.Driver>
      2017-01-03 10:51:00,672 ERROR org.apache.hadoop.hive.ql.Driver: [HiveServer2-Background-Pool: Thread-218]: FAILED: query select count(*) from manyparttbl has been cancelled
      2017-01-03 10:51:00,673 INFO  org.apache.hadoop.hive.ql.log.PerfLogger: [HiveServer2-Background-Pool: Thread-218]: <PERFLOG method=releaseLocks from=org.apache.hadoop.hive.ql.Driver>
      2017-01-03 10:51:40,755 INFO  org.apache.hadoop.hive.ql.log.PerfLogger: [HiveServer2-Background-Pool: Thread-215]: </PERFLOG method=releaseLocks start=1483469419487 end=1483469500755 duration=81268 from=org.apache.hadoop.hive.ql.Driver>
      

        Attachments

        1. HIVE-15572.2.patch
          26 kB
          Yongzhi Chen
        2. HIVE-15572.1.patch
          26 kB
          Yongzhi Chen

          Issue Links

            Activity

              People

              • Assignee:
                ychena Yongzhi Chen
                Reporter:
                ychena Yongzhi Chen
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: