Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-15572

Improve the response time for query canceling when it happens during acquiring locks

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • None
    • None
    • None

    Description

      When query canceling command sent during Hive Acquire locks (from zookeeper), hive will finish acquiring all the locks and release them. As it is shown in the following log:
      It took 165 s to finish acquire the lock,then spend 81s to release them.
      We can improve the performance by not acquiring any more locks and releasing held locks when the query canceling command is received.

      Background-Pool: Thread-224]: <PERFLOG method=TimeToSubmit from=org.apache.hadoop.hive.ql.Driver>
      2017-01-03 10:50:35,413 INFO  org.apache.hadoop.hive.ql.log.PerfLogger: [HiveServer2-Background-Pool: Thread-224]: <PERFLOG method=acquireReadWriteLocks from=org.apache.hadoop.hive.ql.Driver>
      2017-01-03 10:51:00,671 INFO  org.apache.hadoop.hive.ql.log.PerfLogger: [HiveServer2-Background-Pool: Thread-218]: </PERFLOG method=acquireReadWriteLocks start=1483469295080 end=1483469460671 duration=165591 from=org.apache.hadoop.hive.ql.Driver>
      2017-01-03 10:51:00,672 INFO  org.apache.hadoop.hive.ql.log.PerfLogger: [HiveServer2-Background-Pool: Thread-218]: <PERFLOG method=Driver.execute from=org.apache.hadoop.hive.ql.Driver>
      2017-01-03 10:51:00,672 ERROR org.apache.hadoop.hive.ql.Driver: [HiveServer2-Background-Pool: Thread-218]: FAILED: query select count(*) from manyparttbl has been cancelled
      2017-01-03 10:51:00,673 INFO  org.apache.hadoop.hive.ql.log.PerfLogger: [HiveServer2-Background-Pool: Thread-218]: <PERFLOG method=releaseLocks from=org.apache.hadoop.hive.ql.Driver>
      2017-01-03 10:51:40,755 INFO  org.apache.hadoop.hive.ql.log.PerfLogger: [HiveServer2-Background-Pool: Thread-215]: </PERFLOG method=releaseLocks start=1483469419487 end=1483469500755 duration=81268 from=org.apache.hadoop.hive.ql.Driver>
      

      Attachments

        1. HIVE-15572.2.patch
          26 kB
          Yongzhi Chen
        2. HIVE-15572.1.patch
          26 kB
          Yongzhi Chen

        Issue Links

          Activity

            People

              ychena Yongzhi Chen
              ychena Yongzhi Chen
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: