Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-15441

Provide a config to timeout long compiling queries

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Patch Available
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Query Planning
    • Labels:
      None

      Description

      Sometimes Hive users have long compiling queries which may need to scan thousands or even more partitions (perhaps by accident). The compilation process may take a very long time, especially in getInputSummary where it need to make NN calls to get info about each input path.

      This is bad because it may block many other queries. Parallel compilation may be useful but still getInputSummary has a global lock. In this case, it makes sense to provide Hive admin with a config to put a timeout limit for compilation, so that these "bad" queries can be blocked.

      Note https://issues.apache.org/jira/browse/HIVE-12431 also tries to address similar issue. However it cancels those queries that are waiting for the compile lock, which I think is not so useful for our case since the query under compile is the one to be blamed.

        Attachments

        1. HIVE-15441.1.patch
          4 kB
          Chao Sun

          Issue Links

            Activity

              People

              • Assignee:
                csun Chao Sun
                Reporter:
                csun Chao Sun
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated: