Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-11141

Improve RuleRegExp when the Expression node stack gets huge

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.0.0
    • Component/s: None
    • Labels:
      None

      Description

      Hive occasionally gets bottlenecked on generating plans for large queries, the majority of the cases time is spent in fetching metadata, partitions and other optimizer transformation related rules

      I have attached the query for the test case which needs to be tested after we setup database as shown below.

      create database dataset_3;
      use database dataset_3;
      

      createtable.rtf - create table command
      SQLQuery10.sql.mssql - explain query

      It seems that the most problematic part of the code as the stack gets arbitrary long, in RuleRegExp.java

        @Override
        public int cost(Stack<Node> stack) throws SemanticException {
          int numElems = (stack != null ? stack.size() : 0);
          String name = "";
          for (int pos = numElems - 1; pos >= 0; pos--) {
            name = stack.get(pos).getName() + "%" + name;
            Matcher m = pattern.matcher(name);
            if (m.matches()) {
              return m.group().length();
            }
          }
          return -1;
        }
      

        Attachments

        1. createtable.rtf
          3 kB
          Hari Sankar Sivarama Subramaniyan
        2. HIVE-11141.1.patch
          19 kB
          Hari Sankar Sivarama Subramaniyan
        3. HIVE-11141.2.patch
          30 kB
          Hari Sankar Sivarama Subramaniyan
        4. HIVE-11141.3.patch
          95 kB
          Hari Sankar Sivarama Subramaniyan
        5. HIVE-11141.4.patch
          10 kB
          Hari Sankar Sivarama Subramaniyan
        6. HIVE-11141.5.patch
          10 kB
          Hari Sankar Sivarama Subramaniyan
        7. HIVE-11141.7.patch
          12 kB
          Hari Sankar Sivarama Subramaniyan
        8. HIVE-11141.branch-1.0.patch
          12 kB
          Hari Sankar Sivarama Subramaniyan
        9. HIVE-11141.branch-1.2.patch
          12 kB
          Hari Sankar Sivarama Subramaniyan

          Issue Links

            Activity

              People

              • Assignee:
                hsubramaniyan Hari Sankar Sivarama Subramaniyan
                Reporter:
                hsubramaniyan Hari Sankar Sivarama Subramaniyan
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: