Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-21980

Parsing time can be high in case of deeply nested subqueries

    XMLWordPrintableJSON

Details

    Description

      for queries which are recursively doing:

      SELECT ...
      FROM (SELECT ...
                   FROM ( [...]
                   ) JOIN
                  (SELECT ...
                  FROM ( [...] )
      JOIN ( [...]
      )
      

      when the issue is present hs2 stops right after starting compiling and the following stack is there

      	at org.apache.hadoop.hive.ql.parse.HiveParser_FromClauseParser.subQuerySource(HiveParser_FromClauseParser.java:4745)
      	at org.apache.hadoop.hive.ql.parse.HiveParser_FromClauseParser.synpred1_FromClauseParser_fragment(HiveParser_FromClauseParser.java:6513)
      	at org.apache.hadoop.hive.ql.parse.HiveParser_FromClauseParser.synpred1_FromClauseParser(HiveParser_FromClauseParser.java:6660)
      	at org.apache.hadoop.hive.ql.parse.HiveParser_FromClauseParser$DFA10.specialStateTransition(HiveParser_FromClauseParser.java:6942)
      	at org.antlr.runtime.DFA.predict(DFA.java:80)
      	at org.apache.hadoop.hive.ql.parse.HiveParser_FromClauseParser.atomjoinSource(HiveParser_FromClauseParser.java:1596)
      	at org.apache.hadoop.hive.ql.parse.HiveParser_FromClauseParser.joinSource(HiveParser_FromClauseParser.java:1859)
      	at org.apache.hadoop.hive.ql.parse.HiveParser_FromClauseParser.fromSource(HiveParser_FromClauseParser.java:1532)
      	at org.apache.hadoop.hive.ql.parse.HiveParser_FromClauseParser.fromClause(HiveParser_FromClauseParser.java:1375)
      	at org.apache.hadoop.hive.ql.parse.HiveParser.fromClause(HiveParser.java:45755)
      	at org.apache.hadoop.hive.ql.parse.HiveParser.atomSelectStatement(HiveParser.java:40337)
      	at org.apache.hadoop.hive.ql.parse.HiveParser.selectStatement(HiveParser.java:40589)
      	at org.apache.hadoop.hive.ql.parse.HiveParser.regularBody(HiveParser.java:40235)
      	at org.apache.hadoop.hive.ql.parse.HiveParser.queryStatementExpressionBody(HiveParser.java:39445)
      	at org.apache.hadoop.hive.ql.parse.HiveParser.queryStatementExpression(HiveParser.java:39333)
      	at org.apache.hadoop.hive.ql.parse.HiveParser_FromClauseParser.subQuerySource(HiveParser_FromClauseParser.java:4745)
      	at org.apache.hadoop.hive.ql.parse.HiveParser_FromClauseParser.atomjoinSource(HiveParser_FromClauseParser.java:1699)
      	at org.apache.hadoop.hive.ql.parse.HiveParser_FromClauseParser.joinSource(HiveParser_FromClauseParser.java:1859)
      	at org.apache.hadoop.hive.ql.parse.HiveParser_FromClauseParser.fromSource(HiveParser_FromClauseParser.java:1532)
      	at org.apache.hadoop.hive.ql.parse.HiveParser_FromClauseParser.fromClause(HiveParser_FromClauseParser.java:1375)
      	at org.apache.hadoop.hive.ql.parse.HiveParser.fromClause(HiveParser.java:45755)
      	at org.apache.hadoop.hive.ql.parse.HiveParser.atomSelectStatement(HiveParser.java:40337)
      	at org.apache.hadoop.hive.ql.parse.HiveParser.selectStatement(HiveParser.java:40589)
      	at org.apache.hadoop.hive.ql.parse.HiveParser.regularBody(HiveParser.java:40235)
      	at org.apache.hadoop.hive.ql.parse.HiveParser.queryStatementExpressionBody(HiveParser.java:39445)
      	at org.apache.hadoop.hive.ql.parse.HiveParser.queryStatementExpression(HiveParser.java:39333)
      	at org.apache.hadoop.hive.ql.parse.HiveParser_FromClauseParser.subQuerySource(HiveParser_FromClauseParser.java:4745)
      	at org.apache.hadoop.hive.ql.parse.HiveParser_FromClauseParser.synpred1_FromClauseParser_fragment(HiveParser_FromClauseParser.java:6513)
      	at org.apache.hadoop.hive.ql.parse.HiveParser_FromClauseParser.synpred1_FromClauseParser(HiveParser_FromClauseParser.java:6660)
      	at org.apache.hadoop.hive.ql.parse.HiveParser_FromClauseParser$DFA10.specialStateTransition(HiveParser_FromClauseParser.java:6942)
      	at org.antlr.runtime.DFA.predict(DFA.java:80)
      	at org.apache.hadoop.hive.ql.parse.HiveParser_FromClauseParser.atomjoinSource(HiveParser_FromClauseParser.java:1596)
      	at org.apache.hadoop.hive.ql.parse.HiveParser_FromClauseParser.joinSource(HiveParser_FromClauseParser.java:1859)
      	at org.apache.hadoop.hive.ql.parse.HiveParser_FromClauseParser.fromSource(HiveParser_FromClauseParser.java:1532)
      	at org.apache.hadoop.hive.ql.parse.HiveParser_FromClauseParser.fromClause(HiveParser_FromClauseParser.java:1375)
      	at org.apache.hadoop.hive.ql.parse.HiveParser.fromClause(HiveParser.java:45755)
      	at org.apache.hadoop.hive.ql.parse.HiveParser.atomSelectStatement(HiveParser.java:40337)
      	at org.apache.hadoop.hive.ql.parse.HiveParser.selectStatement(HiveParser.java:40589)
      	at org.apache.hadoop.hive.ql.parse.HiveParser.regularBody(HiveParser.java:40235)
      	at org.apache.hadoop.hive.ql.parse.HiveParser.queryStatementExpressionBody(HiveParser.java:39445)
      	at org.apache.hadoop.hive.ql.parse.HiveParser.queryStatementExpression(HiveParser.java:39333)
      	at org.apache.hadoop.hive.ql.parse.HiveParser_FromClauseParser.subQuerySource(HiveParser_FromClauseParser.java:4745)
      	at org.apache.hadoop.hive.ql.parse.HiveParser_FromClauseParser.atomjoinSource(HiveParser_FromClauseParser.java:1699)
      	at org.apache.hadoop.hive.ql.parse.HiveParser_FromClauseParser.joinSource(HiveParser_FromClauseParser.java:1859)
      	at org.apache.hadoop.hive.ql.parse.HiveParser_FromClauseParser.fromSource(HiveParser_FromClauseParser.java:1532)
      	at org.apache.hadoop.hive.ql.parse.HiveParser_FromClauseParser.fromClause(HiveParser_FromClauseParser.java:1375)
      	at org.apache.hadoop.hive.ql.parse.HiveParser.fromClause(HiveParser.java:45755)
      	at org.apache.hadoop.hive.ql.parse.HiveParser.atomSelectStatement(HiveParser.java:40337)
      	at org.apache.hadoop.hive.ql.parse.HiveParser.selectStatement(HiveParser.java:40589)
      	at org.apache.hadoop.hive.ql.parse.HiveParser.regularBody(HiveParser.java:40235)
      	at org.apache.hadoop.hive.ql.parse.HiveParser.queryStatementExpressionBody(HiveParser.java:39445)
      	at org.apache.hadoop.hive.ql.parse.HiveParser.queryStatementExpression(HiveParser.java:39333)
      	at org.apache.hadoop.hive.ql.parse.HiveParser_FromClauseParser.subQuerySource(HiveParser_FromClauseParser.java:4745)
      	at org.apache.hadoop.hive.ql.parse.HiveParser_FromClauseParser.atomjoinSource(HiveParser_FromClauseParser.java:1699)
      	at org.apache.hadoop.hive.ql.parse.HiveParser_FromClauseParser.joinSource(HiveParser_FromClauseParser.java:1859)
      	at org.apache.hadoop.hive.ql.parse.HiveParser_FromClauseParser.fromSource(HiveParser_FromClauseParser.java:1532)
      	at org.apache.hadoop.hive.ql.parse.HiveParser_FromClauseParser.fromClause(HiveParser_FromClauseParser.java:1375)
      	at org.apache.hadoop.hive.ql.parse.HiveParser.fromClause(HiveParser.java:45755)
      	at org.apache.hadoop.hive.ql.parse.HiveParser.atomSelectStatement(HiveParser.java:40337)
      	at org.apache.hadoop.hive.ql.parse.HiveParser.selectStatement(HiveParser.java:40589)
      	at org.apache.hadoop.hive.ql.parse.HiveParser.regularBody(HiveParser.java:40235)
      	at org.apache.hadoop.hive.ql.parse.HiveParser.queryStatementExpressionBody(HiveParser.java:39445)
      	at org.apache.hadoop.hive.ql.parse.HiveParser.queryStatementExpression(HiveParser.java:39333)
      	at org.apache.hadoop.hive.ql.parse.HiveParser_FromClauseParser.subQuerySource(HiveParser_FromClauseParser.java:4745)
      	at org.apache.hadoop.hive.ql.parse.HiveParser_FromClauseParser.synpred1_FromClauseParser_fragment(HiveParser_FromClauseParser.java:6513)
      	at org.apache.hadoop.hive.ql.parse.HiveParser_FromClauseParser.synpred1_FromClauseParser(HiveParser_FromClauseParser.java:6660)
      	at org.apache.hadoop.hive.ql.parse.HiveParser_FromClauseParser$DFA10.specialStateTransition(HiveParser_FromClauseParser.java:6942)
      	at org.antlr.runtime.DFA.predict(DFA.java:80)
      

      Attachments

        1. HIVE-21980.01.patch
          4 kB
          Zoltan Haindrich
        2. HIVE-21980.01.patch
          4 kB
          Zoltan Haindrich
        3. HIVE-21980.02.patch
          5 kB
          Zoltan Haindrich
        4. HIVE-21980.03.patch
          5 kB
          Zoltan Haindrich

        Issue Links

          Activity

            People

              kgyrtkirk Zoltan Haindrich
              kgyrtkirk Zoltan Haindrich
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: