Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-12864

StackOverflowError parsing queries with very large predicates

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.0.0, 2.1.0
    • Fix Version/s: 2.0.0
    • Component/s: Parser
    • Labels:
      None

      Description

      We have seen that queries with very large predicates might fail with the following stacktrace:

      016-01-12 05:47:36,516|beaver.machine|INFO|552|5072|Thread-22|Exception in thread "main" java.lang.StackOverflowError
      
      2016-01-12 05:47:36,517|beaver.machine|INFO|552|5072|Thread-22|at org.antlr.runtime.tree.CommonTree.setUnknownTokenBoundaries(CommonTree.java:145)
      
      2016-01-12 05:47:36,517|beaver.machine|INFO|552|5072|Thread-22|at org.antlr.runtime.tree.CommonTree.setUnknownTokenBoundaries(CommonTree.java:146)
      2016-01-12 05:47:36,517|beaver.machine|INFO|552|5072|Thread-22|at org.antlr.runtime.tree.CommonTree.setUnknownTokenBoundaries(CommonTree.java:146)
      2016-01-12 05:47:36,517|beaver.machine|INFO|552|5072|Thread-22|at org.antlr.runtime.tree.CommonTree.setUnknownTokenBoundaries(CommonTree.java:146)
      2016-01-12 05:47:36,517|beaver.machine|INFO|552|5072|Thread-22|at org.antlr.runtime.tree.CommonTree.setUnknownTokenBoundaries(CommonTree.java:146)
      2016-01-12 05:47:36,519|beaver.machine|INFO|552|5072|Thread-22|at org.antlr.runtime.tree.CommonTree.setUnknownTokenBoundaries(CommonTree.java:146)
      2016-01-12 05:47:36,519|beaver.machine|INFO|552|5072|Thread-22|at org.antlr.runtime.tree.CommonTree.setUnknownTokenBoundaries(CommonTree.java:146)
      2016-01-12 05:47:36,519|beaver.machine|INFO|552|5072|Thread-22|at org.antlr.runtime.tree.CommonTree.setUnknownTokenBoundaries(CommonTree.java:146)
      2016-01-12 05:47:36,519|beaver.machine|INFO|552|5072|Thread-22|at org.antlr.runtime.tree.CommonTree.setUnknownTokenBoundaries(CommonTree.java:146)
      2016-01-12 05:47:36,519|beaver.machine|INFO|552|5072|Thread-22|at org.antlr.runtime.tree.CommonTree.setUnknownTokenBoundaries(CommonTree.java:146)
      2016-01-12 05:47:36,519|beaver.machine|INFO|552|5072|Thread-22|at org.antlr.runtime.tree.CommonTree.setUnknownTokenBoundaries(CommonTree.java:146)
      2016-01-12 05:47:36,519|beaver.machine|INFO|552|5072|Thread-22|at org.antlr.runtime.tree.CommonTree.setUnknownTokenBoundaries(CommonTree.java:146)
      2016-01-12 05:47:36,519|beaver.machine|INFO|552|5072|Thread-22|at org.antlr.runtime.tree.CommonTree.setUnknownTokenBoundaries(CommonTree.java:146)
      
      2016-01-12 05:47:36,519|beaver.machine|INFO|552|5072|Thread-22|at org.antlr.runtime.tree.CommonTree.setUnknownTokenBoundaries(CommonTree.java:146)
      
      2016-01-12 05:47:36,519|beaver.machine|INFO|552|5072|Thread-22|at org.antlr.runtime.tree.CommonTree.setUnknownTokenBoundaries(CommonTree.java:146)
      2016-01-12 05:47:36,519|beaver.machine|INFO|552|5072|Thread-22|at org.antlr.runtime.tree.CommonTree.setUnknownTokenBoundaries(CommonTree.java:146)
      2016-01-12 05:47:36,519|beaver.machine|INFO|552|5072|Thread-22|at org.antlr.runtime.tree.CommonTree.setUnknownTokenBoundaries(CommonTree.java:146)
      2016-01-12 05:47:36,519|beaver.machine|INFO|552|5072|Thread-22|at org.antlr.runtime.tree.CommonTree.setUnknownTokenBoundaries(CommonTree.java:146)
      2016-01-12 05:47:36,519|beaver.machine|INFO|552|5072|Thread-22|at org.antlr.runtime.tree.CommonTree.setUnknownTokenBoundaries(CommonTree.java:146)
      2016-01-12 05:47:36,519|beaver.machine|INFO|552|5072|Thread-22|at org.antlr.runtime.tree.CommonTree.setUnknownTokenBoundaries(CommonTree.java:146)
      2016-01-12 05:47:36,519|beaver.machine|INFO|552|5072|Thread-22|at org.antlr.runtime.tree.CommonTree.setUnknownTokenBoundaries(CommonTree.java:146)
      2016-01-12 05:47:36,519|beaver.machine|INFO|552|5072|Thread-22|at org.antlr.runtime.tree.CommonTree.setUnknownTokenBoundaries(CommonTree.java:146)
      2016-01-12 05:47:36,520|beaver.machine|INFO|552|5072|Thread-22|at org.antlr.runtime.tree.CommonTree.setUnknownTokenBoundaries(CommonTree.java:146)
      2016-01-12 05:47:36,520|beaver.machine|INFO|552|5072|Thread-22|at org.antlr.runtime.tree.CommonTree.setUnknownTokenBoundaries(CommonTree.java:146)
      2016-01-12 05:47:36,520|beaver.machine|INFO|552|5072|Thread-22|at org.antlr.runtime.tree.CommonTree.setUnknownTokenBoundaries(CommonTree.java:146)
      2016-01-12 05:47:36,520|beaver.machine|INFO|552|5072|Thread-22|at org.antlr.runtime.tree.CommonTree.setUnknownTokenBoundaries(CommonTree.java:146)
      2016-01-12 05:47:36,520|beaver.machine|INFO|552|5072|Thread-22|at org.antlr.runtime.tree.CommonTree.setUnknownTokenBoundaries(CommonTree.java:146)
      2016-01-12 05:47:36,520|beaver.machine|INFO|552|5072|Thread-22|at org.antlr.runtime.tree.CommonTree.setUnknownTokenBoundaries(CommonTree.java:146)
      2016-01-12 05:47:36,520|beaver.machine|INFO|552|5072|Thread-22|at org.antlr.runtime.tree.CommonTree.setUnknownTokenBoundaries(CommonTree.java:146)
      2016-01-12 05:47:36,522|beaver.machine|INFO|552|5072|Thread-22|at org.antlr.runtime.tree.CommonTree.setUnknownTokenBoundaries(CommonTree.java:146)
      2016-01-12 05:47:36,522|beaver.machine|INFO|552|5072|Thread-22|at org.antlr.runtime.tree.CommonTree.setUnknownTokenBoundaries(CommonTree.java:146)
      2016-01-12 05:47:36,522|beaver.machine|INFO|552|5072|Thread-22|at org.antlr.runtime.tree.CommonTree.setUnknownTokenBoundaries(CommonTree.java:146)
      2016-01-12 05:47:36,522|beaver.machine|INFO|552|5072|Thread-22|at org.antlr.runtime.tree.CommonTree.setUnknownTokenBoundaries(CommonTree.java:146)
      2016-01-12 05:47:36,522|beaver.machine|INFO|552|5072|Thread-22|at org.antlr.runtime.tree.CommonTree.setUnknownTokenBoundaries(CommonTree.java:146)
      2016-01-12 05:47:36,522|beaver.machine|INFO|552|5072|Thread-22|at org.antlr.runtime.tree.CommonTree.setUnknownTokenBoundaries(CommonTree.java:146)
      2016-01-12 05:47:36,522|beaver.machine|INFO|552|5072|Thread-22|at org.antlr.runtime.tree.CommonTree.setUnknownTokenBoundaries(CommonTree.java:146)
      2016-01-12 05:47:36,522|beaver.machine|INFO|552|5072|Thread-22|at org.antlr.runtime.tree.CommonTree.setUnknownTokenBoundaries(CommonTree.java:146)
      2016-01-12 05:47:36,523|beaver.machine|INFO|552|5072|Thread-22|at org.antlr.runtime.tree.CommonTree.setUnknownTokenBoundaries(CommonTree.java:146)
      2016-01-12 05:47:36,523|beaver.machine|INFO|552|5072|Thread-22|at org.antlr.runtime.tree.CommonTree.setUnknownTokenBoundaries(CommonTree.java:146)
      2016-01-12 05:47:36,523|beaver.machine|INFO|552|5072|Thread-22|at org.antlr.runtime.tree.CommonTree.setUnknownTokenBoundaries(CommonTree.java:146)
      2016-01-12 05:47:36,523|beaver.machine|INFO|552|5072|Thread-22|at org.antlr.runtime.tree.CommonTree.setUnknownTokenBoundaries(CommonTree.java:146)
      
      2016-01-12 05:47:36,523|beaver.machine|INFO|552|5072|Thread-22|at org.antlr.runtime.tree.CommonTree.setUnknownTokenBoundaries(CommonTree.java:146)
      
      2016-01-12 05:47:36,523|beaver.machine|INFO|552|5072|Thread-22|at org.antlr.runtime.tree.CommonTree.setUnknownTokenBoundaries(CommonTree.java:146)
      2016-01-12 05:47:36,525|beaver.machine|INFO|552|5072|Thread-22|at org.antlr.runtime.tree.CommonTree.setUnknownTokenBoundaries(CommonTree.java:146)
      2016-01-12 05:47:36,525|beaver.machine|INFO|552|5072|Thread-22|at org.antlr.runtime.tree.CommonTree.setUnknownTokenBoundaries(CommonTree.java:146)
      2016-01-12 05:47:36,525|beaver.machine|INFO|552|5072|Thread-22|at org.antlr.runtime.tree.CommonTree.setUnknownTokenBoundaries(CommonTree.java:146)
      
      2016-01-12 05:47:36,525|beaver.machine|INFO|552|5072|Thread-22|at org.antlr.runtime.tree.CommonTree.setUnknownTokenBoundaries(CommonTree.java:146)
      
      2016-01-12 05:47:36,525|beaver.machine|INFO|552|5072|Thread-22|at org.antlr.runtime.tree.CommonTree.setUnknownTokenBoundaries(CommonTree.java:146)
      
      2016-01-12 05:47:36,525|beaver.machine|INFO|552|5072|Thread-22|at org.antlr.runtime.tree.CommonTree.setUnknownTokenBoundaries(CommonTree.java:146)
      
      2016-01-12 05:47:36,526|beaver.machine|INFO|552|5072|Thread-22|at org.antlr.runtime.tree.CommonTree.setUnknownTokenBoundaries(CommonTree.java:146)
      
      2016-01-12 05:47:36,526|beaver.machine|INFO|552|5072|Thread-22|at org.antlr.runtime.tree.CommonTree.setUnknownTokenBoundaries(CommonTree.java:146)
      
      2016-01-12 05:47:36,526|beaver.machine|INFO|552|5072|Thread-22|at org.antlr.runtime.tree.CommonTree.setUnknownTokenBoundaries(CommonTree.java:146)
      
      2016-01-12 05:47:36,526|beaver.machine|INFO|552|5072|Thread-22|at org.antlr.runtime.tree.CommonTree.setUnknownTokenBoundaries(CommonTree.java:146)
      
      2016-01-12 05:47:36,526|beaver.machine|INFO|552|5072|Thread-22|at org.antlr.runtime.tree.CommonTree.setUnknownTokenBoundaries(CommonTree.java:146)
      
      2016-01-12 05:47:36,526|beaver.machine|INFO|552|5072|Thread-22|at org.antlr.runtime.tree.CommonTree.setUnknownTokenBoundaries(CommonTree.java:146)
      
      2016-01-12 05:47:36,526|beaver.machine|INFO|552|5072|Thread-22|at org.antlr.runtime.tree.CommonTree.setUnknownTokenBoundaries(CommonTree.java:146)
      
      2016-01-12 05:47:36,526|beaver.machine|INFO|552|5072|Thread-22|at org.antlr.runtime.tree.CommonTree.setUnknownTokenBoundaries(CommonTree.java:146)
      
      2016-01-12 05:47:36,526|beaver.machine|INFO|552|5072|Thread-22|at org.antlr.runtime.tree.CommonTree.setUnknownTokenBoundaries(CommonTree.java:146)
      
      2016-01-12 05:47:36,526|beaver.machine|INFO|552|5072|Thread-22|at org.antlr.runtime.tree.CommonTree.setUnknownTokenBoundaries(CommonTree.java:146)
      
      2016-01-12 05:47:36,526|beaver.machine|INFO|552|5072|Thread-22|at org.antlr.runtime.tree.CommonTree.setUnknownTokenBoundaries(CommonTree.java:146)
      2016-01-12 05:47:36,526|beaver.machine|INFO|552|5072|Thread-22|at org.antlr.runtime.tree.CommonTree.setUnknownTokenBoundaries(CommonTree.java:146)
      2016-01-12 05:47:36,526|beaver.machine|INFO|552|5072|Thread-22|at org.antlr.runtime.tree.CommonTree.setUnknownTokenBoundaries(CommonTree.java:146)
      2016-01-12 05:47:36,526|beaver.machine|INFO|552|5072|Thread-22|at org.antlr.runtime.tree.CommonTree.setUnknownTokenBoundaries(CommonTree.java:146)
      2016-01-12 05:47:36,526|beaver.machine|INFO|552|5072|Thread-22|at org.antlr.runtime.tree.CommonTree.setUnknownTokenBoundaries(CommonTree.java:146)
      2016-01-12 05:47:36,634|beaver.machine|INFO|552|5072|Thread-22|at org.antlr.runtime.tree.CommonTree.setUnknownTokenBoundaries(CommonTree.java:146)
      2016-01-12 05:47:37,582|main|INFO|552|4568|MainThread|TEST "test_WideQuery" FAILED in 10.95 seconds
      

      The problem could be solved by reimplementing some of the parsing methods so they are iterative instead of recursive.

        Attachments

        1. HIVE-12864.patch
          17 kB
          Jesus Camacho Rodriguez
        2. HIVE-12864.01.patch
          17 kB
          Jesus Camacho Rodriguez

          Issue Links

            Activity

              People

              • Assignee:
                jcamachorodriguez Jesus Camacho Rodriguez
                Reporter:
                jcamachorodriguez Jesus Camacho Rodriguez
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: