Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-11412

StackOverFlow in SemanticAnalyzer for huge filters (~5000)

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Duplicate
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      Queries with ~5000 filter conditions fails in SemanticAnalysis

      Stack trace:

      Exception in thread "main" java.lang.StackOverflowError
      	at java.util.HashMap.hash(HashMap.java:366)
      	at java.util.HashMap.getEntry(HashMap.java:466)
      	at java.util.HashMap.containsKey(HashMap.java:453)
      	at org.apache.commons.collections.map.AbstractMapDecorator.containsKey(AbstractMapDecorator.java:83)
      	at org.apache.hadoop.conf.Configuration.isDeprecated(Configuration.java:558)
      	at org.apache.hadoop.conf.Configuration.handleDeprecation(Configuration.java:605)
      	at org.apache.hadoop.conf.Configuration.get(Configuration.java:885)
      	at org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:907)
      	at org.apache.hadoop.conf.Configuration.getBoolean(Configuration.java:1308)
      	at org.apache.hadoop.hive.conf.HiveConf.getBoolVar(HiveConf.java:2641)
      	at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.processPositionAlias(SemanticAnalyzer.java:11132)
      	at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.processPositionAlias(SemanticAnalyzer.java:11226)
      	at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.processPositionAlias(SemanticAnalyzer.java:11226)
      	at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.processPositionAlias(SemanticAnalyzer.java:11226)
      
      

      Query:

      explain select count(*) from over1k where (
      (t=1 and si=2)
      or (t=2 and si=3)
      or (t=3 and si=4) 
      or (t=4 and si=5) 
      or (t=5 and si=6) 
      or (t=6 and si=7) 
      or (t=7 and si=8)
      or (t=7 and si=8)
      or (t=7 and si=8)
      ...
      

      Repeat the filter around 5000 times.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                hsubramaniyan Hari Sankar Sivarama Subramaniyan
                Reporter:
                prasanth_j Prasanth Jayachandran
              • Votes:
                0 Vote for this issue
                Watchers:
                6 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: