Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-5775 Introduce Cost Based Optimizer to Hive
  3. HIVE-7310

Turning CBO on results in NPE on some queries

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • None
    • None

    Description

      On the CBO branch if I do the following:

      hive> set hive.cbo.enable=true;
      hive> select i_item_id,
      > s_state,
      > GROUPING__ID,
      > avg(ss_quantity) agg1,
      > avg(ss_list_price) agg2,
      > avg(ss_coupon_amt) agg3,
      > avg(ss_sales_price) agg4
      > from store_sales ss
      > join customer_demographics cd on (ss.ss_cdemo_sk = cd.cd_demo_sk)
      > join date_dim d on (ss.ss_sold_date_sk = d.d_date_sk)
      > join store s on (ss.ss_store_sk = s.s_store_sk)
      > join item i on (ss.ss_item_sk = i.i_item_sk)
      > where cd_gender = 'M' and
      > cd_marital_status = 'S' and
      > cd_education_status = 'Secondary' and
      > d_year = 2002 and
      > s_state in ('OH','SD', 'LA', 'MO', 'WA', 'MN')
      > group by i_item_id, s_state with rollup
      > order by i_item_id
      > ,s_state
      > limit 100
      > ;

      I get an NPE. The stack trace is:

      java.lang.RuntimeException: java.lang.RuntimeException: java.lang.RuntimeException: java.lang.NullPointerException
      at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:9555)
      at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:328)
      at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:412)
      at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:308)
      at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:962)
      at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1027)
      at org.apache.hadoop.hive.ql.Driver.run(Driver.java:898)
      at org.apache.hadoop.hive.ql.Driver.run(Driver.java:888)
      at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:277)
      at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:229)
      at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:439)
      at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:812)
      at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:706)
      at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:645)
      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      at java.lang.reflect.Method.invoke(Method.java:606)
      at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
      Caused by: java.lang.RuntimeException: java.lang.RuntimeException: java.lang.NullPointerException
      at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer$OptiqBasedPlanner.getOptimizedAST(SemanticAnalyzer.java:11732)
      at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer$OptiqBasedPlanner.access$200(SemanticAnalyzer.java:11711)
      at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:9532)
      ... 18 more
      Caused by: java.lang.RuntimeException: java.lang.NullPointerException
      at net.hydromatic.optiq.tools.Frameworks.withPrepare(Frameworks.java:170)
      at net.hydromatic.optiq.tools.Frameworks.withPlanner(Frameworks.java:142)
      at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer$OptiqBasedPlanner.getOptimizedAST(SemanticAnalyzer.java:11727)
      ... 20 more
      Caused by: java.lang.NullPointerException
      at org.eigenbase.reltype.RelDataTypeImpl.getField(RelDataTypeImpl.java:79)
      at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer$OptiqBasedPlanner.convertAgg(SemanticAnalyzer.java:12129)
      at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer$OptiqBasedPlanner.genGBRelNode(SemanticAnalyzer.java:12184)
      at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer$OptiqBasedPlanner.genGBLogicalPlan(SemanticAnalyzer.java:12324)
      at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer$OptiqBasedPlanner.genLogicalPlan(SemanticAnalyzer.java:12749)
      at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer$OptiqBasedPlanner.apply(SemanticAnalyzer.java:11758)
      at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer$OptiqBasedPlanner.apply(SemanticAnalyzer.java:11711)
      at net.hydromatic.optiq.tools.Frameworks$1.apply(Frameworks.java:146)
      at net.hydromatic.optiq.prepare.OptiqPrepareImpl.perform(OptiqPrepareImpl.java:536)
      at net.hydromatic.optiq.tools.Frameworks.withPrepare(Frameworks.java:168)
      ... 22 more

      Attachments

        1. HIVE-7310.patch
          0.8 kB
          Laljo John Pullokkaran

        Activity

          People

            jpullokkaran Laljo John Pullokkaran
            hagleitn Gunther Hagleitner
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: