[HIVE-10627] Queries fail with Failed to breakup Windowing invocations into Groups

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.2.1
• Assignee: Jesús Camacho Rodríguez
• Reporter: Jesús Camacho Rodríguez
• Created: 06/May/15 14:28
• Updated: 27/Feb/24 22:24
• Resolved: 20/May/15 19:29

Description

TPC-DS queries 51 fails with Failed to breakup Windowing invocations into Groups. At least 1 group must only depend on input columns. Also check for circular dependencies.

explain  
WITH web_v1 as (
select
  ws_item_sk item_sk, d_date, sum(ws_sales_price),
  sum(sum(ws_sales_price))
      over (partition by ws_item_sk order by d_date rows between unbounded preceding and current row) cume_sales
from web_sales
    ,date_dim
where ws_sold_date_sk=d_date_sk
  and d_month_seq between 1193 and 1193+11
  and ws_item_sk is not NULL
group by ws_item_sk, d_date),
store_v1 as (
select
  ss_item_sk item_sk, d_date, sum(ss_sales_price),
  sum(sum(ss_sales_price))
      over (partition by ss_item_sk order by d_date rows between unbounded preceding and current row) cume_sales
from store_sales
    ,date_dim
where ss_sold_date_sk=d_date_sk
  and d_month_seq between 1193 and 1193+11
  and ss_item_sk is not NULL
group by ss_item_sk, d_date)
 select  *
from (select item_sk
     ,d_date
     ,web_sales
     ,store_sales
     ,max(web_sales)
         over (partition by item_sk order by d_date rows between unbounded preceding and current row) web_cumulative
     ,max(store_sales)
         over (partition by item_sk order by d_date rows between unbounded preceding and current row) store_cumulative
     from (select case when web.item_sk is not null then web.item_sk else store.item_sk end item_sk
                 ,case when web.d_date is not null then web.d_date else store.d_date end d_date
                 ,web.cume_sales web_sales
                 ,store.cume_sales store_sales
           from web_v1 web full outer join store_v1 store on (web.item_sk = store.item_sk
                                                          and web.d_date = store.d_date)
          )x )y
where web_cumulative > store_cumulative
order by item_sk
        ,d_date
limit 100;

Exception

org.apache.hadoop.hive.ql.parse.SemanticException: Failed to breakup Windowing invocations into Groups. At least 1 group must only depend on input columns. Also check for circular dependencies.	
Underlying error: org.apache.hadoop.hive.ql.parse.SemanticException: Line 0:-1 Invalid column reference '$f2'	
	at org.apache.hadoop.hive.ql.parse.WindowingComponentizer.next(WindowingComponentizer.java:94)
	at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genWindowingPlan(SemanticAnalyzer.java:11538)
	at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPostGroupByBodyPlan(SemanticAnalyzer.java:8514)
	at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genBodyPlan(SemanticAnalyzer.java:8472)
	at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:9304)
	at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:9189)
	at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:9210)
	at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:9189)
	at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:9210)
	at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:9189)
	at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:9210)
	at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:9189)
	at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:9210)
	at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:9189)
	at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:9210)
	at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:9592)
	at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:208)
	at org.apache.hadoop.hive.ql.parse.ExplainSemanticAnalyzer.analyzeInternal(ExplainSemanticAnalyzer.java:74)
	at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:208)
	at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:414)
	at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:310)
	at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1005)
	at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1070)
	at org.apache.hadoop.hive.ql.Driver.run(Driver.java:942)
	at org.apache.hadoop.hive.ql.Driver.run(Driver.java:932)
	at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:246)
	at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:198)
	at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:408)
	at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:344)
	at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:731)
	at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:675)
	at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:614)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
FAILED: UnsupportedOperationException null	
14/08/19 16:50:25 ERROR ql.Driver: FAILED: UnsupportedOperationException null	
java.lang.UnsupportedOperationException	
	at com.google.common.collect.ImmutableMap.clear(ImmutableMap.java:332)
	at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.reset(SemanticAnalyzer.java:402)
	at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.init(SemanticAnalyzer.java:9498)
	at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:9612)
	at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:208)
	at org.apache.hadoop.hive.ql.parse.ExplainSemanticAnalyzer.analyzeInternal(ExplainSemanticAnalyzer.java:74)
	at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:208)
	at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:414)
	at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:310)
	at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1005)
	at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1070)
	at org.apache.hadoop.hive.ql.Driver.run(Driver.java:942)
	at org.apache.hadoop.hive.ql.Driver.run(Driver.java:932)
	at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:246)
	at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:198)
	at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:408)
	at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:344)
	at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:731)
	at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:675)
	at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:614)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at org.apache.hadoop.util.RunJar.main(RunJar.java:212)

Issue Links

• mentioned in: Wiki Page Loading...