[HIVE-26621] TPC-DS query 2 fails with java.lang.RuntimeException: cannot find field _col0 from []

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 4.0.0-alpha-1
• Fix Version/s: 4.0.0
• Assignee: Seonggon Namgung
• Reporter: Sungwoo Park
• Created: 11/Oct/22 13:52
• Updated: 29/Mar/24 17:50
• Resolved: 23/Nov/23 13:44

Description

How to reproduce:

1. Build Hive using the master branch. Tez version is 0.10.2.

2. Generate 100GB TPC-DS text data.

3. Load the TPC-DS data using ORC format.

4. Run TPC-DS queries. Query 2 fails with the error shown below.

 

Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.RuntimeException: cannot find field _col0 from []
    at org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.process(ReduceSinkOperator.java:384)
    at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:888)
    at org.apache.hadoop.hive.ql.exec.SelectOperator.process(SelectOperator.java:94)
    at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource$GroupIterator.next(ReduceRecordSource.java:370)
    ... 19 more
Caused by: java.lang.RuntimeException: cannot find field _col0 from []
    at org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorUtils.getStandardStructFieldRef(ObjectInspectorUtils.java:550)
    at org.apache.hadoop.hive.serde2.objectinspector.StandardStructObjectInspector.getStructFieldRef(StandardStructObjectInspector.java:153)
    at org.apache.hadoop.hive.ql.exec.ExprNodeColumnEvaluator.initialize(ExprNodeColumnEvaluator.java:56)
    at org.apache.hadoop.hive.ql.exec.Operator.initEvaluators(Operator.java:1073)
    at org.apache.hadoop.hive.ql.exec.Operator.initEvaluatorsAndReturnStruct(Operator.java:1099)
    at org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.process(ReduceSinkOperator.java:305)
    ... 22 more
 

 

A slightly simplified query that produces the same error is:

with wscs as
 (select sold_date_sk
        ,sales_price
  from (select ws_sold_date_sk sold_date_sk
              ,ws_ext_sales_price sales_price
        from web_sales) x
        union all
       (select cs_sold_date_sk sold_date_sk
              ,cs_ext_sales_price sales_price
        from catalog_sales)),
 wswscs as
 (select d_week_seq
 from wscs
     ,date_dim
 where d_date_sk = sold_date_sk
 group by d_week_seq)
 select d_week_seq1
 from
 (select wswscs.d_week_seq d_week_seq1
  from wswscs,date_dim
  where date_dim.d_week_seq = wswscs.d_week_seq and d_year = 2001) y,
 (select wswscs.d_week_seq d_week_seq2
  from wswscs,date_dim
  where date_dim.d_week_seq = wswscs.d_week_seq and d_year = 2001+1) z
 where d_week_seq1=d_week_seq2-53
 order by d_week_seq1;

 

 

Issue Links

• is caused by: HIVE-24231 Enhance shared work optimizer to merge scans with filters on both sides
• is fixed by: HIVE-27006 ParallelEdgeFixer inserts misconfigured operator and does not connect it in Tez DAG