Details
-
Type:
Bug
-
Status: Closed
-
Priority:
Major
-
Resolution: Fixed
-
Affects Version/s: 0.5.0
-
Fix Version/s: 0.6.0
-
Component/s: Query Processor
-
Labels:None
-
Hadoop Flags:Reviewed
Description
Predicate push down does not work with UDTF's in lateral views
hive> SELECT * FROM src LATERAL VIEW explode(array(1,2,3)) myTable AS k WHERE k=1;
FAILED: Unknown exception: null
hive>
Activity
- All
- Comments
- Work Log
- History
- Activity
- Transitions
- Enables UDTF's to emit rows on close. Although this was disabled initially, it should be allowed. Without this, the test with lateral views on partitioned tables will fail.
- Enables PPD for LV. To make pruning easier, a new operator (LateralViewForward) is placed at the head of every LV DAG. Otherwise, it is difficult for the pruner to handle the special case for LV's.
Will take a look.
Message from: "Carl Steinbach" <carl@cloudera.com>
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
http://review.hbase.org/r/198/
-----------------------------------------------------------
Review request for Hive Developers.
Summary
-------
Review for patch https://issues.apache.org/jira/secure/attachment/12441230/HIVE-1056.1.patch
This addresses bug HIVE-1056.
http://issues.apache.org/jira/browse/HIVE-1056
Diffs
contrib/src/java/org/apache/hadoop/hive/contrib/udtf/example/GenericUDTFCount2.java PRE-CREATION
contrib/src/test/queries/clientpositive/udtf_output_on_close.q PRE-CREATION
contrib/src/test/results/clientpositive/udtf_output_on_close.q.out PRE-CREATION
ql/if/queryplan.thrift d387e8e
ql/src/gen-javabean/org/apache/hadoop/hive/ql/plan/api/OperatorType.java 99d1f2d
ql/src/gen-php/queryplan_types.php 334b4f8
ql/src/gen-py/queryplan/ttypes.py d228e68
ql/src/java/org/apache/hadoop/hive/ql/exec/LateralViewForwardOperator.java PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/exec/LateralViewJoinOperator.java 371a7ac
ql/src/java/org/apache/hadoop/hive/ql/exec/OperatorFactory.java 03bd0bb
ql/src/java/org/apache/hadoop/hive/ql/exec/UDTFOperator.java cab3057
ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java a2f9dba
ql/src/java/org/apache/hadoop/hive/ql/plan/LateralViewForwardDesc.java PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/ppd/OpProcFactory.java 6e1dfe4
ql/src/java/org/apache/hadoop/hive/ql/ppd/PredicatePushDown.java 19fe5f4
ql/src/test/queries/clientpositive/lateral_view_ppd.q PRE-CREATION
ql/src/test/results/clientpositive/lateral_view.q.out 8931455
ql/src/test/results/clientpositive/lateral_view_ppd.q.out PRE-CREATION
Diff: http://review.hbase.org/r/198/diff
Testing
-------
Thanks,
Carl
Added comment for LVF PPD rule.
running tests now.
Committed! Thanks Paul!
We should backport this to 0.6
Attaching patch for branch-0.6
Committed to branch 0.6 together with HIVE-1452. Thanks Paul and Carl!