[HIVE-9341] Apply ColumnPrunning for noop PTFs

• Type: Improvement
• Status: Resolved
• Priority: Trivial
• Resolution: Fixed
• Fix Version/s: 1.1.0
• Component/s: PTF-Windowing
• Assignee: Navis Ryu
• Reporter: Navis Ryu
• Created: 12/Jan/15 05:00
• Updated: 12/Feb/15 23:40
• Resolved: 24/Jan/15 00:17

Description

Currently, PTF disables CP optimization, which can make a huge burden. For example,

select p_mfgr, p_name, p_size,
rank() over (partition by p_mfgr order by p_name) as r,
dense_rank() over (partition by p_mfgr order by p_name) as dr,
sum(p_retailprice) over (partition by p_mfgr order by p_name rows between unbounded preceding and current row) as s1
from noop(on part 
  partition by p_mfgr
  order by p_name
  );

STAGE PLANS:
  Stage: Stage-1
    Map Reduce
      Map Operator Tree:
          TableScan
            alias: part
            Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
            Reduce Output Operator
              key expressions: p_mfgr (type: string), p_name (type: string)
              sort order: ++
              Map-reduce partition columns: p_mfgr (type: string)
              Statistics: Num rows: 26 Data size: 3147 Basic stats: COMPLETE Column stats: NONE
              value expressions: p_partkey (type: int), p_name (type: string), p_mfgr (type: string), p_brand (type: string), p_type (type: string), p_size (type: int), p_container (type: string), p_retailprice (type: double), p_comment (type: string), BLOCK__OFFSET__INSIDE__FILE (type: bigint), INPUT__FILE__NAME (type: string), ROW__ID (type: struct<transactionid:bigint,bucketid:int,rowid:bigint>)
...

There should be a generic way to discern referenced columns but before that, we know CP can be safely applied to noop functions.

Issue Links

• links to: review board (source only)