[IMPALA-225] compound predicate ranges on partition key crash impalad

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: Impala 0.7
• Fix Version/s: Impala 1.0
• Assignee: Marcel Kinard
• Reporter: Greg Rahn
• Created: 11/Apr/13 00:03
• Updated: 20/Dec/15 00:05
• Resolved: 13/Apr/13 01:55

Description

The following crashes impalad.
Partition key is store_sales.ss_date(string).

Build version: Impala v0.7 (6bd9c48) built on Wed Apr 10 10:00:56 PDT 2013

select
  count(*)
from
  store_sales
where
    (ss_date between '1998-12-01' and '1998-12-02'
  or ss_date between '1998-12-30' and '1998-12-31')
;

impala.INFO

...
INFO0410 16:53:54.043000 Thread-4 com.cloudera.impala.catalog.HdfsTable] loaded partition PartitionBlockMetadata{#blocks=6, #filenames=1, totalStringLen=102}
INFO0410 16:53:54.044000 Thread-4 com.cloudera.impala.catalog.HdfsTable] loaded partition PartitionBlockMetadata{#blocks=6, #filenames=1, totalStringLen=102}
INFO0410 16:53:54.532000 Thread-4 com.cloudera.impala.catalog.HdfsTable] loaded disk ids for table tpcds3000.store_sales
INFO0410 16:53:54.532000 Thread-4 com.cloudera.impala.catalog.HdfsTable] 8
INFO0410 16:53:54.532000 Thread-4 com.cloudera.impala.catalog.HdfsTable] table #rows=0
INFO0410 16:53:54.538000 Thread-4 com.cloudera.impala.analysis.BinaryPredicate] ss_date >= '1998-12-01' selectivity: 0.1
INFO0410 16:53:54.538000 Thread-4 com.cloudera.impala.analysis.BinaryPredicate] ss_date <= '1998-12-02' selectivity: 0.1
INFO0410 16:53:54.539000 Thread-4 com.cloudera.impala.analysis.CompoundPredicate] ss_date >= '1998-12-01' AND ss_date <= '1998-12-02' selectivity: 0.010000000000000002
INFO0410 16:53:54.539000 Thread-4 com.cloudera.impala.analysis.BinaryPredicate] ss_date >= '1998-12-30' selectivity: 0.1
INFO0410 16:53:54.539000 Thread-4 com.cloudera.impala.analysis.BinaryPredicate] ss_date <= '1998-12-31' selectivity: 0.1
INFO0410 16:53:54.539000 Thread-4 com.cloudera.impala.analysis.CompoundPredicate] ss_date >= '1998-12-30' AND ss_date <= '1998-12-31' selectivity: 0.010000000000000002
INFO0410 16:53:54.539000 Thread-4 com.cloudera.impala.analysis.CompoundPredicate] ss_date BETWEEN '1998-12-01' AND '1998-12-02' OR ss_date BETWEEN '1998-12-30' AND '1998-12-31' selectivity: 0.19
INFO0410 16:53:54.542000 Thread-4 com.cloudera.impala.analysis.AggregateInfo] agg info:
AggregateInfo{grouping_exprs=, aggregate_exprs=(AggregateExpr{op=COUNT, isStar=true, isDistinct=false, }), agg_tuple=TupleDescriptor{id=1, tbl=null, byte_size=0, is_materialized=true, slots=[SlotDescriptor{id=1, col=null, type=BIGINT, materialized=false, byteSize=0, byteOffset=-1, nullIndicatorByte=0, nullIndicatorBit=0, slotIdx=0}]}, smap=smap(AggregateExpr{op=COUNT, isStar=true, isDistinct=false, }:SlotRef{tblName=null, col=null, id=1})}
mergeAggInfo:
AggregateInfo{grouping_exprs=, aggregate_exprs=(AggregateExpr{op=SUM, isStar=false, isDistinct=false, (SlotRef{tblName=null, col=null, id=1})}), agg_tuple=TupleDescriptor{id=1, tbl=null, byte_size=0, is_materialized=true, slots=[SlotDescriptor{id=1, col=null, type=BIGINT, materialized=false, byteSize=0, byteOffset=-1, nullIndicatorByte=0, nullIndicatorBit=0, slotIdx=0}]}, smap=smap(AggregateExpr{op=COUNT, isStar=true, isDistinct=false, }:SlotRef{tblName=null, col=null, id=1})}
INFO0410 16:53:54.543000 Thread-4 com.cloudera.impala.service.Frontend] create plan
INFO0410 16:53:54.543000 Thread-4 com.cloudera.impala.planner.Planner] create single-node plan
INFO0410 16:53:54.546000 Thread-4 com.cloudera.impala.planner.AggregationNode] Agg: cardinality=1
INFO0410 16:53:54.547000 Thread-4 com.cloudera.impala.planner.AggregationNode] sel=1.0
INFO0410 16:53:54.547000 Thread-4 com.cloudera.impala.planner.AggregationNode] stats Agg: cardinality=1
INFO0410 16:53:54.547000 Thread-4 com.cloudera.impala.planner.HdfsScanNode] collecting partitions for table store_sales
INFO0410 16:53:54.547000 Thread-4 com.cloudera.impala.analysis.BinaryPredicate] '1998-01-02' >= '1998-12-01' selectivity: 0.1
INFO0410 16:53:54.547000 Thread-4 com.cloudera.impala.analysis.BinaryPredicate] '1998-01-02' <= '1998-12-02' selectivity: 0.1
INFO0410 16:53:54.547000 Thread-4 com.cloudera.impala.analysis.BinaryPredicate] ss_date >= '1998-12-01' selectivity: 0.1
INFO0410 16:53:54.547000 Thread-4 com.cloudera.impala.analysis.BinaryPredicate] ss_date <= '1998-12-02' selectivity: 0.1
INFO0410 16:53:54.547000 Thread-4 com.cloudera.impala.analysis.CompoundPredicate] ss_date >= '1998-12-01' AND ss_date <= '1998-12-02' selectivity: 0.010000000000000002
INFO0410 16:53:54.548000 Thread-4 com.cloudera.impala.analysis.BinaryPredicate] '1998-01-02' >= '1998-12-30' selectivity: 0.1
INFO0410 16:53:54.548000 Thread-4 com.cloudera.impala.analysis.BinaryPredicate] '1998-01-02' <= '1998-12-31' selectivity: 0.1
INFO0410 16:53:54.548000 Thread-4 com.cloudera.impala.analysis.BinaryPredicate] ss_date >= '1998-12-30' selectivity: 0.1
INFO0410 16:53:54.548000 Thread-4 com.cloudera.impala.analysis.BinaryPredicate] ss_date <= '1998-12-31' selectivity: 0.1
INFO0410 16:53:54.548000 Thread-4 com.cloudera.impala.analysis.CompoundPredicate] ss_date >= '1998-12-30' AND ss_date <= '1998-12-31' selectivity: 0.010000000000000002
INFO0410 16:53:54.548000 Thread-4 com.cloudera.impala.analysis.CompoundPredicate] ss_date BETWEEN '1998-12-01' AND '1998-12-02' OR ss_date BETWEEN '1998-12-30' AND '1998-12-31' selectivity: 0.19