[IMPALA-9023] select statement fails with "ERROR: IllegalStateException: null" after expression rewrite

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: Impala 3.4.0
• Fix Version/s: Impala 3.4.0
• Component/s: Frontend
• Labels: planner
• Epic Color: ghx-label-14
• Assignee: Alice Fan
• Reporter: Alice Fan
• Created: 08/Oct/19 07:31
• Updated: 14/May/20 03:29
• Resolved: 10/Nov/19 03:51

Description

Steps to repro:

create table t (year int, make string);

insert into t 
values
(2005, 'Toyota'),
(2010, 'Toyota'),
(2005, 'Honda'),
(2010, 'Honda'),
(2005, 'Ford'),
(2010, 'Ford');

select * 
from t
where 
  case when make = 'Toyota' then year < 2008
  end 
and make = 'Toyota';

It will fail with following errors:

I1007 23:48:37.523810 15479 Frontend.java:1295] 7d47c4acb54a7131:bca9c9ef00000000] Analysis and authorization finished.
I1007 23:48:37.574666 15479 jni-util.cc:288] 7d47c4acb54a7131:bca9c9ef00000000] java.lang.IllegalStateException
        at com.google.common.base.Preconditions.checkState(Preconditions.java:133)
        at org.apache.impala.analysis.SlotRef.isBoundBySlotIds(SlotRef.java:222)
        at org.apache.impala.analysis.Expr.isBoundBySlotIds(Expr.java:1256)
        at org.apache.impala.analysis.Expr.isBoundBySlotIds(Expr.java:1256)
        at org.apache.impala.planner.HdfsPartitionPruner.prunePartitions(HdfsPartitionPruner.java:124)
        at org.apache.impala.planner.SingleNodePlanner.createHdfsScanPlan(SingleNodePlanner.java:1269)
        at org.apache.impala.planner.SingleNodePlanner.createScanNode(SingleNodePlanner.java:1395)
        at org.apache.impala.planner.SingleNodePlanner.createTableRefNode(SingleNodePlanner.java:1582)
        at org.apache.impala.planner.SingleNodePlanner.createTableRefsPlan(SingleNodePlanner.java:826)
        at org.apache.impala.planner.SingleNodePlanner.createSelectPlan(SingleNodePlanner.java:662)
        at org.apache.impala.planner.SingleNodePlanner.createQueryPlan(SingleNodePlanner.java:261)
        at org.apache.impala.planner.SingleNodePlanner.createSingleNodePlan(SingleNodePlanner.java:151)
        at org.apache.impala.planner.Planner.createPlan(Planner.java:117)
        at org.apache.impala.service.Frontend.createExecRequest(Frontend.java:1180)
        at org.apache.impala.service.Frontend.getPlannedExecRequest(Frontend.java:1506)
        at org.apache.impala.service.Frontend.doCreateExecRequest(Frontend.java:1370)
        at org.apache.impala.service.Frontend.getTExecRequest(Frontend.java:1261)
        at org.apache.impala.service.Frontend.createExecRequest(Frontend.java:1231)
        at org.apache.impala.service.JniFrontend.createExecRequest(JniFrontend.java:154)
I1007 23:48:37.717686 15479 status.cc:124] 7d47c4acb54a7131:bca9c9ef00000000] IllegalStateException: null
    @          0x1b90190  impala::Status::Status()
    @          0x238b21c  impala::JniUtil::GetJniExceptionMsg()
    @          0x21b18b5  impala::JniCall::Call<>()
    @          0x21ae9f3  impala::JniUtil::CallJniMethod<>()
    @          0x21acc96  impala::Frontend::GetExecRequest()
    @          0x21d9fe0  impala::ImpalaServer::ExecuteInternal()
    @          0x21d9a0e  impala::ImpalaServer::Execute()
    @          0x226698a  impala::ImpalaServer::query()
    @          0x27ce3fd  beeswax::BeeswaxServiceProcessor::process_query()
    @          0x27ce14b  beeswax::BeeswaxServiceProcessor::dispatchCall()
    @          0x279bb78  impala::ImpalaServiceProcessor::dispatchCall()
    @          0x1b3f553  apache::thrift::TDispatchProcessor::process()
    @          0x1f9ee57  apache::thrift::server::TAcceptQueueServer::Task::run()
    @          0x1f9459c  impala::ThriftThread::RunRunnable()
    @          0x1f95cc2  boost::_mfi::mf2<>::operator()()
    @          0x1f95b58  boost::_bi::list3<>::operator()<>()
    @          0x1f958a4  boost::_bi::bind_t<>::operator()()
    @          0x1f957b7  boost::detail::function::void_function_obj_invoker0<>::invoke()
    @          0x1eb271f  boost::function0<>::operator()()
    @          0x2405a34  impala::Thread::SuperviseThread()
    @          0x240ddb8  boost::_bi::list5<>::operator()<>()
    @          0x240dcdc  boost::_bi::bind_t<>::operator()()
    @          0x240dc9f  boost::detail::thread_data<>::run()
    @          0x3bea089  thread_proxy
    @     0x7fea2747f6b9  start_thread
    @     0x7fea23c5e41c  clone

Workarounds:
1. Disable expression rewrites (SET ENABLE_EXPR_REWRITES=0), which can fix the issue.
2. Rewrite query with the same logic:

select * from t where make = 'Toyota' and year < 2008;