[IMPALA-6454] CTAS into Kudu fails with mixed-case partition and/pr primary key column names

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: Impala 2.7.0, Impala 2.8.0, Impala 2.9.0, Impala 2.10.0, Impala 2.11.0
• Fix Version/s: Impala 3.0, Impala 2.12.0
• Component/s: Frontend
• Labels: kudu
• Target Version: Impala 2.12.0
• Epic Color: ghx-label-1
• Assignee: Pranay Singh
• Reporter: Alexander Behm
• Created: 30/Jan/18 17:34
• Updated: 12/Apr/18 05:09
• Resolved: 12/Feb/18 19:11

Description

CTAS into Kudu mail fail if the primary key and/or partition column names are not specified in lower case.

The observed error is:

ERROR: AnalysisException: null
CAUSED BY: IllegalStateException: null

Reproduction:

CREATE TABLE ctas_test
PRIMARY KEY (PK1, PK2)
PARTITION BY HASH (PK1) PARTITIONS 10
STORED AS KUDU
AS
SELECT 1 PK1, 2 PK2, 3 C1, 4 C2

IMPALA Log:

===========
I0122 13:39:15.829866 7557 Frontend.java:927] Compiled query.
I0122 13:39:15.830080 7557 jni-util.cc:196] org.apache.impala.common.AnalysisException
at org.apache.impala.analysis.AnalysisContext.analyze(AnalysisContext.java:433)
at org.apache.impala.analysis.AnalysisContext.analyze(AnalysisContext.java:369)
at org.apache.impala.service.Frontend.analyzeStmt(Frontend.java:899)
at org.apache.impala.service.Frontend.createExecRequest(Frontend.java:1048)
at org.apache.impala.service.JniFrontend.createExecRequest(JniFrontend.java:156)
Caused by: java.lang.IllegalStateException
at com.google.common.base.Preconditions.checkState(Preconditions.java:129)
at org.apache.impala.analysis.InsertStmt.prepareExpressions(InsertStmt.java:770)
at org.apache.impala.analysis.InsertStmt.analyze(InsertStmt.java:383)
at org.apache.impala.analysis.CreateTableAsSelectStmt.analyze(CreateTableAsSelectStmt.java:212)
at org.apache.impala.analysis.AnalysisContext.analyze(AnalysisContext.java:388)
... 4 more
I0122 13:39:15.845522 7557 status.cc:122] AnalysisException: null
CAUSED BY: IllegalStateException: null
@ 0x83e979 impala::Status::Status()
@ 0xb71ec9 impala::JniUtil::GetJniExceptionMsg()
@ 0xa7cc0b impala::Frontend::GetExecRequest()
@ 0xa97368 impala::ImpalaServer::ExecuteInternal()
@ 0xa9e408 impala::ImpalaServer::Execute()
@ 0xadd27e impala::ImpalaServer::query()
@ 0xd6ee05 beeswax::BeeswaxServiceProcessor::process_query()
@ 0xd72104 beeswax::BeeswaxServiceProcessor::dispatchCall()
@ 0x80e69c apache::thrift::TDispatchProcessor::process()
@ 0x1b5666b apache::thrift::server::TThreadPoolServer::Task::run()
@ 0x1b3df89 apache::thrift::concurrency::ThreadManager::Worker::run()
@ 0x9d4ca9 impala::ThriftThread::RunRunnable()
@ 0x9d5a82 boost::detail::function::void_function_obj_invoker0<>::invoke()
@ 0xbd4722 impala::Thread::SuperviseThread()
@ 0xbd4e84 boost::detail::thread_data<>::run()
@ 0xe6113a (unknown)
@ 0x7fd1efc19dc5 start_thread
@ 0x7fd1ef946ced __clone
I0122 13:39:15.845556 7557 impala-server.cc:989] UnregisterQuery(): query_id=42439790ebd1f31f:772bd00b00000000
I0122 13:39:15.845562 7557 impala-server.cc:1075] Cancel(): query_id=42439790ebd1f31f:772bd00b00000000

Rootcause: In the below call graph in the function KuduTable.isPrimaryKeyColumn() primaryKeyColumnNames_ does not have a matching case which causes
primaryKeyExprs_ to be empty and hence we hit an Exception. Changing the case fixes the problem.
Problem: In the InsertStmt.prepareExpressions() , primaryKeyExprs_ is empty, which is causing Preconditions.checkState(!primaryKeyExprs_.isEmpty()); to fail and as a result
Exception is generated.
Call graph
-----------------
AnalysisContext.analyze(0
--->CreateTableAsSelectStmt.analyze()
------->InsertStmt.analyze
----------->InsertStmt.prepareExpressions
-------------->KuduTable.isPrimaryKeyColumn