Details
-
Bug
-
Status: Resolved
-
Critical
-
Resolution: Fixed
-
Impala 2.7.0, Impala 2.8.0, Impala 2.9.0, Impala 2.10.0, Impala 2.11.0
-
ghx-label-1
Description
CTAS into Kudu mail fail if the primary key and/or partition column names are not specified in lower case.
The observed error is:
ERROR: AnalysisException: null CAUSED BY: IllegalStateException: null
Reproduction:
CREATE TABLE ctas_test PRIMARY KEY (PK1, PK2) PARTITION BY HASH (PK1) PARTITIONS 10 STORED AS KUDU AS SELECT 1 PK1, 2 PK2, 3 C1, 4 C2
IMPALA Log:
=========== I0122 13:39:15.829866 7557 Frontend.java:927] Compiled query. I0122 13:39:15.830080 7557 jni-util.cc:196] org.apache.impala.common.AnalysisException at org.apache.impala.analysis.AnalysisContext.analyze(AnalysisContext.java:433) at org.apache.impala.analysis.AnalysisContext.analyze(AnalysisContext.java:369) at org.apache.impala.service.Frontend.analyzeStmt(Frontend.java:899) at org.apache.impala.service.Frontend.createExecRequest(Frontend.java:1048) at org.apache.impala.service.JniFrontend.createExecRequest(JniFrontend.java:156) Caused by: java.lang.IllegalStateException at com.google.common.base.Preconditions.checkState(Preconditions.java:129) at org.apache.impala.analysis.InsertStmt.prepareExpressions(InsertStmt.java:770) at org.apache.impala.analysis.InsertStmt.analyze(InsertStmt.java:383) at org.apache.impala.analysis.CreateTableAsSelectStmt.analyze(CreateTableAsSelectStmt.java:212) at org.apache.impala.analysis.AnalysisContext.analyze(AnalysisContext.java:388) ... 4 more I0122 13:39:15.845522 7557 status.cc:122] AnalysisException: null CAUSED BY: IllegalStateException: null @ 0x83e979 impala::Status::Status() @ 0xb71ec9 impala::JniUtil::GetJniExceptionMsg() @ 0xa7cc0b impala::Frontend::GetExecRequest() @ 0xa97368 impala::ImpalaServer::ExecuteInternal() @ 0xa9e408 impala::ImpalaServer::Execute() @ 0xadd27e impala::ImpalaServer::query() @ 0xd6ee05 beeswax::BeeswaxServiceProcessor::process_query() @ 0xd72104 beeswax::BeeswaxServiceProcessor::dispatchCall() @ 0x80e69c apache::thrift::TDispatchProcessor::process() @ 0x1b5666b apache::thrift::server::TThreadPoolServer::Task::run() @ 0x1b3df89 apache::thrift::concurrency::ThreadManager::Worker::run() @ 0x9d4ca9 impala::ThriftThread::RunRunnable() @ 0x9d5a82 boost::detail::function::void_function_obj_invoker0<>::invoke() @ 0xbd4722 impala::Thread::SuperviseThread() @ 0xbd4e84 boost::detail::thread_data<>::run() @ 0xe6113a (unknown) @ 0x7fd1efc19dc5 start_thread @ 0x7fd1ef946ced __clone I0122 13:39:15.845556 7557 impala-server.cc:989] UnregisterQuery(): query_id=42439790ebd1f31f:772bd00b00000000 I0122 13:39:15.845562 7557 impala-server.cc:1075] Cancel(): query_id=42439790ebd1f31f:772bd00b00000000
Rootcause: In the below call graph in the function KuduTable.isPrimaryKeyColumn() primaryKeyColumnNames_ does not have a matching case which causes
primaryKeyExprs_ to be empty and hence we hit an Exception. Changing the case fixes the problem.
Problem: In the InsertStmt.prepareExpressions() , primaryKeyExprs_ is empty, which is causing Preconditions.checkState(!primaryKeyExprs_.isEmpty()); to fail and as a result
Exception is generated.
Call graph
-----------------
AnalysisContext.analyze(0
--->CreateTableAsSelectStmt.analyze()
------->InsertStmt.analyze
----------->InsertStmt.prepareExpressions
-------------->KuduTable.isPrimaryKeyColumn
The issue has been fixed by the below PATCH
IMPALA-6454: CTAS into Kudu fails with mixed-case partition or primary key column names.CTAS into Kudu fails if the primary key and/or the partition column names are not
specified in lower case.The problem is that we pass in the primary key column names
directly from the parser instead we should be passing the post-analysis ColumnDefs
as primary keys. So it is fixed by making changes to createCtasTarget()
in KuduTable class to take a list of ColumnDef class associated with the primary keys
from getPrimaryKeyColumnDefs() in CreateTableStmt class.
ColumnDef class has column names stored in lower case that are used by createCtasTarget()
to populate primaryKeyColumnNames_ in KuduTable class that resolves the issue.
Testing
-------
Verified against the newly added test case that reproduces the issue without the fix.
Change-Id: Ica1c8ec1544339e9e80733a7a0c78594e0a727d2
Reviewed-on: http://gerrit.cloudera.org:8080/9147
Reviewed-by: Alex Behm <alex.behm@cloudera.com>
Tested-by: Impala Public Jenkins