[IMPALA-5111] IllegalArgumentException when using explicit "NOT NULL" on pk column

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: Impala 2.9.0
• Fix Version/s: Impala 2.9.0
• Component/s: Catalog
• Labels: kudu query_generator regression
• Target Version: Impala 2.9.0
• Epic Color: ghx-label-1
• Assignee: Matthew Jacobs
• Reporter: Michael Brown
• Created: 23/Mar/17 19:49
• Updated: 24/Mar/17 21:34
• Resolved: 24/Mar/17 21:34

Description

If I create a Kudu table and explicitly use NOT NULL for a primary key column, I get an IllegalArgumentException.

CREATE TABLE t (
  c SMALLINT NOT NULL,
  PRIMARY KEY (c)
)
PARTITION BY HASH (c) PARTITIONS 2
STORED AS KUDU
TBLPROPERTIES ('kudu.master_addresses'='127.0.0.1');

I0323 12:39:38.180569 28753 jni-util.cc:176] org.apache.impala.common.ImpalaRuntimeException: Error creating Kudu table 'impala::default.t'
        at org.apache.impala.service.KuduCatalogOpExecutor.createManagedTable(KuduCatalogOpExecutor.java:89)
        at org.apache.impala.service.CatalogOpExecutor.createKuduTable(CatalogOpExecutor.java:1653)
        at org.apache.impala.service.CatalogOpExecutor.createTable(CatalogOpExecutor.java:1575)
        at org.apache.impala.service.CatalogOpExecutor.execDdlRequest(CatalogOpExecutor.java:273)
        at org.apache.impala.service.JniCatalog.execDdl(JniCatalog.java:142)
Caused by: java.lang.IllegalArgumentException
        at com.google.common.base.Preconditions.checkArgument(Preconditions.java:72)
        at org.apache.impala.service.KuduCatalogOpExecutor.createColumnSchema(KuduCatalogOpExecutor.java:103)
        at org.apache.impala.service.KuduCatalogOpExecutor.createTableSchema(KuduCatalogOpExecutor.java:134)
        at org.apache.impala.service.KuduCatalogOpExecutor.createManagedTable(KuduCatalogOpExecutor.java:85)
        ... 4 more
I0323 12:39:38.217900 28753 status.cc:114] ImpalaRuntimeException: Error creating Kudu table 'impala::default.t'
CAUSED BY: IllegalArgumentException: null
    @          0x1209591  impala::Status::Status()
    @          0x15e5a94  impala::JniUtil::GetJniExceptionMsg()
    @          0x11fae31  impala::JniUtil::CallJniMethod<>()
    @          0x11f95a9  impala::Catalog::ExecDdl()
    @          0x11c4b50  CatalogServiceThriftIf::ExecDdl()
    @          0x123fa34  impala::CatalogServiceProcessor::process_ExecDdl()
    @          0x123f782  impala::CatalogServiceProcessor::dispatchCall()
    @          0x11b0564  apache::thrift::TDispatchProcessor::process()
    @          0x1381f5f  apache::thrift::server::TAcceptQueueServer::Task::run()
    @          0x137e049  impala::ThriftThread::RunRunnable()
    @          0x137f775  boost::_mfi::mf2<>::operator()()
    @          0x137f60b  boost::_bi::list3<>::operator()<>()
    @          0x137f357  boost::_bi::bind_t<>::operator()()
    @          0x137f26a  boost::detail::function::void_function_obj_invoker0<>::invoke()
    @          0x138ce2e  boost::function0<>::operator()()
    @          0x163a631  impala::Thread::SuperviseThread()
    @          0x164300c  boost::_bi::list4<>::operator()<>()
    @          0x1642f4f  boost::_bi::bind_t<>::operator()()
    @          0x1642f12  boost::detail::thread_data<>::run()
    @          0x1aee81a  thread_proxy
    @     0x7fc40964c184  start_thread
    @     0x7fc40937937d  clone
E0323 12:39:38.217911 28753 catalog-server.cc:69] ImpalaRuntimeException: Error creating Kudu table 'impala::default.t'
CAUSED BY: IllegalArgumentException: null

Note that a workaround is to remove NOT NULL here, and it works:

Query: create TABLE t (
  c SMALLINT,
  PRIMARY KEY (c)
)
PARTITION BY HASH (c) PARTITIONS 2
STORED AS KUDU
TBLPROPERTIES ('kudu.master_addresses'='127.0.0.1')

Fetched 0 row(s) in 0.87s

... but know that SHOW CREATE TABLE will include NOT NULL:

Query: show create table t
+-------------------------------------------------------------------------------+
| result                                                                        |
+-------------------------------------------------------------------------------+
| CREATE TABLE default.t (                                                      |
|   c SMALLINT NOT NULL ENCODING AUTO_ENCODING COMPRESSION DEFAULT_COMPRESSION, |
|   PRIMARY KEY (c)                                                             |
| )                                                                             |
| PARTITION BY HASH (c) PARTITIONS 2                                            |
| STORED AS KUDU                                                                |
| TBLPROPERTIES ('kudu.master_addresses'='127.0.0.1')                           |
+-------------------------------------------------------------------------------+
Fetched 1 row(s) in 4.57s

I found this because automation uses the output of SHOW CREATE TABLE to create a table copy.