Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-5111

IllegalArgumentException when using explicit "NOT NULL" on pk column

    Details

      Description

      If I create a Kudu table and explicitly use NOT NULL for a primary key column, I get an IllegalArgumentException.

      CREATE TABLE t (
        c SMALLINT NOT NULL,
        PRIMARY KEY (c)
      )
      PARTITION BY HASH (c) PARTITIONS 2
      STORED AS KUDU
      TBLPROPERTIES ('kudu.master_addresses'='127.0.0.1');
      
      I0323 12:39:38.180569 28753 jni-util.cc:176] org.apache.impala.common.ImpalaRuntimeException: Error creating Kudu table 'impala::default.t'
              at org.apache.impala.service.KuduCatalogOpExecutor.createManagedTable(KuduCatalogOpExecutor.java:89)
              at org.apache.impala.service.CatalogOpExecutor.createKuduTable(CatalogOpExecutor.java:1653)
              at org.apache.impala.service.CatalogOpExecutor.createTable(CatalogOpExecutor.java:1575)
              at org.apache.impala.service.CatalogOpExecutor.execDdlRequest(CatalogOpExecutor.java:273)
              at org.apache.impala.service.JniCatalog.execDdl(JniCatalog.java:142)
      Caused by: java.lang.IllegalArgumentException
              at com.google.common.base.Preconditions.checkArgument(Preconditions.java:72)
              at org.apache.impala.service.KuduCatalogOpExecutor.createColumnSchema(KuduCatalogOpExecutor.java:103)
              at org.apache.impala.service.KuduCatalogOpExecutor.createTableSchema(KuduCatalogOpExecutor.java:134)
              at org.apache.impala.service.KuduCatalogOpExecutor.createManagedTable(KuduCatalogOpExecutor.java:85)
              ... 4 more
      I0323 12:39:38.217900 28753 status.cc:114] ImpalaRuntimeException: Error creating Kudu table 'impala::default.t'
      CAUSED BY: IllegalArgumentException: null
          @          0x1209591  impala::Status::Status()
          @          0x15e5a94  impala::JniUtil::GetJniExceptionMsg()
          @          0x11fae31  impala::JniUtil::CallJniMethod<>()
          @          0x11f95a9  impala::Catalog::ExecDdl()
          @          0x11c4b50  CatalogServiceThriftIf::ExecDdl()
          @          0x123fa34  impala::CatalogServiceProcessor::process_ExecDdl()
          @          0x123f782  impala::CatalogServiceProcessor::dispatchCall()
          @          0x11b0564  apache::thrift::TDispatchProcessor::process()
          @          0x1381f5f  apache::thrift::server::TAcceptQueueServer::Task::run()
          @          0x137e049  impala::ThriftThread::RunRunnable()
          @          0x137f775  boost::_mfi::mf2<>::operator()()
          @          0x137f60b  boost::_bi::list3<>::operator()<>()
          @          0x137f357  boost::_bi::bind_t<>::operator()()
          @          0x137f26a  boost::detail::function::void_function_obj_invoker0<>::invoke()
          @          0x138ce2e  boost::function0<>::operator()()
          @          0x163a631  impala::Thread::SuperviseThread()
          @          0x164300c  boost::_bi::list4<>::operator()<>()
          @          0x1642f4f  boost::_bi::bind_t<>::operator()()
          @          0x1642f12  boost::detail::thread_data<>::run()
          @          0x1aee81a  thread_proxy
          @     0x7fc40964c184  start_thread
          @     0x7fc40937937d  clone
      E0323 12:39:38.217911 28753 catalog-server.cc:69] ImpalaRuntimeException: Error creating Kudu table 'impala::default.t'
      CAUSED BY: IllegalArgumentException: null
      

      Note that a workaround is to remove NOT NULL here, and it works:

      Query: create TABLE t (
        c SMALLINT,
        PRIMARY KEY (c)
      )
      PARTITION BY HASH (c) PARTITIONS 2
      STORED AS KUDU
      TBLPROPERTIES ('kudu.master_addresses'='127.0.0.1')
      
      Fetched 0 row(s) in 0.87s
      

      ... but know that SHOW CREATE TABLE will include NOT NULL:

      Query: show create table t
      +-------------------------------------------------------------------------------+
      | result                                                                        |
      +-------------------------------------------------------------------------------+
      | CREATE TABLE default.t (                                                      |
      |   c SMALLINT NOT NULL ENCODING AUTO_ENCODING COMPRESSION DEFAULT_COMPRESSION, |
      |   PRIMARY KEY (c)                                                             |
      | )                                                                             |
      | PARTITION BY HASH (c) PARTITIONS 2                                            |
      | STORED AS KUDU                                                                |
      | TBLPROPERTIES ('kudu.master_addresses'='127.0.0.1')                           |
      +-------------------------------------------------------------------------------+
      Fetched 1 row(s) in 4.57s
      

      I found this because automation uses the output of SHOW CREATE TABLE to create a table copy.

        Activity

        Hide
        mjacobs Matthew Jacobs added a comment -

        commit 878fcf5a746153bcaf38dab2ecd6fa6a462ceb13
        Author: Matthew Jacobs <mj@cloudera.com>
        Date: Thu Mar 23 13:16:42 2017 -0700

        IMPALA-5111: Fix check when creating NOT NULL PK col in Kudu

        The fix for IMPALA-4616 broke the ability to create a PK key
        col in a Kudu table as explicitly 'NOT NULL'. While this is
        the default, it should be possible to specify.

        The precondition that was failing was fixed, and some tests
        were added/modified.

        Change-Id: I557eea7cd994d6a2ed38893d283d08107e78f789
        Reviewed-on: http://gerrit.cloudera.org:8080/6465
        Reviewed-by: Matthew Jacobs <mj@cloudera.com>
        Tested-by: Impala Public Jenkins

        Show
        mjacobs Matthew Jacobs added a comment - commit 878fcf5a746153bcaf38dab2ecd6fa6a462ceb13 Author: Matthew Jacobs <mj@cloudera.com> Date: Thu Mar 23 13:16:42 2017 -0700 IMPALA-5111 : Fix check when creating NOT NULL PK col in Kudu The fix for IMPALA-4616 broke the ability to create a PK key col in a Kudu table as explicitly 'NOT NULL'. While this is the default, it should be possible to specify. The precondition that was failing was fixed, and some tests were added/modified. Change-Id: I557eea7cd994d6a2ed38893d283d08107e78f789 Reviewed-on: http://gerrit.cloudera.org:8080/6465 Reviewed-by: Matthew Jacobs <mj@cloudera.com> Tested-by: Impala Public Jenkins
        Hide
        mjacobs Matthew Jacobs added a comment - - edited

        Yup, this looks like it was introduced by this commit:
        IMPALA-4616: Add missing Kudu column options

        Adds support for missing Kudu column options in ALTER TABLE
        (it was there in CREATE TABLE already):

        • encoding
        • compression
        • block_size

        Also adds support for adding nullable columns with default
        values.

        All of the above was not originally implemented due to
        limitations in the Kudu client, but have since been fixed:
        KUDU-1746, KUDU-1747

        Testing: Updates and adds relevant test cases.

        Change-Id: I96a0fce7f6bc0c086b259d3119daa72c94b0af7b
        Reviewed-on: http://gerrit.cloudera.org:8080/6220
        Reviewed-by: Marcel Kornacker <marcel@cloudera.com>
        Tested-by: Impala Public Jenkins

        Should be easy to fix, thanks for filing!

        Show
        mjacobs Matthew Jacobs added a comment - - edited Yup, this looks like it was introduced by this commit: IMPALA-4616 : Add missing Kudu column options Adds support for missing Kudu column options in ALTER TABLE (it was there in CREATE TABLE already): encoding compression block_size Also adds support for adding nullable columns with default values. All of the above was not originally implemented due to limitations in the Kudu client, but have since been fixed: KUDU-1746 , KUDU-1747 Testing: Updates and adds relevant test cases. Change-Id: I96a0fce7f6bc0c086b259d3119daa72c94b0af7b Reviewed-on: http://gerrit.cloudera.org:8080/6220 Reviewed-by: Marcel Kornacker <marcel@cloudera.com> Tested-by: Impala Public Jenkins Should be easy to fix, thanks for filing!
        Hide
        mikesbrown Michael Brown added a comment -

        Matthew Jacobs I assigned to you initially. From what I can tell this is a regression.

        Show
        mikesbrown Michael Brown added a comment - Matthew Jacobs I assigned to you initially. From what I can tell this is a regression.

          People

          • Assignee:
            mjacobs Matthew Jacobs
            Reporter:
            mikesbrown Michael Brown
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development