Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-4662

with Kudu tables, ClassCastException with NULL literal in IN list

    Details

      Description

      This seems to be a Kudu-specific bug, since the error is about initializing the Kudu scan node, and the equivalent query works fine in functional.

      explain select id from functional_kudu.alltypestiny where id in (null);
      

      This query is OK:

      explain select id from functional_kudu.alltypestiny where id = null;
      

      The backtrace is:

      I1214 15:12:41.465106 25897 jni-util.cc:169] org.apache.impala.common.ImpalaRuntimeException: Unable to initialize the Kudu scan node
              at org.apache.impala.planner.KuduScanNode.init(KuduScanNode.java:135)
              at org.apache.impala.planner.SingleNodePlanner.createScanNode(SingleNodePlanner.java:1285)
              at org.apache.impala.planner.SingleNodePlanner.createTableRefNode(SingleNodePlanner.java:1498)
              at org.apache.impala.planner.SingleNodePlanner.createTableRefsPlan(SingleNodePlanner.java:773)
              at org.apache.impala.planner.SingleNodePlanner.createSelectPlan(SingleNodePlanner.java:613)
              at org.apache.impala.planner.SingleNodePlanner.createQueryPlan(SingleNodePlanner.java:254)
              at org.apache.impala.planner.SingleNodePlanner.createSingleNodePlan(SingleNodePlanner.java:147)
              at org.apache.impala.planner.Planner.createPlan(Planner.java:87)
              at org.apache.impala.service.Frontend.createExecRequest(Frontend.java:998)
              at org.apache.impala.service.Frontend.createExecRequest(Frontend.java:1090)
              at org.apache.impala.service.JniFrontend.createExecRequest(JniFrontend.java:160)
      Caused by: java.lang.ClassCastException: org.apache.impala.analysis.NullLiteral cannot be cast to org.apache.impala.analysis.NumericLiteral
              at org.apache.impala.planner.KuduScanNode.getKuduInListValue(KuduScanNode.java:404)
              at org.apache.impala.planner.KuduScanNode.tryConvertInListKuduPredicate(KuduScanNode.java:382)
              at org.apache.impala.planner.KuduScanNode.extractKuduConjuncts(KuduScanNode.java:287)
              at org.apache.impala.planner.KuduScanNode.init(KuduScanNode.java:123)
              ... 10 more
      I1214 15:12:41.512305 25897 status.cc:114] ImpalaRuntimeException: Unable to initialize the Kudu scan node
      CAUSED BY: ClassCastException: org.apache.impala.analysis.NullLiteral cannot be cast to org.apache.impala.analysis.NumericLiteral
          @          0x11c8de5  impala::Status::Status()
          @          0x15a0dfe  impala::JniUtil::GetJniExceptionMsg()
          @          0x1462fbe  impala::JniUtil::CallJniMethod<>()
          @          0x145f6a9  impala::Frontend::GetExecRequest()
          @          0x147ccc9  impala::ImpalaServer::ExecuteInternal()
          @          0x147c870  impala::ImpalaServer::Execute()
          @          0x14e547e  impala::ImpalaServer::query()
          @          0x1943d96  beeswax::BeeswaxServiceProcessor::process_query()
          @          0x1943ae4  beeswax::BeeswaxServiceProcessor::dispatchCall()
          @          0x192c4ef  impala::ImpalaServiceProcessor::dispatchCall()
          @          0x116fcac  apache::thrift::TDispatchProcessor::process()
          @          0x27d058b  apache::thrift::server::TThreadPoolServer::Task::run()
          @          0x27b8df9  apache::thrift::concurrency::ThreadManager::Worker::run()
          @          0x1336995  impala::ThriftThread::RunRunnable()
          @          0x13380c1  boost::_mfi::mf2<>::operator()()
          @          0x1337f57  boost::_bi::list3<>::operator()<>()
          @          0x1337ca3  boost::_bi::bind_t<>::operator()()
          @          0x1337bb6  boost::detail::function::void_function_obj_invoker0<>::invoke()
          @          0x1345534  boost::function0<>::operator()()
          @          0x15f21b5  impala::Thread::SuperviseThread()
          @          0x15f918e  boost::_bi::list4<>::operator()<>()
          @          0x15f90d1  boost::_bi::bind_t<>::operator()()
          @          0x15f902c  boost::detail::thread_data<>::run()
          @          0x1a4ecca  thread_proxy
          @     0x7f7619a75184  start_thread
          @     0x7f76197a237d  (unknown)
      

        Activity

        Hide
        mjacobs Matthew Jacobs added a comment -

        Thanks! should be an easy fix

        Show
        mjacobs Matthew Jacobs added a comment - Thanks! should be an easy fix
        Hide
        mjacobs Matthew Jacobs added a comment -

        commit c2faf4a8a13ba89f2f4f2bc4fbd878cffda53d1f
        Author: Matthew Jacobs <mj@cloudera.com>
        Date: Wed Dec 14 15:58:59 2016 -0800

        IMPALA-4662: Fix NULL literal handling in Kudu IN list predicates

        The KuduScanNode attempts to push IN list predicates to the
        Kudu scan, but NULL literals cannot be pushed. The code in
        KuduScanNode needed to check if the Literals in the
        InPredicate is a NullLiteral, in which case the entire IN
        list should not be pushed to Kudu.

        The same handling is already in place for binary predicate
        pushdown.

        Change-Id: Iaf2c10a326373ad80aef51a85cec64071daefa7b
        Reviewed-on: http://gerrit.cloudera.org:8080/5505
        Reviewed-by: Michael Brown <mikeb@cloudera.com>
        Reviewed-by: Matthew Jacobs <mj@cloudera.com>
        Tested-by: Internal Jenkins

        Show
        mjacobs Matthew Jacobs added a comment - commit c2faf4a8a13ba89f2f4f2bc4fbd878cffda53d1f Author: Matthew Jacobs <mj@cloudera.com> Date: Wed Dec 14 15:58:59 2016 -0800 IMPALA-4662 : Fix NULL literal handling in Kudu IN list predicates The KuduScanNode attempts to push IN list predicates to the Kudu scan, but NULL literals cannot be pushed. The code in KuduScanNode needed to check if the Literals in the InPredicate is a NullLiteral, in which case the entire IN list should not be pushed to Kudu. The same handling is already in place for binary predicate pushdown. Change-Id: Iaf2c10a326373ad80aef51a85cec64071daefa7b Reviewed-on: http://gerrit.cloudera.org:8080/5505 Reviewed-by: Michael Brown <mikeb@cloudera.com> Reviewed-by: Matthew Jacobs <mj@cloudera.com> Tested-by: Internal Jenkins
        Hide
        jbapple Jim Apple added a comment -

        This is a bulk comment on all issues with Fix Version 2.8.0 that were resolved on or after 2016-12-09.

        2.8.0 was branched on December 9, with only two changes to master cherry-picked to the 2.8.0 release branch after that:

        https://github.com/apache/incubator-impala/commits/2.8.0

        Issues fixed after December 9 might not be fixed in 2.8.0. If you are the one who marked this issue Resolved, can you check to see if the patch is in 2.8.0 by using the link above? If the patch is not in 2.8.0, can you change the Fix Version to 2.9.0?

        Thank you!

        Show
        jbapple Jim Apple added a comment - This is a bulk comment on all issues with Fix Version 2.8.0 that were resolved on or after 2016-12-09. 2.8.0 was branched on December 9, with only two changes to master cherry-picked to the 2.8.0 release branch after that: https://github.com/apache/incubator-impala/commits/2.8.0 Issues fixed after December 9 might not be fixed in 2.8.0. If you are the one who marked this issue Resolved, can you check to see if the patch is in 2.8.0 by using the link above? If the patch is not in 2.8.0, can you change the Fix Version to 2.9.0? Thank you!

          People

          • Assignee:
            alex.behm Alexander Behm
            Reporter:
            mikesbrown Michael Brown
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development