Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-2603

Incorrect results and plan for inline view referencing several collection types correlated with different ancestor blocks

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    Description

      Problem
      Queries with multiple nested inline views that have correlated references to nested collections (relative table references), can return incorrect results in RELEASE and hit a DCHECK in debug under the following condition:

      • There is an inline view that references multiple nested collections which come from different ancestor blocks at different levels of nesting.
      • In the example below, Impala fails to generate a correct plan for the "a" inline view because the references "t5" and "t6" reference different ancestor query blocks at different nesting levels.

      Query:

      SELECT
        1
      FROM 
        customer t1
        INNER JOIN (
          SELECT 
            1
          FROM
            t1.c_orders t2
            INNER JOIN (
              SELECT
                1
              FROM 
                t2.o_lineitems t5 
                INNER JOIN t1.c_orders t6
             ) as a
        ) as b;
      

      Wrong Query Plan:

      +------------------------------------------------------------------------------------+
      | Explain String                                                                     |
      +------------------------------------------------------------------------------------+
      | Estimated Per-Host Requirements: Memory=176.00MB VCores=1                          |
      | WARNING: The following tables are missing relevant table and/or column statistics. |
      | tpch_nested_parquet.customer                                                       |
      |                                                                                    |
      | 05:EXCHANGE [UNPARTITIONED]                                                        |
      | |                                                                                  |
      | 01:SUBPLAN                                                                         |
      | |                                                                                  |
      | |--04:NESTED LOOP JOIN [CROSS JOIN]                                                |
      | |  |                                                                               |
      | |  |--02:SINGULAR ROW SRC                                                          |
      | |  |                                                                               |
      | |  03:UNNEST [t1.c_orders t2]                                                      |
      | |                                                                                  |
      | 00:SCAN HDFS [tpch_nested_parquet.customer t1]                                     |
      |    partitions=1/1 files=4 size=554.13MB                                            |
      +------------------------------------------------------------------------------------+
      

      Stack Trace:

      #0  0x00007f5c10cf5cc9 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
      #1  0x00007f5c10cf90d8 in __GI_abort () at abort.c:89
      #2  0x0000000002144d09 in google::DumpStackTraceAndExit () at src/utilities.cc:147
      #3  0x000000000213ddbd in google::LogMessage::Fail () at src/logging.cc:1315
      #4  0x000000000213fc45 in google::LogMessage::SendToLog (this=0x7f5b9ef08e00) at src/logging.cc:1269
      #5  0x000000000213d913 in google::LogMessage::Flush (this=this@entry=0x7f5b9ef08e00) at src/logging.cc:1138
      #6  0x000000000214059e in google::LogMessageFatal::~LogMessageFatal (this=0x7f5b9ef08e00, __in_chrg=<optimized out>) at src/logging.cc:1836
      #7  0x0000000001586657 in impala::Coordinator::ValidateCollectionSlots (this=0xc49ca00, batch=0xc3b3e00) at /home/dev/Impala/be/src/runtime/coordinator.cc:911
      #8  0x000000000158638d in impala::Coordinator::GetNext (this=0xc49ca00, batch=0x7dd3bd0, state=0xd934400) at /home/dev/Impala/be/src/runtime/coordinator.cc:890
      #9  0x00000000013710c3 in impala::ImpalaServer::QueryExecState::FetchNextBatch (this=0x7dd2000) at /home/dev/Impala/be/src/service/query-exec-state.cc:877
      #10 0x000000000136f169 in impala::ImpalaServer::QueryExecState::FetchRowsInternal (this=0x7dd2000, max_rows=1024, fetched_rows=0x7f5b9ef09370) at /home/dev/Impala/be/src/service/query-exec-state.cc:706
      #11 0x000000000136e80d in impala::ImpalaServer::QueryExecState::FetchRows (this=0x7dd2000, max_rows=1024, fetched_rows=0x7f5b9ef09370) at /home/dev/Impala/be/src/service/query-exec-state.cc:616
      #12 0x0000000001364418 in impala::ImpalaServer::FetchInternal (this=0x910ac00, query_id=..., start_over=false, fetch_size=1024, query_results=0x7f5b9ef09668) at /home/dev/Impala/be/src/service/impala-beeswax-server.cc:613
      #13 0x000000000135fd34 in impala::ImpalaServer::fetch (this=0x910ac00, query_results=..., query_handle=..., start_over=false, fetch_size=1024) at /home/dev/Impala/be/src/service/impala-beeswax-server.cc:288
      #14 0x00000000014fded4 in beeswax::BeeswaxServiceProcessor::process_fetch (this=0xc9fdf40, seqid=0, iprot=0x7d789c0, oprot=0x8fe8340, callContext=0x7efe9c0) at /home/dev/Impala/be/generated-sources/gen-cpp/BeeswaxService.cpp:3150
      #15 0x00000000014fc312 in beeswax::BeeswaxServiceProcessor::dispatchCall (this=0xc9fdf40, iprot=0x7d789c0, oprot=0x8fe8340, fname=..., seqid=0, callContext=0x7efe9c0) at /home/dev/Impala/be/generated-sources/gen-cpp/BeeswaxService.cpp:2952
      #16 0x00000000014e577b in impala::ImpalaServiceProcessor::dispatchCall (this=0xc9fdf40, iprot=0x7d789c0, oprot=0x8fe8340, fname=..., seqid=0, callContext=0x7efe9c0) at /home/dev/Impala/be/generated-sources/gen-cpp/ImpalaService.cpp:1673
      #17 0x00000000012fac15 in apache::thrift::TDispatchProcessor::process (this=0xc9fdf40, in=..., out=..., connectionContext=0x7efe9c0) at /home/dev/Impala/thirdparty/thrift-0.9.0/build/include/thrift/TDispatchProcessor.h:121
      #18 0x00000000020e39ea in apache::thrift::server::TThreadPoolServer::Task::run (this=0xcd9bbc0) at src/thrift/server/TThreadPoolServer.cpp:70
      #19 0x00000000020d012f in apache::thrift::concurrency::ThreadManager::Task::run (this=0x8fe8300) at src/thrift/concurrency/ThreadManager.cpp:187
      #20 0x00000000020d2d9a in apache::thrift::concurrency::ThreadManager::Worker::run (this=0xce50f90) at src/thrift/concurrency/ThreadManager.cpp:316
      #21 0x00000000012241a5 in impala::ThriftThread::RunRunnable (this=0x90d5a80, runnable=..., promise=0x7fffce7e4120) at /home/dev/Impala/be/src/rpc/thrift-thread.cc:61
      #22 0x000000000122596d in boost::_mfi::mf2<void, impala::ThriftThread, boost::shared_ptr<apache::thrift::concurrency::Runnable>, impala::Promise<unsigned long>*>::operator() (this=0x7f00780, p=0x90d5a80, a1=..., a2=0x7fffce7e4120) at /usr/include/boost/bind/mem_fn_template.hpp:280
      #23 0x00000000012257c8 in boost::_bi::list3<boost::_bi::value<impala::ThriftThread*>, boost::_bi::value<boost::shared_ptr<apache::thrift::concurrency::Runnable> >, boost::_bi::value<impala::Promise<unsigned long>*> >::operator()<boost::_mfi::mf2<void, impala::ThriftThread, boost::shared_ptr<apache::thrift::concurrency::Runnable>, impala::Promise<unsigned long>*>, boost::_bi::list0> (this=0x7f00790, f=..., a=...) at /usr/include/boost/bind/bind.hpp:392
      #24 0x0000000001225559 in boost::_bi::bind_t<void, boost::_mfi::mf2<void, impala::ThriftThread, boost::shared_ptr<apache::thrift::concurrency::Runnable>, impala::Promise<unsigned long>*>, boost::_bi::list3<boost::_bi::value<impala::ThriftThread*>, boost::_bi::value<boost::shared_ptr<apache::thrift::concurrency::Runnable> >, boost::_bi::value<impala::Promise<unsigned long>*> > >::operator() (this=0x7f00780) at /usr/include/boost/bind/bind_template.hpp:20
      #25 0x000000000122547a in boost::detail::function::void_function_obj_invoker0<boost::_bi::bind_t<void, boost::_mfi::mf2<void, impala::ThriftThread, boost::shared_ptr<apache::thrift::concurrency::Runnable>, impala::Promise<unsigned long>*>, boost::_bi::list3<boost::_bi::value<impala::ThriftThread*>, boost::_bi::value<boost::shared_ptr<apache::thrift::concurrency::Runnable> >, boost::_bi::value<impala::Promise<unsigned long>*> > >, void>::invoke (function_obj_ptr=...) at /usr/include/boost/function/function_template.hpp:153
      #26 0x000000000125b4a8 in boost::function0<void>::operator() (this=0x7f5b9ef09e00) at /usr/include/boost/function/function_template.hpp:767
      #27 0x000000000146e091 in impala::Thread::SuperviseThread(std::string const&, std::string const&, boost::function<void ()>, impala::Promise<long>*) (name=..., category=..., functor=..., thread_started=0x7fffce7e3f60) at /home/dev/Impala/be/src/util/thread.cc:314
      #28 0x00000000014772c7 in boost::_bi::list4<boost::_bi::value<std::string>, boost::_bi::value<std::string>, boost::_bi::value<boost::function<void ()> >, boost::_bi::value<impala::Promise<long>*> >::operator()<void (*)(std::string const&, std::string const&, boost::function<void ()>, impala::Promise<long>*), boost::_bi::list0>(boost::_bi::type<void>, void (*&)(std::string const&, std::string const&, boost::function<void ()>, impala::Promise<long>*), boost::_bi::list0&, int) (this=0x90d85c0, f=@0x90d85b8: 0x146dd88 <impala::Thread::SuperviseThread(std::string const&, std::string const&, boost::function<void ()>, impala::Promise<long>*)>, a=...) at /usr/include/boost/bind/bind.hpp:457
      #29 0x0000000001477211 in boost::_bi::bind_t<void, void (*)(std::string const&, std::string const&, boost::function<void ()>, impala::Promise<long>*), boost::_bi::list4<boost::_bi::value<std::string>, boost::_bi::value<std::string>, boost::_bi::value<boost::function<void ()> >, boost::_bi::value<impala::Promise<long>*> > >::operator()() (this=0x90d85b8) at /usr/include/boost/bind/bind_template.hpp:20
      #30 0x00000000014771d6 in boost::detail::thread_data<boost::_bi::bind_t<void, void (*)(std::string const&, std::string const&, boost::function<void ()>, impala::Promise<long>*), boost::_bi::list4<boost::_bi::value<std::string>, boost::_bi::value<std::string>, boost::_bi::value<boost::function<void ()> >, boost::_bi::value<impala::Promise<long>*> > > >::run() (this=0x90d8400) at /usr/include/boost/thread/detail/thread.hpp:117
      #31 0x00007f5c13cbfa4a in ?? () from /usr/lib/x86_64-linux-gnu/libboost_thread.so.1.54.0
      #32 0x00007f5c1316c182 in start_thread (arg=0x7f5b9ef0a700) at pthread_create.c:312
      #33 0x00007f5c10db947d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
      

      impalad.FATAL:

      Check failed: tuple->IsNull(slot_desc->null_indicator_offset())
      

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            Unassigned Unassigned
            tarasbob Taras Bobrovytsky
            Votes:
            1 Vote for this issue
            Watchers:
            8 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment