Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-4915

Unbounded DECIMAL casts from floating point to decimal trigger undefined behavior

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: Impala 2.9.0
    • Fix Version/s: Impala 2.9.0
    • Component/s: Backend
    • Labels:
      None

      Description

      Here's an example of using Impala as an ASCII art generator:

      [localhost:21000] > select power(2, 1023) * 100;
      Query: select power(2, 1023) * 100
      Query submitted at: 2017-02-11 02:58:50 (Coordinator: http://impala-dev:25000)
      Query progress can be monitored at: http://impala-dev:25000/query_plan?query_id=2346bde04782aabd:3466c05900000000
      +----------------------+
      | power(2, 1023) * 100 |
      +----------------------+
      | Infinity             |
      +----------------------+
      Fetched 1 row(s) in 0.01s
      [localhost:21000] > select cast(power(2, 1023) * 100 AS decimal(10,10));
      Query: select cast(power(2, 1023) * 100 AS decimal(10,10))
      Query submitted at: 2017-02-11 02:59:20 (Coordinator: http://impala-dev:25000)
      Query progress can be monitored at: http://impala-dev:25000/query_plan?query_id=de46ef0a381bc968:66f76aa600000000
      +----------------------------------------------+
      | cast(power(2, 1023) * 100 as decimal(10,10)) |
      +----------------------------------------------+
      | --.*(+,))+(0(                                |
      +----------------------------------------------+
      Fetched 1 row(s) in 0.01s
      [localhost:21000] > set disable_codegen=1;
      DISABLE_CODEGEN set to 1
      [localhost:21000] > select cast(power(2, 1023) * 100 AS decimal(10,10));
      Query: select cast(power(2, 1023) * 100 AS decimal(10,10))
      Query submitted at: 2017-02-11 03:00:00 (Coordinator: http://impala-dev:25000)
      Query progress can be monitored at: http://impala-dev:25000/query_plan?query_id=b64823307d95c477:5d8b586b00000000
      +----------------------------------------------+
      | cast(power(2, 1023) * 100 as decimal(10,10)) |
      +----------------------------------------------+
      | --.*(+,))+(0(                                |
      +----------------------------------------------+
      Fetched 1 row(s) in 0.01s
      [localhost:21000] > set decimal_v2=1;
      DECIMAL_V2 set to 1
      [localhost:21000] > select cast(power(2, 1023) * 100 AS decimal(10,10));
      Query: select cast(power(2, 1023) * 100 AS decimal(10,10))
      Query submitted at: 2017-02-11 03:00:34 (Coordinator: http://impala-dev:25000)
      Query progress can be monitored at: http://impala-dev:25000/query_plan?query_id=b24376cb151d7203:35baf9b300000000
      +----------------------------------------------+
      | cast(power(2, 1023) * 100 as decimal(10,10)) |
      +----------------------------------------------+
      | --.*(+,))+(0(                                |
      +----------------------------------------------+
      Fetched 1 row(s) in 0.01s
      

        Activity

        Hide
        jbapple Jim Apple added a comment -

        I changed fix version because the next version of Apache Impala is likely to be called 2.9.0. We don't have an RM yet, and that RM could decide to call it "Apache Impala (incubating) 2000" or something, so this is just an educated guess.

        Show
        jbapple Jim Apple added a comment - I changed fix version because the next version of Apache Impala is likely to be called 2.9.0. We don't have an RM yet, and that RM could decide to call it "Apache Impala (incubating) 2000" or something, so this is just an educated guess.
        Show
        zamsden_impala_ad21 Zachary added a comment - https://gerrit.cloudera.org/#/c/5951/
        Hide
        zamsden_impala_ad21 Zachary added a comment -
        +---------------------------------------------+
        Fetched 1 row(s) in 0.01s
        [localhost:21000] > select cast(power(2, 1023) * 100 AS decimal(19,1));
        Query: select cast(power(2, 1023) * 100 AS decimal(19,1))
        Query submitted at: 2017-02-13 20:52:40 (Coordinator: http://impala-dev:25000)
        Query progress can be monitored at: http://impala-dev:25000/query_plan?query_id=b248a0934c206c5a:911f3cd100000000
        +---------------------------------------------+
        | cast(power(2, 1023) * 100 as decimal(19,1)) |
        +---------------------------------------------+
        | 0.0                                         |
        +---------------------------------------------+
        Fetched 1 row(s) in 0.01s
        [localhost:21000] > select cast(power(2, 1023) * 100 AS decimal(18,1));
        Query: select cast(power(2, 1023) * 100 AS decimal(18,1))
        Query submitted at: 2017-02-13 20:52:43 (Coordinator: http://impala-dev:25000)
        Query progress can be monitored at: http://impala-dev:25000/query_plan?query_id=e4bd53b5e5811e2:4b19dd0c00000000
        +---------------------------------------------+
        | cast(power(2, 1023) * 100 as decimal(18,1)) |
        +---------------------------------------------+
        | -..--).0-*(+,))+(0.(                        |
        +---------------------------------------------+
        Fetched 1 row(s) in 0.01s
        
        Show
        zamsden_impala_ad21 Zachary added a comment - +---------------------------------------------+ Fetched 1 row(s) in 0.01s [localhost:21000] > select cast(power(2, 1023) * 100 AS decimal(19,1)); Query: select cast(power(2, 1023) * 100 AS decimal(19,1)) Query submitted at: 2017-02-13 20:52:40 (Coordinator: http://impala-dev:25000) Query progress can be monitored at: http://impala-dev:25000/query_plan?query_id=b248a0934c206c5a:911f3cd100000000 +---------------------------------------------+ | cast(power(2, 1023) * 100 as decimal(19,1)) | +---------------------------------------------+ | 0.0 | +---------------------------------------------+ Fetched 1 row(s) in 0.01s [localhost:21000] > select cast(power(2, 1023) * 100 AS decimal(18,1)); Query: select cast(power(2, 1023) * 100 AS decimal(18,1)) Query submitted at: 2017-02-13 20:52:43 (Coordinator: http://impala-dev:25000) Query progress can be monitored at: http://impala-dev:25000/query_plan?query_id=e4bd53b5e5811e2:4b19dd0c00000000 +---------------------------------------------+ | cast(power(2, 1023) * 100 as decimal(18,1)) | +---------------------------------------------+ | -..--).0-*(+,))+(0.( | +---------------------------------------------+ Fetched 1 row(s) in 0.01s
        Hide
        jbapple Jim Apple added a comment -

        Using UBSAN (https://gerrit.cloudera.org/#/c/5082/):

        be/src/runtime/decimal-value.inline.h:371:23: runtime error: negation of -9223372036854775808 cannot be represented in type 'long'; cast to an unsigned type to negate this value to itself
            #0 0x7fde508a0c2d in impala::DecimalValue<long>::ToString(int, int) const /tmp/be/src/runtime/decimal-value.inline.h:371:23
            #1 0x7fde5089f35c in impala::DecimalValue<long>::ToString(impala::ColumnType const&) const /tmp/be/src/runtime/decimal-value.inline.h:352:10
            #2 0x7fde50899798 in impala::RawValue::PrintValue(void const*, impala::ColumnType const&, int, std::basic_stringstream<char, std::char_traits<char>, std::allocator<char> >*) /tmp/be/src/runtime/raw-value.cc:295:22
            #3 0x7fde4a20517b in impala::AsciiQueryResultSet::AddOneRow(std::vector<void*, std::allocator<void*> > const&, std::vector<int, std::allocator<int> > const&) /tmp/be/src/service/query-result-set.cc:171:5
            #4 0x7fde4e4a10f8 in impala::PlanRootSink::Send(impala::RuntimeState*, impala::RowBatch*) /tmp/be/src/exec/plan-root-sink.cc:114:33
            #5 0x7fde5082654b in impala::PlanFragmentExecutor::ExecInternal() /tmp/be/src/runtime/plan-fragment-executor.cc:367:31
            #6 0x7fde50823ccd in impala::PlanFragmentExecutor::Exec() /tmp/be/src/runtime/plan-fragment-executor.cc:338:14
            #7 0x7fde5075252a in impala::FragmentInstanceState::Exec() /tmp/be/src/runtime/fragment-instance-state.cc:66:7
            #8 0x7fde50842aac in impala::QueryExecMgr::ExecFInstance(impala::FragmentInstanceState*) /tmp/be/src/runtime/query-exec-mgr.cc:100:3
            #9 0x7fde50851aae in boost::_mfi::mf1<void, impala::QueryExecMgr, impala::FragmentInstanceState*>::operator()(impala::QueryExecMgr*, impala::FragmentInstanceState*) const /opt/Impala-Toolchain/boost-1.57.0/include/boost/bind/mem_fn_tem
        plate.hpp:165:16
            #10 0x7fde50851908 in void boost::_bi::list2<boost::_bi::value<impala::QueryExecMgr*>, boost::_bi::value<impala::FragmentInstanceState*> >::operator()<boost::_mfi::mf1<void, impala::QueryExecMgr, impala::FragmentInstanceState*>, boost:
        :_bi::list0>(boost::_bi::type<void>, boost::_mfi::mf1<void, impala::QueryExecMgr, impala::FragmentInstanceState*>&, boost::_bi::list0&, int) /opt/Impala-Toolchain/boost-1.57.0/include/boost/bind/bind.hpp:313:9
            #11 0x7fde508515bb in boost::_bi::bind_t<void, boost::_mfi::mf1<void, impala::QueryExecMgr, impala::FragmentInstanceState*>, boost::_bi::list2<boost::_bi::value<impala::QueryExecMgr*>, boost::_bi::value<impala::FragmentInstanceState*> 
        > >::operator()() /opt/Impala-Toolchain/boost-1.57.0/include/boost/bind/bind_template.hpp:20:16
            #12 0x7fde50850999 in boost::detail::function::void_function_obj_invoker0<boost::_bi::bind_t<void, boost::_mfi::mf1<void, impala::QueryExecMgr, impala::FragmentInstanceState*>, boost::_bi::list2<boost::_bi::value<impala::QueryExecMgr*>
        , boost::_bi::value<impala::FragmentInstanceState*> > >, void>::invoke(boost::detail::function::function_buffer&) /opt/Impala-Toolchain/boost-1.57.0/include/boost/function/function_template.hpp:153:11
            #13 0x7fde52099044 in boost::function0<void>::operator()() const /opt/Impala-Toolchain/boost-1.57.0/include/boost/function/function_template.hpp:766:14
            #14 0x7fde52093020 in impala::Thread::SuperviseThread(std::string const&, std::string const&, boost::function<void ()>, impala::Promise<long>*) /tmp/be/src/util/thread.cc:317:3
            #15 0x7fde520b3923 in void boost::_bi::list4<boost::_bi::value<std::string>, boost::_bi::value<std::string>, boost::_bi::value<boost::function<void ()> >, boost::_bi::value<impala::Promise<long>*> >::operator()<void (*)(std::string con
        st&, std::string const&, boost::function<void ()>, impala::Promise<long>*), boost::_bi::list0>(boost::_bi::type<void>, void (*&)(std::string const&, std::string const&, boost::function<void ()>, impala::Promise<long>*), boost::_bi::list0&,
         int) /opt/Impala-Toolchain/boost-1.57.0/include/boost/bind/bind.hpp:457:9
            #16 0x7fde520b324b in boost::_bi::bind_t<void, void (*)(std::string const&, std::string const&, boost::function<void ()>, impala::Promise<long>*), boost::_bi::list4<boost::_bi::value<std::string>, boost::_bi::value<std::string>, boost:
        :_bi::value<boost::function<void ()> >, boost::_bi::value<impala::Promise<long>*> > >::operator()() /opt/Impala-Toolchain/boost-1.57.0/include/boost/bind/bind_template.hpp:20:16
            #17 0x7fde520b1c95 in boost::detail::thread_data<boost::_bi::bind_t<void, void (*)(std::string const&, std::string const&, boost::function<void ()>, impala::Promise<long>*), boost::_bi::list4<boost::_bi::value<std::string>, boost::_bi:
        :value<std::string>, boost::_bi::value<boost::function<void ()> >, boost::_bi::value<impala::Promise<long>*> > > >::run() /opt/Impala-Toolchain/boost-1.57.0/include/boost/thread/detail/thread.hpp:116:17
            #18 0xa46dc9 in thread_proxy (/home/jbapple/Impala/be/build/debug/service/impalad+0xa46dc9)
            #19 0x7fde43173183 in start_thread /build/eglibc-oGUzwX/eglibc-2.19/nptl/pthread_create.c:312
            #20 0x7fde42ea037c in clone /build/eglibc-oGUzwX/eglibc-2.19/misc/../sysdeps/unix/sysv/linux/x86_64/clone.S:111
        
        Show
        jbapple Jim Apple added a comment - Using UBSAN ( https://gerrit.cloudera.org/#/c/5082/): be/src/runtime/decimal-value.inline.h:371:23: runtime error: negation of -9223372036854775808 cannot be represented in type 'long'; cast to an unsigned type to negate this value to itself #0 0x7fde508a0c2d in impala::DecimalValue<long>::ToString(int, int) const /tmp/be/src/runtime/decimal-value.inline.h:371:23 #1 0x7fde5089f35c in impala::DecimalValue<long>::ToString(impala::ColumnType const&) const /tmp/be/src/runtime/decimal-value.inline.h:352:10 #2 0x7fde50899798 in impala::RawValue::PrintValue(void const*, impala::ColumnType const&, int, std::basic_stringstream<char, std::char_traits<char>, std::allocator<char> >*) /tmp/be/src/runtime/raw-value.cc:295:22 #3 0x7fde4a20517b in impala::AsciiQueryResultSet::AddOneRow(std::vector<void*, std::allocator<void*> > const&, std::vector<int, std::allocator<int> > const&) /tmp/be/src/service/query-result-set.cc:171:5 #4 0x7fde4e4a10f8 in impala::PlanRootSink::Send(impala::RuntimeState*, impala::RowBatch*) /tmp/be/src/exec/plan-root-sink.cc:114:33 #5 0x7fde5082654b in impala::PlanFragmentExecutor::ExecInternal() /tmp/be/src/runtime/plan-fragment-executor.cc:367:31 #6 0x7fde50823ccd in impala::PlanFragmentExecutor::Exec() /tmp/be/src/runtime/plan-fragment-executor.cc:338:14 #7 0x7fde5075252a in impala::FragmentInstanceState::Exec() /tmp/be/src/runtime/fragment-instance-state.cc:66:7 #8 0x7fde50842aac in impala::QueryExecMgr::ExecFInstance(impala::FragmentInstanceState*) /tmp/be/src/runtime/query-exec-mgr.cc:100:3 #9 0x7fde50851aae in boost::_mfi::mf1<void, impala::QueryExecMgr, impala::FragmentInstanceState*>::operator()(impala::QueryExecMgr*, impala::FragmentInstanceState*) const /opt/Impala-Toolchain/boost-1.57.0/include/boost/bind/mem_fn_tem plate.hpp:165:16 #10 0x7fde50851908 in void boost::_bi::list2<boost::_bi::value<impala::QueryExecMgr*>, boost::_bi::value<impala::FragmentInstanceState*> >::operator()<boost::_mfi::mf1<void, impala::QueryExecMgr, impala::FragmentInstanceState*>, boost: :_bi::list0>(boost::_bi::type<void>, boost::_mfi::mf1<void, impala::QueryExecMgr, impala::FragmentInstanceState*>&, boost::_bi::list0&, int) /opt/Impala-Toolchain/boost-1.57.0/include/boost/bind/bind.hpp:313:9 #11 0x7fde508515bb in boost::_bi::bind_t<void, boost::_mfi::mf1<void, impala::QueryExecMgr, impala::FragmentInstanceState*>, boost::_bi::list2<boost::_bi::value<impala::QueryExecMgr*>, boost::_bi::value<impala::FragmentInstanceState*> > >::operator()() /opt/Impala-Toolchain/boost-1.57.0/include/boost/bind/bind_template.hpp:20:16 #12 0x7fde50850999 in boost::detail::function::void_function_obj_invoker0<boost::_bi::bind_t<void, boost::_mfi::mf1<void, impala::QueryExecMgr, impala::FragmentInstanceState*>, boost::_bi::list2<boost::_bi::value<impala::QueryExecMgr*> , boost::_bi::value<impala::FragmentInstanceState*> > >, void>::invoke(boost::detail::function::function_buffer&) /opt/Impala-Toolchain/boost-1.57.0/include/boost/function/function_template.hpp:153:11 #13 0x7fde52099044 in boost::function0<void>::operator()() const /opt/Impala-Toolchain/boost-1.57.0/include/boost/function/function_template.hpp:766:14 #14 0x7fde52093020 in impala::Thread::SuperviseThread(std::string const&, std::string const&, boost::function<void ()>, impala::Promise<long>*) /tmp/be/src/util/thread.cc:317:3 #15 0x7fde520b3923 in void boost::_bi::list4<boost::_bi::value<std::string>, boost::_bi::value<std::string>, boost::_bi::value<boost::function<void ()> >, boost::_bi::value<impala::Promise<long>*> >::operator()<void (*)(std::string con st&, std::string const&, boost::function<void ()>, impala::Promise<long>*), boost::_bi::list0>(boost::_bi::type<void>, void (*&)(std::string const&, std::string const&, boost::function<void ()>, impala::Promise<long>*), boost::_bi::list0&, int) /opt/Impala-Toolchain/boost-1.57.0/include/boost/bind/bind.hpp:457:9 #16 0x7fde520b324b in boost::_bi::bind_t<void, void (*)(std::string const&, std::string const&, boost::function<void ()>, impala::Promise<long>*), boost::_bi::list4<boost::_bi::value<std::string>, boost::_bi::value<std::string>, boost: :_bi::value<boost::function<void ()> >, boost::_bi::value<impala::Promise<long>*> > >::operator()() /opt/Impala-Toolchain/boost-1.57.0/include/boost/bind/bind_template.hpp:20:16 #17 0x7fde520b1c95 in boost::detail::thread_data<boost::_bi::bind_t<void, void (*)(std::string const&, std::string const&, boost::function<void ()>, impala::Promise<long>*), boost::_bi::list4<boost::_bi::value<std::string>, boost::_bi: :value<std::string>, boost::_bi::value<boost::function<void ()> >, boost::_bi::value<impala::Promise<long>*> > > >::run() /opt/Impala-Toolchain/boost-1.57.0/include/boost/thread/detail/thread.hpp:116:17 #18 0xa46dc9 in thread_proxy (/home/jbapple/Impala/be/build/debug/service/impalad+0xa46dc9) #19 0x7fde43173183 in start_thread /build/eglibc-oGUzwX/eglibc-2.19/nptl/pthread_create.c:312 #20 0x7fde42ea037c in clone /build/eglibc-oGUzwX/eglibc-2.19/misc/../sysdeps/unix/sysv/linux/x86_64/clone.S:111

          People

          • Assignee:
            zamsden Zach Amsden
            Reporter:
            zamsden_impala_ad21 Zachary
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development