Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-5457

OOM during profile serialization leads to crash

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Critical
    • Resolution: Duplicate
    • Affects Version/s: Impala 2.8.0
    • Fix Version/s: Impala 2.10.0
    • Component/s: Backend
    • Labels:
    • Epic Color:
      ghx-label-3

      Description

      It looks like an oom exception during thrift serialization is not handled correctly and can lead to a crash. Here is the stack trace of the crashing thread from a minidump.

       0  libc-2.12.so!__GI_raise + 0x35
       1  libc-2.12.so!__GI_abort + 0x175
       2  libstdc++.so.6!__gnu_cxx::__verbose_terminate_handler() + 0x15d
       3  libstdc++.so.6!std::rethrow_exception(std::__exception_ptr::exception_ptr) + 0x76
       4  libstdc++.so.6!std::terminate() + 0x11
       5  libstdc++.so.6!__cxa_throw + 0x68
       6  impalad!(anonymous namespace)::handle_oom(void* (*)(void*), void*, bool, bool) [clone .constprop.53] + 0x3a8
       7  impalad!operator new(unsigned long) + 0x2b0
       8  impalad!std::vector<impala::TRuntimeProfileNode, std::allocator<impala::TRuntimeProfileNode> >::reserve(unsigned long) [new_allocator.h : 104 + 0x10]
       9  impalad!impala::RuntimeProfile::ToThrift(std::vector<impala::TRuntimeProfileNode, std::allocator<impala::TRuntimeProfileNode> >*) const [runtime-profile.cc : 755 + 0x8]
      10  impalad!impala::RuntimeProfile::SerializeToArchiveString(std::basic_stringstream<char, std::char_traits<char>, std::allocator<char> >*) const [runtime-profile.cc : 724 + 0x5]
      11  impalad!impala::ImpalaServer::GetRuntimeProfileStr(impala::TUniqueId const&, bool, std::basic_stringstream<char, std::char_traits<char>, std::allocator<char> >*) [impala-server.cc : 571 + 0x5]
      12  impalad!impala::ImpalaHttpHandler::QueryProfileEncodedHandler(std::map<std::string, std::string, std::less<std::string>, std::allocator<std::pair<std::string const, std::string> > > const&, rapidjson::GenericDocument<rapidjson::UTF8<char>, rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator> >*) [impala-http-handler.cc : 219 + 0x1c]
      13  impalad!impala::Webserver::RenderUrlWithTemplate(std::map<std::string, std::string, std::less<std::string>, std::allocator<std::pair<std::string const, std::string> > > const&, impala::Webserver::UrlHandler const&, std::basic_stringstream<char, std::char_traits<char>, std::allocator<char> >*, impala::ContentType*) [function_template.hpp : 767 + 0x11]
      14  impalad!impala::Webserver::BeginRequestCallback(sq_connection*, sq_request_info*) [webserver.cc : 418 + 0x1c]
      15  impalad!handle_request + 0x740
      16  impalad!process_new_connection + 0x16d
      17  impalad!worker_thread + 0x2dd
      18  libpthread-2.12.so!start_thread + 0xd1
      19  libc-2.12.so!clone + 0x6d
      

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                lv Lars Volker
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: