Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-13347

TSAN failure in backend tests after IMPALA-12737

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Critical
    • Resolution: Fixed
    • Impala 4.5.0
    • Impala 4.5.0
    • Backend
    • None

    Description

      Impala backend tests (expr-test, session-expiry-test, internal-server-test) started detecting a thread leak in TSAN builds after IMPALA-12737 "Refactor the Workload Management Initialization Process" was merged. Example:

       WARNING: ThreadSanitizer: thread leak (pid=475)
        Thread T566 (tid=1135, finished) created by main thread at:
          #0 pthread_create <null> (unifiedbetests+0x23d7083)
          #1 boost::thread::start_thread_noexcept() <null> (unifiedbetests+0x521607d)
          #2 boost::thread::thread<void (*)(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, boost::function<void ()>, impala::ThreadDebugInfo const*, impala::Promise<long, (impala::PromiseMode)0>*), std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, boost::function<void ()>, impala::ThreadDebugInfo*, impala::Promise<long, (impala::PromiseMode)0>*>(void (*)(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, boost::function<void ()>, impala::ThreadDebugInfo const*, impala::Promise<long, (impala::PromiseMode)0>*), std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, boost::function<void ()>, impala::ThreadDebugInfo*, impala::Promise<long, (impala::PromiseMode)0>*) /data/jenkins/workspace/impala-cdw-master-staging-core-tsan/Impala-Toolchain/toolchain-packages-gcc10.4.0/boost-1.74.0-p1/include/boost/thread/detail/thread.hpp:424:13 (unifiedbetests+0x4f8b214)
          #3 impala::Thread::StartThread(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, boost::function<void ()> const&, std::unique_ptr<impala::Thread, std::default_delete<impala::Thread> >*, bool) /data/jenkins/workspace/impala-cdw-master-staging-core-tsan/repos/Impala/be/src/util/thread.cc:317:13 (unifiedbetests+0x4f8761c)
          #4 impala::Status impala::Thread::Create<boost::_bi::bind_t<void, boost::_mfi::mf0<void, impala::ImpalaServer>, boost::_bi::list1<boost::_bi::value<impala::ImpalaServer*> > > >(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, boost::_bi::bind_t<void, boost::_mfi::mf0<void, impala::ImpalaServer>, boost::_bi::list1<boost::_bi::value<impala::ImpalaServer*> > > const&, std::unique_ptr<impala::Thread, std::default_delete<impala::Thread> >*, bool) /data/jenkins/workspace/impala-cdw-master-staging-core-tsan/repos/Impala/be/src/util/thread.h:74:12 (unifiedbetests+0x4c695f2)
          #5 impala::ImpalaServer::Start(int, int, int, int) /data/jenkins/workspace/impala-cdw-master-staging-core-tsan/repos/Impala/be/src/service/impala-server.cc:3220:5 (unifiedbetests+0x4c63e4a)
          #6 impala::InProcessImpalaServer::StartWithClientServers(int, int, int) /data/jenkins/workspace/impala-cdw-master-staging-core-tsan/repos/Impala/be/src/testutil/in-process-servers.cc:97:3 (unifiedbetests+0x501bd3b)
          #7 impala::InProcessImpalaServer::StartWithEphemeralPorts(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, int, impala::InProcessImpalaServer**) /data/jenkins/workspace/impala-cdw-master-staging-core-tsan/repos/Impala/be/src/testutil/in-process-servers.cc:69:21 (unifiedbetests+0x501bc22)
          #8 impala::ExprTest::SetUpTestCase() /data/jenkins/workspace/impala-cdw-master-staging-core-tsan/repos/Impala/be/src/exprs/expr-test.cc:235:5 (unifiedbetests+0x26d42f9)
          #9 void testing::internal::HandleExceptionsInMethodIfSupported<testing::TestSuite, void>(testing::TestSuite*, void (testing::TestSuite::*)(), char const*) <null> (unifiedbetests+0x6876a5c)
          #10 main /data/jenkins/workspace/impala-cdw-master-staging-core-tsan/repos/Impala/be/src/service/unified-betest-main.cc:48:10 (unifiedbetests+0x24363e0)
      
      SUMMARY: ThreadSanitizer: thread leak (/data0/jenkins/workspace/impala-cdw-master-staging-core-tsan/repos/Impala/be/build/debug/service/unifiedbetests+0x23d7083) in __interceptor_pthread_create

      Attachments

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            jasonmfehr Jason Fehr
            MikaelSmith Michael Smith
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment