Details
-
Sub-task
-
Status: Open
-
Major
-
Resolution: Unresolved
-
None
-
None
-
None
-
ghx-label-9
Description
TSAN reports the following data race:
WARNING: ThreadSanitizer: data race (pid=20418)
Read of size 8 at 0x7b100015f150 by thread T1101:
#0 impala::BloomFilter::BucketFindAVX2(unsigned int, unsigned int) const /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/util/bloom-filter.cc:185:53 (impalad+0x21b1009)
#1 <null> <null> (0x7fe7b58dd882)
#2 impala::HdfsParquetScanner::AssembleRows(std::vector<impala::ParquetColumnReader*, std::allocator<impala::ParquetColumnReader*> > const&, impala::RowBatch*, bool*) /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/exec/parquet/hdfs-parquet-scanner.cc:1140:29 (impalad+0x285aa2e)
#3 impala::HdfsParquetScanner::GetNextInternal(impala::RowBatch*) /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/exec/parquet/hdfs-parquet-scanner.cc:456:19 (impalad+0x2858410)
#4 impala::HdfsParquetScanner::ProcessSplit() /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/exec/parquet/hdfs-parquet-scanner.cc:353:21 (impalad+0x285740b)
#5 impala::HdfsScanNode::ProcessSplit(std::vector<impala::FilterContext, std::allocator<impala::FilterContext> > const&, impala::MemPool*, impala::io::ScanRange*, long*) /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/exec/hdfs-scan-node.cc:514:21 (impalad+0x249e0bb)
#6 impala::HdfsScanNode::ScannerThread(bool, long) /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/exec/hdfs-scan-node.cc:415:7 (impalad+0x249d6b7)
#7 impala::HdfsScanNode::ThreadTokenAvailableCb(impala::ThreadResourcePool*)::$_0::operator()() const /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/exec/hdfs-scan-node.cc:337:13 (impalad+0x249ec06)
#8 boost::detail::function::void_function_obj_invoker0<impala::HdfsScanNode::ThreadTokenAvailableCb(impala::ThreadResourcePool*)::$_0, void>::invoke(boost::detail::function::function_buffer&) /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/boost-1.57.0-p3/include/boost/function/function_template.hpp:153:11 (impalad+0x249ea19)
#9 boost::function0<void>::operator()() const /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/boost-1.57.0-p3/include/boost/function/function_template.hpp:766:14 (impalad+0x1cf2e21)
#10 impala::Thread::SuperviseThread(std::string const&, std::string const&, boost::function<void ()>, impala::ThreadDebugInfo const*, impala::Promise<long, (impala::PromiseMode)0>*) /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/util/thread.cc:360:3 (impalad+0x22a3c06)
#11 void boost::_bi::list5<boost::_bi::value<std::string>, boost::_bi::value<std::string>, boost::_bi::value<boost::function<void ()> >, boost::_bi::value<impala::ThreadDebugInfo*>, boost::_bi::value<impala::Promise<long, (impala::PromiseMode)0>*> >::operator()<void (*)(std::string const&, std::string const&, boost::function<void ()>, impala::ThreadDebugInfo const*, impala::Promise<long, (impala::PromiseMode)0>*), boost::_bi::list0>(boost::_bi::type<void>, void (*&)(std::string const&, std::string const&, boost::function<void ()>, impala::ThreadDebugInfo const*, impala::Promise<long, (impala::PromiseMode)0>*), boost::_bi::list0&, int) /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/boost-1.57.0-p3/include/boost/bind/bind.hpp:525:9 (impalad+0x22abcbc)
#12 boost::_bi::bind_t<void, void (*)(std::string const&, std::string const&, boost::function<void ()>, impala::ThreadDebugInfo const*, impala::Promise<long, (impala::PromiseMode)0>*), boost::_bi::list5<boost::_bi::value<std::string>, boost::_bi::value<std::string>, boost::_bi::value<boost::function<void ()> >, boost::_bi::value<impala::ThreadDebugInfo*>, boost::_bi::value<impala::Promise<long, (impala::PromiseMode)0>*> > >::operator()() /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/boost-1.57.0-p3/include/boost/bind/bind_template.hpp:20:27 (impalad+0x22abbd3)
#13 boost::detail::thread_data<boost::_bi::bind_t<void, void (*)(std::string const&, std::string const&, boost::function<void ()>, impala::ThreadDebugInfo const*, impala::Promise<long, (impala::PromiseMode)0>*), boost::_bi::list5<boost::_bi::value<std::string>, boost::_bi::value<std::string>, boost::_bi::value<boost::function<void ()> >, boost::_bi::value<impala::ThreadDebugInfo*>, boost::_bi::value<impala::Promise<long, (impala::PromiseMode)0>*> > > >::run() /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/boost-1.57.0-p3/include/boost/thread/detail/thread.hpp:116:17 (impalad+0x22ab8c0)
#14 thread_proxy <null> (impalad+0x2f7ccc9)
Previous write of size 8 at 0x7b100015f150 by thread T126 (mutexes: write M940824749541653120):
#0 impala::BloomFilter::Init(int) /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/util/bloom-filter.cc:56:14 (impalad+0x21b0647)
#1 impala::BloomFilter::Init(impala::BloomFilterPB const&, unsigned char const*, unsigned long) /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/util/bloom-filter.cc:63:3 (impalad+0x21b080f)
#2 impala::RuntimeFilterBank::PublishGlobalFilter(impala::PublishFilterParamsPB const&, kudu::rpc::RpcContext*) /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/runtime/runtime-filter-bank.cc:339:39 (impalad+0x1f3f88f)
#3 impala::QueryState::PublishFilter(impala::PublishFilterParamsPB const&, kudu::rpc::RpcContext*) /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/runtime/query-state.cc:722:17 (impalad+0x1f2cd2b)
#4 impala::DataStreamService::PublishFilter(impala::PublishFilterParamsPB const*, impala::PublishFilterResultPB*, kudu::rpc::RpcContext*) /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/service/data-stream-service.cc:132:9 (impalad+0x205a237)
#5 impala::DataStreamServiceIf::DataStreamServiceIf(scoped_refptr<kudu::MetricEntity> const&, scoped_refptr<kudu::rpc::ResultTracker> const&)::$_7::operator()(google::protobuf::Message const*, google::protobuf::Message*, kudu::rpc::RpcContext*) const /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/generated-sources/gen-cpp/data_stream_service.service.cc:130:13 (impalad+0x2102d44)
#6 std::_Function_handler<void (google::protobuf::Message const*, google::protobuf::Message*, kudu::rpc::RpcContext*), impala::DataStreamServiceIf::DataStreamServiceIf(scoped_refptr<kudu::MetricEntity> const&, scoped_refptr<kudu::rpc::ResultTracker> const&)::$_7>::_M_invoke(std::_Any_data const&, google::protobuf::Message const*, google::protobuf::Message*, kudu::rpc::RpcContext*) /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/gcc-4.9.2/lib/gcc/x86_64-unknown-linux-gnu/4.9.2/../../../../include/c++/4.9.2/functional:2039:2 (impalad+0x2102b2e)
#7 std::function<void (google::protobuf::Message const*, google::protobuf::Message*, kudu::rpc::RpcContext*)>::operator()(google::protobuf::Message const*, google::protobuf::Message*, kudu::rpc::RpcContext*) const /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/gcc-4.9.2/lib/gcc/x86_64-unknown-linux-gnu/4.9.2/../../../../include/c++/4.9.2/functional:2439:14 (impalad+0x278aa38)
#8 kudu::rpc::GeneratedServiceIf::Handle(kudu::rpc::InboundCall*) /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/kudu/rpc/service_if.cc:139:3 (impalad+0x278a39e)
#9 impala::ImpalaServicePool::RunThread() /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/rpc/impala-service-pool.cc:271:15 (impalad+0x1e199df)
#10 boost::_mfi::mf0<void, impala::ImpalaServicePool>::operator()(impala::ImpalaServicePool*) const /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/boost-1.57.0-p3/include/boost/bind/mem_fn_template.hpp:49:29 (impalad+0x1e1fe36)
#11 void boost::_bi::list1<boost::_bi::value<impala::ImpalaServicePool*> >::operator()<boost::_mfi::mf0<void, impala::ImpalaServicePool>, boost::_bi::list0>(boost::_bi::type<void>, boost::_mfi::mf0<void, impala::ImpalaServicePool>&, boost::_bi::list0&, int) /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/boost-1.57.0-p3/include/boost/bind/bind.hpp:253:9 (impalad+0x1e1fd8a)
#12 boost::_bi::bind_t<void, boost::_mfi::mf0<void, impala::ImpalaServicePool>, boost::_bi::list1<boost::_bi::value<impala::ImpalaServicePool*> > >::operator()() /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/boost-1.57.0-p3/include/boost/bind/bind_template.hpp:20:27 (impalad+0x1e1fd13)
#13 boost::detail::function::void_function_obj_invoker0<boost::_bi::bind_t<void, boost::_mfi::mf0<void, impala::ImpalaServicePool>, boost::_bi::list1<boost::_bi::value<impala::ImpalaServicePool*> > >, void>::invoke(boost::detail::function::function_buffer&) /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/boost-1.57.0-p3/include/boost/function/function_template.hpp:153:11 (impalad+0x1e1fb09)
#14 boost::function0<void>::operator()() const /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/boost-1.57.0-p3/include/boost/function/function_template.hpp:766:14 (impalad+0x1cf2e21)
#15 impala::Thread::SuperviseThread(std::string const&, std::string const&, boost::function<void ()>, impala::ThreadDebugInfo const*, impala::Promise<long, (impala::PromiseMode)0>*) /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/util/thread.cc:360:3 (impalad+0x22a3c06)
#16 void boost::_bi::list5<boost::_bi::value<std::string>, boost::_bi::value<std::string>, boost::_bi::value<boost::function<void ()> >, boost::_bi::value<impala::ThreadDebugInfo*>, boost::_bi::value<impala::Promise<long, (impala::PromiseMode)0>*> >::operator()<void (*)(std::string const&, std::string const&, boost::function<void ()>, impala::ThreadDebugInfo const*, impala::Promise<long, (impala::PromiseMode)0>*), boost::_bi::list0>(boost::_bi::type<void>, void (*&)(std::string const&, std::string const&, boost::function<void ()>, impala::ThreadDebugInfo const*, impala::Promise<long, (impala::PromiseMode)0>*), boost::_bi::list0&, int) /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/boost-1.57.0-p3/include/boost/bind/bind.hpp:525:9 (impalad+0x22abcbc)
#17 boost::_bi::bind_t<void, void (*)(std::string const&, std::string const&, boost::function<void ()>, impala::ThreadDebugInfo const*, impala::Promise<long, (impala::PromiseMode)0>*), boost::_bi::list5<boost::_bi::value<std::string>, boost::_bi::value<std::string>, boost::_bi::value<boost::function<void ()> >, boost::_bi::value<impala::ThreadDebugInfo*>, boost::_bi::value<impala::Promise<long, (impala::PromiseMode)0>*> > >::operator()() /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/boost-1.57.0-p3/include/boost/bind/bind_template.hpp:20:27 (impalad+0x22abbd3)
#18 boost::detail::thread_data<boost::_bi::bind_t<void, void (*)(std::string const&, std::string const&, boost::function<void ()>, impala::ThreadDebugInfo const*, impala::Promise<long, (impala::PromiseMode)0>*), boost::_bi::list5<boost::_bi::value<std::string>, boost::_bi::value<std::string>, boost::_bi::value<boost::function<void ()> >, boost::_bi::value<impala::ThreadDebugInfo*>, boost::_bi::value<impala::Promise<long, (impala::PromiseMode)0>*> > > >::run() /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/boost-1.57.0-p3/include/boost/thread/detail/thread.hpp:116:17 (impalad+0x22ab8c0)
#19 thread_proxy <null> (impalad+0x2f7ccc9)
Location is heap block of size 64 at 0x7b100015f140 allocated by thread T126:
#0 operator new(unsigned long) /mnt/source/llvm/llvm-5.0.1.src-p2/projects/compiler-rt/lib/tsan/rtl/tsan_new_delete.cc:44 (impalad+0x191d233)
#1 impala::RuntimeFilterBank::PublishGlobalFilter(impala::PublishFilterParamsPB const&, kudu::rpc::RpcContext*) /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/runtime/runtime-filter-bank.cc:323:36 (impalad+0x1f3f6ec)
#2 impala::QueryState::PublishFilter(impala::PublishFilterParamsPB const&, kudu::rpc::RpcContext*) /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/runtime/query-state.cc:722:17 (impalad+0x1f2cd2b)
#3 impala::DataStreamService::PublishFilter(impala::PublishFilterParamsPB const*, impala::PublishFilterResultPB*, kudu::rpc::RpcContext*) /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/service/data-stream-service.cc:132:9 (impalad+0x205a237)
#4 impala::DataStreamServiceIf::DataStreamServiceIf(scoped_refptr<kudu::MetricEntity> const&, scoped_refptr<kudu::rpc::ResultTracker> const&)::$_7::operator()(google::protobuf::Message const*, google::protobuf::Message*, kudu::rpc::RpcContext*) const /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/generated-sources/gen-cpp/data_stream_service.service.cc:130:13 (impalad+0x2102d44)
#5 std::_Function_handler<void (google::protobuf::Message const*, google::protobuf::Message*, kudu::rpc::RpcContext*), impala::DataStreamServiceIf::DataStreamServiceIf(scoped_refptr<kudu::MetricEntity> const&, scoped_refptr<kudu::rpc::ResultTracker> const&)::$_7>::_M_invoke(std::_Any_data const&, google::protobuf::Message const*, google::protobuf::Message*, kudu::rpc::RpcContext*) /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/gcc-4.9.2/lib/gcc/x86_64-unknown-linux-gnu/4.9.2/../../../../include/c++/4.9.2/functional:2039:2 (impalad+0x2102b2e)
#6 std::function<void (google::protobuf::Message const*, google::protobuf::Message*, kudu::rpc::RpcContext*)>::operator()(google::protobuf::Message const*, google::protobuf::Message*, kudu::rpc::RpcContext*) const /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/gcc-4.9.2/lib/gcc/x86_64-unknown-linux-gnu/4.9.2/../../../../include/c++/4.9.2/functional:2439:14 (impalad+0x278aa38)
#7 kudu::rpc::GeneratedServiceIf::Handle(kudu::rpc::InboundCall*) /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/kudu/rpc/service_if.cc:139:3 (impalad+0x278a39e)
#8 impala::ImpalaServicePool::RunThread() /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/rpc/impala-service-pool.cc:271:15 (impalad+0x1e199df)
#9 boost::_mfi::mf0<void, impala::ImpalaServicePool>::operator()(impala::ImpalaServicePool*) const /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/boost-1.57.0-p3/include/boost/bind/mem_fn_template.hpp:49:29 (impalad+0x1e1fe36)
#10 void boost::_bi::list1<boost::_bi::value<impala::ImpalaServicePool*> >::operator()<boost::_mfi::mf0<void, impala::ImpalaServicePool>, boost::_bi::list0>(boost::_bi::type<void>, boost::_mfi::mf0<void, impala::ImpalaServicePool>&, boost::_bi::list0&, int) /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/boost-1.57.0-p3/include/boost/bind/bind.hpp:253:9 (impalad+0x1e1fd8a)
#11 boost::_bi::bind_t<void, boost::_mfi::mf0<void, impala::ImpalaServicePool>, boost::_bi::list1<boost::_bi::value<impala::ImpalaServicePool*> > >::operator()() /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/boost-1.57.0-p3/include/boost/bind/bind_template.hpp:20:27 (impalad+0x1e1fd13)
#12 boost::detail::function::void_function_obj_invoker0<boost::_bi::bind_t<void, boost::_mfi::mf0<void, impala::ImpalaServicePool>, boost::_bi::list1<boost::_bi::value<impala::ImpalaServicePool*> > >, void>::invoke(boost::detail::function::function_buffer&) /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/boost-1.57.0-p3/include/boost/function/function_template.hpp:153:11 (impalad+0x1e1fb09)
#13 boost::function0<void>::operator()() const /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/boost-1.57.0-p3/include/boost/function/function_template.hpp:766:14 (impalad+0x1cf2e21)
#14 impala::Thread::SuperviseThread(std::string const&, std::string const&, boost::function<void ()>, impala::ThreadDebugInfo const*, impala::Promise<long, (impala::PromiseMode)0>*) /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/util/thread.cc:360:3 (impalad+0x22a3c06)
#15 void boost::_bi::list5<boost::_bi::value<std::string>, boost::_bi::value<std::string>, boost::_bi::value<boost::function<void ()> >, boost::_bi::value<impala::ThreadDebugInfo*>, boost::_bi::value<impala::Promise<long, (impala::PromiseMode)0>*> >::operator()<void (*)(std::string const&, std::string const&, boost::function<void ()>, impala::ThreadDebugInfo const*, impala::Promise<long, (impala::PromiseMode)0>*), boost::_bi::list0>(boost::_bi::type<void>, void (*&)(std::string const&, std::string const&, boost::function<void ()>, impala::ThreadDebugInfo const*, impala::Promise<long, (impala::PromiseMode)0>*), boost::_bi::list0&, int) /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/boost-1.57.0-p3/include/boost/bind/bind.hpp:525:9 (impalad+0x22abcbc)
#16 boost::_bi::bind_t<void, void (*)(std::string const&, std::string const&, boost::function<void ()>, impala::ThreadDebugInfo const*, impala::Promise<long, (impala::PromiseMode)0>*), boost::_bi::list5<boost::_bi::value<std::string>, boost::_bi::value<std::string>, boost::_bi::value<boost::function<void ()> >, boost::_bi::value<impala::ThreadDebugInfo*>, boost::_bi::value<impala::Promise<long, (impala::PromiseMode)0>*> > >::operator()() /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/boost-1.57.0-p3/include/boost/bind/bind_template.hpp:20:27 (impalad+0x22abbd3)
#17 boost::detail::thread_data<boost::_bi::bind_t<void, void (*)(std::string const&, std::string const&, boost::function<void ()>, impala::ThreadDebugInfo const*, impala::Promise<long, (impala::PromiseMode)0>*), boost::_bi::list5<boost::_bi::value<std::string>, boost::_bi::value<std::string>, boost::_bi::value<boost::function<void ()> >, boost::_bi::value<impala::ThreadDebugInfo*>, boost::_bi::value<impala::Promise<long, (impala::PromiseMode)0>*> > > >::run() /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/boost-1.57.0-p3/include/boost/thread/detail/thread.hpp:116:17 (impalad+0x22ab8c0)
#18 thread_proxy <null> (impalad+0x2f7ccc9)
Mutex M940824749541653120 is already destroyed.
Thread T1101 (tid=31826, running) created by thread T443 at:
#0 pthread_create /mnt/source/llvm/llvm-5.0.1.src-p2/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors.cc:889 (impalad+0x18b4c1b)
#1 boost::thread::start_thread_noexcept() <null> (impalad+0x2f7c0a9)
#2 boost::thread::thread<void (*)(std::string const&, std::string const&, boost::function<void ()>, impala::ThreadDebugInfo const*, impala::Promise<long, (impala::PromiseMode)0>*), std::string, std::string, boost::function<void ()>, impala::ThreadDebugInfo*, impala::Promise<long, (impala::PromiseMode)0>*>(void (*)(std::string const&, std::string const&, boost::function<void ()>, impala::ThreadDebugInfo const*, impala::Promise<long, (impala::PromiseMode)0>*), std::string, std::string, boost::function<void ()>, impala::ThreadDebugInfo*, impala::Promise<long, (impala::PromiseMode)0>*) /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/boost-1.57.0-p3/include/boost/thread/detail/thread.hpp:419:13 (impalad+0x22a62bb)
#3 impala::Thread::StartThread(std::string const&, std::string const&, boost::function<void ()> const&, std::unique_ptr<impala::Thread, std::default_delete<impala::Thread> >*, bool) /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/util/thread.cc:317:13 (impalad+0x22a35a5)
#4 impala::Status impala::Thread::Create<impala::HdfsScanNode::ThreadTokenAvailableCb(impala::ThreadResourcePool*)::$_0>(std::string const&, std::string const&, impala::HdfsScanNode::ThreadTokenAvailableCb(impala::ThreadResourcePool*)::$_0 const&, std::unique_ptr<impala::Thread, std::default_delete<impala::Thread> >*, bool) /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/util/thread.h:74:12 (impalad+0x249d21d)
#5 impala::HdfsScanNode::ThreadTokenAvailableCb(impala::ThreadResourcePool*) /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/exec/hdfs-scan-node.cc:340:14 (impalad+0x249c86f)
#6 impala::HdfsScanNode::AddDiskIoRanges(std::vector<impala::io::ScanRange*, std::allocator<impala::io::ScanRange*> > const&, impala::EnqueueLocation) /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/exec/hdfs-scan-node.cc:212:24 (impalad+0x249ce8e)
#7 impala::HdfsScanner::IssueFooterRanges(impala::HdfsScanNodeBase*, impala::THdfsFileFormat::type const&, std::vector<impala::HdfsFileDesc*, std::allocator<impala::HdfsFileDesc*> > const&) /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/exec/hdfs-scanner.cc:836:5 (impalad+0x2389be8)
#8 impala::HdfsParquetScanner::IssueInitialRanges(impala::HdfsScanNodeBase*, std::vector<impala::HdfsFileDesc*, std::allocator<impala::HdfsFileDesc*> > const&) /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/exec/parquet/hdfs-parquet-scanner.cc:84:10 (impalad+0x2852ddb)
#9 impala::HdfsScanNodeBase::IssueInitialScanRanges(impala::RuntimeState*) /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/exec/hdfs-scan-node-base.cc:600:9 (impalad+0x235ce29)
#10 impala::HdfsScanNode::GetNext(impala::RuntimeState*, impala::RowBatch*, bool*) /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/exec/hdfs-scan-node.cc:90:21 (impalad+0x249ba2c)
#11 impala::FragmentInstanceState::ExecInternal() /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/runtime/fragment-instance-state.cc:382:7 (impalad+0x1f1c514)
#12 impala::FragmentInstanceState::Exec() /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/runtime/fragment-instance-state.cc:95:14 (impalad+0x1f1a3a6)
#13 impala::QueryState::ExecFInstance(impala::FragmentInstanceState*) /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/runtime/query-state.cc:694:24 (impalad+0x1f2ca53)
#14 impala::QueryState::StartFInstances()::$_7::operator()() const /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/runtime/query-state.cc:603:35 (impalad+0x1f2eba2)
#15 boost::detail::function::void_function_obj_invoker0<impala::QueryState::StartFInstances()::$_7, void>::invoke(boost::detail::function::function_buffer&) /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/boost-1.57.0-p3/include/boost/function/function_template.hpp:153:11 (impalad+0x1f2e9c9)
#16 boost::function0<void>::operator()() const /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/boost-1.57.0-p3/include/boost/function/function_template.hpp:766:14 (impalad+0x1cf2e21)
#17 impala::Thread::SuperviseThread(std::string const&, std::string const&, boost::function<void ()>, impala::ThreadDebugInfo const*, impala::Promise<long, (impala::PromiseMode)0>*) /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/util/thread.cc:360:3 (impalad+0x22a3c06)
#18 void boost::_bi::list5<boost::_bi::value<std::string>, boost::_bi::value<std::string>, boost::_bi::value<boost::function<void ()> >, boost::_bi::value<impala::ThreadDebugInfo*>, boost::_bi::value<impala::Promise<long, (impala::PromiseMode)0>*> >::operator()<void (*)(std::string const&, std::string const&, boost::function<void ()>, impala::ThreadDebugInfo const*, impala::Promise<long, (impala::PromiseMode)0>*), boost::_bi::list0>(boost::_bi::type<void>, void (*&)(std::string const&, std::string const&, boost::function<void ()>, impala::ThreadDebugInfo const*, impala::Promise<long, (impala::PromiseMode)0>*), boost::_bi::list0&, int) /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/boost-1.57.0-p3/include/boost/bind/bind.hpp:525:9 (impalad+0x22abcbc)
#19 boost::_bi::bind_t<void, void (*)(std::string const&, std::string const&, boost::function<void ()>, impala::ThreadDebugInfo const*, impala::Promise<long, (impala::PromiseMode)0>*), boost::_bi::list5<boost::_bi::value<std::string>, boost::_bi::value<std::string>, boost::_bi::value<boost::function<void ()> >, boost::_bi::value<impala::ThreadDebugInfo*>, boost::_bi::value<impala::Promise<long, (impala::PromiseMode)0>*> > >::operator()() /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/boost-1.57.0-p3/include/boost/bind/bind_template.hpp:20:27 (impalad+0x22abbd3)
#20 boost::detail::thread_data<boost::_bi::bind_t<void, void (*)(std::string const&, std::string const&, boost::function<void ()>, impala::ThreadDebugInfo const*, impala::Promise<long, (impala::PromiseMode)0>*), boost::_bi::list5<boost::_bi::value<std::string>, boost::_bi::value<std::string>, boost::_bi::value<boost::function<void ()> >, boost::_bi::value<impala::ThreadDebugInfo*>, boost::_bi::value<impala::Promise<long, (impala::PromiseMode)0>*> > > >::run() /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/boost-1.57.0-p3/include/boost/thread/detail/thread.hpp:116:17 (impalad+0x22ab8c0)
#21 thread_proxy <null> (impalad+0x2f7ccc9)
Thread T126 (tid=20900, running) created by main thread at:
#0 pthread_create /mnt/source/llvm/llvm-5.0.1.src-p2/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors.cc:889 (impalad+0x18b4c1b)
#1 boost::thread::start_thread_noexcept() <null> (impalad+0x2f7c0a9)
#2 boost::thread::thread<void (*)(std::string const&, std::string const&, boost::function<void ()>, impala::ThreadDebugInfo const*, impala::Promise<long, (impala::PromiseMode)0>*), std::string, std::string, boost::function<void ()>, impala::ThreadDebugInfo*, impala::Promise<long, (impala::PromiseMode)0>*>(void (*)(std::string const&, std::string const&, boost::function<void ()>, impala::ThreadDebugInfo const*, impala::Promise<long, (impala::PromiseMode)0>*), std::string, std::string, boost::function<void ()>, impala::ThreadDebugInfo*, impala::Promise<long, (impala::PromiseMode)0>*) /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/boost-1.57.0-p3/include/boost/thread/detail/thread.hpp:419:13 (impalad+0x22a62bb)
#3 impala::Thread::StartThread(std::string const&, std::string const&, boost::function<void ()> const&, std::unique_ptr<impala::Thread, std::default_delete<impala::Thread> >*, bool) /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/util/thread.cc:317:13 (impalad+0x22a35a5)
#4 impala::Status impala::Thread::Create<void (impala::ImpalaServicePool::*)(), impala::ImpalaServicePool*>(std::string const&, std::string const&, void (impala::ImpalaServicePool::* const&)(), impala::ImpalaServicePool* const&, std::unique_ptr<impala::Thread, std::default_delete<impala::Thread> >*, bool) /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/util/thread.h:81:12 (impalad+0x1e1ba47)
#5 impala::ImpalaServicePool::Init(int) /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/rpc/impala-service-pool.cc:91:5 (impalad+0x1e19725)
#6 impala::RpcMgr::RegisterService(int, int, kudu::rpc::GeneratedServiceIf*, impala::MemTracker*) /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/rpc/rpc-mgr.cc:163:3 (impalad+0x1e08ba4)
#7 impala::DataStreamService::Init() /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/service/data-stream-service.cc:78:3 (impalad+0x2059b2f)
#8 impala::ExecEnv::Init() /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/runtime/exec-env.cc:391:3 (impalad+0x1e50481)
#9 ImpaladMain(int, char**) /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/service/impalad-main.cc:72:3 (impalad+0x2075af7)
#10 main /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/service/daemon-main.cc:37:12 (impalad+0x191f96a)SUMMARY: ThreadSanitizer: data race /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/util/bloom-filter.cc:185:53 in impala::BloomFilter::BucketFindAVX2(unsigned int, unsigned int) const
Notice that part of the stack right before BloomFilter::BucketFindAVX2 is missing. I think because the Bloom Filter functions calls are codegened. AFAICT, the stack should be something like: HdfsParquetScanner::AssembleRows -> HdfsParquetScanner::TransferScratchTuples -> codegend_process_scratch_batch_fn_ -> FilterContext::Eval -> RuntimeFilter::Eval -> RuntimeFilter::Eval -> BloomFilter::Find
I haven't been able to successfully get codegen to run with -fsanitize=thread yet, which is why part of the stack is missing.
Attachments
Issue Links
- is related to
-
-
- Open
-