Uploaded image for project: 'Parquet'
  1. Parquet
  2. PARQUET-1640

[C++] parquet-encoding-benchmark crashes

Details

    Description

      [...]
      BM_DictDecodingByteArray/DecodeArrowNonNull_Dict/65536     42199236 ns   42186639 ns          1   8.91692MB/s
      pure virtual method called
      terminate called without an active exception
      

      Attachments

        Issue Links

          Activity

            apitrou Antoine Pitrou created issue -
            apitrou Antoine Pitrou added a comment -

            gdb backtrace:

            (gdb) bt
            #0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
            #1  0x00007ffff4349801 in __GI_abort () at abort.c:79
            #2  0x00007ffff4d6484a in __gnu_cxx::__verbose_terminate_handler ()
                at /home/nwani/m3/conda-bld/compilers_linux-64_1560109574129/work/.build/x86_64-conda_cos6-linux-gnu/src/gcc/libstdc++-v3/libsupc++/vterminate.cc:95
            #3  0x00007ffff4d62f47 in __cxxabiv1::__terminate (handler=<optimized out>)
                at /home/nwani/m3/conda-bld/compilers_linux-64_1560109574129/work/.build/x86_64-conda_cos6-linux-gnu/src/gcc/libstdc++-v3/libsupc++/eh_terminate.cc:47
            #4  0x00007ffff4d62f7d in std::terminate ()
                at /home/nwani/m3/conda-bld/compilers_linux-64_1560109574129/work/.build/x86_64-conda_cos6-linux-gnu/src/gcc/libstdc++-v3/libsupc++/eh_terminate.cc:57
            #5  0x00007ffff4d63988 in __cxxabiv1::__cxa_pure_virtual ()
                at /home/nwani/m3/conda-bld/compilers_linux-64_1560109574129/work/.build/x86_64-conda_cos6-linux-gnu/src/gcc/libstdc++-v3/libsupc++/pure.cc:50
            #6  0x00007ffff610539a in arrow::PoolBuffer::~PoolBuffer (this=0x5555558eb470, __in_chrg=<optimized out>) at ../src/arrow/buffer.cc:114
            #7  0x00007ffff61087e1 in __gnu_cxx::new_allocator<arrow::PoolBuffer>::destroy<arrow::PoolBuffer> (this=0x5555558eb470, __p=0x5555558eb470)
                at /usr/include/c++/7/ext/new_allocator.h:140
            #8  0x00007ffff610872d in std::allocator_traits<std::allocator<arrow::PoolBuffer> >::destroy<arrow::PoolBuffer> (__a=..., __p=0x5555558eb470)
                at /usr/include/c++/7/bits/alloc_traits.h:487
            #9  0x00007ffff610832f in std::_Sp_counted_ptr_inplace<arrow::PoolBuffer, std::allocator<arrow::PoolBuffer>, (__gnu_cxx::_Lock_policy)2>::_M_dispose (
                this=0x5555558eb460) at /usr/include/c++/7/bits/shared_ptr_base.h:535
            #10 0x000055555561883a in std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release (this=0x5555558eb460) at /usr/include/c++/7/bits/shared_ptr_base.h:154
            #11 0x0000555555615a4b in std::__shared_count<(__gnu_cxx::_Lock_policy)2>::~__shared_count (this=0x5555558cd678, __in_chrg=<optimized out>)
                at /usr/include/c++/7/bits/shared_ptr_base.h:684
            #12 0x00005555556130fc in std::__shared_ptr<arrow::Buffer, (__gnu_cxx::_Lock_policy)2>::~__shared_ptr (this=0x5555558cd670, __in_chrg=<optimized out>)
                at /usr/include/c++/7/bits/shared_ptr_base.h:1123
            #13 0x0000555555613118 in std::shared_ptr<arrow::Buffer>::~shared_ptr (this=0x5555558cd670, __in_chrg=<optimized out>) at /usr/include/c++/7/bits/shared_ptr.h:93
            #14 0x0000555555614bc0 in parquet::BenchmarkDecodeArrow::~BenchmarkDecodeArrow (this=0x5555558cd570, __in_chrg=<optimized out>)
                at ../src/parquet/encoding_benchmark.cc:268
            #15 0x0000555555614c6a in parquet::BM_PlainDecodingByteArray::~BM_PlainDecodingByteArray (this=0x5555558cd570, __in_chrg=<optimized out>)
                at ../src/parquet/encoding_benchmark.cc:357
            #16 0x00005555556207b4 in parquet::BM_PlainDecodingByteArray_DecodeArrow_Dense_Benchmark::~BM_PlainDecodingByteArray_DecodeArrow_Dense_Benchmark (this=0x5555558cd570, 
                __in_chrg=<optimized out>) at ../src/parquet/encoding_benchmark.cc:372
            #17 0x00005555556207d0 in parquet::BM_PlainDecodingByteArray_DecodeArrow_Dense_Benchmark::~BM_PlainDecodingByteArray_DecodeArrow_Dense_Benchmark (this=0x5555558cd570, 
                __in_chrg=<optimized out>) at ../src/parquet/encoding_benchmark.cc:372
            #18 0x0000555555634626 in std::default_delete<benchmark::internal::Benchmark>::operator() (this=0x5555558d1b10, __ptr=0x5555558cd570)
                at /usr/include/c++/7/bits/unique_ptr.h:78
            #19 0x000055555563229f in std::unique_ptr<benchmark::internal::Benchmark, std::default_delete<benchmark::internal::Benchmark> >::~unique_ptr (this=0x5555558d1b10, 
                __in_chrg=<optimized out>) at /usr/include/c++/7/bits/unique_ptr.h:268
            #20 0x000055555563aff3 in std::_Destroy<std::unique_ptr<benchmark::internal::Benchmark, std::default_delete<benchmark::internal::Benchmark> > > (
                __pointer=0x5555558d1b10) at /usr/include/c++/7/bits/stl_construct.h:98
            #21 0x0000555555638f71 in std::_Destroy_aux<false>::__destroy<std::unique_ptr<benchmark::internal::Benchmark, std::default_delete<benchmark::internal::Benchmark> >*> (
                __first=0x5555558d1b10, __last=0x5555558d1b50) at /usr/include/c++/7/bits/stl_construct.h:108
            #22 0x000055555563642c in std::_Destroy<std::unique_ptr<benchmark::internal::Benchmark, std::default_delete<benchmark::internal::Benchmark> >*> (
                __first=0x5555558d1ab0, __last=0x5555558d1b50) at /usr/include/c++/7/bits/stl_construct.h:137
            #23 0x0000555555633d31 in std::_Destroy<std::unique_ptr<benchmark::internal::Benchmark, std::default_delete<benchmark::internal::Benchmark> >*, std::unique_ptr<benchmark::internal::Benchmark, std::default_delete<benchmark::internal::Benchmark> > > (__first=0x5555558d1ab0, __last=0x5555558d1b50)
                at /usr/include/c++/7/bits/stl_construct.h:206
            #24 0x0000555555631607 in std::vector<std::unique_ptr<benchmark::internal::Benchmark, std::default_delete<benchmark::internal::Benchmark> >, std::allocator<std::unique_ptr<benchmark::internal::Benchmark, std::default_delete<benchmark::internal::Benchmark> > > >::~vector (
                this=0x5555558b5660 <benchmark::internal::BenchmarkFamilies::GetInstance()::instance>, __in_chrg=<optimized out>) at /usr/include/c++/7/bits/stl_vector.h:434
            #25 0x00005555556312a4 in benchmark::internal::BenchmarkFamilies::~BenchmarkFamilies (
                this=0x5555558b5660 <benchmark::internal::BenchmarkFamilies::GetInstance()::instance>, __in_chrg=<optimized out>)
                at /home/antoine/arrow/dev/cpp/build-test/gbenchmark_ep-prefix/src/gbenchmark_ep/src/benchmark_register.cc:68
            #26 0x00007ffff434c041 in __run_exit_handlers (status=0, listp=0x7ffff46f4718 <__exit_funcs>, run_list_atexit=run_list_atexit@entry=true, 
                run_dtors=run_dtors@entry=true) at exit.c:108
            #27 0x00007ffff434c13a in __GI_exit (status=<optimized out>) at exit.c:139
            #28 0x00007ffff432ab9e in __libc_start_main (main=0x555555622139 <main(int, char**)>, argc=2, argv=0x7fffffffd3b8, init=<optimized out>, fini=<optimized out>, 
                rtld_fini=<optimized out>, stack_end=0x7fffffffd3a8) at ../csu/libc-start.c:344
            #29 0x000055555560e33a in _start ()
            
            apitrou Antoine Pitrou added a comment - gdb backtrace: (gdb) bt #0 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51 #1 0x00007ffff4349801 in __GI_abort () at abort.c:79 #2 0x00007ffff4d6484a in __gnu_cxx::__verbose_terminate_handler () at /home/nwani/m3/conda-bld/compilers_linux-64_1560109574129/work/.build/x86_64-conda_cos6-linux-gnu/src/gcc/libstdc++-v3/libsupc++/vterminate.cc:95 #3 0x00007ffff4d62f47 in __cxxabiv1::__terminate (handler=<optimized out>) at /home/nwani/m3/conda-bld/compilers_linux-64_1560109574129/work/.build/x86_64-conda_cos6-linux-gnu/src/gcc/libstdc++-v3/libsupc++/eh_terminate.cc:47 #4 0x00007ffff4d62f7d in std::terminate () at /home/nwani/m3/conda-bld/compilers_linux-64_1560109574129/work/.build/x86_64-conda_cos6-linux-gnu/src/gcc/libstdc++-v3/libsupc++/eh_terminate.cc:57 #5 0x00007ffff4d63988 in __cxxabiv1::__cxa_pure_virtual () at /home/nwani/m3/conda-bld/compilers_linux-64_1560109574129/work/.build/x86_64-conda_cos6-linux-gnu/src/gcc/libstdc++-v3/libsupc++/pure.cc:50 #6 0x00007ffff610539a in arrow::PoolBuffer::~PoolBuffer ( this =0x5555558eb470, __in_chrg=<optimized out>) at ../src/arrow/buffer.cc:114 #7 0x00007ffff61087e1 in __gnu_cxx::new_allocator<arrow::PoolBuffer>::destroy<arrow::PoolBuffer> ( this =0x5555558eb470, __p=0x5555558eb470) at /usr/include/c++/7/ext/new_allocator.h:140 #8 0x00007ffff610872d in std::allocator_traits<std::allocator<arrow::PoolBuffer> >::destroy<arrow::PoolBuffer> (__a=..., __p=0x5555558eb470) at /usr/include/c++/7/bits/alloc_traits.h:487 #9 0x00007ffff610832f in std::_Sp_counted_ptr_inplace<arrow::PoolBuffer, std::allocator<arrow::PoolBuffer>, (__gnu_cxx::_Lock_policy)2>::_M_dispose ( this =0x5555558eb460) at /usr/include/c++/7/bits/shared_ptr_base.h:535 #10 0x000055555561883a in std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release ( this =0x5555558eb460) at /usr/include/c++/7/bits/shared_ptr_base.h:154 #11 0x0000555555615a4b in std::__shared_count<(__gnu_cxx::_Lock_policy)2>::~__shared_count ( this =0x5555558cd678, __in_chrg=<optimized out>) at /usr/include/c++/7/bits/shared_ptr_base.h:684 #12 0x00005555556130fc in std::__shared_ptr<arrow::Buffer, (__gnu_cxx::_Lock_policy)2>::~__shared_ptr ( this =0x5555558cd670, __in_chrg=<optimized out>) at /usr/include/c++/7/bits/shared_ptr_base.h:1123 #13 0x0000555555613118 in std::shared_ptr<arrow::Buffer>::~shared_ptr ( this =0x5555558cd670, __in_chrg=<optimized out>) at /usr/include/c++/7/bits/shared_ptr.h:93 #14 0x0000555555614bc0 in parquet::BenchmarkDecodeArrow::~BenchmarkDecodeArrow ( this =0x5555558cd570, __in_chrg=<optimized out>) at ../src/parquet/encoding_benchmark.cc:268 #15 0x0000555555614c6a in parquet::BM_PlainDecodingByteArray::~BM_PlainDecodingByteArray ( this =0x5555558cd570, __in_chrg=<optimized out>) at ../src/parquet/encoding_benchmark.cc:357 #16 0x00005555556207b4 in parquet::BM_PlainDecodingByteArray_DecodeArrow_Dense_Benchmark::~BM_PlainDecodingByteArray_DecodeArrow_Dense_Benchmark ( this =0x5555558cd570, __in_chrg=<optimized out>) at ../src/parquet/encoding_benchmark.cc:372 #17 0x00005555556207d0 in parquet::BM_PlainDecodingByteArray_DecodeArrow_Dense_Benchmark::~BM_PlainDecodingByteArray_DecodeArrow_Dense_Benchmark ( this =0x5555558cd570, __in_chrg=<optimized out>) at ../src/parquet/encoding_benchmark.cc:372 #18 0x0000555555634626 in std::default_delete<benchmark::internal::Benchmark>:: operator () ( this =0x5555558d1b10, __ptr=0x5555558cd570) at /usr/include/c++/7/bits/unique_ptr.h:78 #19 0x000055555563229f in std::unique_ptr<benchmark::internal::Benchmark, std::default_delete<benchmark::internal::Benchmark> >::~unique_ptr ( this =0x5555558d1b10, __in_chrg=<optimized out>) at /usr/include/c++/7/bits/unique_ptr.h:268 #20 0x000055555563aff3 in std::_Destroy<std::unique_ptr<benchmark::internal::Benchmark, std::default_delete<benchmark::internal::Benchmark> > > ( __pointer=0x5555558d1b10) at /usr/include/c++/7/bits/stl_construct.h:98 #21 0x0000555555638f71 in std::_Destroy_aux< false >::__destroy<std::unique_ptr<benchmark::internal::Benchmark, std::default_delete<benchmark::internal::Benchmark> >*> ( __first=0x5555558d1b10, __last=0x5555558d1b50) at /usr/include/c++/7/bits/stl_construct.h:108 #22 0x000055555563642c in std::_Destroy<std::unique_ptr<benchmark::internal::Benchmark, std::default_delete<benchmark::internal::Benchmark> >*> ( __first=0x5555558d1ab0, __last=0x5555558d1b50) at /usr/include/c++/7/bits/stl_construct.h:137 #23 0x0000555555633d31 in std::_Destroy<std::unique_ptr<benchmark::internal::Benchmark, std::default_delete<benchmark::internal::Benchmark> >*, std::unique_ptr<benchmark::internal::Benchmark, std::default_delete<benchmark::internal::Benchmark> > > (__first=0x5555558d1ab0, __last=0x5555558d1b50) at /usr/include/c++/7/bits/stl_construct.h:206 #24 0x0000555555631607 in std::vector<std::unique_ptr<benchmark::internal::Benchmark, std::default_delete<benchmark::internal::Benchmark> >, std::allocator<std::unique_ptr<benchmark::internal::Benchmark, std::default_delete<benchmark::internal::Benchmark> > > >::~vector ( this =0x5555558b5660 <benchmark::internal::BenchmarkFamilies::GetInstance()::instance>, __in_chrg=<optimized out>) at /usr/include/c++/7/bits/stl_vector.h:434 #25 0x00005555556312a4 in benchmark::internal::BenchmarkFamilies::~BenchmarkFamilies ( this =0x5555558b5660 <benchmark::internal::BenchmarkFamilies::GetInstance()::instance>, __in_chrg=<optimized out>) at /home/antoine/arrow/dev/cpp/build-test/gbenchmark_ep-prefix/src/gbenchmark_ep/src/benchmark_register.cc:68 #26 0x00007ffff434c041 in __run_exit_handlers (status=0, listp=0x7ffff46f4718 <__exit_funcs>, run_list_atexit=run_list_atexit@entry= true , run_dtors=run_dtors@entry= true ) at exit.c:108 #27 0x00007ffff434c13a in __GI_exit (status=<optimized out>) at exit.c:139 #28 0x00007ffff432ab9e in __libc_start_main (main=0x555555622139 <main( int , char **)>, argc=2, argv=0x7fffffffd3b8, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7fffffffd3a8) at ../csu/libc-start.c:344 #29 0x000055555560e33a in _start ()
            apitrou Antoine Pitrou made changes -
            Field Original Value New Value
            Summary [C++] parquet-encoding-benchmark crashes in debug mode [C++] parquet-encoding-benchmark crashes
            apitrou Antoine Pitrou added a comment -

            Also crashes in release mode ("free(): invalid pointer"). Backtrace:

            #0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
            #1  0x00007ffff55a8801 in __GI_abort () at abort.c:79
            #2  0x00007ffff55f1897 in __libc_message (action=action@entry=do_abort, fmt=fmt@entry=0x7ffff571eb9a "%s\n") at ../sysdeps/posix/libc_fatal.c:181
            #3  0x00007ffff55f890a in malloc_printerr (str=str@entry=0x7ffff571cd88 "free(): invalid pointer") at malloc.c:5350
            #4  0x00007ffff55ffe1c in _int_free (have_lock=0, p=0x7ffff6dc2440 <guard variable for arrow::default_memory_pool()::default_memory_pool_>, 
                av=0x7ffff5953c40 <main_arena>) at malloc.c:4157
            #5  __GI___libc_free (mem=0x7ffff6dc2450 <arrow::default_memory_pool()::default_memory_pool_>) at malloc.c:3124
            #6  0x00007ffff712fb3a in __gnu_cxx::new_allocator<int>::deallocate (this=0x5555557dfc38, __p=<optimized out>) at /usr/include/c++/7/ext/new_allocator.h:125
            #7  std::allocator_traits<std::allocator<int> >::deallocate (__a=..., __n=<optimized out>, __p=<optimized out>) at /usr/include/c++/7/bits/alloc_traits.h:462
            #8  std::_Vector_base<int, std::allocator<int> >::_M_deallocate (this=0x5555557dfc38, __n=<optimized out>, __p=<optimized out>)
                at /usr/include/c++/7/bits/stl_vector.h:180
            #9  std::_Vector_base<int, std::allocator<int> >::~_Vector_base (this=0x5555557dfc38, __in_chrg=<optimized out>) at /usr/include/c++/7/bits/stl_vector.h:162
            #10 std::vector<int, std::allocator<int> >::~vector (this=0x5555557dfc38, __in_chrg=<optimized out>) at /usr/include/c++/7/bits/stl_vector.h:435
            #11 arrow::internal::BinaryMemoTable::~BinaryMemoTable (this=0x5555557dfc00, __in_chrg=<optimized out>) at ../src/arrow/util/hashing.h:556
            #12 arrow::internal::BinaryMemoTable::~BinaryMemoTable (this=0x5555557dfc00, __in_chrg=<optimized out>) at ../src/arrow/util/hashing.h:556
            #13 0x00007ffff6534790 in std::default_delete<arrow::internal::MemoTable>::operator() (this=<optimized out>, __ptr=<optimized out>)
                at /usr/include/c++/7/bits/unique_ptr.h:78
            #14 std::unique_ptr<arrow::internal::MemoTable, std::default_delete<arrow::internal::MemoTable> >::~unique_ptr (this=0x5555557e1a10, __in_chrg=<optimized out>)
                at /usr/include/c++/7/bits/unique_ptr.h:268
            #15 arrow::internal::DictionaryMemoTable::DictionaryMemoTableImpl::~DictionaryMemoTableImpl (this=0x5555557e1a00, __in_chrg=<optimized out>)
                at ../src/arrow/array/builder_dict.cc:165
            #16 std::default_delete<arrow::internal::DictionaryMemoTable::DictionaryMemoTableImpl>::operator() (this=<optimized out>, __ptr=0x5555557e1a00)
                at /usr/include/c++/7/bits/unique_ptr.h:78
            #17 std::unique_ptr<arrow::internal::DictionaryMemoTable::DictionaryMemoTableImpl, std::default_delete<arrow::internal::DictionaryMemoTable::DictionaryMemoTableImpl> >::~unique_ptr (this=<optimized out>, __in_chrg=<optimized out>) at /usr/include/c++/7/bits/unique_ptr.h:268
            #18 arrow::internal::DictionaryMemoTable::~DictionaryMemoTable (this=<optimized out>, __in_chrg=<optimized out>) at ../src/arrow/array/builder_dict.h:56
            #19 0x00005555555773ec in std::default_delete<arrow::internal::DictionaryMemoTable>::operator() (this=<optimized out>, __ptr=0x5555557e1930)
                at /usr/include/c++/7/bits/unique_ptr.h:78
            #20 std::unique_ptr<arrow::internal::DictionaryMemoTable, std::default_delete<arrow::internal::DictionaryMemoTable> >::~unique_ptr (this=0x5555557dfb20, 
                __in_chrg=<optimized out>) at /usr/include/c++/7/bits/unique_ptr.h:268
            #21 arrow::internal::DictionaryBuilderBase<arrow::NumericBuilder<arrow::Int32Type>, arrow::BinaryType>::~DictionaryBuilderBase (this=0x5555557dfa90, 
                __in_chrg=<optimized out>) at ../src/arrow/array/builder_dict.h:140
            #22 arrow::Dictionary32Builder<arrow::BinaryType>::~Dictionary32Builder (this=0x5555557dfa90, __in_chrg=<optimized out>) at ../src/arrow/array/builder_dict.h:383
            #23 arrow::internal::BinaryDictionary32BuilderImpl<arrow::BinaryType>::~BinaryDictionary32BuilderImpl (this=0x5555557dfa90, __in_chrg=<optimized out>)
                at ../src/arrow/array/builder_dict.h:426
            #24 arrow::BinaryDictionary32Builder::~BinaryDictionary32Builder (this=0x5555557dfa90, __in_chrg=<optimized out>) at ../src/arrow/array/builder_dict.h:457
            #25 arrow::BinaryDictionary32Builder::~BinaryDictionary32Builder (this=0x5555557dfa90, __in_chrg=<optimized out>) at ../src/arrow/array/builder_dict.h:457
            #26 0x000055555557a668 in std::default_delete<arrow::BinaryDictionary32Builder>::operator() (this=<optimized out>, __ptr=0x5555557dfa90)
                at /usr/include/c++/7/bits/unique_ptr.h:78
            #27 std::unique_ptr<arrow::BinaryDictionary32Builder, std::default_delete<arrow::BinaryDictionary32Builder> >::~unique_ptr (this=<synthetic pointer>, 
                __in_chrg=<optimized out>) at /usr/include/c++/7/bits/unique_ptr.h:268
            #28 parquet::BenchmarkDecodeArrow::DecodeArrowBenchmark<arrow::BinaryDictionary32Builder> (this=0x5555557da1a0, state=...) at ../src/parquet/encoding-benchmark.cc:312
            #29 0x000055555557f5a4 in benchmark::RunSpecifiedBenchmarks(benchmark::BenchmarkReporter*, benchmark::BenchmarkReporter*) ()
            #30 0x0000555555570540 in main ()
            
            apitrou Antoine Pitrou added a comment - Also crashes in release mode ("free(): invalid pointer"). Backtrace: #0 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51 #1 0x00007ffff55a8801 in __GI_abort () at abort.c:79 #2 0x00007ffff55f1897 in __libc_message (action=action@entry=do_abort, fmt=fmt@entry=0x7ffff571eb9a "%s\n" ) at ../sysdeps/posix/libc_fatal.c:181 #3 0x00007ffff55f890a in malloc_printerr (str=str@entry=0x7ffff571cd88 "free(): invalid pointer" ) at malloc.c:5350 #4 0x00007ffff55ffe1c in _int_free (have_lock=0, p=0x7ffff6dc2440 <guard variable for arrow::default_memory_pool()::default_memory_pool_>, av=0x7ffff5953c40 <main_arena>) at malloc.c:4157 #5 __GI___libc_free (mem=0x7ffff6dc2450 <arrow::default_memory_pool()::default_memory_pool_>) at malloc.c:3124 #6 0x00007ffff712fb3a in __gnu_cxx::new_allocator< int >::deallocate ( this =0x5555557dfc38, __p=<optimized out>) at /usr/include/c++/7/ext/new_allocator.h:125 #7 std::allocator_traits<std::allocator< int > >::deallocate (__a=..., __n=<optimized out>, __p=<optimized out>) at /usr/include/c++/7/bits/alloc_traits.h:462 #8 std::_Vector_base< int , std::allocator< int > >::_M_deallocate ( this =0x5555557dfc38, __n=<optimized out>, __p=<optimized out>) at /usr/include/c++/7/bits/stl_vector.h:180 #9 std::_Vector_base< int , std::allocator< int > >::~_Vector_base ( this =0x5555557dfc38, __in_chrg=<optimized out>) at /usr/include/c++/7/bits/stl_vector.h:162 #10 std::vector< int , std::allocator< int > >::~vector ( this =0x5555557dfc38, __in_chrg=<optimized out>) at /usr/include/c++/7/bits/stl_vector.h:435 #11 arrow::internal::BinaryMemoTable::~BinaryMemoTable ( this =0x5555557dfc00, __in_chrg=<optimized out>) at ../src/arrow/util/hashing.h:556 #12 arrow::internal::BinaryMemoTable::~BinaryMemoTable ( this =0x5555557dfc00, __in_chrg=<optimized out>) at ../src/arrow/util/hashing.h:556 #13 0x00007ffff6534790 in std::default_delete<arrow::internal::MemoTable>:: operator () ( this =<optimized out>, __ptr=<optimized out>) at /usr/include/c++/7/bits/unique_ptr.h:78 #14 std::unique_ptr<arrow::internal::MemoTable, std::default_delete<arrow::internal::MemoTable> >::~unique_ptr ( this =0x5555557e1a10, __in_chrg=<optimized out>) at /usr/include/c++/7/bits/unique_ptr.h:268 #15 arrow::internal::DictionaryMemoTable::DictionaryMemoTableImpl::~DictionaryMemoTableImpl ( this =0x5555557e1a00, __in_chrg=<optimized out>) at ../src/arrow/array/builder_dict.cc:165 #16 std::default_delete<arrow::internal::DictionaryMemoTable::DictionaryMemoTableImpl>:: operator () ( this =<optimized out>, __ptr=0x5555557e1a00) at /usr/include/c++/7/bits/unique_ptr.h:78 #17 std::unique_ptr<arrow::internal::DictionaryMemoTable::DictionaryMemoTableImpl, std::default_delete<arrow::internal::DictionaryMemoTable::DictionaryMemoTableImpl> >::~unique_ptr ( this =<optimized out>, __in_chrg=<optimized out>) at /usr/include/c++/7/bits/unique_ptr.h:268 #18 arrow::internal::DictionaryMemoTable::~DictionaryMemoTable ( this =<optimized out>, __in_chrg=<optimized out>) at ../src/arrow/array/builder_dict.h:56 #19 0x00005555555773ec in std::default_delete<arrow::internal::DictionaryMemoTable>:: operator () ( this =<optimized out>, __ptr=0x5555557e1930) at /usr/include/c++/7/bits/unique_ptr.h:78 #20 std::unique_ptr<arrow::internal::DictionaryMemoTable, std::default_delete<arrow::internal::DictionaryMemoTable> >::~unique_ptr ( this =0x5555557dfb20, __in_chrg=<optimized out>) at /usr/include/c++/7/bits/unique_ptr.h:268 #21 arrow::internal::DictionaryBuilderBase<arrow::NumericBuilder<arrow::Int32Type>, arrow::BinaryType>::~DictionaryBuilderBase ( this =0x5555557dfa90, __in_chrg=<optimized out>) at ../src/arrow/array/builder_dict.h:140 #22 arrow::Dictionary32Builder<arrow::BinaryType>::~Dictionary32Builder ( this =0x5555557dfa90, __in_chrg=<optimized out>) at ../src/arrow/array/builder_dict.h:383 #23 arrow::internal::BinaryDictionary32BuilderImpl<arrow::BinaryType>::~BinaryDictionary32BuilderImpl ( this =0x5555557dfa90, __in_chrg=<optimized out>) at ../src/arrow/array/builder_dict.h:426 #24 arrow::BinaryDictionary32Builder::~BinaryDictionary32Builder ( this =0x5555557dfa90, __in_chrg=<optimized out>) at ../src/arrow/array/builder_dict.h:457 #25 arrow::BinaryDictionary32Builder::~BinaryDictionary32Builder ( this =0x5555557dfa90, __in_chrg=<optimized out>) at ../src/arrow/array/builder_dict.h:457 #26 0x000055555557a668 in std::default_delete<arrow::BinaryDictionary32Builder>:: operator () ( this =<optimized out>, __ptr=0x5555557dfa90) at /usr/include/c++/7/bits/unique_ptr.h:78 #27 std::unique_ptr<arrow::BinaryDictionary32Builder, std::default_delete<arrow::BinaryDictionary32Builder> >::~unique_ptr ( this =<synthetic pointer>, __in_chrg=<optimized out>) at /usr/include/c++/7/bits/unique_ptr.h:268 #28 parquet::BenchmarkDecodeArrow::DecodeArrowBenchmark<arrow::BinaryDictionary32Builder> ( this =0x5555557da1a0, state=...) at ../src/parquet/encoding-benchmark.cc:312 #29 0x000055555557f5a4 in benchmark::RunSpecifiedBenchmarks(benchmark::BenchmarkReporter*, benchmark::BenchmarkReporter*) () #30 0x0000555555570540 in main ()
            apitrou Antoine Pitrou added a comment -

            Given the symptoms, I'd say this looks like memory corruption.

            apitrou Antoine Pitrou added a comment - Given the symptoms, I'd say this looks like memory corruption.
            wesm Wes McKinney added a comment -

            Thanks. I will take a look since I have worked on this code path most recently

            wesm Wes McKinney added a comment - Thanks. I will take a look since I have worked on this code path most recently
            wesm Wes McKinney made changes -
            Fix Version/s cpp-1.6.0 [ 12343927 ]
            wesm Wes McKinney made changes -
            Assignee Wes McKinney [ wesmckinn ]
            wesm Wes McKinney added a comment -

            I can't reproduce this. Do you have an old libparquet.so lying around?

            wesm Wes McKinney added a comment - I can't reproduce this. Do you have an old libparquet.so lying around?
            apitrou Antoine Pitrou added a comment -

            No. I made sure I cleaned up the CMake build directory. Note this is on Ubuntu 18.04, depending on the allocator memory corruption may pass silently.

            apitrou Antoine Pitrou added a comment - No. I made sure I cleaned up the CMake build directory. Note this is on Ubuntu 18.04, depending on the allocator memory corruption may pass silently.
            wesm Wes McKinney added a comment -

            OK, the

            pure virtual method called
            terminate called without an active exception
            

            would indicate an ABI mismatch at the vtable level. I'll try jemalloc on/off to see if I can reproduce

            wesm Wes McKinney added a comment - OK, the pure virtual method called terminate called without an active exception would indicate an ABI mismatch at the vtable level. I'll try jemalloc on/off to see if I can reproduce
            apitrou Antoine Pitrou added a comment -

            I've tried to selectively disable benchmarks and any of the Arrow-related benchmarks (e.g. BM_PlainDecodingByteArray/DecodeArrowNonNull_Dense/1024) is enough to reproduce here.

            apitrou Antoine Pitrou added a comment - I've tried to selectively disable benchmarks and any of the Arrow-related benchmarks (e.g. BM_PlainDecodingByteArray/DecodeArrowNonNull_Dense/1024 ) is enough to reproduce here.
            wesm Wes McKinney added a comment - - edited

            I've tried a few things and no dice. If you can determine the origin of the memory corruption I can try to help fix (I'm also on Ubuntu 18.04)

            wesm Wes McKinney added a comment - - edited I've tried a few things and no dice. If you can determine the origin of the memory corruption I can try to help fix (I'm also on Ubuntu 18.04)
            wesm Wes McKinney added a comment -

            OK, I have a repro. It only occurs with gcc 7.x but not with clang-7.

            wesm Wes McKinney added a comment - OK, I have a repro. It only occurs with gcc 7.x but not with clang-7.
            apitrou Antoine Pitrou added a comment -

            Hmm, I'm trying to rebase, but it seems this has been occurring for a long time...

            apitrou Antoine Pitrou added a comment - Hmm, I'm trying to rebase, but it seems this has been occurring for a long time...
            apitrou Antoine Pitrou made changes -
            Assignee Wes McKinney [ wesmckinn ] Antoine Pitrou [ pitrou ]
            githubbot ASF GitHub Bot made changes -
            Remote Link This issue links to "GitHub Pull Request #5120 (Web Link)" [ 166666 ]
            githubbot ASF GitHub Bot made changes -
            Labels pull-request-available
            githubbot ASF GitHub Bot logged work - 19/Aug/19 19:01
            • Time Spent:
              10m
               
              pitrou commented on pull request #5120: PARQUET-1640: [C++] Fix crash in parquet-encoding-benchmark
              URL: https://github.com/apache/arrow/pull/5120
               
               
                 The crash is due to undefined order of global variable destruction.
                 But destroying a PoolBuffer relies on its MemoryPool being still alive.
                 Release resources in TearDown() instead.
               
              ----------------------------------------------------------------
              This is an automated message from the Apache Git Service.
              To respond to the message, please log on to GitHub and use the
              URL above to go to the specific comment.
               
              For queries about this service, please contact Infrastructure at:
              users@infra.apache.org
            githubbot ASF GitHub Bot made changes -
            Worklog Id 297355 [ 297355 ]
            Remaining Estimate 0h [ 0 ]
            Time Spent 10m [ 600 ]
            githubbot ASF GitHub Bot made changes -
            Remote Link This issue links to "GitHub Pull Request #5120 (Web Link)" [ 166666 ] This issue links to "GitHub Pull Request #5120 (Web Link)" [ 166666 ]
            githubbot ASF GitHub Bot logged work - 19/Aug/19 21:06
            githubbot ASF GitHub Bot made changes -
            Worklog Id 297420 [ 297420 ]
            Time Spent 10m [ 600 ] 20m [ 1200 ]
            githubbot ASF GitHub Bot made changes -
            Remote Link This issue links to "GitHub Pull Request #5120 (Web Link)" [ 166666 ] This issue links to "GitHub Pull Request #5120 (Web Link)" [ 166666 ]
            githubbot ASF GitHub Bot logged work - 19/Aug/19 21:16
            • Time Spent:
              10m
               
              wesm commented on pull request #5120: PARQUET-1640: [C++] Fix crash in parquet-encoding-benchmark
              URL: https://github.com/apache/arrow/pull/5120
               
               
                 
               
              ----------------------------------------------------------------
              This is an automated message from the Apache Git Service.
              To respond to the message, please log on to GitHub and use the
              URL above to go to the specific comment.
               
              For queries about this service, please contact Infrastructure at:
              users@infra.apache.org
            githubbot ASF GitHub Bot made changes -
            Worklog Id 297423 [ 297423 ]
            Time Spent 20m [ 1200 ] 0.5h [ 1800 ]
            wesm Wes McKinney added a comment -

            Issue resolved by pull request 5120
            https://github.com/apache/arrow/pull/5120

            wesm Wes McKinney added a comment - Issue resolved by pull request 5120 https://github.com/apache/arrow/pull/5120
            wesm Wes McKinney made changes -
            Resolution Fixed [ 1 ]
            Status Open [ 1 ] Resolved [ 5 ]
            rok Rok Mihevc added a comment -

            This issue has been migrated to issue #42902 on GitHub. Please see the migration documentation for further details.

            rok Rok Mihevc added a comment - This issue has been migrated to issue #42902 on GitHub. Please see the migration documentation for further details.

            People

              apitrou Antoine Pitrou
              apitrou Antoine Pitrou
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 0.5h
                  0.5h