[IMPALA-5123] ASAN failure: heap-use-after-free in timezone_db.cc:683

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: Impala 2.9.0
• Fix Version/s: Impala 2.9.0
• Component/s: Backend
• Epic Color: ghx-label-3
• Assignee: Zach Amsden
• Reporter: Lars Volker
• Created: 28/Mar/17 13:43
• Updated: 15/Jun/17 17:14
• Resolved: 15/Jun/17 17:14

Description

Looks like the char *filestr in line 674 points to a temporary object and the underlying memory is free'd right after it's initialization. This was introduced by this change: https://gerrit.cloudera.org/#/c/5523/

Here's the ASAN output:

Log file created at: 2017/03/27 21:22:06
Running on machine: impala-boost-static-burst-slave-15d8.vpc.cloudera.com
Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg
E0327 21:22:06.348176  4077 logging.cc:124] stderr will be logged to this file.
=================================================================
==4077==ERROR: AddressSanitizer: heap-use-after-free on address 0x6060000d6658 at pc 0x000000fab738 bp 0x7fff105e5970 sp 0x7fff105e5120
READ of size 25 at 0x6060000d6658 thread T0
    #0 0xfab737 in fopen /data/jenkins/workspace/verify-impala-toolchain-package-build/label/ec2-package-centos-6/toolchain/source/llvm/llvm-3.8.0.src-p1/projects/compiler-rt/lib/asan/../sanitizer_common/sanitizer_common_interceptors.inc:4780
    #1 0x1b13a54 in impala::TimezoneDatabase::Initialize() /data/jenkins/workspace/impala-umbrella-build-and-test/repos/Impala/be/src/exprs/timezone_db.cc:683:15
    #2 0x15832f8 in ImpaladMain(int, char**) /data/jenkins/workspace/impala-umbrella-build-and-test/repos/Impala/be/src/service/impalad-main.cc:63:29
    #3 0x1032548 in main /data/jenkins/workspace/impala-umbrella-build-and-test/repos/Impala/be/src/service/daemon-main.cc:37:12
    #4 0x38de01ecdc in __libc_start_main (/lib64/libc.so.6+0x38de01ecdc)
    #5 0xf589dc in _start (/data/jenkins/workspace/impala-umbrella-build-and-test/repos/Impala/be/build/debug/service/impalad+0xf589dc)

0x6060000d6658 is located 24 bytes inside of 49-byte region [0x6060000d6640,0x6060000d6671)
freed by thread T0 here:
    #0 0x102fd30 in operator delete(void*) /data/jenkins/workspace/verify-impala-toolchain-package-build/label/ec2-package-centos-6/toolchain/source/llvm/llvm-3.8.0.src-p1/projects/compiler-rt/lib/asan/asan_new_delete.cc:94
    #1 0x1b13a16 in impala::TimezoneDatabase::Initialize() /data/jenkins/workspace/impala-umbrella-build-and-test/repos/Impala/be/src/exprs/timezone_db.cc:674:19
    #2 0x15832f8 in ImpaladMain(int, char**) /data/jenkins/workspace/impala-umbrella-build-and-test/repos/Impala/be/src/service/impalad-main.cc:63:29
    #3 0x1032548 in main /data/jenkins/workspace/impala-umbrella-build-and-test/repos/Impala/be/src/service/daemon-main.cc:37:12
    #4 0x38de01ecdc in __libc_start_main (/lib64/libc.so.6+0x38de01ecdc)

previously allocated by thread T0 here:
    #0 0x102f730 in operator new(unsigned long) /data/jenkins/workspace/verify-impala-toolchain-package-build/label/ec2-package-centos-6/toolchain/source/llvm/llvm-3.8.0.src-p1/projects/compiler-rt/lib/asan/asan_new_delete.cc:62
    #1 0x7f827a5fcc48 in __gnu_cxx::new_allocator<char>::allocate(unsigned long, void const*) /data/jenkins/workspace/verify-impala-toolchain-package-build/label/ec2-package-centos-6/toolchain/source/gcc/build/x86_64-unknown-linux-gnu/libstdc++-v3/include/ext/new_allocator.h:104
    #2 0x7f827a5fcc48 in std::string::_Rep::_S_create(unsigned long, unsigned long, std::allocator<char> const&) /data/jenkins/workspace/verify-impala-toolchain-package-build/label/ec2-package-centos-6/toolchain/source/gcc/build/x86_64-unknown-linux-gnu/libstdc++-v3/include/bits/basic_string.tcc:607

SUMMARY: AddressSanitizer: heap-use-after-free /data/jenkins/workspace/verify-impala-toolchain-package-build/label/ec2-package-centos-6/toolchain/source/llvm/llvm-3.8.0.src-p1/projects/compiler-rt/lib/asan/../sanitizer_common/sanitizer_common_interceptors.inc:4780 in fopen

Issue Links

• is duplicated by: IMPALA-5132 ASAN detected memory problem with Timezone db
• is required by: IMPALA-4673 Use --local_library_dir for timestamp db scratch space