Author: Alex Behm <email@example.com>
Date: Fri Oct 28 16:33:40 2016 -0700
IMPALA-4377: Fix Java UDF-arg buffer use-after-free in UdfExecutorTest.
The bug is simplest to explain with the old buggy test code below.
I added comments in the code to explain the bug.
// We used this for creating Text UDF arguments before invoking a UDF.
Writable createText(String v)
// Note that 'w' does not own the native buffer backing the string,
// explained below.
ImpalaTextWritable w = new ImpalaTextWritable(createStringValue(v));
long createStringValue(String v)
byte array = v.getBytes();
long ptr = allocate(16);
UnsafeUtil.UNSAFE.putInt(ptr + 8, 0);
ImpalaStringWritable sw = new ImpalaStringWritable(ptr);
// This allocates a new native buffer and sets it as a member
// of 'sw'. The native buffer is freed in sw.finalize().
// However, after this function there are no more references
// to 'sw', so the GC is allowed to free it. When that happens
// the UDF argument's native memory is gone and we get garbage
// UDF evaluations.
sw.set(array, 0, array.length);
This change also includes logging improvements to make similar
issues easier to diagnose in the future.
Testing: The bug was easy to reproduce by increasing the number
of runs in TestUdfImpl() to a large number. After this patch
I could not reproduce the issue anymore.
Reviewed-by: Matthew Jacobs <firstname.lastname@example.org>
Tested-by: Internal Jenkins