[IMPALA-10371] test_java_udfs crash impalad if result spooling is enabled

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: Impala 3.4.0
• Component/s: Backend
• Epic Color: ghx-label-6
• Assignee: Daniel Becker
• Reporter: Riza Suminto
• Created: 02/Dec/20 00:14
• Updated: 25/Feb/21 02:10
• Resolved: 24/Feb/21 11:43

Description

The following test query from TestUdfExecution::test_java_udfs crash impalad when result spooling is enabled.

select throws_exception() from functional.alltypestiny

The following is a truncated JVM crash log related to the crash

---------------  T H R E A D  ---------------Current thread (0x000000000fb4c000):  JavaThread "Thread-700" [_thread_in_native, id=30853, stack(0x00007f79715ff000,0x00007f7971dff000)]Stack: [0x00007f79715ff000,0x00007f7971dff000],  sp=0x00007f7971dfa280,  free space=8172k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
V  [libjvm.so+0xb6b032]
V  [libjvm.so+0x4f14bd]
V  [libjvm.so+0x80fa8f]
V  [libjvm.so+0x7e0991]
V  [libjvm.so+0x69fa10]
j  org.apache.impala.TestUdfException.evaluate()Lorg/apache/hadoop/io/BooleanWritable;+9
v  ~StubRoutines::call_stub
V  [libjvm.so+0x6af9ba]
V  [libjvm.so+0xa1def8]
V  [libjvm.so+0xa1f8d5]
V  [libjvm.so+0x7610f8]  JVM_InvokeMethod+0x128
J 2286  sun.reflect.NativeMethodAccessorImpl.invoke0(Ljava/lang/reflect/Method;Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object; (0 bytes) @ 0x00007f7acb553ced [0x00007f7acb553c00+0xed]
J 6921 C2 sun.reflect.NativeMethodAccessorImpl.invoke(Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object; (104 bytes) @ 0x00007f7acbd1de38 [0x00007f7acbd1ddc0+0x78]
J 3645 C2 org.apache.impala.hive.executor.UdfExecutor.evaluate()V (396 bytes) @ 0x00007f7acaf6e894 [0x00007f7acaf6e640+0x254]
v  ~StubRoutines::call_stub
V  [libjvm.so+0x6af9ba]
V  [libjvm.so+0x72c046]
V  [libjvm.so+0x730523]
C  0x00007f7ab4c5d0d0
C  [impalad+0x26a2648]  impala::ScalarExprEvaluator::GetValue(impala::ScalarExpr const&, impala::TupleRow const*)+0x7a
C  [impalad+0x26a25cb]  impala::ScalarExprEvaluator::GetValue(impala::TupleRow const*)+0x2b
C  [impalad+0x21f4f78]  impala::AsciiQueryResultSet::AddRows(std::vector<impala::ScalarExprEvaluator*, std::allocator<impala::ScalarExprEvaluator*> > const&, impala::RowBatch*, int, int)+0x4c2
C  [impalad+0x25c5862]  impala::BufferedPlanRootSink::GetNext(impala::RuntimeState*, impala::QueryResultSet*, int, bool*, long)+0x70c
C  [impalad+0x296cf17]  impala::Coordinator::GetNext(impala::QueryResultSet*, int, bool*, long)+0x557
C  [impalad+0x219f5fe]  impala::ClientRequestState::FetchRowsInternal(int, impala::QueryResultSet*, long)+0x6b2
C  [impalad+0x219d98e]  impala::ClientRequestState::FetchRows(int, impala::QueryResultSet*, long)+0x46
C  [impalad+0x21c1d29]  impala::ImpalaServer::FetchInternal(impala::TUniqueId, bool, int, beeswax::Results*)+0x717
C  [impalad+0x21bbde9]  impala::ImpalaServer::fetch(beeswax::Results&, beeswax::QueryHandle const&, bool, int)+0x577

If result spooling is enabled, BufferedPlanRootSink will be used and ScalarExprEvaluation will be called in BufferedPlanRootSink::GetNext, leading to this crash.

Without result spooling, BlockingPlanRootSink will be used and ScalarExprEvaluation is called in BlockingPlanRootSink::Send. No crash happen when result spooling is disabled.

Attached is the full JVM crash log and resolved minidump.

 

Issue Links

• is related to: IMPALA-9856 Enable result spooling by default