[IMPALA-9782] KuduPartitionExpr is not thread-safe

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: Impala 4.0.0
• Component/s: Backend
• Epic Link: Impala Multi-threaded query execution
• Epic Color: ghx-label-11
• Assignee: Tim Armstrong
• Reporter: Tim Armstrong
• Created: 28/May/20 03:53
• Updated: 09/Aug/23 00:42
• Resolved: 05/Jun/20 16:25

Description

This is a blocker for supporting Kudu DML with mt_dop. The expression has some mutable objects in the ScalarExpr object that needs to be moved to the evaluator.

KuduPartitioner is not thread-safe, because it uses an internal buffer. We should consider making it thread-safe or clonable, because the initialisation is expensive and does RPCs and things - https://github.com/cloudera/kudu/blob/master/src/kudu/client/partitioner-internal.cc#L40

KuduPartialRow is inherently not thread-safe because it contains thread-local data.

Issue Links

• causes: IMPALA-10829 Memory leak while inserting kudu with rand()
• depends upon: KUDU-3139 Add thread-safe KuduPartitioner::PartitionRow() method