[IMPALA-10956] datasketches UDFS: memory leak and merge overhead

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: Impala 4.0.0
• Fix Version/s: Impala 4.1.0
• Component/s: Backend
• Epic Color: ghx-label-11
• Assignee: Alexander Saydakov
• Reporter: Alexander Saydakov
• Created: 07/Oct/21 21:47
• Updated: 17/Apr/22 12:33
• Resolved: 13/Dec/21 23:24

Description

I believe that there are memory leaks in aggregate UDFs that are using Apache Datasketches. Sketch or union objects are placed in a buffer to maintain the state of the aggregation, and later deallocated without calling destructors for those objects.

Also there is unnecessary overhead during the merge phase of some of those UDFs, when an instance of a union is created for every merge operation and finalized by calling a quite expensive get_result() every time as well. This second issue is about performance, not a bug.