[IMPALA-7366] Remove MarkNeedsDeepCopy() from UnionNode

• Type: Sub-task
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: Backend
• Labels: resource-management
• Target Version: Product Backlog
• Epic Color: ghx-label-8
• Assignee: Unassigned
• Reporter: Tim Armstrong
• Created: 27/Jul/18 22:46
• Updated: 09/Aug/18 17:31

Description

This case is a bit trickier than the others. The issue is that some plan nodes, like HdfsScanNode, hold onto memory that is referenced by returned batches and don't necessarily attach the memory before returning *eos.

E.g. if you have a scan with a limit feeding into a UnionNode, the scan may terminate earlier before it attaches all memory to its output batches, then HdfsScanNode::Close() will free that memory. Thus it's not safe for the UnionNode to close its child until all referenced data has been copied out.

The fix is either to attach the memory to the final batch in Close() or to have a new method like FlushResources() that is called before Close().