Details

    • Type: Sub-task
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Backend

      Description

      This case is a bit trickier than the others. The issue is that some plan nodes, like HdfsScanNode, hold onto memory that is referenced by returned batches and don't necessarily attach the memory before returning *eos.

      E.g. if you have a scan with a limit feeding into a UnionNode, the scan may terminate earlier before it attaches all memory to its output batches, then HdfsScanNode::Close() will free that memory. Thus it's not safe for the UnionNode to close its child until all referenced data has been copied out.

      The fix is either to attach the memory to the final batch in Close() or to have a new method like FlushResources() that is called before Close().

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              tarmstrong Tim Armstrong
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated: