Apache Drill
  1. Apache Drill
  2. DRILL-641

Better semantic and handling for kill()

    Details

    • Type: Improvement Improvement
    • Status: Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: Future
    • Component/s: None
    • Labels:
      None

      Description

      Currently when an operator invokes kill(), only the upstream operators within that fragment get terminated. This causes problems if the upstream operator in a different fragment is still continuously producing batches, and it then becomes the responsibility of the operator to drain all the batches even though a kill was issued.

      While dealing with plans with multiple fragments we would need to inform the sender that a kill has been invoked. It would then be the responsibility of the sender to not push any more batches to the corresponding receiver.

        Activity

        Hide
        Jacques Nadeau added a comment -

        as part of this, we need to remove the cleanup method from the recordbatch interface. each operator should know when it is done with memory and release rather than waiting for someone else to tell it.

        Show
        Jacques Nadeau added a comment - as part of this, we need to remove the cleanup method from the recordbatch interface. each operator should know when it is done with memory and release rather than waiting for someone else to tell it.
        Hide
        Steven Phillips added a comment -

        before we can do that, we will need to change how we do memory accounting. With the change from DRILL-620, each operator has its own buffer allocator, along with its own accountor. Right now this only works if all downstream operators cleanup first.

        We would need to transfer accounting between operators for this to work.

        Show
        Steven Phillips added a comment - before we can do that, we will need to change how we do memory accounting. With the change from DRILL-620 , each operator has its own buffer allocator, along with its own accountor. Right now this only works if all downstream operators cleanup first. We would need to transfer accounting between operators for this to work.

          People

          • Assignee:
            Unassigned
            Reporter:
            Mehant Baid
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:

              Development