Uploaded image for project: 'Apache Drill'
  1. Apache Drill
  2. DRILL-1055

Implement ProducerConsumer operator

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.4.0
    • Component/s: None
    • Labels:
      None

      Description

      Currently, a minor fragment is run in a single thread, and parallelization is achieved by running multiple minor fragments. There are cases where we are limited in how much we can parallelize the leaf fragments due to the limited number of files/row groups. This can result in under-utilization of resources.

      This operator will allow the scan operator to run in parallel with the downstream operators in the leaf fragments, thus increasing cluster utilization, and reducing the duration of the leaf fragments in certain cases.

        Attachments

          Activity

            People

            • Assignee:
              jnadeau Jacques Nadeau
              Reporter:
              sphillips Steven Phillips
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: