Uploaded image for project: 'Apache Drill'
  1. Apache Drill
  2. DRILL-4147

Union All operator runs in a single fragment

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.8.0
    • Component/s: None
    • Labels:
      None

      Description

      A User noticed that running select from a single directory is much faster than union all on two directories.
      (https://drill.apache.org/blog/2014/12/09/running-sql-queries-on-amazon-s3/#comment-2349732267)

      It seems like UNION ALL operator doesn't parallelize sub scans (its using SINGLETON for distribution type). Everything is ran in single fragment.

      We may have to use SubsetTransformer in UnionAllPrule.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                amansinha100 Aman Sinha
                Reporter:
                aah amit hadke
                Reviewer:
                Robert Hou
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: