Uploaded image for project: 'Apache Drill'
  1. Apache Drill
  2. DRILL-4147

Union All operator runs in a single fragment

VotersWatch issueWatchersLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 1.8.0
    • None
    • None

    Description

      A User noticed that running select from a single directory is much faster than union all on two directories.
      (https://drill.apache.org/blog/2014/12/09/running-sql-queries-on-amazon-s3/#comment-2349732267)

      It seems like UNION ALL operator doesn't parallelize sub scans (its using SINGLETON for distribution type). Everything is ran in single fragment.

      We may have to use SubsetTransformer in UnionAllPrule.

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            amansinha100 Aman Sinha
            aah amit hadke
            Robert Hou Robert Hou
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment