Uploaded image for project: 'Apache Drill'
  1. Apache Drill
  2. DRILL-6099

Drill does not push limit past project (flatten) if it cannot be pushed into scan

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.12.0
    • Fix Version/s: 1.13.0
    • Component/s: None
    • Labels:

      Description

      It would be useful to have pushdown occur past flatten(project). Here is an example to illustrate the issue:

      explain plan without implementation for }}{{select name, flatten(categories) as category from dfs.`/tmp/t_json_20` LIMIT 1;

      DrillScreenRel{{  }}

        DrillLimitRel(fetch=[1]){{    }}

          DrillProjectRel(name=[$0], category=[FLATTEN($1)])

            DrillScanRel(table=[[dfs, /tmp/t_json_20]], groupscan=[EasyGroupScan [selectionRoot=maprfs:/tmp/t_json_20, numFiles=1, columns=[`name`, `categories`], files=[maprfs:///tmp/t_json_20/0_0_0.json]]])

      ================================================================= 

      Content of 0_0_0.json

      =================================================================

      {   "name" : "Eric Goldberg, MD",   "categories" : [ "Doctors", "Health & Medical" ] } {   "name" : "Pine Cone Restaurant",   "categories" : [ "Restaurants" ] } {   "name" : "Deforest Family Restaurant",   "categories" : [ "American (Traditional)", "Restaurants" ] } {   "name" : "Culver's",   "categories" : [ "Food", "Ice Cream & Frozen Yogurt", "Fast Food", "Restaurants" ] } {   "name" : "Chang Jiang Chinese Kitchen",   "categories" : [ "Chinese", "Restaurants" ] }

       

        Attachments

          Activity

            People

            • Assignee:
              gparai Gautam Kumar Parai
              Reporter:
              gparai Gautam Kumar Parai
              Reviewer:
              Chunhui Shi
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: