Uploaded image for project: 'Apache Drill'
  1. Apache Drill
  2. DRILL-6635

PartitionLimit for Lateral/Unnest

    XMLWordPrintableJSON

    Details

      Description

      With batch processing changes in Lateral/Unnest the limit/TopN clause within Lateral-Unnest subquery will not work as expected since it will impose limit/TopN across RowId's. We need a new mechanism to apply these operators at rowId boundary.
      For now we are planning to add support for only limit and hence need to have a new operator PartitionLimit which will get the partitionColumn on which the limit should be imposed. This will currently only support queries between lateral and unnest. 
      For TopN we can still achieve that using combination of Sort and Partition Limit and later we can figure out how to address it directly within TopN or is it needed at all. Since the number of rows across EMIT boundary on which SORT will operate should not be big enough and mostly be done in memory.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                shamirwasia Sorabh Hamirwasia
                Reporter:
                shamirwasia Sorabh Hamirwasia
                Reviewer:
                Parth Chandra
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: