Uploaded image for project: 'Apache Drill'
  1. Apache Drill
  2. DRILL-6678

Improve SelectionVectorRemover to pack output batch based on BatchSizing

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Reopened
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 1.14.0
    • Fix Version/s: None
    • Labels:
      None

      Description

      SelectionVectorRemover in most of the cases is downstream to Filter which reduces the number of records to be copied in output container. In those cases if SelectionVectorRemover can pack the output batch to approximate maximum utilization based on RecordBatchSizer target record count, that will reduce the number of output batches from it and will help to improve performance. During Lateral & Unnest Performance evaluation we have noticed a significant decrease in performance as number of batches increases for same number of rows (i.e. Batch is not fully packed)

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                shamirwasia Sorabh Hamirwasia
                Reporter:
                shamirwasia Sorabh Hamirwasia
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated: