Uploaded image for project: 'Apache Drill'
  1. Apache Drill
  2. DRILL-6678

Improve SelectionVectorRemover to pack output batch based on BatchSizing

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Reopened
    • Major
    • Resolution: Unresolved
    • 1.14.0
    • None
    • Execution - Relational Operators
    • None

    Description

      SelectionVectorRemover in most of the cases is downstream to Filter which reduces the number of records to be copied in output container. In those cases if SelectionVectorRemover can pack the output batch to approximate maximum utilization based on RecordBatchSizer target record count, that will reduce the number of output batches from it and will help to improve performance. During Lateral & Unnest Performance evaluation we have noticed a significant decrease in performance as number of batches increases for same number of rows (i.e. Batch is not fully packed)

      Attachments

        Issue Links

          is required by

          Bug - A problem which impairs or prevents the functions of the product. DRILL-6211 Optimizations for SelectionVectorRemover

          • Major - Major loss of function.
          • Resolved

          Activity

            People

              shamirwasia Sorabh Hamirwasia
              shamirwasia Sorabh Hamirwasia
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated: