Uploaded image for project: 'Apache Drill'
  1. Apache Drill
  2. DRILL-6160

Limit batch size for streaming aggregate based on memory

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Critical
    • Resolution: Unresolved
    • Affects Version/s: 1.12.0
    • Fix Version/s: None
    • Component/s: Execution - Flow
    • Labels:
      None

      Description

      Streaming aggregate has batch size of 32k rows. Change it to figure out number of rows in the outgoing batch based on memory configured with the option outputBatchSize and actual incoming batch size. Limit it to max of 64k and min of 1. 

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                ppenumarthy Padma Penumarthy
                Reporter:
                ppenumarthy Padma Penumarthy
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated: