Uploaded image for project: 'Apache Drill'
  1. Apache Drill
  2. DRILL-6180

Use System Option "output_batch_size" for External Sort

    XMLWordPrintableJSON

Details

    Description

      External Sort has boot time configuration for output batch size "drill.exec.sort.external.spill.merge_batch_size" which is defaulted to 16M.

      To make batch sizing configuration uniform across all operators, change this to use new system option that is added "drill.exec.memory.operator.output_batch_size". This option has default value of 32M.

      So, what are the implications if default is changed to 32M for external sort ?

      Instead, should we change the output batch size default to 16M for all operators ?

       

       

      Attachments

        Issue Links

          Activity

            People

              ppenumarthy Padma Penumarthy
              ppenumarthy Padma Penumarthy
              Paul Rogers Paul Rogers
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: