Uploaded image for project: 'Apache Drill'
  1. Apache Drill
  2. DRILL-2728

Merge spill files in ExternalSort

    XMLWordPrintableJSON

    Details

      Description

      When external sort spills, it first merges all of the in-memory batches, and then keeps one batch in memory, while spilling the rest to disk. It's possible that we will still run out of memory, due to the single batch held in memory for each spill. When the memory due to these batches becomes too large, we should combine the spill files.

        Attachments

        1. DRILL-2728.patch
          7 kB
          Steven Phillips

          Activity

            People

            • Assignee:
              sphillips Steven Phillips
              Reporter:
              sphillips Steven Phillips
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: