Details
-
Sub-task
-
Status: Resolved
-
Minor
-
Resolution: Fixed
-
1.8.0
-
None
-
None
Description
The ExternalSortBatch (ESB) operator performs a sort while spilling to disk to stay within a defined memory budget.
ESB uses two memory allocators:
- The operator allocator for most operations,
- The copier allocator (child of the operator allocator) for merge operations.
The copier allocator is used only when merging prior to spill.
However, since the ESB must manage within a single memory budget, the code is made more complex by using two allocators, and keeping track of the memory used in both.
Better to use a single allocator. When checking for the need to spill, the code already checks if enough memory is available for an in-memory sort, if needed. Similarly, the code should also check if sufficient memory is available for the copy step when needed.