Details
-
Sub-task
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
1.10.0
-
None
-
None
Description
The new "record batch sizer" computes the actual data size of a record given a batch of vectors. For most purposes, the record width must include the overhead of the offset vectors for variable-sized vectors. The initial code drop included only the character data, but not the offset vector size when computing row width.
Since the "managed" external sort relies on the computed row size to determine memory usage, the underestimation of row count width can cause an OOM under certain low-memory conditions.