Details
-
Wish
-
Status: Open
-
Major
-
Resolution: Unresolved
-
None
-
None
-
None
Description
In the code of DeltaBinaryPackingValuesWriter, the parameters are always DEFAULT_NUM_BLOCK_VALUES(which is 128) and DEFAULT_NUM_MINIBLOCKS(which is 4). So if the file is written by parquet-mr, the number of values in a miniblock is always 32. It is consistent with the spec.
However, the code in DeltaBinaryPackingConfig indicates that the number of values in a miniblock must be multiple of 8. Would it be better if the limitation was changed to 32?