Details
-
Improvement
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
None
-
None
Description
There were a couple patches related to 64-byte alignment for tensor messages at the beginning of April that I've been concerned about:
ARROW-2308: https://github.com/apache/arrow/commit/26bc4ab5a31e3430e1d545068e5a5a5ba5bc7a22#diff-bf4eb56f3cb806c32f60ae67afbb1bf0
ARROW-2437: https://github.com/apache/arrow/commit/0f87c12d45250ee763ac8c43b7e57e8f06a0b9f3
In my opinion, the alignment issue should be encapsulated in the protocol if possible; to have argument that toggles whether the file is to be aligned after reading the metadata seems wrong to me. The bytes written for alignment purposes should be accounted for in the metadata size prefix in the IPC message.
Since there are unit tests for this, I'd like to take a look and see if I can improve this without breaking the tests.