Uploaded image for project: 'Apache Arrow'
  1. Apache Arrow
  2. ARROW-2840

[C++] See if stream alignment logic can be simplified

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 0.11.0
    • C++
    • None

    Description

      There were a couple patches related to 64-byte alignment for tensor messages at the beginning of April that I've been concerned about:

      ARROW-2308: https://github.com/apache/arrow/commit/26bc4ab5a31e3430e1d545068e5a5a5ba5bc7a22#diff-bf4eb56f3cb806c32f60ae67afbb1bf0

      ARROW-2437: https://github.com/apache/arrow/commit/0f87c12d45250ee763ac8c43b7e57e8f06a0b9f3

      In my opinion, the alignment issue should be encapsulated in the protocol if possible; to have argument that toggles whether the file is to be aligned after reading the metadata seems wrong to me. The bytes written for alignment purposes should be accounted for in the metadata size prefix in the IPC message.

      Since there are unit tests for this, I'd like to take a look and see if I can improve this without breaking the tests.

      Attachments

        Activity

          People

            wesm Wes McKinney
            wesm Wes McKinney
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: