Uploaded image for project: 'Apache Arrow'
  1. Apache Arrow
  2. ARROW-725

[Format] Constant length list type

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Trivial
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.3.0
    • Component/s: Format
    • Labels:
      None

      Description

      It makes sense to store some data in a row-based format. For example, a position might be stored as two or three coordinates per row, and all of them will almost always be accessed simultaneously. Currently, arrow must store these as two or three separate vectors, but cache performance could potentially be improved if every coordinate for a given row were in the same location in memory.

      The List type could satisfy this requirement, but it requires an additional offset vector which isn't necessary when every element is the same size. I think it would be helpful to define a new type that is essentially a List with every element having the same length. I think "Tuple" would be a natural fit for this type but I'm open to other suggestions.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                elahrvivaz Emilio Lahr-Vivaz
                Reporter:
                bhulette Brian Hulette
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: