Uploaded image for project: 'Apache Arrow'
  1. Apache Arrow
  2. ARROW-5510

[Format] Feather V2

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: 1.0.0
    • Component/s: Format
    • Labels:
      None

      Description

      The initial Feather file format is a minimal subset of the Arrow IPC format. It has a number of limitations (see https://wesmckinney.com/blog/feather-arrow-future/). 

      We want to retain "feather" as the name of the on-disk representation of Arrow memory, so in order to support everything that Arrow supports, we need a "feather 2.0" format.

      IIUC, defining the file format is "done" (dump the memory to disk). Remaining issues include upgrading "feather" readers and writers in all languages to support both feather 1.0 and feather 2.0. (e.g. https://issues.apache.org/jira/browse/ARROW-5501)

        Attachments

          Activity

            People

            • Assignee:
              wesm Wes McKinney
              Reporter:
              npr Neal Richardson
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated: