Uploaded image for project: 'Apache Arrow'
  1. Apache Arrow
  2. ARROW-5510

[Format] Feather V2 based on Arrow IPC file format, with compression support

    XMLWordPrintableJSON

    Details

      Description

      The initial Feather file format is a minimal subset of the Arrow IPC format. It has a number of limitations (see https://wesmckinney.com/blog/feather-arrow-future/). 

      We want to retain "feather" as the name of the on-disk representation of Arrow memory, so in order to support everything that Arrow supports, we need a "feather 2.0" format.

      IIUC, defining the file format is "done" (dump the memory to disk). Remaining issues include upgrading "feather" readers and writers in all languages to support both feather 1.0 and feather 2.0. (e.g. https://issues.apache.org/jira/browse/ARROW-5501)

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                wesm Wes McKinney
                Reporter:
                npr Neal Richardson
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 6.5h
                  6.5h