Uploaded image for project: 'Apache Arrow'
  1. Apache Arrow
  2. ARROW-5510

[Format] Feather V2 based on Arrow IPC file format, with compression support

    XMLWordPrintableJSON

Details

    Description

      The initial Feather file format is a minimal subset of the Arrow IPC format. It has a number of limitations (see https://wesmckinney.com/blog/feather-arrow-future/). 

      We want to retain "feather" as the name of the on-disk representation of Arrow memory, so in order to support everything that Arrow supports, we need a "feather 2.0" format.

      IIUC, defining the file format is "done" (dump the memory to disk). Remaining issues include upgrading "feather" readers and writers in all languages to support both feather 1.0 and feather 2.0. (e.g. https://issues.apache.org/jira/browse/ARROW-5501)

      Attachments

        Issue Links

          Activity

            People

              wesm Wes McKinney
              npr Neal Richardson
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 6.5h
                  6.5h