Uploaded image for project: 'Apache Arrow'
  1. Apache Arrow
  2. ARROW-6823

[C++][Python][R] Support metadata in the feather format?

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: C++, Python, R
    • Labels:

      Description

      This might need to wait / could be enabled by the feather v2 (ARROW-5510), but thought to open a specific issue about it: do we want to support saving metadata in feather files?

      With Parquet files, you can have file-level metadata (which we currently use to eg store the pandas_metadata). I think it would be useful to have a similar mechanism for Feather files.

      A use case where this came up is in GeoPandas where we would like to store the Coordinate Reference System identifier of the geometry data inside the file, to avoid needing a sidecar file just for that.

      In a v2 world (using the IPC format), I suppose this could be the metadata of the Schema.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              jorisvandenbossche Joris Van den Bossche
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated: