Uploaded image for project: 'Apache Arrow'
  1. Apache Arrow
  2. ARROW-1784

[Python] Read and write pandas.DataFrame in pyarrow.serialize by decomposing the BlockManager rather than coercing to Arrow format

    Details

    • Type: New Feature
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.8.0
    • Component/s: Python

      Description

      See discussion in https://github.com/dask/distributed/pull/931

      This will permit zero-copy reads for DataFrames not containing Python objects. In the event of an ObjectBlock these arrays will not be worse than pickle to reconstruct on the receiving side

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                wesmckinn Wes McKinney
                Reporter:
                wesmckinn Wes McKinney
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: