Uploaded image for project: 'Apache Arrow'
  1. Apache Arrow
  2. ARROW-3789

[Python] Enable calling object in Table.to_pandas to "self-destruct" for improved memory use

    XMLWordPrintableJSON

Details

    Description

      One issue with using Table.to_pandas is that it results in a memory doubling (at least, more if there are a lot of Python objects created). It would be useful if there was an option to destroy the arrow::Column references once they've been transferred into the target data frame. This would render the pyarrow.Table object useless afterward

      Attachments

        Issue Links

          Activity

            People

              wesm Wes McKinney
              wesm Wes McKinney
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 3.5h
                  3.5h