Uploaded image for project: 'Apache Arrow'
  1. Apache Arrow
  2. ARROW-2090

[Python] Add context manager methods to ParquetWriter

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.9.0
    • Component/s: Python

      Description

      Add the ability to use python with syntax on the ParquetWriter object.

      For example:

      with pq.ParquetWriter(foo, schema) as writer:
          writer.write_table(table)
      

      The benefit of this syntax is that it removes the chances of you writing out a partial (invalid) parquet file, which is currently possible if you forget to call the close method, or more likely the close method is not called due to an exception being thrown.

      It should still be possible to use the previous syntax for backwards compatibility and fine grained control reasons.
      Similarly, the parquet module level write_table method should be able to use the new syntax without changing previous behaviour.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                aposney Alec Posney
                Reporter:
                aposney Alec Posney
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: