Uploaded image for project: 'Apache Arrow'
  1. Apache Arrow
  2. ARROW-2090

[Python] Add context manager methods to ParquetWriter

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • None
    • 0.9.0
    • Python

    Description

      Add the ability to use python with syntax on the ParquetWriter object.

      For example:

      with pq.ParquetWriter(foo, schema) as writer:
          writer.write_table(table)
      

      The benefit of this syntax is that it removes the chances of you writing out a partial (invalid) parquet file, which is currently possible if you forget to call the close method, or more likely the close method is not called due to an exception being thrown.

      It should still be possible to use the previous syntax for backwards compatibility and fine grained control reasons.
      Similarly, the parquet module level write_table method should be able to use the new syntax without changing previous behaviour.

      Attachments

        Issue Links

          Activity

            People

              aposney Alec Posney
              aposney Alec Posney
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: