[BEAM-12545] WriteToFile should support mime/encoding.

• Type: Improvement
• Status: Resolved
• Priority: P3
• Resolution: Fixed
• Affects Version/s: 2.30.0
• Fix Version/s: 2.33.0
• Component/s: io-py-files
• Assignee: Unassigned
• Reporter: Andreas Bergmeier
• Created: 28/Jun/21 14:59
• Updated: 15/Jan/22 05:13
• Resolved: 15/Jan/22 05:13

Description

For our use case, we need to write blobs to GCS. Also, we want to have both Content-Type and Content-Encoding set for these blobs.
Since WriteToFiles currently only calls Filesystem.create without mime_type and compression_type, the resulting blobs are of type octet-stream.

We imagine this to look something like:

 metadata = sink.metadata()
 full_file_name, writer = _create_writer(base_path=self.base_path.get(),
 writer_key=(destination, w),
 mime_type=metadata.mime_type,
 compression_type=metadata.compression_type)

WDYT?

Issue Links

• links to: GitHub Pull Request #15265