[HOP-3557] Parquet File output transform not writing data to GCS when running on Beam Dataflow

• Type: Bug
• Status: Resolved
• Priority: Normal
• Resolution: Fixed
• Affects Version/s: 1.0.0, 1.1.0
• Fix Version/s: 2.1.0, 2.2.0
• Component/s: Transforms
• Labels: GCP dataflow parquetWriter
• Assignee: Unassigned
• Reporter: Yancarlo Perez
• Created: 16/Dec/21 16:56
• Updated: 28/Oct/22 12:34
• Resolved: 28/Oct/22 12:34

Description

When executing a pipeline with the Beam Dataflow pipeline engine and using the Parquet File output transform it creates the file in GCS but with no contents.

If you run the same attached pipeline using the local engine it creates the file with the correct data.