[BEAM-12071] DataFrame IO sinks do not correctly partition by window

• Type: Bug
• Status: Resolved
• Priority: P1
• Resolution: Fixed
• Affects Version/s: 2.26.0, 2.27.0, 2.28.0
• Fix Version/s: 2.29.0
• Component/s: dsl-dataframe, sdk-py-core
• Labels: dataframe-api
• Assignee: Brian Hulette
• Reporter: Brian Hulette
• Created: 30/Mar/21 15:53
• Updated: 22/Jun/21 19:22
• Resolved: 01/Apr/21 00:52

Description

I just discovered that when processing windowed data with DataFrameTransform, only one partition is written to every window, and a single window gets every other partition/window combination.

Issue Links

• is caused by: BEAM-12072 WriteToFiles with a concrete sink can shunt data to a single window
• links to: GitHub Pull Request #14374
• links to: GitHub Pull Request #14383