[BEAM-5028] getPerDestinationOutputFilenames() is getting processed before write is finished

• Type: Bug
• Status: Resolved
• Priority: P2
• Resolution: Fixed
• Affects Version/s: 2.5.0
• Fix Version/s: 2.7.0
• Component/s: io-java-files
• Assignee: Jean-Baptiste Onofré
• Reporter: Jozef Vilcek
• Created: 26/Jul/18 09:55
• Updated: 24/Jul/20 20:15
• Resolved: 01/Nov/18 12:34

Description

I believe I am hitting a race condition on finalising writes vs notification events about frites being done.

It looks to me that WriteFilesResult.getPerDestinationOutputFilenames() can announce files before they are finished:

https://github.com/apache/beam/blob/release-2.5.0/sdks/java/core/src/main/java/org/apache/beam/sdk/io/WriteFiles.java#L799

This is related to BEAM-3268 where it appears to not be fully fixed.

Issue Links

• links to: GitHub Pull Request #6075