[BEAM-7689] Temporary directory for WriteOperation may not be unique in FileBaseSink

• Type: Bug
• Status: Resolved
• Priority: P0
• Resolution: Fixed
• Affects Version/s: 2.5.0, 2.6.0, 2.7.0, 2.8.0, 2.9.0, 2.10.0, 2.11.0, 2.12.0, 2.13.0
• Fix Version/s: 2.14.0
• Component/s: io-java-files
• Assignee: Heejong Lee
• Reporter: Heejong Lee
• Created: 04/Jul/19 00:33
• Updated: 24/Jul/20 19:56
• Resolved: 12/Jul/19 19:57

Description

Temporary directory for WriteOperation in FileBasedSink is generated from a second-granularity timestamp (yyyy-MM-dd_HH-mm-ss) and unique increasing index. Such granularity is not enough to make temporary directories unique between different jobs. When two jobs share the same temporary directory, output file may not be produced in one job since the required temporary file can be deleted from another job.

Issue Links

• links to: GitHub Pull Request #9013
• links to: GitHub Pull Request #9039
• links to: GitHub Pull Request #9071