Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-7689

Temporary directory for WriteOperation may not be unique in FileBaseSink

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: P0
    • Resolution: Fixed
    • Affects Version/s: 2.5.0, 2.6.0, 2.7.0, 2.8.0, 2.9.0, 2.10.0, 2.11.0, 2.12.0, 2.13.0
    • Fix Version/s: 2.14.0
    • Component/s: io-java-files
    • Labels:
      None

      Description

      Temporary directory for WriteOperation in FileBasedSink is generated from a second-granularity timestamp (yyyy-MM-dd_HH-mm-ss) and unique increasing index. Such granularity is not enough to make temporary directories unique between different jobs. When two jobs share the same temporary directory, output file may not be produced in one job since the required temporary file can be deleted from another job.

        Attachments

          Activity

            People

            • Assignee:
              heejong Heejong Lee
              Reporter:
              heejong Heejong Lee
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 3h 40m
                3h 40m