Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-5895

FileAlreadyExistsException was thrown : Temporary Index File can not be cleaned up because OutputStream doesn't close properly

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 3.0.0-alpha1
    • Fix Version/s: 2.5.0
    • Component/s: client
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      In TaskLog.java, Temporary Index File is created by following code.

      BufferedOutputStream bos =
        new BufferedOutputStream(
          SecureIOUtils.createForWrite(tmpIndexFile, 0644));
      DataOutputStream dos = new DataOutputStream(bos);
      

      The code is surrounded by try-finally so if some Exception/ERROR is thrown between constructing bos and dos, temporary file is not cleaned up.
      I met the situation that when a thread ran, OOM was thrown after bos created and temporary file is not cleaned up. At different time, another thread executed same logic and fail because of FileAlreadyExistsException.

        Attachments

        1. MAPREDUCE-5895.patch
          2 kB
          Kousuke Saruta
        2. MAPREDUCE-5895.patch
          2 kB
          Kousuke Saruta

          Activity

            People

            • Assignee:
              sarutak Kousuke Saruta
              Reporter:
              sarutak Kousuke Saruta
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: