Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-6280

Reject directory vs file path conflict resolution in FileOutputCommitter

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 2.6.0
    • None
    • mrv2
    • None

    Description

      If one task commits a directory foo, and then another task commits file foo, the directory foo with potentially many files will be wiped out. While this is a very unlikely scenario, due to tasks being homogeneous in nature, it's so much more important to alert the user by failing the commit. This came up in jlowe's review for MAPREDUCE-6275 and seems to be the behavior in branch-1 as well.

      Attachments

        Issue Links

          relates to

          Bug - A problem which impairs or prevents the functions of the product. MAPREDUCE-6275 Race condition in FileOutputCommitter v2 for user-specified task output subdirs

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Activity

            People

              Unassigned Unassigned
              jira.shegalov Gera Shegalov
              Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

                Created:
                Updated: