Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-6512

FileOutputCommitter tasks unconditionally create parent directories

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Won't Fix
    • None
    • None
    • None
    • None

    Description

      If the output directory is deleted then subsequent tasks should fail. Instead they blindly create the missing parent directories, leading the job to be "succesful" despite potentially missing almost all of the output. Task attempts should fail if the parent app attempt directory is missing when they go to create their task attempt directory.

      Attachments

        1. MAPREDUCE-6512.2.patch
          16 kB
          Chang Li
        2. MAPREDUCE-6512.2.patch
          16 kB
          Chang Li
        3. MAPREDUCE-6512.patch
          6 kB
          Chang Li
        4. MAPREDUCE-6512.patch
          6 kB
          Chang Li

        Activity

          People

            lichangleo Chang Li
            lichangleo Chang Li
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: