Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-4654

remove temporary output directory of failed tasks

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 0.17.2, 0.18.1
    • 0.18.3
    • None
    • None
    • Reviewed

    Description

      When dfs is getting full (80+% of reserved space), the rate of write failures increases, such that more map-reduce tasks can fail. By not cleaning up the temporary output directory of tasks the situation worsens over the lifetime of a job, increasing the probability of the whole job failing.

      Attachments

        1. patch-4654-1-0.18.txt
          3 kB
          Amareshwari Sriramadasu
        2. patch-4654-0.18.txt
          3 kB
          Amareshwari Sriramadasu

        Issue Links

          is related to

          Bug - A problem which impairs or prevents the functions of the product. HADOOP-4759 HADOOP-4654 to be fixed for branches >= 0.19

          • Major - Major loss of function.
          • Closed

          Activity

            People

              amareshwari Amareshwari Sriramadasu
              ckunz Christian Kunz
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: