Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-336

The task tracker should track disk space used, and have a configurable cap

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Major
    • Resolution: Duplicate
    • None
    • 0.4.0
    • None
    • None

    Description

      We've been having problems where the task tracker is destabilizing HDFS by filling disks and then releasing them. Various tasks can also interfere with each other this way.

      We should have a configurable max working space for a task (and allow the setting of lower per task limits). The task tracker should track use against this limit and terminate a job if it overruns it.

      Attachments

        Issue Links

          duplicates

          Bug - A problem which impairs or prevents the functions of the product. HADOOP-27 MapRed tries to allocate tasks to nodes that have no available disk space

          • Major - Major loss of function.
          • Closed
          is related to

          Bug - A problem which impairs or prevents the functions of the product. HADOOP-324 "IOException: No space left on device" is handled incorrectly

          • Major - Major loss of function.
          • Closed

          Activity

            People

              Unassigned Unassigned
              eric14 Eric Baldeschwieler
              Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: