Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-336

The task tracker should track disk space used, and have a configurable cap

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Duplicate
    • Affects Version/s: None
    • Fix Version/s: 0.4.0
    • Component/s: None
    • Labels:
      None

      Description

      We've been having problems where the task tracker is destabilizing HDFS by filling disks and then releasing them. Various tasks can also interfere with each other this way.

      We should have a configurable max working space for a task (and allow the setting of lower per task limits). The task tracker should track use against this limit and terminate a job if it overruns it.

        Issue Links

          Activity

          Hide
          owen.omalley Owen O'Malley added a comment -

          this was fixed by HADOOP-27

          Show
          owen.omalley Owen O'Malley added a comment - this was fixed by HADOOP-27
          Hide
          eric14 eric baldeschwieler added a comment -

          Konstantin mentions that tasks filling disks cause HDFS ioExceptions

          Show
          eric14 eric baldeschwieler added a comment - Konstantin mentions that tasks filling disks cause HDFS ioExceptions

            People

            • Assignee:
              Unassigned
              Reporter:
              eric14 eric baldeschwieler
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development