Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-1028

Cleanup tasks are scheduled using high memory configuration, leaving tasks in unassigned state.

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Blocker Blocker
    • Resolution: Fixed
    • Affects Version/s: 0.21.0
    • Fix Version/s: 0.21.0
    • Component/s: jobtracker
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Makes taskCleanup tasks to use 1 slot even for high memory jobs.

      Description

      A cleanup task is launched for a failed task of a job. This task is created based on the TIP of the failed task, and so is marked as requiring as many slots to run as the original task itself. For instance, if a high RAM job requires 2 slots per task, a cleanup task of the high RAM jobs requires 2 slots as well.

      Further, a cleanup task is scheduled to a tasktracker by the jobtracker itself and not the scheduler. While doing so, the JT doesn't check if the TT has enough slots free to run a high RAM cleanup task - always assuming 1 slot is enough. Thus, a task is oversubscribed to the TT.

      However, on the TT, before launch, we check that the task can actually run, and wait for so many slots to become available. If the slots don't get freed quickly, we will have tasks stuck in an unassigned state.

      1. yhadoop-0.20-MR1028.patch
        1 kB
        Jothi Padmanabhan
      2. MR-1028.v1.1.patch
        13 kB
        Ravi Gummadi
      3. MR-1028.v1.patch
        12 kB
        Ravi Gummadi
      4. MR-1028.patch
        1 kB
        Ravi Gummadi

        Activity

        Hemanth Yamijala created issue -
        Ravi Gummadi made changes -
        Field Original Value New Value
        Attachment MR-1028.patch [ 12420466 ]
        Ravi Gummadi made changes -
        Attachment MR-1028.v1.patch [ 12420542 ]
        Ravi Gummadi made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Release Note Makes taskCleanup tasks to use 1 slot even for high memory jobs.
        Jothi Padmanabhan made changes -
        Status Patch Available [ 10002 ] Open [ 1 ]
        Ravi Gummadi made changes -
        Attachment MR-1028.v1.1.patch [ 12420550 ]
        Ravi Gummadi made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Jothi Padmanabhan made changes -
        Status Patch Available [ 10002 ] Open [ 1 ]
        Jothi Padmanabhan made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Jothi Padmanabhan made changes -
        Attachment yhadoop-0.20-MR1028.patch [ 12420581 ]
        Hemanth Yamijala made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Hadoop Flags [Reviewed]
        Resolution Fixed [ 1 ]
        Tom White made changes -
        Status Resolved [ 5 ] Closed [ 6 ]

          People

          • Assignee:
            Ravi Gummadi
            Reporter:
            Hemanth Yamijala
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development