Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-3483

CapacityScheduler reserves container on same node as AM but can't ever use due to never enough avail memory

    Details

    • Type: Bug Bug
    • Status: Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: 2.3.0
    • Fix Version/s: None
    • Component/s: mrv2
    • Labels:
      None
    • Target Version/s:

      Description

      Saw a case where a job was stuck trying to get reducers. The issue is the capacity scheduler reserved a container on the same node as the application master but there wasn't ever enough memory to run the reducer on that node. Node total memory was 8G, Reducer needed 8G, AM was using 2G. This particular job had 10 reducers and it was stuck waiting on the one because the AM + reserved reducer memory was already over the queue limit.

        Activity

        No work has yet been logged on this issue.

          People

          • Assignee:
            Arun C Murthy
            Reporter:
            Thomas Graves
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:

              Development