Hadoop YARN
  1. Hadoop YARN
  2. YARN-110

AM releases too many containers due to the protocol

    Details

    • Type: Bug Bug
    • Status: Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: resourcemanager, scheduler
    • Labels:
      None

      Description

      • AM sends request asking 4 containers on host H1.
      • Asynchronously, host H1 reaches RM and gets assigned 4 containers. RM at this point, sets the value against H1 to
        zero in its aggregate request-table for all apps.
      • In the mean-while AM gets to need 3 more containers, so a total of 7 including the 4 from previous request.
      • Today, AM sends the absolute number of 7 against H1 to RM as part of its request table.
      • RM seems to be overriding its earlier value of zero against H1 to 7 against H1. And thus allocating 7 more
        containers.
      • AM already gets 4 in this scheduling iteration, but gets 7 more, a total of 11 instead of the required 7.
      1. YARN-110.patch
        6 kB
        Arun C Murthy

        Issue Links

          Activity

          Arun C Murthy created issue -
          Vinod Kumar Vavilapalli made changes -
          Field Original Value New Value
          Component/s mrv2 [ 12314301 ]
          Arun C Murthy made changes -
          Assignee Sharad Agarwal [ sharadag ] Arun C Murthy [ acmurthy ]
          Arun C Murthy made changes -
          Status Open [ 1 ] Resolved [ 5 ]
          Resolution Not A Problem [ 8 ]
          Vinod Kumar Vavilapalli made changes -
          Resolution Not A Problem [ 8 ]
          Status Resolved [ 5 ] Reopened [ 4 ]
          Assignee Arun C Murthy [ acmurthy ]
          Anupam Seth made changes -
          Assignee Anupam Seth [ anupamseth ]
          Anupam Seth made changes -
          Assignee Anupam Seth [ anupamseth ]
          Arun C Murthy made changes -
          Project Hadoop Map/Reduce [ 12310941 ] Hadoop YARN [ 12313722 ]
          Key MAPREDUCE-2694 YARN-110
          Component/s resourcemanager [ 12319322 ]
          Component/s scheduler [ 12319325 ]
          Component/s mrv2 [ 12314301 ]
          Arun C Murthy made changes -
          Assignee Arun C Murthy [ acmurthy ]
          Karthik Kambatla (Inactive) made changes -
          Link This issue is related to MAPREDUCE-4671 [ MAPREDUCE-4671 ]
          Arun C Murthy made changes -
          Attachment YARN-110.patch [ 12546372 ]
          Arun C Murthy made changes -
          Status Reopened [ 4 ] Patch Available [ 10002 ]
          Arun C Murthy made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]

            People

            • Assignee:
              Arun C Murthy
              Reporter:
              Arun C Murthy
            • Votes:
              0 Vote for this issue
              Watchers:
              22 Start watching this issue

              Dates

              • Created:
                Updated:

                Development