Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-3020

n similar addContainerRequest()s produce n*(n+1)/2 containers

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Duplicate
    • 2.5.0, 2.6.0, 2.5.1, 2.5.2
    • None
    • client
    • None

    Description

      BUG: If the application master calls addContainerRequest() n times, but with the same priority, I get up to 1+2+3+...+n containers = n*(n+1)/2 . The most containers are requested when the interval between calls to addContainerRequest() exceeds the heartbeat interval of calls to allocate() (in AMRMClientImpl's run() method).

      If the application master calls addContainerRequest() n times, but with a unique priority each time, I get n containers (as I intended).

      Analysis:
      There is a logic problem in AMRMClientImpl.java.
      Although AMRMClientImpl.java, allocate() does an ask.clear() , on subsequent calls to addContainerRequest(), addResourceRequest() finds the previous matching remoteRequest and increments the container count rather than starting anew, and does an addResourceRequestToAsk() which defeats the ask.clear().

      From documentation and code comments, it was hard for me to discern the intended behavior of the API, but the inconsistency reported in this issue suggests one case or the other is implemented incorrectly.

      Attachments

        Activity

          People

            Unassigned Unassigned
            peterdkirchner Peter D Kirchner
            Votes:
            0 Vote for this issue
            Watchers:
            13 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Time Tracking

                Estimated:
                Original Estimate - 24h
                24h
                Remaining:
                Remaining Estimate - 24h
                24h
                Logged:
                Time Spent - Not Specified
                Not Specified