Uploaded image for project: 'Slider'
  1. Slider
  2. SLIDER-955

fail to track the outstandingRequest when submit an application that yarn.memory is not a multiple of minimum-allocation-mb

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: Slider 0.80
    • Fix Version/s: Slider 0.90.2
    • Component/s: appmaster
    • Labels:
      None

      Description

      "yarn.scheduler.minimum-allocation-mb” is set to 256 in my cluster.
      I submit an application that “yarn.memory” is set to 7169 (not a multiple of 256)
      the resource of allocated container will be 7424(256*29) memory because the resource requirement is normalized to a multiple of "yarn.scheduler.minimum-allocation-mb” by RM.
      the resource requirement of outstandingRequest that is tracked by Slider AM does not match the resource of the allocated container.
      therefore, Slider AM can’t clean up the outstandingRequest and requests in AMRMClient.

      I think it should fix as following 1 or 2
      1. an application should be able to submit only if “yarn.memory” is a multiple of "yarn.scheduler.minimum-allocation-mb”
      2. the resource requirement of outstandingRequest is normalized to a multiple of "yarn.scheduler.minimum-allocation-mb”

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                stevel@apache.org Steve Loughran
                Reporter:
                kyungwan nam kyungwan nam
              • Votes:
                1 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: