Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-6351

Have RM match relaxedLocality request via time instead of missedOpportunities

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Open
    • Major
    • Resolution: Unresolved
    • None
    • None
    • capacityscheduler, yarn
    • None

    Description

      When using relaxLocality=true, the current CapacityScheduler strategy is to wait a certain amount of missedOpportunities to schedule a request in a node, a rack or off_switch. This means that the missedOpportunities param is dependent on the number of nodes in the cluster and the duration of each container.

      A different strategy would be to wait a configurable amount of time before deciding to go to different location.

      This JIRA proposal is to extract the current behavior into a pluggable strategy pattern and create a new strategy that is simply based on time.

      Attachments

        Activity

          People

            Unassigned Unassigned
            roniburd Roni Burd
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated: