Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-5597 YARN Federation improvements
  3. YARN-8697

LocalityMulticastAMRMProxyPolicy should fallback to random sub-cluster when cannot resolve resource

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.2.0
    • Component/s: None
    • Labels:
      None

      Description

      Right now in LocalityMulticastAMRMProxyPolicy, whenever we cannot resolve the resource name (node or rack), we always route the request to home sub-cluster. However, home sub-cluster might not be always be ready to use (timed out YARN-8581) or enabled (by AMRMProxyPolicy weights). It might also be overwhelmed by the requests if sub-cluster resolver has some issue. In this Jira, we are changing it to pick a random active and enabled sub-cluster for resource request we cannot resolve.

        Attachments

        1. YARN-8697.v1.patch
          13 kB
          Botong Huang
        2. YARN-8697.v2.patch
          13 kB
          Botong Huang

          Activity

            People

            • Assignee:
              botong Botong Huang
              Reporter:
              botong Botong Huang
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: