Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-2915 Enable YARN RM scale out via federation using multiple RM's
  3. YARN-6955

Handle concurrent register AM requests in FederationInterceptor

    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.9.0, 3.0.0-beta1
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      The timeout between AM and AMRMProxy is shorter than the timeout + failOver between FederationInterceptor (AMRMProxy) and RM. When the first register thread in FI is blocked because of an RM failover, AM can timeout and resend register call, leading to two outstanding register call inside FI.

      Eventually when RM comes back up, one thread succeeds register and the other thread got an application already registered exception. FI should swallow the exception and return success back to AM in both threads.

        Attachments

        1. YARN-6955.v1.patch
          12 kB
          Botong Huang
        2. YARN-6955.v2.patch
          13 kB
          Botong Huang

          Activity

            People

            • Assignee:
              botong Botong Huang
              Reporter:
              botong Botong Huang
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: