Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-10229

[Federation] Client should be able to submit application to RM directly using normal client conf

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 3.1.1
    • Fix Version/s: 3.4.0, 3.3.1
    • Component/s: amrmproxy, federation
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed
    • Flags:
      Patch

      Description

      Scenario: When enable the yarn federation feature with multi yarn clusters, one can submit their job to yarn-router by modified their client configuration with yarn router address.

      But if one still wants to submit their jobs via the original client (before enable federation) to RM directly, it will encounter the AMRMToken exception.  That means once enable federation ,if some one want to submit job, they have to  modify the client conf.

       

      one possible solution for this Scenario is:

      In NodeManger, when the client ApplicationMaster request comes:

      • get the client job.xml  from HDFS "".
      • parse the "yarn.resourcemanager.scheduler.address" parameter in job.xml
      • if the value of the parameter is "localhost:8049"(AMRM address),then do the AMRMToken valid process
      • if the value of the parameter is "rm:port"(rm address),then skip the AMRMToken valid process

       

       

       

        Attachments

        1. YARN-10229.008.patch
          5 kB
          Bilwa S T
        2. YARN-10229.007.patch
          5 kB
          Bilwa S T
        3. YARN-10229.006.patch
          14 kB
          Bilwa S T
        4. YARN-10229.005.patch
          15 kB
          Bilwa S T
        5. YARN-10229.004.patch
          15 kB
          Bilwa S T
        6. YARN-10229.003.patch
          15 kB
          Bilwa S T
        7. YARN-10229.002.patch
          15 kB
          Bilwa S T
        8. YARN-10229.001.patch
          15 kB
          Bilwa S T

          Activity

            People

            • Assignee:
              BilwaST Bilwa S T
              Reporter:
              122512597@qq.com JohnsonGuo
            • Votes:
              1 Vote for this issue
              Watchers:
              12 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: