Uploaded image for project: 'REEF'
  1. REEF
  2. REEF-1747

Workaround to allow two AMs in one YARN container

    XMLWordPrintableJSON

    Details

      Description

      The current version of YARN Java client does not allow us to register two Application Masters running in the same process. Technically, YARN Resource Manager does not care which process each AM runs in. However, there is a problem with the YARN Java client implementation: this library contains a singleton UserGroupInformation object that holds the user credentials of the current RM session. This data structure is shared by all AMs, and when REEF application tries to register the second (unmanaged) AM, the client library presents to YARN RM all credentials, including the security token of the first (managed) AM. YARN rejects such registration request, throwing InvalidApplicationMasterRequestException "Application Master is already registered".

      A proper fix for this issue would be a patch for Hadoop YARN Java client, that would allow us to pass the required security token into the AMRMClientAsync.registerApplicationMaster() call. We also need a quick workaround in REEF so we can run REEF-on-REEF and Spark+REEF applications using unpatched Hadoop libraries.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                motus Sergiy Matusevych
                Reporter:
                motus Sergiy Matusevych
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - 168h
                  168h
                  Remaining:
                  Remaining Estimate - 168h
                  168h
                  Logged:
                  Time Spent - Not Specified
                  Not Specified