Uploaded image for project: 'REEF (Retired)'
  1. REEF (Retired)
  2. REEF-1669

Removing YarnClient from YarnContainerManager

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 0.16
    • REEF
    • None

    Description

      Currently we have both YarnClient and resourceManager (AMRMClientAsync) in YarnContainerManager. We call init() and start() for both of them. But inside driver, we are not able to get correct YarnConfiguration for YarnClient, that makes yarnClient.getNodeReports() fail to connect to RM.

      In YarnContainerManager, we already have onNodesUpdated() which gives the list of the NodeReports on the fly. yarnClient.getNodeReports() was called at beginning and it only gives a static list. So looks like we don't need to call it anyway.

      Based on RM folks, REEF should not use YarnClient at driver side. So the proposal it to to remove YarnClient from YarnContainerManager.

      Attachments

        Activity

          People

            juliaw Julia Wang
            juliaw Julia Wang
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: