Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-420

Enable the RM to work with AM's that are not managed by it

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    Description

      Currently, the RM itself manages the AM by allocating a container for it and negotiating the launch on the NodeManager and manages the AM lifecycle. Thereafter, the AM negotiates resources with the RM and launches tasks to do the real work.
      It would be a useful improvement to enhance this model by allowing the AM to be launched independently by the client without requiring the RM. These AM's would be launched on a gateway machine that can talk to the cluster. This would open up new use cases such as the following
      1) Easy debugging of AM, specially during initial development. Having the AM launched on an arbitrary cluster node makes it hard to looks at logs or attach a debugger to the AM. If it can be launched locally then these tasks would be easier.
      2) Running AM's that need special privileges that may not be available on machines managed by the NodeManager

      Attachments

        1. MAPREDUCE-4427-3.patch
          56 kB
          Bikas Saha
        2. MAPREDUCE-4427-2.patch
          57 kB
          Bikas Saha
        3. MAPREDUCE-4427-1.patch
          58 kB
          Bikas Saha

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            bikassaha Bikas Saha
            bikassaha Bikas Saha
            Votes:
            0 Vote for this issue
            Watchers:
            17 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment