Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-420

Enable the RM to work with AM's that are not managed by it

    XMLWordPrintableJSON

Details

    Description

      Currently, the RM itself manages the AM by allocating a container for it and negotiating the launch on the NodeManager and manages the AM lifecycle. Thereafter, the AM negotiates resources with the RM and launches tasks to do the real work.
      It would be a useful improvement to enhance this model by allowing the AM to be launched independently by the client without requiring the RM. These AM's would be launched on a gateway machine that can talk to the cluster. This would open up new use cases such as the following
      1) Easy debugging of AM, specially during initial development. Having the AM launched on an arbitrary cluster node makes it hard to looks at logs or attach a debugger to the AM. If it can be launched locally then these tasks would be easier.
      2) Running AM's that need special privileges that may not be available on machines managed by the NodeManager

      Attachments

        1. MAPREDUCE-4427-3.patch
          56 kB
          Bikas Saha
        2. MAPREDUCE-4427-2.patch
          57 kB
          Bikas Saha
        3. MAPREDUCE-4427-1.patch
          58 kB
          Bikas Saha

        Issue Links

          Activity

            People

              bikassaha Bikas Saha
              bikassaha Bikas Saha
              Votes:
              0 Vote for this issue
              Watchers:
              17 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: