Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-6338

MR AppMaster does not honor ephemeral port range

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 2.6.0
    • 2.9.0, 3.0.0-alpha4
    • mr-am, mrv2
    • None
    • Reviewed

    Description

      1. The MR AppMaster should only use port ranges defined in the yarn.app.mapreduce.am.job.client.port-range property. On initial startup of the MRAppMaster, it does use the port range defined in the property. However, it also opens up a listener on a random ephemeral port. This is not the Jetty listener. It is another listener opened by the MRAppMaster via another thread and is recognized by the RM. Other nodes will try to communicate to it via that random port. With firewall settings on, the MR job will fail because the random port is not opened. This problem has caused others to have all OS ephemeral ports opened to have MR jobs run.

      This is related to MAPREDUCE-4079

      Attachments

        1. MAPREDUCE-6338.003.patch
          1 kB
          John
        2. MAPREDUCE-6338.002.patch
          3 kB
          Frank Nguyen

        Issue Links

          Activity

            People

              bleuleon Frank Nguyen
              bleuleon Frank Nguyen
              Votes:
              5 Vote for this issue
              Watchers:
              14 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: