Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-3532

When 0 is provided as port number in yarn.nodemanager.webapp.address, NMs webserver component picks up random port, NM keeps on Reporting 0 port to RM

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 0.23.1
    • Fix Version/s: 0.23.1
    • Component/s: mrv2, nodemanager
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Modified NM to report correct http address when an ephemeral web port is configured.

      Description

      I tried following -:
      yarn.nodemanager.address=0.0.0.0:0
      yarn.nodemanager.webapp.address=0.0.0.0:0
      yarn.nodemanager.localizer.address=0.0.0.0:0
      mapreduce.shuffle.port=0

      When 0 is provided as number in yarn.nodemanager.webapp.address.
      NM instantiate WebServer as 0 piort e.g.

      2011-12-08 11:33:02,467 INFO org.apache.hadoop.yarn.server.nodemanager.webapp.WebServer: Instantiating NMWebApp at 0.0.0.0:0
      

      After that WebServer pick up some random port e.g.

      2011-12-08 11:33:02,562 INFO org.apache.hadoop.http.HttpServer: Jetty bound to port 36272
      2011-12-08 11:33:02,562 INFO org.mortbay.log: jetty-6.1.26
      2011-12-08 11:33:02,831 INFO org.mortbay.log: Started SelectChannelConnector@0.0.0.0:36272
      2011-12-08 11:33:02,831 INFO org.apache.hadoop.yarn.webapp.WebApps: Web app /node started at 36272
      

      And NM WebServer responds correctly but
      RM's cluster/Nodes page shows the following -:

      /Rack RUNNING NM:57963 NM:0 Healthy 8-Dec-2011 11:33:01 Healthy 8 12 GB 0 KB
      

      Whereas NM:0 is not clickable.
      Seems even NM's webserver pick random port but it never gets updated and so NM report 0 as HTTP port to RM causing NM Hyperlinks un-clickable
      But verified that MR job runs successfully with random.

        Attachments

        1. MAPREDUCE-3532-1.patch
          4 kB
          Bhallamudi Venkata Siva Kamesh
        2. MAPREDUCE-3532.patch
          1 kB
          Bhallamudi Venkata Siva Kamesh

          Activity

            People

            • Assignee:
              kamesh Bhallamudi Venkata Siva Kamesh
              Reporter:
              karams Karam Singh
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: