Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-3030

RM is not processing heartbeat and continuously giving the message 'Node not found rebooting'

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Blocker Blocker
    • Resolution: Fixed
    • Affects Version/s: 0.23.0
    • Fix Version/s: 0.23.0, 0.24.0
    • Component/s: mrv2, resourcemanager
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Node Manager Logs
      2011-09-19 13:39:29,816 INFO  webapp.WebApps (WebApps.java:start(162)) - Registered webapp guice modules
      2011-09-19 13:39:29,817 INFO  service.AbstractService (AbstractService.java:start(61)) - Service:org.apache.hadoop.yarn.server.nodemanager.webapp.WebServer is started.
      2011-09-19 13:39:29,818 INFO  service.AbstractService (AbstractService.java:start(61)) - Service:Dispatcher is started.
      2011-09-19 13:39:29,819 INFO  nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:start(133)) - Configured ContainerManager Address is 10.18.52.124:45454
      2011-09-19 13:39:29,819 INFO  ipc.YarnRPC (YarnRPC.java:create(47)) - Creating YarnRPC for org.apache.hadoop.yarn.ipc.HadoopYarnProtoRPC
      2011-09-19 13:39:29,822 INFO  ipc.HadoopYarnRPC (HadoopYarnProtoRPC.java:getProxy(49)) - Creating a HadoopYarnProtoRpc proxy for protocol interface org.apache.hadoop.yarn.server.api.ResourceTracker
      2011-09-19 13:39:29,862 INFO  nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:registerWithRM(165)) - Connected to ResourceManager at 0.0.0.0:8025
      2011-09-19 13:39:30,369 INFO  nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:registerWithRM(189)) - Registered with ResourceManager as 10.18.52.124:45454 with total resource of memory: 8192, 
      2011-09-19 13:39:30,369 INFO  service.AbstractService (AbstractService.java:start(61)) - Service:org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl is started.
      2011-09-19 13:39:30,371 INFO  service.AbstractService (AbstractService.java:start(61)) - Service:org.apache.hadoop.yarn.server.nodemanager.NodeManager is started.
      
      Resource Manager Logs
      2011-09-19 14:01:03,238 INFO  resourcemanager.ResourceTrackerService (ResourceTrackerService.java:nodeHeartbeat(201)) - Node not found rebooting 10.18.52.124:45454
      Call: protocol=org.apache.hadoop.yarn.proto.ResourceTracker$ResourceTrackerService$BlockingInterface, method=nodeHeartbeat
      2011-09-19 14:01:04,240 INFO  resourcemanager.ResourceTrackerService (ResourceTrackerService.java:nodeHeartbeat(201)) - Node not found rebooting 10.18.52.124:45454
      Call: protocol=org.apache.hadoop.yarn.proto.ResourceTracker$ResourceTrackerService$BlockingInterface, method=nodeHeartbeat
      2011-09-19 14:01:05,242 INFO  resourcemanager.ResourceTrackerService (ResourceTrackerService.java:nodeHeartbeat(201)) - Node not found rebooting 10.18.52.124:45454
      Call: protocol=org.apache.hadoop.yarn.proto.ResourceTracker$ResourceTrackerService$BlockingInterface, method=nodeHeartbeat
      2011-09-19 14:01:06,244 INFO  resourcemanager.ResourceTrackerService (ResourceTrackerService.java:nodeHeartbeat(201)) - Node not found rebooting 10.18.52.124:45454
      Call: protocol=org.apache.hadoop.yarn.proto.ResourceTracker$ResourceTrackerService$BlockingInterface, method=nodeHeartbeat
      2011-09-19 14:01:07,246 INFO  resourcemanager.ResourceTrackerService (ResourceTrackerService.java:nodeHeartbeat(201)) - Node not found rebooting 10.18.52.124:45454
      Call: protocol=org.apache.hadoop.yarn.proto.ResourceTracker$ResourceTrackerService$BlockingInterface, method=nodeHeartbeat
      2011-09-19 14:01:08,247 INFO  resourcemanager.ResourceTrackerService (ResourceTrackerService.java:nodeHeartbeat(201)) - Node not found rebooting 10.18.52.124:45454
      

      Node Manager is registered with Resource manager and the for every heartbeat, it is printing the above message.

        Issue Links

          Activity

          Devaraj K created issue -
          Vinod Kumar Vavilapalli made changes -
          Field Original Value New Value
          Link This issue is related to MAPREDUCE-2965 [ MAPREDUCE-2965 ]
          Vinod Kumar Vavilapalli made changes -
          Fix Version/s 0.23.0 [ 12315570 ]
          Affects Version/s 0.23.0 [ 12315570 ]
          Affects Version/s 0.24.0 [ 12317654 ]
          Component/s mrv2 [ 12314301 ]
          Devaraj K made changes -
          Attachment MAPREDUCE-3030.patch [ 12495062 ]
          Devaraj K made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Fix Version/s 0.24.0 [ 12317654 ]
          Vinod Kumar Vavilapalli made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Hadoop Flags [Reviewed]
          Resolution Fixed [ 1 ]
          Arun C Murthy made changes -
          Status Resolved [ 5 ] Closed [ 6 ]

            People

            • Assignee:
              Devaraj K
              Reporter:
              Devaraj K
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development