Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-3530

Sometimes NODE_UPDATE to the scheduler throws an NPE causing the scheduling to stop

    Details

    • Target Version/s:
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Fixed an NPE occuring during scheduling in the ResourceManager.

      Description

      Sometimes NODE_UPDATE to the scheduler throws NPE causes scheduling to stop but ResourceManager keeps on running.
      I have been observing intermitently for last 3 weeks.
      But with latest svn code. I tried to run sort twice and both times Job got stuck due to NPE.

      java.lang.NullPointerException
              at org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerApp.containerLaunchedOnNode(SchedulerApp.java:181)
              at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.containerLaunchedOnNode(CapacityScheduler.java:596)
              at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.nodeUpdate(CapacityScheduler.java:539)
              at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.handle(CapacityScheduler.java:617)
              at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.handle(CapacityScheduler.java:77)
              at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$SchedulerEventDispatcher$EventProcessor.run(ResourceManager.java:294)
              at java.lang.Thread.run(Thread.java:619)
      
      1. MAPREDUCE-3530.patch
        2 kB
        Arun C Murthy
      2. MAPREDUCE-3530.patch
        12 kB
        Arun C Murthy
      3. MAPREDUCE-3530-20111214.txt
        9 kB
        Vinod Kumar Vavilapalli

        Activity

        Karam Singh created issue -
        Mahadev konar made changes -
        Field Original Value New Value
        Priority Major [ 3 ] Blocker [ 1 ]
        Karam Singh made changes -
        Affects Version/s 0.23.1 [ 12318883 ]
        Arun C Murthy made changes -
        Assignee Vinod Kumar Vavilapalli [ vinodkv ]
        Arun C Murthy made changes -
        Assignee Vinod Kumar Vavilapalli [ vinodkv ] Arun C Murthy [ acmurthy ]
        Vinod Kumar Vavilapalli made changes -
        Summary Sometimes NODE_UPDATE to the scheduler throws NPE causes scheduling to stop Sometimes NODE_UPDATE to the scheduler throws an NPE causing the scheduling to stop
        Arun C Murthy made changes -
        Attachment MAPREDUCE-3530.patch [ 12507272 ]
        Arun C Murthy made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Target Version/s 0.23.1 [ 12318883 ]
        Arun C Murthy made changes -
        Status Patch Available [ 10002 ] Open [ 1 ]
        Arun C Murthy made changes -
        Attachment MAPREDUCE-3530.patch [ 12507322 ]
        Arun C Murthy made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Vinod Kumar Vavilapalli made changes -
        Status Patch Available [ 10002 ] Open [ 1 ]
        Vinod Kumar Vavilapalli made changes -
        Attachment MAPREDUCE-3530-20111214.txt [ 12507398 ]
        Vinod Kumar Vavilapalli made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Hadoop Flags Reviewed [ 10343 ]
        Fix Version/s 0.23.1 [ 12318883 ]
        Vinod Kumar Vavilapalli made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Release Note Fixed an NPE occuring during scheduling in the ResourceManager.
        Resolution Fixed [ 1 ]
        Arun C Murthy made changes -
        Status Resolved [ 5 ] Closed [ 6 ]

          People

          • Assignee:
            Arun C Murthy
            Reporter:
            Karam Singh
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development