Uploaded image for project: 'Chukwa'
  1. Chukwa
  2. CHUKWA-40

NPE in ChukwaJobTrackerInstrumentation.completeJob()

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None
    1. CHUKWA-40.patch
      0.6 kB
      Jerome Boulon

      Activity

      Hide
      vinodkv Vinod Kumar Vavilapalli added a comment -

      I see a lot of these in the JobTracker's log as well as stdout, when ChukwaJobTrackerInstrumentation is used for JT.

      2009-03-19 04:08:57,252 WARN org.apache.hadoop.mapred.JobTrackerInstrumentation: could not remove adaptor for this job: job_200903180931_1837
      java.lang.NullPointerException
              at org.apache.hadoop.chukwa.datacollection.controller.ChukwaAgentController.remove(ChukwaAgentController.java:259)
              at org.apache.hadoop.mapred.ChukwaJobTrackerInstrumentation.completeJob(ChukwaJobTrackerInstrumentation.java:93)
              at org.apache.hadoop.mapred.JobInProgress.jobComplete(JobInProgress.java:2109)
              at org.apache.hadoop.mapred.JobInProgress.completedTask(JobInProgress.java:2050)
              at org.apache.hadoop.mapred.JobInProgress.updateTaskStatus(JobInProgress.java:867)
              at org.apache.hadoop.mapred.JobTracker.updateTaskStatuses(JobTracker.java:3252)
              at org.apache.hadoop.mapred.JobTracker.processHeartbeat(JobTracker.java:2553)
              at org.apache.hadoop.mapred.JobTracker.heartbeat(JobTracker.java:2348)
              at sun.reflect.GeneratedMethodAccessor10.invoke(Unknown Source)
              at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
              at java.lang.reflect.Method.invoke(Method.java:597)
              at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:508)
              at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:959)
              at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:955)
              at java.security.AccessController.doPrivileged(Native Method)
              at javax.security.auth.Subject.doAs(Subject.java:396)
              at org.apache.hadoop.ipc.Server$Handler.run(Server.java:953)
      

      The instrumentation update arises out of JobTracker's heartbeat, but the hearbeat doesn't seem to be affected as all Throwables are caught inside the method.
      Don't know for sure if this exception causes some problems with metrics or even if this is harmful.

      Show
      vinodkv Vinod Kumar Vavilapalli added a comment - I see a lot of these in the JobTracker's log as well as stdout, when ChukwaJobTrackerInstrumentation is used for JT. 2009-03-19 04:08:57,252 WARN org.apache.hadoop.mapred.JobTrackerInstrumentation: could not remove adaptor for this job: job_200903180931_1837 java.lang.NullPointerException at org.apache.hadoop.chukwa.datacollection.controller.ChukwaAgentController.remove(ChukwaAgentController.java:259) at org.apache.hadoop.mapred.ChukwaJobTrackerInstrumentation.completeJob(ChukwaJobTrackerInstrumentation.java:93) at org.apache.hadoop.mapred.JobInProgress.jobComplete(JobInProgress.java:2109) at org.apache.hadoop.mapred.JobInProgress.completedTask(JobInProgress.java:2050) at org.apache.hadoop.mapred.JobInProgress.updateTaskStatus(JobInProgress.java:867) at org.apache.hadoop.mapred.JobTracker.updateTaskStatuses(JobTracker.java:3252) at org.apache.hadoop.mapred.JobTracker.processHeartbeat(JobTracker.java:2553) at org.apache.hadoop.mapred.JobTracker.heartbeat(JobTracker.java:2348) at sun.reflect.GeneratedMethodAccessor10.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:508) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:959) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:955) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:953) The instrumentation update arises out of JobTracker's heartbeat, but the hearbeat doesn't seem to be affected as all Throwables are caught inside the method. Don't know for sure if this exception causes some problems with metrics or even if this is harmful.
      Hide
      eyang Eric Yang added a comment -

      This is not harmful. This exception happens when a job history or job conf file was not streamed over to Chukwa. The most likely cause is that the file does not exist for streaming. However, the ChukwaAgentController should check for null pointer.

      Show
      eyang Eric Yang added a comment - This is not harmful. This exception happens when a job history or job conf file was not streamed over to Chukwa. The most likely cause is that the file does not exist for streaming. However, the ChukwaAgentController should check for null pointer.
      Hide
      jboulon Jerome Boulon added a comment -

      test for null before calling a.unregister();

      Show
      jboulon Jerome Boulon added a comment - test for null before calling a.unregister();
      Hide
      eyang Eric Yang added a comment -

      +1 Looks good.

      Show
      eyang Eric Yang added a comment - +1 Looks good.
      Hide
      eyang Eric Yang added a comment -

      I just committed this. Thanks Jerome.

      Show
      eyang Eric Yang added a comment - I just committed this. Thanks Jerome.

        People

        • Assignee:
          jboulon Jerome Boulon
          Reporter:
          vinodkv Vinod Kumar Vavilapalli
        • Votes:
          0 Vote for this issue
          Watchers:
          0 Start watching this issue

          Dates

          • Created:
            Updated:
            Resolved:

            Development