Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-3965

JobConf.findContainingJar NPEs if it can't find the classloader of a class

    Details

    • Type: Bug Bug
    • Status: Open
    • Priority: Minor Minor
    • Resolution: Unresolved
    • Affects Version/s: 0.23.0, 1.0.0, 2.0.0-alpha
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None
    • Environment:

      Setting up a job under Groovy, OSX10.7, job started by ant's <java> with fork==true

      Description

      I never knew that classes may not have classloaders, but it turns out they can -in which case Object.getClass().getClassLoader() will be null "This method will return null in such implementations if this class was loaded by the bootstrap class loader.""

      When this (rare) situation arises, JobConf.findContainingJar() NPEs, when failing with a slightly more useful error message would be preferred.

        Activity

        Hide
        Steve Loughran added a comment -

        stack trace

        
             [java] Exception in thread "main" java.lang.NullPointerException
             [java] 	at org.apache.hadoop.mapred.JobConf.findContainingJar(JobConf.java:1792)
             [java] 	at org.apache.hadoop.mapred.JobConf.setJarByClass(JobConf.java:426)
             [java] 	at org.apache.hadoop.mapreduce.Job.setJarByClass(Job.java:133)
             [java] 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
             [java] 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
             [java] 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
             [java] 	at java.lang.reflect.Method.invoke(Method.java:597)
             [java] 	at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:90)
             [java] 	at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:233)
             [java] 	at groovy.lang.MetaClassImpl.setProperty(MetaClassImpl.java:2388)
             [java] 	at groovy.lang.MetaClassImpl.setProperty(MetaClassImpl.java:3312)
             [java] 	at org.smartfrog.services.hadoop.grumpy.GrumpyJob.setProperty(GrumpyJob.groovy)
             [java] 	at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.setGroovyObjectProperty(ScriptBytecodeAdapter.java:528)
             [java] 	at org.smartfrog.services.hadoop.bluemine.jobs.BluemineJob.createBasicJob(BluemineJob.groovy:51)
             [java] 	at org.smartfrog.services.hadoop.bluemine.jobs.BluemineJob$createBasicJob.call(Unknown Source)
             [java] 	at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:42)
             [java] 	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:108)
             [java] 	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:128)
             [java] 	at org.smartfrog.services.hadoop.bluemine.jobs.DevCount.main(DevCount.groovy:95)
        
        Show
        Steve Loughran added a comment - stack trace [java] Exception in thread "main" java.lang.NullPointerException [java] at org.apache.hadoop.mapred.JobConf.findContainingJar(JobConf.java:1792) [java] at org.apache.hadoop.mapred.JobConf.setJarByClass(JobConf.java:426) [java] at org.apache.hadoop.mapreduce.Job.setJarByClass(Job.java:133) [java] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [java] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) [java] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [java] at java.lang.reflect.Method.invoke(Method.java:597) [java] at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:90) [java] at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:233) [java] at groovy.lang.MetaClassImpl.setProperty(MetaClassImpl.java:2388) [java] at groovy.lang.MetaClassImpl.setProperty(MetaClassImpl.java:3312) [java] at org.smartfrog.services.hadoop.grumpy.GrumpyJob.setProperty(GrumpyJob.groovy) [java] at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.setGroovyObjectProperty(ScriptBytecodeAdapter.java:528) [java] at org.smartfrog.services.hadoop.bluemine.jobs.BluemineJob.createBasicJob(BluemineJob.groovy:51) [java] at org.smartfrog.services.hadoop.bluemine.jobs.BluemineJob$createBasicJob.call(Unknown Source) [java] at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:42) [java] at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:108) [java] at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:128) [java] at org.smartfrog.services.hadoop.bluemine.jobs.DevCount.main(DevCount.groovy:95)
        Hide
        Steve Loughran added a comment -

        This is triggered if the Class passed down to the map class is class java.lang.Class

        job.setJarByClass(Class.class);
        

        Groovy seems to trigger it for other reasons

        Show
        Steve Loughran added a comment - This is triggered if the Class passed down to the map class is class java.lang.Class job.setJarByClass( Class .class); Groovy seems to trigger it for other reasons
        Hide
        Steve Loughran added a comment -

        no, groovy triggers it for the same reason -user error.

        Propose

        1. a check in findContainingJar to handle a null class
          # if found, an IllegalArgumentException is raised -including the classname at fault
        2. a test that tries to set the class of a job to java.lang.Class and expects the relevant error
        Show
        Steve Loughran added a comment - no, groovy triggers it for the same reason -user error. Propose a check in findContainingJar to handle a null class # if found, an IllegalArgumentException is raised -including the classname at fault a test that tries to set the class of a job to java.lang.Class and expects the relevant error

          People

          • Assignee:
            Unassigned
            Reporter:
            Steve Loughran
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:

              Development