Uploaded image for project: 'Pig'
  1. Pig
  2. PIG-2533

Pig MR job exceptions masked on frontend

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.10.0, 0.11
    • Component/s: None
    • Labels:
      None

      Description

      Pig MR jobs failures produce misleading error messages because just the first line of the error message is reported. Printing the whole error message is very useful when debugging issues as the current version is very misleading about the source of an issue.

      PRINTING WHOLE ERROR MESSAGE:

      2012-02-14 21:55:53,936 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 6017: java.io.IOException: Deserialization error: org.apache.hcatalog.data.schema.HCatSchema
      	at org.apache.pig.impl.util.ObjectSerializer.deserialize(ObjectSerializer.java:55)
      	at org.apache.pig.impl.util.UDFContext.deserialize(UDFContext.java:181)
      	at org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil.setupUDFContext(MapRedUtil.java:159)
      	at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.setupUdfEnvAndStores(PigOutputFormat.java:229)
      	at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.checkOutputSpecs(PigOutputFormat.java:186)
      	at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:811)
      	at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:771)
      	at org.apache.hadoop.mapred.jobcontrol.Job.submit(Job.java:378)
      	at org.apache.hadoop.mapred.jobcontrol.JobControl.startReadyJobs(JobControl.java:247)
      	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
      	at java.lang.reflect.Method.invoke(Method.java:597)
      	at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigJobControl.mainLoopAction(PigJobControl.java:144)
      	at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigJobControl.run(PigJobControl.java:121)
      	at java.lang.Thread.run(Thread.java:662)
      Caused by: java.lang.ClassNotFoundException: org.apache.hcatalog.data.schema.HCatSchema
      	at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
      	at java.security.AccessController.doPrivileged(Native Method)
      	at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
      	at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
      	at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
      	at java.lang.Class.forName0(Native Method)
      	at java.lang.Class.forName(Class.java:247)
      	at java.io.ObjectInputStream.resolveClass(ObjectInputStream.java:603)
      	at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1574)
      	at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1495)
      	at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1731)
      	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1328)
      	at java.io.ObjectInputStream.readObject(ObjectInputStream.java:350)
      	at java.util.Hashtable.readObject(Hashtable.java:859)
      	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
      	at java.lang.reflect.Method.invoke(Method.java:597)
      	at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:974)
      	at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1848)
      	at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1752)
      	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1328)
      	at java.io.ObjectInputStream.readObject(ObjectInputStream.java:350)
      	at java.util.HashMap.readObject(HashMap.java:1030)
      	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
      	at java.lang.reflect.Method.invoke(Method.java:597)
      	at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:974)
      	at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1848)
      	at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1752)
      	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1328)
      	at java.io.ObjectInputStream.readObject(ObjectInputStream.java:350)
      	at org.apache.pig.impl.util.ObjectSerializer.deserialize(ObjectSerializer.java:53)
      	... 15 more
      

      CURRENT ERROR MESSAGE:

      Pig Stack Trace
      ---------------
      ERROR 6017: java.io.IOException: Deserialization error: org.apache.hcatalog.data.schema.HCatSchema
      
      org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066: Unable to open iterator for alias l
              at org.apache.pig.PigServer.openIterator(PigServer.java:857)
              at org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:655)
              at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:303)
              at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:188)
              at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:164)
              at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:84)
              at org.apache.pig.Main.run(Main.java:561)
              at org.apache.pig.Main.main(Main.java:111)
              at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
              at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
              at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
              at java.lang.reflect.Method.invoke(Method.java:597)
              at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
      Caused by: org.apache.pig.PigException: ERROR 1002: Unable to store alias l
              at org.apache.pig.PigServer.storeEx(PigServer.java:956)
              at org.apache.pig.PigServer.store(PigServer.java:919)
              at org.apache.pig.PigServer.openIterator(PigServer.java:832)
              ... 12 more
      Caused by: org.apache.pig.backend.executionengine.ExecException: ERROR 6017: java.io.IOException: Deserialization error: org.apache.hcatalog.data.schema.HCatSchema
              at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:301)
              at org.apache.pig.PigServer.launchPlan(PigServer.java:1270)
              at org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1255)
              at org.apache.pig.PigServer.storeEx(PigServer.java:952)
              ... 14 more
      ================================================================================
      

        Attachments

        1. PIG-2533.patch
          3 kB
          Travis Crawford
        2. PIG-2533.patch
          0.8 kB
          Travis Crawford

          Activity

            People

            • Assignee:
              traviscrawford Travis Crawford
              Reporter:
              traviscrawford Travis Crawford
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: