Pig
  1. Pig
  2. PIG-2429

Better error message in pig_*.log

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 0.9.0
    • Fix Version/s: 0.9.0
    • Component/s: None
    • Labels:
      None

      Description

      Hi,

      The below script sometimes fails giving null error message and some times with correct error message in the pig_*.log

      Script

      register 'a.py' using jython as test;
      A = load 'test.txt' as (x:chararray);
      B = foreach A generate
              test.a(x) as y;
      C = group B by (y);
      store C into 'C';
      

      where a.py is

      @outputSchema("n:chararray")
      def a(n):
        n = n.lower();
        return n;
      

      input- test.txt
      =======
      a

      =======

      I have not put null check in a.py so that the script would error out.

      This script when run with 0.9 version, fails printing in the pig_*.log either null error message or the correct error message

      Null Error message

      Pig Stack Trace
      ---------------
      ERROR 2244: Job failed, hadoop does not return any error message
      
      org.apache.pig.backend.executionengine.ExecException: ERROR 2244: Job failed, hadoop does not return any error message
              at org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:139)
              at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:192)
              at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:164)
              at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:81)
              at org.apache.pig.Main.run(Main.java:561)
              at org.apache.pig.Main.main(Main.java:111)
      

      Correct Error message

      Backend error message
      ---------------------
      org.apache.pig.backend.executionengine.ExecException: ERROR 0: Error executing function
              at org.apache.pig.scripting.jython.JythonFunction.exec(JythonFunction.java:106)
              at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:216)
              at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:305)
              at org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.getNext(PhysicalOperator.java:322)
              at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:332)
              at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:284)
              at org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:290)
              at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLocalRearrange.getNext(POLocalRearrange.java:256)
              at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.runPipeline(PigGenericMapBase.java:267)
              at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:262)
              at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:64)
              at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
              at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
              at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
              at org.apache.hadoop.mapred.Child$4.run(Child.java:261)
              at java.security.AccessController.doPrivileged(Native Method)
              at javax.security.auth.Subject.doAs(Subject.java:396)
              at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059)
              at org.apache.hadoop.mapred.Child.main(Child.java:255)
      Caused by: Traceback (most recent call last):
        File "a.py", line 5, in n
      AttributeError: 'NoneType' object has no attribute 'lower'
      
              at org.python.core.PyException.fillInStackTrace(PyException.java:70)
              at java.lang.Throwable.<init>(Throwable.java:181)
              at java.lang.Exception.<init>(Exception.java:29)
              at java.lang.RuntimeException.<init>(RuntimeException.java:32)
              at org.python.core.PyException.<init>(PyException.java:46)
              at org.python.core.PyException.<init>(PyException.java:43)
              at org.python.core.PyException.<init>(PyException.java:61)
              at org.python.core.Py.AttributeError(Py.java:145)
              at org.python.core.PyObject.noAttributeError(PyObject.java:936)
              at org.python.core.PyObject.__getattr__(PyObject.java:931)
              at org.python.pycode._pyx3.n$1(a.py:6)
              at org.python.pycode._pyx3.call_function(a.py)
              at org.python.core.PyTableCode.call(PyTableCode.java:165)
              at org.python.core.PyBaseCode.call(PyBaseCode.java:297)
              at org.python.core.PyFunction.__call__(PyFunction.java:370)
              at org.python.core.PyFunction.__call__(PyFunction.java:360)
              at org.python.core.PyFunction.__call__(PyFunction.java:355)
              at org.apache.pig.scripting.jython.JythonFunction.exec(JythonFunction.java:103)
              ... 18 more
      		
      

      Putting a couple of sysout in the code, i found it was happening when in Launcher.getStats

      TaskReport[] mapRep = jobClient.getMapTaskReports(MRJobID);
      getErrorMessages(mapRep, "map", errNotDbg, pigContext);
      totalHadoopTimeSpent += computeTimeSpent(mapRep);
      

      whenever mapRed becomes null, it gives null error message.

      Can it be made better?

      Regards,
      Anitha

        Issue Links

          Activity

            People

            • Assignee:
              xuting zhao
              Reporter:
              Anitha Raju
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development