Pig
  1. Pig
  2. PIG-2429

Better error message in pig_*.log

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 0.9.0
    • Fix Version/s: 0.9.0
    • Component/s: None
    • Labels:
      None

      Description

      Hi,

      The below script sometimes fails giving null error message and some times with correct error message in the pig_*.log

      Script

      register 'a.py' using jython as test;
      A = load 'test.txt' as (x:chararray);
      B = foreach A generate
              test.a(x) as y;
      C = group B by (y);
      store C into 'C';
      

      where a.py is

      @outputSchema("n:chararray")
      def a(n):
        n = n.lower();
        return n;
      

      input- test.txt
      =======
      a

      =======

      I have not put null check in a.py so that the script would error out.

      This script when run with 0.9 version, fails printing in the pig_*.log either null error message or the correct error message

      Null Error message

      Pig Stack Trace
      ---------------
      ERROR 2244: Job failed, hadoop does not return any error message
      
      org.apache.pig.backend.executionengine.ExecException: ERROR 2244: Job failed, hadoop does not return any error message
              at org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:139)
              at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:192)
              at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:164)
              at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:81)
              at org.apache.pig.Main.run(Main.java:561)
              at org.apache.pig.Main.main(Main.java:111)
      

      Correct Error message

      Backend error message
      ---------------------
      org.apache.pig.backend.executionengine.ExecException: ERROR 0: Error executing function
              at org.apache.pig.scripting.jython.JythonFunction.exec(JythonFunction.java:106)
              at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:216)
              at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:305)
              at org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.getNext(PhysicalOperator.java:322)
              at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:332)
              at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:284)
              at org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:290)
              at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLocalRearrange.getNext(POLocalRearrange.java:256)
              at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.runPipeline(PigGenericMapBase.java:267)
              at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:262)
              at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:64)
              at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
              at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
              at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
              at org.apache.hadoop.mapred.Child$4.run(Child.java:261)
              at java.security.AccessController.doPrivileged(Native Method)
              at javax.security.auth.Subject.doAs(Subject.java:396)
              at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059)
              at org.apache.hadoop.mapred.Child.main(Child.java:255)
      Caused by: Traceback (most recent call last):
        File "a.py", line 5, in n
      AttributeError: 'NoneType' object has no attribute 'lower'
      
              at org.python.core.PyException.fillInStackTrace(PyException.java:70)
              at java.lang.Throwable.<init>(Throwable.java:181)
              at java.lang.Exception.<init>(Exception.java:29)
              at java.lang.RuntimeException.<init>(RuntimeException.java:32)
              at org.python.core.PyException.<init>(PyException.java:46)
              at org.python.core.PyException.<init>(PyException.java:43)
              at org.python.core.PyException.<init>(PyException.java:61)
              at org.python.core.Py.AttributeError(Py.java:145)
              at org.python.core.PyObject.noAttributeError(PyObject.java:936)
              at org.python.core.PyObject.__getattr__(PyObject.java:931)
              at org.python.pycode._pyx3.n$1(a.py:6)
              at org.python.pycode._pyx3.call_function(a.py)
              at org.python.core.PyTableCode.call(PyTableCode.java:165)
              at org.python.core.PyBaseCode.call(PyBaseCode.java:297)
              at org.python.core.PyFunction.__call__(PyFunction.java:370)
              at org.python.core.PyFunction.__call__(PyFunction.java:360)
              at org.python.core.PyFunction.__call__(PyFunction.java:355)
              at org.apache.pig.scripting.jython.JythonFunction.exec(JythonFunction.java:103)
              ... 18 more
      		
      

      Putting a couple of sysout in the code, i found it was happening when in Launcher.getStats

      TaskReport[] mapRep = jobClient.getMapTaskReports(MRJobID);
      getErrorMessages(mapRep, "map", errNotDbg, pigContext);
      totalHadoopTimeSpent += computeTimeSpent(mapRep);
      

      whenever mapRed becomes null, it gives null error message.

      Can it be made better?

      Regards,
      Anitha

        Issue Links

          Activity

          Hide
          xuting zhao added a comment -

          Hi Anitha,

          I am trying to reproduce the bug and I ran that example in both branch-0.8, release-0.9 and trunk in local mode for 15 times and it gave the correct error message. May I know how often do you get the wrong message? Is there some specific condition for this bug? Thanks!

          Xuting

          The command line I used:
          bin/pig -x local test.pig

          test.pig:
          register 'a.py' using org.apache.pig.scripting.jython.JythonScriptEngine as test;
          A = load 'input.txt' as (x:chararray);
          B = foreach A generate test.a as y;
          C = group B by ;
          store C into 'C';

          a.py:
          #!/usr/bin/python
          @outputSchema("n:chararray")
          def a:
          return n.lower();

          input.txt:

          a

          Show
          xuting zhao added a comment - Hi Anitha, I am trying to reproduce the bug and I ran that example in both branch-0.8, release-0.9 and trunk in local mode for 15 times and it gave the correct error message. May I know how often do you get the wrong message? Is there some specific condition for this bug? Thanks! Xuting The command line I used: bin/pig -x local test.pig test.pig: register 'a.py' using org.apache.pig.scripting.jython.JythonScriptEngine as test; A = load 'input.txt' as (x:chararray); B = foreach A generate test.a as y; C = group B by ; store C into 'C'; a.py: #!/usr/bin/python @outputSchema("n:chararray") def a : return n.lower(); input.txt: a
          Hide
          Anitha Raju added a comment -

          Hi Xuting,

          I had run in Map reduce mode. Like discussed over chat, i have sent you the code to reproduce the same.

          Regards,
          Anitha

          Show
          Anitha Raju added a comment - Hi Xuting, I had run in Map reduce mode. Like discussed over chat, i have sent you the code to reproduce the same. Regards, Anitha
          Hide
          xuting zhao added a comment -

          Hi Anitha,

          Thanks for the help. I have run the tests in MapReduce mode in Pig 0.9 and trunk, it all seems give the correct error message. I think this should be a Hadoop problem or communication problem between Hadoop and Pig.

          The mapRep retrieves the running reports of the Map tasks that are running on Hadoop and then Pig will use the information stored in the reports to create different types of exception and error messages. If the mapRep is null, no exception information can be retrieved and then the wrong error is thrown.

          Show
          xuting zhao added a comment - Hi Anitha, Thanks for the help. I have run the tests in MapReduce mode in Pig 0.9 and trunk, it all seems give the correct error message. I think this should be a Hadoop problem or communication problem between Hadoop and Pig. The mapRep retrieves the running reports of the Map tasks that are running on Hadoop and then Pig will use the information stored in the reports to create different types of exception and error messages. If the mapRep is null, no exception information can be retrieved and then the wrong error is thrown.
          Hide
          xuting zhao added a comment -

          I think this problem is linked to PIG-1829,

          Show
          xuting zhao added a comment - I think this problem is linked to PIG-1829 ,

            People

            • Assignee:
              xuting zhao
              Reporter:
              Anitha Raju
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development