Uploaded image for project: 'Pig'
  1. Pig
  2. PIG-4873

InputSplit.getLocations return null and result a NPE in Pig

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.16.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      A customer report a NPE stack below:
      Caused by: java.lang.NullPointerException
      at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigSplit.toString(PigSplit.java:403)
      at java.lang.String.valueOf(String.java:2854)
      at java.lang.StringBuilder.append(StringBuilder.java:128)
      at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:753)
      at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
      at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
      at java.security.AccessController.doPrivileged(Native Method)
      at javax.security.auth.Subject.doAs(Subject.java:415)
      at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)

      It is be a MapReduce issue to return null in InputSplit.getLocations. But it is unnecessary to fail the job just because Pig needs to print out the information for the split. The job run normally after capture the null.

        Attachments

        1. PIG-4873-1.patch
          1 kB
          Jianyong Dai

          Activity

            People

            • Assignee:
              daijy Jianyong Dai
              Reporter:
              daijy Jianyong Dai
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: