Pig
  1. Pig
  2. PIG-1993

PigStorageSchema throw NPE with ColumnPruning

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.8.0, 0.9.0
    • Fix Version/s: 0.8.1
    • Component/s: impl
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      The following script fail:

      a = load '1.txt' as (a0:int, a1:int, a2:int);
      store a into 'temp' using org.apache.pig.piggybank.storage.PigStorageSchema();
      exec
      a = LOAD 'temp' using org.apache.pig.piggybank.storage.PigStorageSchema();
      b = FOREACH a GENERATE a1;
      dump b;
      

      Error message:
      java.lang.ArrayIndexOutOfBoundsException: 2
      at org.apache.pig.piggybank.storage.PigStorageSchema.getNext(PigStorageSchema.java:94)
      at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigRecordReader.nextKeyValue(PigRecordReader.java:187)
      at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:423)
      at org.apache.hadoop.mapreduce.MapContext.nextKeyValue(MapContext.java:67)
      at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:143)
      at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:621)
      at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
      at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:177)

      1. PIG-1993-0.patch
        0.9 kB
        Daniel Dai
      2. PIG-1993-1.patch
        3 kB
        Daniel Dai

        Activity

        Hide
        Daniel Dai added a comment -

        Attach PIG-1993-0.patch for reference. It is not a complete patch, no test case included.

        Show
        Daniel Dai added a comment - Attach PIG-1993 -0.patch for reference. It is not a complete patch, no test case included.
        Hide
        Daniel Dai added a comment -

        Attach a complete patch PIG-1993-1.patch

        Show
        Daniel Dai added a comment - Attach a complete patch PIG-1993 -1.patch
        Hide
        Daniel Dai added a comment -

        Dmitriy, can you review the patch?

        Show
        Daniel Dai added a comment - Dmitriy, can you review the patch?
        Hide
        Dmitriy V. Ryaboy added a comment -

        looks good, please commit if tests pass.

        Glad to see someone's actually using this!

        Show
        Dmitriy V. Ryaboy added a comment - looks good, please commit if tests pass. Glad to see someone's actually using this!
        Hide
        Daniel Dai added a comment -

        Patch committed to both trunk and 0.8 branch.

        Show
        Daniel Dai added a comment - Patch committed to both trunk and 0.8 branch.

          People

          • Assignee:
            Daniel Dai
            Reporter:
            Daniel Dai
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development