Pig
  1. Pig
  2. PIG-1078

[zebra] merge join with empty table failed

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.6.0, 0.7.0
    • Component/s: None
    • Labels:
      None

      Description

      Got indexOutOfBound exception.

      Here is the pig script:
      register /grid/0/dev/hadoopqa/jars/zebra.jar;
      --a1 = load '1.txt' as (a:int, b:float,c:long,d:double,e:chararray,f:bytearray,r1(f1:chararray,f2:chararray),m1:map[]);

      --a2 = load 'empty.txt' as (a:int, b:float,c:long,d:double,e:chararray,f:bytearray,r1(f1:chararray,f2:chararray),m1:map[]);
      --dump a1;

      --a1order = order a1 by a;
      --a2order = order a2 by a;

      --store a1order into 'a1' using org.apache.hadoop.zebra.pig.TableStorer('[a,b,c];[d,e,f,r1,m1]');
      --store a2order into 'empty' using org.apache.hadoop.zebra.pig.TableStorer('[a,b,c];[d,e,f,r1,m1]');

      rec1 = load 'a1' using org.apache.hadoop.zebra.pig.TableLoader();
      rec2 = load 'empty' using org.apache.hadoop.zebra.pig.TableLoader();
      joina = join rec1 by a, rec2 by a using "merge" ;
      dump joina;

      ======
      please note that table "a1" and "empty" are created correctly.

      Here is the stack trace:
      Backend error message
      ---------------------
      java.lang.ArrayIndexOutOfBoundsException: 0
      at org.apache.hadoop.zebra.mapred.TableInputFormat.getTableRecordReader(TableInputFormat.java:478)
      at org.apache.hadoop.zebra.pig.TableLoader.bindTo(TableLoader.java:166)
      at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POMergeJoin.seekInRightStream(POMergeJoin.java:400)
      at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POMergeJoin.getNext(POMergeJoin.java:181)
      at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.runPipeline(PigMapBase.java:247)
      at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:238)
      at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapOnly$Map.map(PigMapOnly.java:65)
      at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
      at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:358)
      at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
      at org.apache.hadoop.mapred.Child.main(Child.java:159)

      Pig Stack Trace
      ---------------
      ERROR 6015: During execution, encountered a Hadoop error.

      org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066: Unable to open iterator for alias joina
      at org.apache.pig.PigServer.openIterator(PigServer.java:481)
      at org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:539)
      at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:241)
      at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:168)
      at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:144)
      at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:89)
      at org.apache.pig.Main.main(Main.java:386)
      Caused by: org.apache.pig.backend.executionengine.ExecException: ERROR 6015: During execution, encountered a Hadoop error.
      at .apache.hadoop.zebra.mapred.TableInputFormat.getTableRecordReader(TableInputFormat.java:478)
      at .apache.hadoop.zebra.pig.TableLoader.bindTo(TableLoader.java:166)
      at .apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POMergeJoin.seekInRightStream(POMergeJoin.java:400)
      at .apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POMergeJoin.getNext(POMergeJoin.java:181)
      at .apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.runPipeline(PigMapBase.java:247)
      at .apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:238)
      at .apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapOnly$Map.map(PigMapOnly.java:65)
      at .apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
      at .apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:358)
      at .apache.hadoop.mapred.MapTask.run(MapTask.java:307)
      Caused by: java.lang.ArrayIndexOutOfBoundsException: 0
      ... 10 more
      ================================================================================

        Activity

        Hide
        Ashutosh Chauhan added a comment -

        This seems to be related to Zebra. Jing, do you think it has to do with merge join implementation of Pig ?

        Show
        Ashutosh Chauhan added a comment - This seems to be related to Zebra. Jing, do you think it has to do with merge join implementation of Pig ?
        Hide
        Yan Zhou added a comment -

        This pacth is also targeted for the 0.6 release so it needs to be on the 0.6 branch too.

        Show
        Yan Zhou added a comment - This pacth is also targeted for the 0.6 release so it needs to be on the 0.6 branch too.
        Hide
        Hadoop QA added a comment -

        +1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12425632/PIG-1078.patch
        against trunk revision 882340.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 2 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed core unit tests.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: http://hudson.zones.apache.org/hudson/job/Pig-Patch-h7.grid.sp2.yahoo.net/163/testReport/
        Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Pig-Patch-h7.grid.sp2.yahoo.net/163/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Console output: http://hudson.zones.apache.org/hudson/job/Pig-Patch-h7.grid.sp2.yahoo.net/163/console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12425632/PIG-1078.patch against trunk revision 882340. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 2 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Pig-Patch-h7.grid.sp2.yahoo.net/163/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Pig-Patch-h7.grid.sp2.yahoo.net/163/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: http://hudson.zones.apache.org/hudson/job/Pig-Patch-h7.grid.sp2.yahoo.net/163/console This message is automatically generated.
        Hide
        Chao Wang added a comment -

        Patch reviewed. +1

        Show
        Chao Wang added a comment - Patch reviewed. +1
        Hide
        Alan Gates added a comment -

        Patch checked in.

        Show
        Alan Gates added a comment - Patch checked in.
        Hide
        Alan Gates added a comment -

        Patch checked into 0.6 branch.

        Show
        Alan Gates added a comment - Patch checked into 0.6 branch.

          People

          • Assignee:
            Yan Zhou
            Reporter:
            Jing Huang
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development