Uploaded image for project: 'Pig'
  1. Pig
  2. PIG-858

Order By followed by "replicated" join fails while compiling MR-plan from physical plan

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.4.0
    • Fix Version/s: 0.6.0
    • Component/s: impl
    • Labels:
      None

      Description

      Consider the query:

      A = load 'a';
      B = order A by $0;
      C = join A by $0, B by $0;
      explain C;
      

      works. But if replicated join is used instead

      A = load 'a';
      B = order A by $0;
      C = join A by $0, B by $0 using "replicated";
      explain C;
      

      this fails with ERROR org.apache.pig.tools.grunt.Grunt - ERROR 2034: Error compiling operator POFRJoin
      relevant stacktrace:

      Caused by: java.lang.RuntimeException: org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MRCompilerException: ERROR 2034: Error compiling operator POFRJoin
              at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.explain(HExecutionEngine.java:306)
              at org.apache.pig.PigServer.explain(PigServer.java:574)
              ... 8 more
      Caused by: org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MRCompilerException: ERROR 2034: Error compiling operator POFRJoin
              at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MRCompiler.visitFRJoin(MRCompiler.java:942)
              at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POFRJoin.visit(POFRJoin.java:173)
              at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MRCompiler.compile(MRCompiler.java:342)
              at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MRCompiler.compile(MRCompiler.java:327)
              at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MRCompiler.compile(MRCompiler.java:233)
              at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.compile(MapReduceLauncher.java:301)
              at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.explain(MapReduceLauncher.java:278)
              at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.explain(HExecutionEngine.java:303)
              ... 9 more
      Caused by: java.lang.ArrayIndexOutOfBoundsException: -1
              at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MRCompiler.visitFRJoin(MRCompiler.java:901)
              ... 16 more
      

        Attachments

        1. pig-858.patch
          14 kB
          Ashutosh Chauhan

          Activity

            People

            • Assignee:
              ashutoshc Ashutosh Chauhan
              Reporter:
              ashutoshc Ashutosh Chauhan
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: