Uploaded image for project: 'Pig'
  1. Pig
  2. PIG-5031

Tez failing to compile when replicate join is done with a limit vertex on left

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.17.0
    • Component/s: tez
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      a = load 'file:///tmp/input' as (x:int, y:int);
      b = load 'file:///tmp/input' as (x:int, y:int);
      c = limit a 1;
      d = join c by x, b by x using 'replicated';
      store a into 'file:///tmp/pigoutput/a';
      store d into 'file:///tmp/pigoutput/d';
      

      This fails with

      ERROR 2022: The current operator is closed. This is unexpected while compiling.
      
      org.apache.pig.backend.hadoop.executionengine.tez.plan.TezCompilerException: ERROR 2034: Error compiling operator POFRJoin
              at org.apache.pig.backend.hadoop.executionengine.tez.plan.TezCompiler.visitFRJoin(TezCompiler.java:756)
              at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POFRJoin.visit(POFRJoin.java:213)
              at org.apache.pig.backend.hadoop.executionengine.tez.plan.TezCompiler.compile(TezCompiler.java:409)
              at org.apache.pig.backend.hadoop.executionengine.tez.plan.TezCompiler.compile(TezCompiler.java:388)
              at org.apache.pig.backend.hadoop.executionengine.tez.plan.TezCompiler.compile(TezCompiler.java:264)
              at org.apache.pig.backend.hadoop.executionengine.tez.TezLauncher.compile(TezLauncher.java:423)
              at org.apache.pig.backend.hadoop.executionengine.tez.TezLauncher.launchPig(TezLauncher.java:171)
              at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.launchPig(HExecutionEngine.java:290)
              at org.apache.pig.PigServer.launchPlan(PigServer.java:1474)
              at org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1459)
              at org.apache.pig.PigServer.execute(PigServer.java:1448)
              at org.apache.pig.PigServer.executeBatch(PigServer.java:488)
              at org.apache.pig.PigServer.executeBatch(PigServer.java:471)
              at org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:172)
              at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:235)
              at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:206)
              at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:81)
              at org.apache.pig.Main.run(Main.java:630)
              at org.apache.pig.Main.main(Main.java:176)
      Caused by: org.apache.pig.backend.hadoop.executionengine.tez.plan.TezCompilerException: ERROR 2022: The current operator is closed. This is unexpected while compiling.
              at org.apache.pig.backend.hadoop.executionengine.tez.plan.TezCompiler.visitFRJoin(TezCompiler.java:739)
              ... 18 more
      

        Attachments

        1. pig-5031-v01.patch
          4 kB
          Koji Noguchi

          Activity

            People

            • Assignee:
              knoguchi Koji Noguchi
              Reporter:
              knoguchi Koji Noguchi
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: