[PIG-2164] CombinerOptimizer exception with multi-query

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.9.0
• Fix Version/s: 0.9.1, 0.10.0
• Assignee: Thejas Nair
• Reporter: Thejas Nair
• Created: 14/Jul/11 21:00
• Updated: 05/Oct/11 17:20
• Resolved: 21/Sep/11 21:44

Description

l = load 'x' as (a,b,c);
f1 = foreach (group l by (a,b)) generate COUNT_STAR(*);
f2 = foreach (group l by (a,null)) generate COUNT_STAR(*);
u = union f1, f2; 
explain u;  

Pig Stack Trace
---------------
ERROR 2018: Internal error. Unable to introduce the combiner for optimization.

org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1067: Unable to explain alias u
        at org.apache.pig.PigServer.explain(PigServer.java:1063)
        at org.apache.pig.tools.grunt.GruntParser.explainCurrentBatch(GruntParser.java:393)
        at org.apache.pig.tools.grunt.GruntParser.processExplain(GruntParser.java:325)
        at org.apache.pig.tools.grunt.GruntParser.processExplain(GruntParser.java:288)
        at org.apache.pig.tools.pigscript.parser.PigScriptParser.Explain(PigScriptParser.java:665)
        at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:325)
        at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:188)
        at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:164)
        at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:67)
        at org.apache.pig.Main.run(Main.java:487)
        at org.apache.pig.Main.main(Main.java:108)
Caused by: org.apache.pig.impl.plan.optimizer.OptimizerException: ERROR 2018: Internal error. Unable to introduce the combiner for optimization.
        at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.CombinerOptimizer.visitMROp(CombinerOptimizer.java:313)
        at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceOper.visit(MapReduceOper.java:252)
        at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceOper.visit(MapReduceOper.java:42)
        at org.apache.pig.impl.plan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:69)
        at org.apache.pig.impl.plan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:71)
        at org.apache.pig.impl.plan.DepthFirstWalker.walk(DepthFirstWalker.java:52)
        at org.apache.pig.impl.plan.PlanVisitor.visit(PlanVisitor.java:51)
        at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.compile(MapReduceLauncher.java:492)
        at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.explain(MapReduceLauncher.java:454)
        at org.apache.pig.PigServer.explain(PigServer.java:1055)
        ... 10 more
Caused by: java.lang.ArrayIndexOutOfBoundsException: 2
        at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.CombinerOptimizer.addAlgebraicFuncToCombineFE(CombinerOptimizer.java:515)
        at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.CombinerOptimizer.visitMROp(CombinerOptimizer.java:227)
        ... 19 more
================================================================================