Details
-
Bug
-
Status: Closed
-
Major
-
Resolution: Fixed
-
0.2.0
-
None
-
None
-
Patch Available
Description
Use of expressions that use the same sub-expressions in foreach lead to translation errors. This issue is caused due to sharing operators across nested plans. To remedy this issue, logical operators should be cloned and not shared across plans.
grunt> a = load 'a' as (x, y, z); grunt> b = foreach a { >> exp1 = x + y; >> exp2 = exp1 + x; >> generate exp1, exp2; >> } grunt> explain b; 2008-10-30 15:38:40,257 [main] WARN org.apache.pig.PigServer - bytearray is implicitly casted to double under LOAdd Operator 2008-10-30 15:38:40,258 [main] WARN org.apache.pig.PigServer - bytearray is implicitly casted to double under LOAdd Operator 2008-10-30 15:38:40,258 [main] WARN org.apache.pig.PigServer - bytearray is implicitly casted to double under LOAdd Operator Logical Plan: Store sms-Thu Oct 30 11:27:27 PDT 2008-2609 Schema: {double,double} Type: Unknown | |---ForEach sms-Thu Oct 30 11:27:27 PDT 2008-2605 Schema: {double,double} Type: bag | | | Add sms-Thu Oct 30 11:27:27 PDT 2008-2600 FieldSchema: double Type: double | | | |---Cast sms-Thu Oct 30 11:27:27 PDT 2008-2606 FieldSchema: double Type: double | | | | | |---Project sms-Thu Oct 30 11:27:27 PDT 2008-2598 Projections: [0] Overloaded: false FieldSchema: x: bytearray Type: bytearray | | Input: Load sms-Thu Oct 30 11:27:27 PDT 2008-2597 | | | |---Cast sms-Thu Oct 30 11:27:27 PDT 2008-2607 FieldSchema: double Type: double | | | |---Project sms-Thu Oct 30 11:27:27 PDT 2008-2599 Projections: [1] Overloaded: false FieldSchema: y: bytearray Type: bytearray | Input: Load sms-Thu Oct 30 11:27:27 PDT 2008-2597 | | | Add sms-Thu Oct 30 11:27:27 PDT 2008-2603 FieldSchema: double Type: double | | | |---Project sms-Thu Oct 30 11:27:27 PDT 2008-2601 Projections: [*] Overloaded: false FieldSchema: double Type: double | | Input: Add sms-Thu Oct 30 11:27:27 PDT 2008-2600| | | |---Add sms-Thu Oct 30 11:27:27 PDT 2008-2600 FieldSchema: double Type: double | | | | | |---Project sms-Thu Oct 30 11:27:27 PDT 2008-2598 Projections: [0] Overloaded: false FieldSchema: x: bytearray Type: bytearray | | | Input: Load sms-Thu Oct 30 11:27:27 PDT 2008-2597 | | | | | |---Project sms-Thu Oct 30 11:27:27 PDT 2008-2599 Projections: [1] Overloaded: false FieldSchema: y: bytearray Type: bytearray | | Input: Load sms-Thu Oct 30 11:27:27 PDT 2008-2597 | | | |---Cast sms-Thu Oct 30 11:27:27 PDT 2008-2608 FieldSchema: double Type: double | | | |---Project sms-Thu Oct 30 11:27:27 PDT 2008-2602 Projections: [0] Overloaded: false FieldSchema: x: bytearray Type: bytearray | Input: Load sms-Thu Oct 30 11:27:27 PDT 2008-2597 | |---Load sms-Thu Oct 30 11:27:27 PDT 2008-2597 Schema: {x: bytearray,y: bytearray,z: bytearray} Type: bag 2008-10-30 15:38:40,272 [main] ERROR org.apache.pig.impl.plan.OperatorPlan - Attempt to give operator of type org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject multiple outputs. This operator does not support multiple outputs. 2008-10-30 15:38:40,272 [main] ERROR org.apache.pig.backend.hadoop.executionengine.physicalLayer.LogToPhyTranslationVisitor - Invalid physical operators in the physical planAttempt to give operator of type org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject multiple outputs. This operator does not support multiple outputs. 2008-10-30 15:38:40,273 [main] ERROR org.apache.pig.tools.grunt.GruntParser - java.io.IOException: Unable to explain alias b [org.apache.pig.impl.plan.VisitorException] at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.compile(HExecutionEngine.java:235) at org.apache.pig.PigServer.compilePp(PigServer.java:731) at org.apache.pig.PigServer.explain(PigServer.java:495) at org.apache.pig.tools.grunt.GruntParser.processExplain(GruntParser.java:155) at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:193) at org.apache.pig.tools.grunt.GruntParser.parseContOnError(GruntParser.java:94) at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:58) at org.apache.pig.Main.main(Main.java:282) Caused by: org.apache.pig.backend.executionengine.ExecException: org.apache.pig.impl.plan.VisitorException ... 8 more Caused by: org.apache.pig.impl.plan.VisitorException at org.apache.pig.backend.hadoop.executionengine.physicalLayer.LogToPhyTranslationVisitor.visit(LogToPhyTranslationVisitor.java:324) at org.apache.pig.impl.logicalLayer.LOAdd.visit(LOAdd.java:69) at org.apache.pig.impl.logicalLayer.LOAdd.visit(LOAdd.java:29) at org.apache.pig.impl.plan.DependencyOrderWalkerWOSeenChk.walk(DependencyOrderWalkerWOSeenChk.java:68) at org.apache.pig.backend.hadoop.executionengine.physicalLayer.LogToPhyTranslationVisitor.visit(LogToPhyTranslationVisitor.java:805) at org.apache.pig.impl.logicalLayer.LOForEach.visit(LOForEach.java:121) at org.apache.pig.impl.logicalLayer.LOForEach.visit(LOForEach.java:40) at org.apache.pig.impl.plan.DependencyOrderWalker.walk(DependencyOrderWalker.java:68) at org.apache.pig.impl.plan.PlanVisitor.visit(PlanVisitor.java:51) at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.compile(HExecutionEngine.java:232) ... 7 more Caused by: org.apache.pig.impl.plan.PlanException: Attempt to give operator of type org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject multiple outputs. This operator does not support multiple outputs. at org.apache.pig.impl.plan.OperatorPlan.connect(OperatorPlan.java:158) at org.apache.pig.backend.hadoop.executionengine.physicalLayer.plans.PhysicalPlan.connect(PhysicalPlan.java:89) at org.apache.pig.backend.hadoop.executionengine.physicalLayer.LogToPhyTranslationVisitor.visit(LogToPhyTranslationVisitor.java:320) ... 16 more 2008-10-30 15:38:40,274 [main] ERROR org.apache.pig.tools.grunt.GruntParser - Unable to explain alias b [org.apache.pig.impl.plan.VisitorException] 2008-10-30 15:38:40,274 [main] ERROR org.apache.pig.tools.grunt.GruntParser - java.io.IOException: Unable to explain alias b [org.apache.pig.impl.plan.VisitorException]