[PIG-2087] Not able to project into 'group' tuple from FILTER

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Cannot Reproduce
• Affects Version/s: 0.8.0
• Labels: pig
• Assignee: Unassigned
• Reporter: Daniel Eklund
• Created: 22/May/11 22:01
• Updated: 03/Oct/11 07:41
• Resolved: 03/Oct/11 07:41

Description

GROUP creates the 'group' tuple, but subsequent FILTER statements cannot project into it without throwing ClassCastExceptions.

Example:

-----------

my_data = LOAD 'test.txt' using PigStorage(',')
as (name:chararray, age:int, eye_color:chararray, height:int);

by_age_and_color = GROUP my_data BY (age, eye_color);

OUT2 = FILTER by_age_and_color by group.age is not null;
dump OUT2;

– I get a similar problem even if I do something like:

OUT3 = FILTER by_age_and_color by group.age > 9;
dump OUT3;

--------- sample test.txt ---------
ravi,33,blue,43
brendan,33,green,53
ravichandra,15,blue,43
leonor,15,brown,46
caeser,18,blue,23
JCVD,,blue,23
anthony,33,blue,46
xavier,23,blue,13
patrick,18,blue,33
sang,33,brown,44

-------------------------------

java.lang.ClassCastException: java.lang.Integer cannot be cast to org.apache.pig.data.Tuple
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject.getNext(POProject.java:392)
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:276)
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject.getNext(POProject.java:138)
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject.getNext(POProject.java:276)
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.GreaterThanExpr.getNext(GreaterThanExpr.java:72)
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POFilter.getNext(POFilter.java:148)
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:276)
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLocalRearrange.getNext(POLocalRearrange.java:256)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.runPipeline(PigMapBase.java:236)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:231)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:53)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:646)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:322)
at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:210)

Issue Links

• relates to: PIG-2118 ClassCastException when using group.$0