[PIG-3049] Cannot sort on a bag in nested foreach

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: 0.11, 0.12.0
• Fix Version/s: 0.12.0
• Assignee: Daniel Dai
• Reporter: Jonathan Coveney
• Created: 14/Nov/12 18:41
• Updated: 08/Apr/13 20:00
• Resolved: 08/Apr/13 20:00

Description

The following script fails.

a = load 'words_and_numbers' as (word:chararray, number:int);
b = foreach (group a by number) {
  a_bag = a.word;
  ord = order a_bag by word;
  generate group, ord;
}
dump b;

On this data:

$ cat words_and_numbers                                                                                                              
hey	1
hey	2
you	3
you	4
I	5
could	6

it throws the following error:

ava.lang.ClassCastException: java.lang.String cannot be cast to org.apache.pig.data.Tuple
	at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject.getNext(POProject.java:469)
	at org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:308)
	at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject.getNext(POProject.java:160)
	at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POProject.getNext(POProject.java:384)
	at org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.getNext(PhysicalOperator.java:340)
	at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLocalRearrange.getNext(POLocalRearrange.java:333)
	at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.runPipeline(PigGenericMapBase.java:283)
	at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:278)
	at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:64)
	at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
	at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:647)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:323)
	at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:210)

Is this a supported feature of Pig? Seems reasonable, just seems like something weird is going on.

Issue Links

• relates to: PIG-2265 Test case TestSecondarySort failure