[PIG-919] Type mismatch in key from map: expected org.apache.pig.impl.io.NullableBytesWritable, recieved org.apache.pig.impl.io.NullableText when doing simple group

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.3.0
• Fix Version/s: 0.3.0
• Component/s: impl

Release Note

Jeff is right - the issue is in how maps are interpreted by Pig. I agree we need to figure out how best to work with maps but it is a bigger issue than the scope of this bug and I would like to deal with it separately. (In fact it is on the roadmap to address.)

• Assignee: Viraj Bhat
• Reporter: Viraj Bhat
• Created: 13/Aug/09 00:45
• Updated: 08/Feb/11 07:09
• Resolved: 27/Aug/09 23:35

Description

I have a Pig script, which takes in a student file and generates a bag of maps. I later want to group on the value of the key "name0" which corresponds to the first name of the student.

register mymapudf.jar;



data = LOAD '/user/viraj/studenttab10k' AS (somename:chararray,age:long,marks:float);



genmap = foreach data generate flatten(mymapudf.GenHashList(somename,' ')) as bp:map[], age, marks;



getfirstnames = foreach genmap generate bp#'name0' as firstname, age, marks;



filternonnullfirstnames = filter getfirstnames by firstname is not null;




groupgenmap = group filternonnullfirstnames by firstname;



dump groupgenmap;

When I execute this code, I get an error in the Map Phase:
===========================================================================================================
java.io.IOException: Type mismatch in key from map: expected org.apache.pig.impl.io.NullableBytesWritable, recieved org.apache.pig.impl.io.NullableText
at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:415)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Map.collect(PigMapReduce.java:108)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.runPipeline(PigMapBase.java:253)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:242)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Map.map(PigMapReduce.java:93)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:227)
at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2209)
===========================================================================================================