[MAPREDUCE-570] Streaming job fails with with identity mapper class

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Duplicate
• Component/s: contrib/streaming
• Assignee: Unassigned
• Reporter: Suhas
• Created: 05/Dec/08 20:48
• Updated: 05/Jul/10 05:32
• Resolved: 05/Jul/10 05:32

Description

Streaming job command:
$HADOOP_HOME/bin/hadoop jar $HADOOP_HOME/hadoop-streaming.jar \
-input myInputDirs \
-output myOutputDir \
-mapper org.apache.hadoop.mapred.lib.IdentityMapper \
-reducer org.apache.hadoop.mapred.lib.IndentityReducer

Error in job tracker hadoop.log:

2008-11-24 22:18:12,637 INFO org.apache.hadoop.mapred.TaskInProgress: Error from attempt_200811242217_0001_m_000001_3: java.io.IOException: Type mismatch in key from map: expected org.apache.hadoop.io.Text, recieved org.apache.hadoop.io.LongWritable^D
at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:415)^D
at org.apache.hadoop.mapred.lib.IdentityMapper.map(IdentityMapper.java:37)^D
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)^D
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:227)^D
at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2207)^D

Issue Links

• relates to: MAPREDUCE-1766 Hadoop Streaming should not use TextInputFormat class as the default input format class.