Details
-
Improvement
-
Status: Patch Available
-
Minor
-
Resolution: Unresolved
-
2.2.0
-
None
Description
Currently we will get NPE if the serializer/deserializer is not configured correctly.
14/01/14 11:52:35 INFO mapred.JobClient: Task Id : attempt_201401072154_0027_m_000002_2, Status : FAILED java.lang.NullPointerException at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.<init>(MapTask.java:944) at org.apache.hadoop.mapred.MapTask$NewOutputCollector.<init>(MapTask.java:672) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:740) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:368) at org.apache.hadoop.mapred.Child$4.run(Child.java:255) at java.security.AccessController.doPrivileged(AccessController.java:362) at javax.security.auth.Subject.doAs(Subject.java:573) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1502) at org.apache.hadoop.mapred.Child.main(Child.java:249)
serializationFactory.getSerializer and serializationFactory.getDeserializer returns NULL in this case.
Let's check NPE for serializer/deserializer in MapTask so that we don't get meaningless NPE.
Attachments
Attachments
Issue Links
- duplicates
-
MAPREDUCE-2584 Check for serializers early, and give out more information regarding missing serializers
- Resolved