ma/reduce throws the following exception if "io.serializations" is not set:

• Key: HADOOP-3093
• Type: Bug
• Status: Closed
• Resolution: Fixed
• Priority: Major
• Assignee: Amareshwari Sriramadasu
• Reporter: Runping Qi
• Affects Version/s: 0.17.0
• Fix Version/s: 0.17.0

Description

map/reduce throws the following exception if "io.serializations" is not set:

java.lang.NullPointerException
at org.apache.hadoop.io.serializer.SerializationFactory.<init>(SerializationFactory.java:52)
at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.<init>(MapTask.java:325)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:177)
at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2073)

Map/reduce should use org.apache.hadoop.io.serializer.WritableSerialization as the default class as it had in the past.

Comments

[Amareshwari Sriramadasu]

Here is a patch fixing the Null pointer exception.
I added a new api getStrings(String name, String[] defaultValue) in Configuration.java, which returns default value if nothing specified for the configuration property.

[Devaraj Das]

+1

[Tom White]

+1

[Enis Soztutar]

Just a small hit,

lets change

public String[] getStrings(String name, String... defaultValue) {
}

so that both

conf.getStrings(key, new String[] {"foo.bar", "foo.baz"});

and

conf.getStrings(key, "foo.bar", "foo.baz");

works.

Moreover since we are at it, introducing a setStrings() might be a good idea (the patch for HADOOP-449 needs it).

[Hadoop QA]

-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12378768/patch-3093.txt
against trunk revision 619744.

@author +1. The patch does not contain any @author tags.

tests included -1. The patch doesn't appear to include any new or modified tests.
Please justify why no tests are needed for this patch.

javadoc +1. The javadoc tool did not generate any warning messages.

javac +1. The applied patch does not generate any new javac compiler warnings.

release audit +1. The applied patch does not generate any new release audit warnings.

findbugs +1. The patch does not introduce any new Findbugs warnings.

core tests +1. The patch passed core unit tests.

contrib tests +1. The patch passed contrib unit tests.

Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2087/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2087/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2087/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2087/console

This message is automatically generated.

[Amareshwari Sriramadasu]

Canceling the patch to address Enis's suggestion.

[Amareshwari Sriramadasu]

Here is a patch adding
1. String[] Configuration.getStrings(String name, String... defaultValue)
2. void Configuration.setStrings(String name, String... values)

[Hadoop QA]

-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12378920/patch-3093.txt
against trunk revision 619744.

@author +1. The patch does not contain any @author tags.

tests included -1. The patch doesn't appear to include any new or modified tests.
Please justify why no tests are needed for this patch.

patch -1. The patch command could not apply the patch.

Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2099/console

This message is automatically generated.

[Amareshwari Sriramadasu]

trying hudson again.

[Hadoop QA]

-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12378920/patch-3093.txt
against trunk revision 643282.

@author +1. The patch does not contain any @author tags.

tests included -1. The patch doesn't appear to include any new or modified tests.
Please justify why no tests are needed for this patch.

javadoc +1. The javadoc tool did not generate any warning messages.

javac +1. The applied patch does not generate any new javac compiler warnings.

release audit +1. The applied patch does not generate any new release audit warnings.

findbugs +1. The patch does not introduce any new Findbugs warnings.

core tests +1. The patch passed core unit tests.

contrib tests +1. The patch passed contrib unit tests.

Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2111/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2111/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2111/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2111/console

This message is automatically generated.

[Devaraj Das]

I just committed this. Thanks, Amareshwari!