Hadoop Common
  1. Hadoop Common
  2. HADOOP-3093

ma/reduce throws the following exception if "io.serializations" is not set:

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.17.0
    • Fix Version/s: 0.17.0
    • Component/s: None
    • Labels:
      None
    • Release Note:
      Hide
      The following public APIs are added in org.apache.hadoop.conf.Configuration
       String[] Configuration.getStrings(String name, String... defaultValue) and
       void Configuration.setStrings(String name, String... values)
      Show
      The following public APIs are added in org.apache.hadoop.conf.Configuration  String[] Configuration.getStrings(String name, String... defaultValue) and  void Configuration.setStrings(String name, String... values)

      Description

      map/reduce throws the following exception if "io.serializations" is not set:

      java.lang.NullPointerException
      at org.apache.hadoop.io.serializer.SerializationFactory.<init>(SerializationFactory.java:52)
      at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.<init>(MapTask.java:325)
      at org.apache.hadoop.mapred.MapTask.run(MapTask.java:177)
      at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2073)

      Map/reduce should use org.apache.hadoop.io.serializer.WritableSerialization as the default class as it had in the past.

      1. patch-3093.txt
        2 kB
        Amareshwari Sriramadasu
      2. patch-3093.txt
        2 kB
        Amareshwari Sriramadasu

        Activity

        Runping Qi created issue -
        Hide
        Amareshwari Sriramadasu added a comment -

        Here is a patch fixing the Null pointer exception.
        I added a new api getStrings(String name, String[] defaultValue) in Configuration.java, which returns default value if nothing specified for the configuration property.

        Show
        Amareshwari Sriramadasu added a comment - Here is a patch fixing the Null pointer exception. I added a new api getStrings(String name, String[] defaultValue) in Configuration.java, which returns default value if nothing specified for the configuration property.
        Amareshwari Sriramadasu made changes -
        Field Original Value New Value
        Attachment patch-3093.txt [ 12378768 ]
        Amareshwari Sriramadasu made changes -
        Assignee Amareshwari Sriramadasu [ amareshwari ]
        Status Open [ 1 ] Patch Available [ 10002 ]
        Fix Version/s 0.17.0 [ 12312913 ]
        Amareshwari Sriramadasu made changes -
        Description

        map/reduce throws the following exception if "io.serializations" is not set:

        java.lang.NullPointerException
                at org.apache.hadoop.io.serializer.SerializationFactory.<init>(SerializationFactory.java:52)
                at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.<init>(MapTask.java:325)
                at org.apache.hadoop.mapred.MapTask.run(MapTask.java:177)
                at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2073)

        Map/reduce should use org.apache.hadoop.io.serializer.WritableSerialization as the default class as it had in the past.

        map/reduce throws the following exception if "io.serializations" is not set:

        java.lang.NullPointerException
                at org.apache.hadoop.io.serializer.SerializationFactory.<init>(SerializationFactory.java:52)
                at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.<init>(MapTask.java:325)
                at org.apache.hadoop.mapred.MapTask.run(MapTask.java:177)
                at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2073)

        Map/reduce should use org.apache.hadoop.io.serializer.WritableSerialization as the default class as it had in the past.
        Affects Version/s 0.17.0 [ 12312913 ]
        Hide
        Devaraj Das added a comment -

        +1

        Show
        Devaraj Das added a comment - +1
        Hide
        Tom White added a comment -

        +1

        Show
        Tom White added a comment - +1
        Hide
        Enis Soztutar added a comment -

        Just a small hit,

        lets change

        public String[] getStrings(String name, String... defaultValue) {
        }
        

        so that both

        conf.getStrings(key, new String[] {"foo.bar", "foo.baz"}); 
        

        and

        conf.getStrings(key, "foo.bar", "foo.baz"); 
        

        works.

        Moreover since we are at it, introducing a setStrings() might be a good idea (the patch for HADOOP-449 needs it).

        Show
        Enis Soztutar added a comment - Just a small hit, lets change public String [] getStrings( String name, String ... defaultValue) { } so that both conf.getStrings(key, new String [] { "foo.bar" , "foo.baz" }); and conf.getStrings(key, "foo.bar" , "foo.baz" ); works. Moreover since we are at it, introducing a setStrings() might be a good idea (the patch for HADOOP-449 needs it).
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12378768/patch-3093.txt
        against trunk revision 619744.

        @author +1. The patch does not contain any @author tags.

        tests included -1. The patch doesn't appear to include any new or modified tests.
        Please justify why no tests are needed for this patch.

        javadoc +1. The javadoc tool did not generate any warning messages.

        javac +1. The applied patch does not generate any new javac compiler warnings.

        release audit +1. The applied patch does not generate any new release audit warnings.

        findbugs +1. The patch does not introduce any new Findbugs warnings.

        core tests +1. The patch passed core unit tests.

        contrib tests +1. The patch passed contrib unit tests.

        Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2087/testReport/
        Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2087/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2087/artifact/trunk/build/test/checkstyle-errors.html
        Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2087/console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12378768/patch-3093.txt against trunk revision 619744. @author +1. The patch does not contain any @author tags. tests included -1. The patch doesn't appear to include any new or modified tests. Please justify why no tests are needed for this patch. javadoc +1. The javadoc tool did not generate any warning messages. javac +1. The applied patch does not generate any new javac compiler warnings. release audit +1. The applied patch does not generate any new release audit warnings. findbugs +1. The patch does not introduce any new Findbugs warnings. core tests +1. The patch passed core unit tests. contrib tests +1. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2087/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2087/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2087/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2087/console This message is automatically generated.
        Hide
        Amareshwari Sriramadasu added a comment -

        Canceling the patch to address Enis's suggestion.

        Show
        Amareshwari Sriramadasu added a comment - Canceling the patch to address Enis's suggestion.
        Amareshwari Sriramadasu made changes -
        Status Patch Available [ 10002 ] Open [ 1 ]
        Hide
        Amareshwari Sriramadasu added a comment -

        Here is a patch adding
        1. String[] Configuration.getStrings(String name, String... defaultValue)
        2. void Configuration.setStrings(String name, String... values)

        Show
        Amareshwari Sriramadasu added a comment - Here is a patch adding 1. String[] Configuration.getStrings(String name, String... defaultValue) 2. void Configuration.setStrings(String name, String... values)
        Amareshwari Sriramadasu made changes -
        Attachment patch-3093.txt [ 12378920 ]
        Amareshwari Sriramadasu made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12378920/patch-3093.txt
        against trunk revision 619744.

        @author +1. The patch does not contain any @author tags.

        tests included -1. The patch doesn't appear to include any new or modified tests.
        Please justify why no tests are needed for this patch.

        patch -1. The patch command could not apply the patch.

        Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2099/console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12378920/patch-3093.txt against trunk revision 619744. @author +1. The patch does not contain any @author tags. tests included -1. The patch doesn't appear to include any new or modified tests. Please justify why no tests are needed for this patch. patch -1. The patch command could not apply the patch. Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2099/console This message is automatically generated.
        Hide
        Amareshwari Sriramadasu added a comment -

        trying hudson again.

        Show
        Amareshwari Sriramadasu added a comment - trying hudson again.
        Amareshwari Sriramadasu made changes -
        Status Patch Available [ 10002 ] Open [ 1 ]
        Amareshwari Sriramadasu made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12378920/patch-3093.txt
        against trunk revision 643282.

        @author +1. The patch does not contain any @author tags.

        tests included -1. The patch doesn't appear to include any new or modified tests.
        Please justify why no tests are needed for this patch.

        javadoc +1. The javadoc tool did not generate any warning messages.

        javac +1. The applied patch does not generate any new javac compiler warnings.

        release audit +1. The applied patch does not generate any new release audit warnings.

        findbugs +1. The patch does not introduce any new Findbugs warnings.

        core tests +1. The patch passed core unit tests.

        contrib tests +1. The patch passed contrib unit tests.

        Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2111/testReport/
        Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2111/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2111/artifact/trunk/build/test/checkstyle-errors.html
        Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2111/console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12378920/patch-3093.txt against trunk revision 643282. @author +1. The patch does not contain any @author tags. tests included -1. The patch doesn't appear to include any new or modified tests. Please justify why no tests are needed for this patch. javadoc +1. The javadoc tool did not generate any warning messages. javac +1. The applied patch does not generate any new javac compiler warnings. release audit +1. The applied patch does not generate any new release audit warnings. findbugs +1. The patch does not introduce any new Findbugs warnings. core tests +1. The patch passed core unit tests. contrib tests +1. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2111/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2111/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2111/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2111/console This message is automatically generated.
        Hide
        Devaraj Das added a comment -

        I just committed this. Thanks, Amareshwari!

        Show
        Devaraj Das added a comment - I just committed this. Thanks, Amareshwari!
        Devaraj Das made changes -
        Resolution Fixed [ 1 ]
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Amareshwari Sriramadasu made changes -
        Release Note The following public APIs are added in org.apache.hadoop.conf.Configuration
         String[] Configuration.getStrings(String name, String... defaultValue) and
         void Configuration.setStrings(String name, String... values)
        Description
        map/reduce throws the following exception if "io.serializations" is not set:

        java.lang.NullPointerException
                at org.apache.hadoop.io.serializer.SerializationFactory.<init>(SerializationFactory.java:52)
                at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.<init>(MapTask.java:325)
                at org.apache.hadoop.mapred.MapTask.run(MapTask.java:177)
                at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2073)

        Map/reduce should use org.apache.hadoop.io.serializer.WritableSerialization as the default class as it had in the past.
        map/reduce throws the following exception if "io.serializations" is not set:

        java.lang.NullPointerException
                at org.apache.hadoop.io.serializer.SerializationFactory.<init>(SerializationFactory.java:52)
                at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.<init>(MapTask.java:325)
                at org.apache.hadoop.mapred.MapTask.run(MapTask.java:177)
                at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2073)

        Map/reduce should use org.apache.hadoop.io.serializer.WritableSerialization as the default class as it had in the past.
        Nigel Daley made changes -
        Status Resolved [ 5 ] Closed [ 6 ]
        Owen O'Malley made changes -
        Component/s mapred [ 12310690 ]
        Transition Time In Source Status Execution Times Last Executer Last Execution Date
        Patch Available Patch Available Open Open
        3d 23h 17m 2 Amareshwari Sriramadasu 01/Apr/08 06:39
        Open Open Patch Available Patch Available
        1d 15h 34m 3 Amareshwari Sriramadasu 01/Apr/08 06:40
        Patch Available Patch Available Resolved Resolved
        1d 2h 43m 1 Devaraj Das 02/Apr/08 09:23
        Resolved Resolved Closed Closed
        49d 11h 42m 1 Nigel Daley 21/May/08 21:06

          People

          • Assignee:
            Amareshwari Sriramadasu
            Reporter:
            Runping Qi
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development