Hadoop Common
  1. Hadoop Common
  2. HADOOP-1906

JobConf should warn about the existance of obsolete mapred-default.xml.

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.15.0
    • Component/s: conf
    • Labels:
      None

      Description

      Since the mapred-default.xml is ignored after HADOOP-785, we should generate a warning when it is present. Otherwise users forget to move the values to hadoop-site.xml and get confused when things don't work right.

        Activity

        Hide
        Hudson added a comment -
        Show
        Hudson added a comment - Integrated in Hadoop-Nightly #263 (See http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Nightly/263/ )
        Hide
        Owen O'Malley added a comment -

        I just committed this. Thanks, Arun.

        Show
        Owen O'Malley added a comment - I just committed this. Thanks, Arun.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12367144/HADOOP-1906_1_20071005.patch
        against trunk revision r582165.

        @author +1. The patch does not contain any @author tags.

        javadoc +1. The javadoc tool did not generate any warning messages.

        javac +1. The applied patch does not generate any new compiler warnings.

        findbugs +1. The patch does not introduce any new Findbugs warnings.

        core tests +1. The patch passed core unit tests.

        contrib tests -1. The patch failed contrib unit tests.

        Test results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/893/testReport/
        Findbugs warnings: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/893/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Checkstyle results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/893/artifact/trunk/build/test/checkstyle-errors.html
        Console output: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/893/console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12367144/HADOOP-1906_1_20071005.patch against trunk revision r582165. @author +1. The patch does not contain any @author tags. javadoc +1. The javadoc tool did not generate any warning messages. javac +1. The applied patch does not generate any new compiler warnings. findbugs +1. The patch does not introduce any new Findbugs warnings. core tests +1. The patch passed core unit tests. contrib tests -1. The patch failed contrib unit tests. Test results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/893/testReport/ Findbugs warnings: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/893/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/893/artifact/trunk/build/test/checkstyle-errors.html Console output: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/893/console This message is automatically generated.
        Hide
        Doug Cutting added a comment -

        > Although I guess it means you don't have to reconfigure immediately to try 0.15.

        Yes, that's the key point. We generally try to make it so that upgrading between major releases should require no code changes, provided your code compiles and runs without warnings in the prior release. So the first step to removing anything is generating warnings. For API changes, this means they're deprecated for at least one release cycle before they're removed. Other changes are harder to know how to handle, but in this case a warning seems most consistent with this compatibility policy, no?

        Show
        Doug Cutting added a comment - > Although I guess it means you don't have to reconfigure immediately to try 0.15. Yes, that's the key point. We generally try to make it so that upgrading between major releases should require no code changes, provided your code compiles and runs without warnings in the prior release. So the first step to removing anything is generating warnings. For API changes, this means they're deprecated for at least one release cycle before they're removed. Other changes are harder to know how to handle, but in this case a warning seems most consistent with this compatibility policy, no?
        Hide
        Owen O'Malley added a comment -

        I'm not wild about loading the mapred-default.xml. I think it will just prolong the pain like pulling the bandaid off slowly. Although I guess it means you don't have to reconfigure immediately to try 0.15.

        Show
        Owen O'Malley added a comment - I'm not wild about loading the mapred-default.xml. I think it will just prolong the pain like pulling the bandaid off slowly. Although I guess it means you don't have to reconfigure immediately to try 0.15.
        Hide
        Doug Cutting added a comment -

        +1 this looks good to me.

        Show
        Doug Cutting added a comment - +1 this looks good to me.
        Hide
        Arun C Murthy added a comment -

        Patch which checks, warns and loads mapred-default.xml if found on the classpath...

        Show
        Arun C Murthy added a comment - Patch which checks, warns and loads mapred-default.xml if found on the classpath...
        Hide
        Doug Cutting added a comment -

        Another option would be to generate the warning but still load mapred-default.xml, for back-compatibility, in 0.15, and not stop loading it until 0.16. Is there any point in staging things like that?

        Show
        Doug Cutting added a comment - Another option would be to generate the warning but still load mapred-default.xml, for back-compatibility, in 0.15, and not stop loading it until 0.16. Is there any point in staging things like that?

          People

          • Assignee:
            Arun C Murthy
            Reporter:
            Owen O'Malley
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development