Hadoop Common
  1. Hadoop Common
  2. HADOOP-7664

o.a.h.conf.Configuration complains of overriding final parameter even if the value with which its attempting to override is the same.

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 0.20.205.0, 0.23.0
    • Fix Version/s: 1.0.0, 0.23.0
    • Component/s: conf
    • Labels:
      None
    • Environment:

      commit a2f64ee8d9312fe24780ec53b15af439a315796d

    • Hadoop Flags:
      Reviewed
    • Target Version/s:

      Description

      o.a.h.conf.Configuration complains of overriding final parameter even if the value with which its attempting to override is the same.

      1. HADOOP-7664.branch-0.20-security.patch
        0.6 kB
        Ravi Prakash
      2. HADOOP-7664.patch
        0.7 kB
        Ravi Prakash
      3. HADOOP-7664.patch
        0.7 kB
        Ravi Prakash
      4. HADOOP-7664.patch
        0.7 kB
        Ravi Prakash

        Issue Links

          Activity

          Hide
          Ravi Prakash added a comment -

          Not including a test because it is a trivial change to a log message. Please review and commit

          Show
          Ravi Prakash added a comment - Not including a test because it is a trivial change to a log message. Please review and commit
          Hide
          Harsh J added a comment -

          I'd say that the warning is fine because it only warns that something is trying to override, regardless of what value gets put in.

          Agree that it makes a lot of noise though. I'd like to know what others think.

          Show
          Harsh J added a comment - I'd say that the warning is fine because it only warns that something is trying to override, regardless of what value gets put in. Agree that it makes a lot of noise though. I'd like to know what others think.
          Hide
          Harsh J added a comment -

          Marking as PA on Ravi's behalf.

          Show
          Harsh J added a comment - Marking as PA on Ravi's behalf.
          Hide
          Ravi Prakash added a comment -

          Thanks Harsh!

          Users usually tend to use the same config files as were used to start up the cluster. Ideally they should have two different config files, one for bringing up the cluster. And another for submitting jobs. Unfortunately that is cumbersome.

          I think the point of throwing a warning though is that some configuration which the user wanted to set was not able to be set and as a consequence their job might behave differently. If the value of the setting was already what the user wanted, she/he does not need notification in my opinion.

          Show
          Ravi Prakash added a comment - Thanks Harsh! Users usually tend to use the same config files as were used to start up the cluster. Ideally they should have two different config files, one for bringing up the cluster. And another for submitting jobs. Unfortunately that is cumbersome. I think the point of throwing a warning though is that some configuration which the user wanted to set was not able to be set and as a consequence their job might behave differently. If the value of the setting was already what the user wanted, she/he does not need notification in my opinion.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12495375/HADOOP-7664.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          -1 findbugs. The patch appears to introduce 1 new Findbugs (version 1.3.9) warnings.

          -1 release audit. The applied patch generated 1 release audit warnings (more than the trunk's current 0 warnings).

          +1 core tests. The patch passed unit tests in .

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/213//testReport/
          Release audit warnings: https://builds.apache.org/job/PreCommit-HADOOP-Build/213//artifact/trunk/patchprocess/patchReleaseAuditProblems.txt
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HADOOP-Build/213//artifact/trunk/hadoop-common-project/patchprocess/newPatchFindbugsWarningshadoop-common.html
          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/213//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12495375/HADOOP-7664.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 1 new Findbugs (version 1.3.9) warnings. -1 release audit. The applied patch generated 1 release audit warnings (more than the trunk's current 0 warnings). +1 core tests. The patch passed unit tests in . +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/213//testReport/ Release audit warnings: https://builds.apache.org/job/PreCommit-HADOOP-Build/213//artifact/trunk/patchprocess/patchReleaseAuditProblems.txt Findbugs warnings: https://builds.apache.org/job/PreCommit-HADOOP-Build/213//artifact/trunk/hadoop-common-project/patchprocess/newPatchFindbugsWarningshadoop-common.html Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/213//console This message is automatically generated.
          Hide
          Aaron T. Myers added a comment -

          I can't comment on how frequent this occurrence is or how noisy it is in the logs, but at the very least I don't think we should be doing "value != properties.getProperty(attr)" but rather "!value.equals(properties.getProperty(attr))".

          Show
          Aaron T. Myers added a comment - I can't comment on how frequent this occurrence is or how noisy it is in the logs, but at the very least I don't think we should be doing " value != properties.getProperty(attr) " but rather " !value.equals(properties.getProperty(attr)) ".
          Hide
          Ravi Prakash added a comment -

          Gaawwwww!!! I never thought I'd fall into that trap!
          Thanks Aaron!

          Show
          Ravi Prakash added a comment - Gaawwwww!!! I never thought I'd fall into that trap! Thanks Aaron!
          Hide
          Ravi Prakash added a comment -

          Also, we see this on the console when submitting a job. So its not only in the logs

          Show
          Ravi Prakash added a comment - Also, we see this on the console when submitting a job. So its not only in the logs
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12495429/HADOOP-7664.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          -1 release audit. The applied patch generated 1 release audit warnings (more than the trunk's current 0 warnings).

          +1 core tests. The patch passed unit tests in .

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/217//testReport/
          Release audit warnings: https://builds.apache.org/job/PreCommit-HADOOP-Build/217//artifact/trunk/patchprocess/patchReleaseAuditProblems.txt
          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/217//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12495429/HADOOP-7664.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. -1 release audit. The applied patch generated 1 release audit warnings (more than the trunk's current 0 warnings). +1 core tests. The patch passed unit tests in . +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/217//testReport/ Release audit warnings: https://builds.apache.org/job/PreCommit-HADOOP-Build/217//artifact/trunk/patchprocess/patchReleaseAuditProblems.txt Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/217//console This message is automatically generated.
          Hide
          Harsh J added a comment -

          Hello Ravi,

          Please revise the patch to match the code formatting conditions. An if condition need not have whitespace wrapped around it. (This is why you're getting a release audit -1).

          Show
          Harsh J added a comment - Hello Ravi, Please revise the patch to match the code formatting conditions. An if condition need not have whitespace wrapped around it. (This is why you're getting a release audit -1).
          Hide
          Ravi Prakash added a comment -

          Thanks Harsh! I've updated the patch.

          Show
          Ravi Prakash added a comment - Thanks Harsh! I've updated the patch.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12496120/HADOOP-7664.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          -1 release audit. The applied patch generated 1 release audit warnings (more than the trunk's current 0 warnings).

          +1 core tests. The patch passed unit tests in .

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/222//testReport/
          Release audit warnings: https://builds.apache.org/job/PreCommit-HADOOP-Build/222//artifact/trunk/patchprocess/patchReleaseAuditProblems.txt
          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/222//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12496120/HADOOP-7664.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. -1 release audit. The applied patch generated 1 release audit warnings (more than the trunk's current 0 warnings). +1 core tests. The patch passed unit tests in . +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/222//testReport/ Release audit warnings: https://builds.apache.org/job/PreCommit-HADOOP-Build/222//artifact/trunk/patchprocess/patchReleaseAuditProblems.txt Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/222//console This message is automatically generated.
          Hide
          Ravi Prakash added a comment -

          1. Hmm.. I don't understand why there is still a release audit warning. The code directly above has the same code formatting. If that did not generate RAT warnings this new code shouldn't either

          (more than the trunk's current 0 warnings)

          2. I can't run test-patch for rat warnings and I've filed a jira for that https://issues.apache.org/jira/browse/MAPREDUCE-3069?focusedCommentId=13112626#comment-13112626
          3. The link for Release audit warnings is broken

          Release audit warnings: https://builds.apache.org/job/PreCommit-HADOOP-Build/222//artifact/trunk/patchprocess/patchReleaseAuditProblems.txt

          Show
          Ravi Prakash added a comment - 1. Hmm.. I don't understand why there is still a release audit warning. The code directly above has the same code formatting. If that did not generate RAT warnings this new code shouldn't either (more than the trunk's current 0 warnings) 2. I can't run test-patch for rat warnings and I've filed a jira for that https://issues.apache.org/jira/browse/MAPREDUCE-3069?focusedCommentId=13112626#comment-13112626 3. The link for Release audit warnings is broken Release audit warnings: https://builds.apache.org/job/PreCommit-HADOOP-Build/222//artifact/trunk/patchprocess/patchReleaseAuditProblems.txt
          Hide
          Uma Maheswara Rao G added a comment -

          Hi Ravi,
          Release audit warning is not related to this patch.
          I have already filed a JIRA for it HADOOP-7669

          Thanks
          Uma

          Show
          Uma Maheswara Rao G added a comment - Hi Ravi, Release audit warning is not related to this patch. I have already filed a JIRA for it HADOOP-7669 Thanks Uma
          Hide
          Ravi Prakash added a comment -

          Thanks for the heads up Uma!

          Show
          Ravi Prakash added a comment - Thanks for the heads up Uma!
          Hide
          Suresh Srinivas added a comment -

          +1 for the patch.

          Show
          Suresh Srinivas added a comment - +1 for the patch.
          Hide
          Suresh Srinivas added a comment -

          I committed the patch. Thank you Ravi Prakash.

          Show
          Suresh Srinivas added a comment - I committed the patch. Thank you Ravi Prakash.
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-trunk-Commit #1067 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1067/)
          HADOOP-7664. Remove warmings when overriding final parameter configuration if the override value is same as the final parameter value. Contributed by Ravi Prakash.

          suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1182645
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/Configuration.java
          Show
          Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #1067 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1067/ ) HADOOP-7664 . Remove warmings when overriding final parameter configuration if the override value is same as the final parameter value. Contributed by Ravi Prakash. suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1182645 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/Configuration.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk-Commit #1145 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1145/)
          HADOOP-7664. Remove warmings when overriding final parameter configuration if the override value is same as the final parameter value. Contributed by Ravi Prakash.

          suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1182645
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/Configuration.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #1145 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1145/ ) HADOOP-7664 . Remove warmings when overriding final parameter configuration if the override value is same as the final parameter value. Contributed by Ravi Prakash. suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1182645 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/Configuration.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk-Commit #1087 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1087/)
          HADOOP-7664. Remove warmings when overriding final parameter configuration if the override value is same as the final parameter value. Contributed by Ravi Prakash.

          suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1182645
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/Configuration.java
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #1087 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1087/ ) HADOOP-7664 . Remove warmings when overriding final parameter configuration if the override value is same as the final parameter value. Contributed by Ravi Prakash. suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1182645 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/Configuration.java
          Hide
          Ravi Prakash added a comment -

          Thanks Suresh.

          Show
          Ravi Prakash added a comment - Thanks Suresh.
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk #859 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/859/)
          HADOOP-7664. Remove warmings when overriding final parameter configuration if the override value is same as the final parameter value. Contributed by Ravi Prakash.

          suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1182645
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/Configuration.java
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #859 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/859/ ) HADOOP-7664 . Remove warmings when overriding final parameter configuration if the override value is same as the final parameter value. Contributed by Ravi Prakash. suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1182645 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/Configuration.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk #829 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/829/)
          HADOOP-7664. Remove warmings when overriding final parameter configuration if the override value is same as the final parameter value. Contributed by Ravi Prakash.

          suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1182645
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/Configuration.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #829 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/829/ ) HADOOP-7664 . Remove warmings when overriding final parameter configuration if the override value is same as the final parameter value. Contributed by Ravi Prakash. suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1182645 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/Configuration.java
          Hide
          Ravi Prakash added a comment -

          Reopening to fix 0.20-security

          Show
          Ravi Prakash added a comment - Reopening to fix 0.20-security
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12502974/HADOOP-7664.branch-0.20-security.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          -1 patch. The patch command could not apply the patch.

          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/367//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12502974/HADOOP-7664.branch-0.20-security.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/367//console This message is automatically generated.
          Hide
          Tsz Wo Nicholas Sze added a comment -

          Committed the 0.20s patch to 0.20s and 0.20.205. Thanks, Ravi!

          Show
          Tsz Wo Nicholas Sze added a comment - Committed the 0.20s patch to 0.20s and 0.20.205. Thanks, Ravi!
          Hide
          Matt Foley added a comment -

          correct Fixed Versions.

          Show
          Matt Foley added a comment - correct Fixed Versions.

            People

            • Assignee:
              Ravi Prakash
              Reporter:
              Ravi Prakash
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Time Tracking

                Estimated:
                Original Estimate - 1m
                1m
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 1m
                1m

                  Development