Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-12294

Throw an Exception when fs.permissions.umask-mode is misconfigured

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.0.0-alpha1
    • Component/s: conf
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Incompatible change
    • Release Note:
      The support of the deprecated dfs.umask key is removed in Hadoop 3.0.

      Description

      provide better visibility of parsing configuration failure by logging full error message and propagate error message of parsing configuration back to client

      1. HADOOP-12294.2.patch
        1 kB
        Chang Li
      2. HADOOP-12294.patch
        2 kB
        Chang Li

        Issue Links

          Activity

          Hide
          zxu zhihai xu added a comment -

          Chang Li, thanks for working on the issue. It looks like your patch will have regression, which change the current code functionality at FsPermission#getUMask. The current code only throws IllegalArgumentException when oldUmask is equal to Integer.MIN_VALUE.
          The reason is if oldUmask is not equal to Integer.MIN_VALUE(default value), it will use the oldUmask to create FsPermission.
          You patch removed the check, which means: it will throw IllegalArgumentException instead of creating FsPermission with oldUmask when oldUmask is not equal to Integer.MIN_VALUE.

          int umask = DEFAULT_UMASK;
                if(oldUmask != Integer.MIN_VALUE) { // Property was set with old key
                  if (umask != oldUmask) {
                    LOG.warn(DEPRECATED_UMASK_LABEL
                        + " configuration key is deprecated. " + "Convert to "
                        + UMASK_LABEL + ", using octal or symbolic umask "
                        + "specifications.");
                    // Old and new umask values do not match - Use old umask
                    umask = oldUmask;
                  }
                }
              }    
              return new FsPermission((short)umask);
          
          Show
          zxu zhihai xu added a comment - Chang Li , thanks for working on the issue. It looks like your patch will have regression, which change the current code functionality at FsPermission#getUMask . The current code only throws IllegalArgumentException when oldUmask is equal to Integer.MIN_VALUE. The reason is if oldUmask is not equal to Integer.MIN_VALUE(default value), it will use the oldUmask to create FsPermission. You patch removed the check, which means: it will throw IllegalArgumentException instead of creating FsPermission with oldUmask when oldUmask is not equal to Integer.MIN_VALUE. int umask = DEFAULT_UMASK; if (oldUmask != Integer .MIN_VALUE) { // Property was set with old key if (umask != oldUmask) { LOG.warn(DEPRECATED_UMASK_LABEL + " configuration key is deprecated. " + "Convert to " + UMASK_LABEL + ", using octal or symbolic umask " + "specifications." ); // Old and new umask values do not match - Use old umask umask = oldUmask; } } } return new FsPermission(( short )umask);
          Hide
          lichangleo Chang Li added a comment -

          zhihai xu, thanks for comment. You are write, this change technically breaks backwards compatibility if someone was relying on a misconfigured conf. But the question is whether that's supposed to be a valid case in the first place. I remove the change in cluster.java since that's a separate issue. Jason Lowe could comment on this issue?

          Show
          lichangleo Chang Li added a comment - zhihai xu , thanks for comment. You are write, this change technically breaks backwards compatibility if someone was relying on a misconfigured conf. But the question is whether that's supposed to be a valid case in the first place. I remove the change in cluster.java since that's a separate issue. Jason Lowe could comment on this issue?
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 17m 54s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          +1 javac 7m 45s There were no new javac warning messages.
          +1 javadoc 9m 52s There were no new javadoc warning messages.
          +1 release audit 0m 20s The applied patch does not increase the total number of release audit warnings.
          +1 checkstyle 1m 54s There were no new checkstyle issues.
          +1 whitespace 0m 0s The patch has no lines that end in whitespace.
          +1 install 1m 18s mvn install still works.
          +1 eclipse:eclipse 0m 32s The patch built with eclipse:eclipse.
          +1 findbugs 3m 12s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          -1 common tests 21m 57s Tests failed in hadoop-common.
          +1 mapreduce tests 1m 43s Tests passed in hadoop-mapreduce-client-core.
              66m 30s  



          Reason Tests
          Failed unit tests hadoop.fs.TestLocalFsFCStatistics



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12748225/HADOOP-12294.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / d0e0ba8
          hadoop-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7388/artifact/patchprocess/testrun_hadoop-common.txt
          hadoop-mapreduce-client-core test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7388/artifact/patchprocess/testrun_hadoop-mapreduce-client-core.txt
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/7388/testReport/
          Java 1.7.0_55
          uname Linux asf907.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/7388/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 17m 54s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac 7m 45s There were no new javac warning messages. +1 javadoc 9m 52s There were no new javadoc warning messages. +1 release audit 0m 20s The applied patch does not increase the total number of release audit warnings. +1 checkstyle 1m 54s There were no new checkstyle issues. +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 18s mvn install still works. +1 eclipse:eclipse 0m 32s The patch built with eclipse:eclipse. +1 findbugs 3m 12s The patch does not introduce any new Findbugs (version 3.0.0) warnings. -1 common tests 21m 57s Tests failed in hadoop-common. +1 mapreduce tests 1m 43s Tests passed in hadoop-mapreduce-client-core.     66m 30s   Reason Tests Failed unit tests hadoop.fs.TestLocalFsFCStatistics Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12748225/HADOOP-12294.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / d0e0ba8 hadoop-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7388/artifact/patchprocess/testrun_hadoop-common.txt hadoop-mapreduce-client-core test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7388/artifact/patchprocess/testrun_hadoop-mapreduce-client-core.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/7388/testReport/ Java 1.7.0_55 uname Linux asf907.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/7388/console This message was automatically generated.
          Hide
          aw Allen Wittenauer added a comment -

          But the question is whether that's supposed to be a valid case in the first place.

          It's likely too late to change this in 2.x, even though the old decimal format has been deprecated for a while.

          Show
          aw Allen Wittenauer added a comment - But the question is whether that's supposed to be a valid case in the first place. It's likely too late to change this in 2.x, even though the old decimal format has been deprecated for a while.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 16m 37s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          +1 javac 7m 38s There were no new javac warning messages.
          +1 javadoc 9m 44s There were no new javadoc warning messages.
          +1 release audit 0m 21s The applied patch does not increase the total number of release audit warnings.
          +1 checkstyle 1m 6s There were no new checkstyle issues.
          +1 whitespace 0m 0s The patch has no lines that end in whitespace.
          +1 install 1m 22s mvn install still works.
          +1 eclipse:eclipse 0m 31s The patch built with eclipse:eclipse.
          -1 findbugs 1m 54s The patch appears to introduce 1 new Findbugs (version 3.0.0) warnings.
          -1 common tests 21m 41s Tests failed in hadoop-common.
              60m 57s  



          Reason Tests
          FindBugs module:hadoop-common
          Failed unit tests hadoop.fs.permission.TestFsPermission
            hadoop.security.token.delegation.web.TestWebDelegationToken



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12748250/HADOOP-12294.2.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / d311a38
          Findbugs warnings https://builds.apache.org/job/PreCommit-HADOOP-Build/7390/artifact/patchprocess/newPatchFindbugsWarningshadoop-common.html
          hadoop-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7390/artifact/patchprocess/testrun_hadoop-common.txt
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/7390/testReport/
          Java 1.7.0_55
          uname Linux asf907.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/7390/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 16m 37s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac 7m 38s There were no new javac warning messages. +1 javadoc 9m 44s There were no new javadoc warning messages. +1 release audit 0m 21s The applied patch does not increase the total number of release audit warnings. +1 checkstyle 1m 6s There were no new checkstyle issues. +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 22s mvn install still works. +1 eclipse:eclipse 0m 31s The patch built with eclipse:eclipse. -1 findbugs 1m 54s The patch appears to introduce 1 new Findbugs (version 3.0.0) warnings. -1 common tests 21m 41s Tests failed in hadoop-common.     60m 57s   Reason Tests FindBugs module:hadoop-common Failed unit tests hadoop.fs.permission.TestFsPermission   hadoop.security.token.delegation.web.TestWebDelegationToken Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12748250/HADOOP-12294.2.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / d311a38 Findbugs warnings https://builds.apache.org/job/PreCommit-HADOOP-Build/7390/artifact/patchprocess/newPatchFindbugsWarningshadoop-common.html hadoop-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7390/artifact/patchprocess/testrun_hadoop-common.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/7390/testReport/ Java 1.7.0_55 uname Linux asf907.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/7390/console This message was automatically generated.
          Hide
          jlowe Jason Lowe added a comment -

          To me this looks like a bug introduced by HADOOP-6521. It has the confusing behavior that if one tries unsuccessfully to set the new property but the old property has a valid setting, rather than complaining about the bad value it silently falls back to the old property setting. So users can think they set the property when in reality that setting was ignored unless they watched the logs very closely to notice the warning message.

          However it's been this way for a very long time, which makes it hard to reason whether anyone has relied on this unintuitive behavior intentionally or otherwise. So in that sense I agree with zhihai and Allen that we can avoid any potential breakage in 2.x and propose we fix this for 3.x.

          Show
          jlowe Jason Lowe added a comment - To me this looks like a bug introduced by HADOOP-6521 . It has the confusing behavior that if one tries unsuccessfully to set the new property but the old property has a valid setting, rather than complaining about the bad value it silently falls back to the old property setting. So users can think they set the property when in reality that setting was ignored unless they watched the logs very closely to notice the warning message. However it's been this way for a very long time, which makes it hard to reason whether anyone has relied on this unintuitive behavior intentionally or otherwise. So in that sense I agree with zhihai and Allen that we can avoid any potential breakage in 2.x and propose we fix this for 3.x.
          Hide
          zxu zhihai xu added a comment -

          thanks for the explanation jason, +1 for fixing this in 3.x.

          Show
          zxu zhihai xu added a comment - thanks for the explanation jason, +1 for fixing this in 3.x.
          Hide
          wheat9 Haohui Mai added a comment -

          Committed to trunk. Thanks Chang Li for the contribution.

          Show
          wheat9 Haohui Mai added a comment - Committed to trunk. Thanks Chang Li for the contribution.
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #8846 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8846/)
          HADOOP-12294. Remove the support of the deprecated dfs.umask. (wheat9: rev 4492b9e7302b8c84dddec9713d8148cc6183f46b)

          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/permission/FsPermission.java
          • hadoop-common-project/hadoop-common/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #8846 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8846/ ) HADOOP-12294 . Remove the support of the deprecated dfs.umask. (wheat9: rev 4492b9e7302b8c84dddec9713d8148cc6183f46b) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/permission/FsPermission.java hadoop-common-project/hadoop-common/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #697 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/697/)
          HADOOP-12294. Remove the support of the deprecated dfs.umask. (wheat9: rev 4492b9e7302b8c84dddec9713d8148cc6183f46b)

          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/permission/FsPermission.java
          • hadoop-common-project/hadoop-common/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #697 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/697/ ) HADOOP-12294 . Remove the support of the deprecated dfs.umask. (wheat9: rev 4492b9e7302b8c84dddec9713d8148cc6183f46b) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/permission/FsPermission.java hadoop-common-project/hadoop-common/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk #2638 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2638/)
          HADOOP-12294. Remove the support of the deprecated dfs.umask. (wheat9: rev 4492b9e7302b8c84dddec9713d8148cc6183f46b)

          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/permission/FsPermission.java
          • hadoop-common-project/hadoop-common/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #2638 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2638/ ) HADOOP-12294 . Remove the support of the deprecated dfs.umask. (wheat9: rev 4492b9e7302b8c84dddec9713d8148cc6183f46b) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/permission/FsPermission.java hadoop-common-project/hadoop-common/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk #1433 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1433/)
          HADOOP-12294. Remove the support of the deprecated dfs.umask. (wheat9: rev 4492b9e7302b8c84dddec9713d8148cc6183f46b)

          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/permission/FsPermission.java
          • hadoop-common-project/hadoop-common/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk #1433 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1433/ ) HADOOP-12294 . Remove the support of the deprecated dfs.umask. (wheat9: rev 4492b9e7302b8c84dddec9713d8148cc6183f46b) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/permission/FsPermission.java hadoop-common-project/hadoop-common/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #709 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/709/)
          HADOOP-12294. Remove the support of the deprecated dfs.umask. (wheat9: rev 4492b9e7302b8c84dddec9713d8148cc6183f46b)

          • hadoop-common-project/hadoop-common/CHANGES.txt
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/permission/FsPermission.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #709 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/709/ ) HADOOP-12294 . Remove the support of the deprecated dfs.umask. (wheat9: rev 4492b9e7302b8c84dddec9713d8148cc6183f46b) hadoop-common-project/hadoop-common/CHANGES.txt hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/permission/FsPermission.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk #2567 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2567/)
          HADOOP-12294. Remove the support of the deprecated dfs.umask. (wheat9: rev 4492b9e7302b8c84dddec9713d8148cc6183f46b)

          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/permission/FsPermission.java
          • hadoop-common-project/hadoop-common/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2567 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2567/ ) HADOOP-12294 . Remove the support of the deprecated dfs.umask. (wheat9: rev 4492b9e7302b8c84dddec9713d8148cc6183f46b) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/permission/FsPermission.java hadoop-common-project/hadoop-common/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #629 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/629/)
          HADOOP-12294. Remove the support of the deprecated dfs.umask. (wheat9: rev 4492b9e7302b8c84dddec9713d8148cc6183f46b)

          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/permission/FsPermission.java
          • hadoop-common-project/hadoop-common/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #629 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/629/ ) HADOOP-12294 . Remove the support of the deprecated dfs.umask. (wheat9: rev 4492b9e7302b8c84dddec9713d8148cc6183f46b) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/permission/FsPermission.java hadoop-common-project/hadoop-common/CHANGES.txt
          Hide
          xiaochen Xiao Chen added a comment -

          Hi Chang Li, this seems to cause test TestPermission#testBackwardCompatibility to fail. Could you please take a look? Thanks.

          Error Message
          expected:<18> but was:<0>

          Stacktrace

          java.lang.AssertionError: expected:<18> but was:<0>
          	at org.junit.Assert.fail(Assert.java:88)
          	at org.junit.Assert.failNotEquals(Assert.java:743)
          	at org.junit.Assert.assertEquals(Assert.java:118)
          	at org.junit.Assert.assertEquals(Assert.java:555)
          	at org.junit.Assert.assertEquals(Assert.java:542)
          	at org.apache.hadoop.security.TestPermission.testBackwardCompatibility(TestPermission.java:98)
          
          Show
          xiaochen Xiao Chen added a comment - Hi Chang Li , this seems to cause test TestPermission#testBackwardCompatibility to fail. Could you please take a look? Thanks. Error Message expected:<18> but was:<0> Stacktrace java.lang.AssertionError: expected:<18> but was:<0> at org.junit.Assert.fail(Assert.java:88) at org.junit.Assert.failNotEquals(Assert.java:743) at org.junit.Assert.assertEquals(Assert.java:118) at org.junit.Assert.assertEquals(Assert.java:555) at org.junit.Assert.assertEquals(Assert.java:542) at org.apache.hadoop.security.TestPermission.testBackwardCompatibility(TestPermission.java:98)
          Hide
          xiaochen Xiao Chen added a comment -

          FYI - looks like Wei-Chiu Chuang will fix this test via HDFS-9451.

          Show
          xiaochen Xiao Chen added a comment - FYI - looks like Wei-Chiu Chuang will fix this test via HDFS-9451 .
          Hide
          lichangleo Chang Li added a comment -

          Hi Xiao Chen, I am looking at this problem and following HDFS-9451

          Show
          lichangleo Chang Li added a comment - Hi Xiao Chen , I am looking at this problem and following HDFS-9451
          Hide
          xiaochen Xiao Chen added a comment -

          Thanks Chang!

          Show
          xiaochen Xiao Chen added a comment - Thanks Chang!
          Hide
          cnauroth Chris Nauroth added a comment -

          If the intent was to remove support from trunk/3.0, then I think all traces of it should have been removed from the codebase. Right now, I see that FsPermission#DEPRECATED_UMASK_LABEL is still present. If that had been removed, then it also would have forced us to fix the test failures reported in HDFS-9451. Those test suites would have started getting compilation failures after removal of FsPermission#DEPRECATED_UMASK_LABEL.

          I propose that we revert the current patch, create a new patch that really removes all traces from the codebase, and incorporates any test fixes currently tracked in HDFS-9451. Please let me know if I'm missing some reason that FsPermission#DEPRECATED_UMASK_LABEL needs to remain. Thanks!

          Show
          cnauroth Chris Nauroth added a comment - If the intent was to remove support from trunk/3.0, then I think all traces of it should have been removed from the codebase. Right now, I see that FsPermission#DEPRECATED_UMASK_LABEL is still present. If that had been removed, then it also would have forced us to fix the test failures reported in HDFS-9451 . Those test suites would have started getting compilation failures after removal of FsPermission#DEPRECATED_UMASK_LABEL . I propose that we revert the current patch, create a new patch that really removes all traces from the codebase, and incorporates any test fixes currently tracked in HDFS-9451 . Please let me know if I'm missing some reason that FsPermission#DEPRECATED_UMASK_LABEL needs to remain. Thanks!
          Hide
          lichangleo Chang Li added a comment -

          Chris Nauroth, the original intention of this jira is to remove the use of DEPRECATED_UMASK_LABEL in FsPermission#getUMask, the title was "Throw an Exception when fs.permissions.umask-mode is misconfigured" but changed by Haohui before it was commit.

          Show
          lichangleo Chang Li added a comment - Chris Nauroth , the original intention of this jira is to remove the use of DEPRECATED_UMASK_LABEL in FsPermission#getUMask , the title was "Throw an Exception when fs.permissions.umask-mode is misconfigured" but changed by Haohui before it was commit.
          Hide
          lichangleo Chang Li added a comment -

          I suggest change back the title of this jira, and let HDFS-9451 fix the broken tests and open a new jira if desired to do the real work of remove support of deprecated dfs.umask. Thoughts?

          Show
          lichangleo Chang Li added a comment - I suggest change back the title of this jira, and let HDFS-9451 fix the broken tests and open a new jira if desired to do the real work of remove support of deprecated dfs.umask. Thoughts?
          Hide
          cnauroth Chris Nauroth added a comment -

          Chang Li, thanks for the clarification about the original intent of the patch.

          Is it worth considering fully eliminating dfs.umask from trunk? We're already at a point where the change is backwards-incompatible. It has been deprecated for a long time in 2.x, so maybe it's time just to remove the whole thing?

          If not, then I think we need to correct this JIRA title and its CHANGES.txt entry to avoid confusion. Reading the title made me think that dfs.umask would stop working altogether.

          Show
          cnauroth Chris Nauroth added a comment - Chang Li , thanks for the clarification about the original intent of the patch. Is it worth considering fully eliminating dfs.umask from trunk? We're already at a point where the change is backwards-incompatible. It has been deprecated for a long time in 2.x, so maybe it's time just to remove the whole thing? If not, then I think we need to correct this JIRA title and its CHANGES.txt entry to avoid confusion. Reading the title made me think that dfs.umask would stop working altogether.
          Hide
          cnauroth Chris Nauroth added a comment -

          Chang Li, sorry, we crossed comments. Yes, your proposal sounds fine to me.

          Show
          cnauroth Chris Nauroth added a comment - Chang Li , sorry, we crossed comments. Yes, your proposal sounds fine to me.
          Hide
          lichangleo Chang Li added a comment -

          Hi Chris Nauroth, I have changed back the Jira summary also created HADOOP-12595 to track the work of remove the support of the deprecated dfs.umask. Could you help correct the CHANGES.txt?

          Show
          lichangleo Chang Li added a comment - Hi Chris Nauroth , I have changed back the Jira summary also created HADOOP-12595 to track the work of remove the support of the deprecated dfs.umask. Could you help correct the CHANGES.txt?
          Hide
          cnauroth Chris Nauroth added a comment -

          I have pushed up the CHANGES.txt correction.

          Show
          cnauroth Chris Nauroth added a comment - I have pushed up the CHANGES.txt correction.
          Hide
          lichangleo Chang Li added a comment -

          Thanks Chris Nauroth!

          Show
          lichangleo Chang Li added a comment - Thanks Chris Nauroth !
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #8877 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8877/)
          HADOOP-12294. Correct CHANGES.txt description. (cnauroth: rev 0e1c12c1740132778fb29e41fc47d374fb87021e)

          • hadoop-common-project/hadoop-common/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #8877 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8877/ ) HADOOP-12294 . Correct CHANGES.txt description. (cnauroth: rev 0e1c12c1740132778fb29e41fc47d374fb87021e) hadoop-common-project/hadoop-common/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #715 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/715/)
          HADOOP-12294. Correct CHANGES.txt description. (cnauroth: rev 0e1c12c1740132778fb29e41fc47d374fb87021e)

          • hadoop-common-project/hadoop-common/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #715 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/715/ ) HADOOP-12294 . Correct CHANGES.txt description. (cnauroth: rev 0e1c12c1740132778fb29e41fc47d374fb87021e) hadoop-common-project/hadoop-common/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk #1448 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1448/)
          HADOOP-12294. Correct CHANGES.txt description. (cnauroth: rev 0e1c12c1740132778fb29e41fc47d374fb87021e)

          • hadoop-common-project/hadoop-common/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk #1448 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1448/ ) HADOOP-12294 . Correct CHANGES.txt description. (cnauroth: rev 0e1c12c1740132778fb29e41fc47d374fb87021e) hadoop-common-project/hadoop-common/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk #2657 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2657/)
          HADOOP-12294. Correct CHANGES.txt description. (cnauroth: rev 0e1c12c1740132778fb29e41fc47d374fb87021e)

          • hadoop-common-project/hadoop-common/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #2657 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2657/ ) HADOOP-12294 . Correct CHANGES.txt description. (cnauroth: rev 0e1c12c1740132778fb29e41fc47d374fb87021e) hadoop-common-project/hadoop-common/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #638 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/638/)
          HADOOP-12294. Correct CHANGES.txt description. (cnauroth: rev 0e1c12c1740132778fb29e41fc47d374fb87021e)

          • hadoop-common-project/hadoop-common/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #638 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/638/ ) HADOOP-12294 . Correct CHANGES.txt description. (cnauroth: rev 0e1c12c1740132778fb29e41fc47d374fb87021e) hadoop-common-project/hadoop-common/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #726 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/726/)
          HADOOP-12294. Correct CHANGES.txt description. (cnauroth: rev 0e1c12c1740132778fb29e41fc47d374fb87021e)

          • hadoop-common-project/hadoop-common/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #726 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/726/ ) HADOOP-12294 . Correct CHANGES.txt description. (cnauroth: rev 0e1c12c1740132778fb29e41fc47d374fb87021e) hadoop-common-project/hadoop-common/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk #2576 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2576/)
          HADOOP-12294. Correct CHANGES.txt description. (cnauroth: rev 0e1c12c1740132778fb29e41fc47d374fb87021e)

          • hadoop-common-project/hadoop-common/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2576 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2576/ ) HADOOP-12294 . Correct CHANGES.txt description. (cnauroth: rev 0e1c12c1740132778fb29e41fc47d374fb87021e) hadoop-common-project/hadoop-common/CHANGES.txt

            People

            • Assignee:
              lichangleo Chang Li
              Reporter:
              lichangleo Chang Li
            • Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development