Hadoop Common
  1. Hadoop Common
  2. HADOOP-8727

Gracefully deprecate dfs.umaskmode in 2.x onwards

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 2.0.0-alpha
    • Fix Version/s: 0.23.3, 2.0.2-alpha
    • Component/s: conf
    • Labels:
      None

      Description

      While HADOOP-6234 added dfs.umaskmode in 0.21.0, the subsequent HADOOP-6233 simply renamed it, again in 0.21.0, without any deprecation mechanism (understandable).

      However, 1.x now carries dfs.umaskmode but there isn't a graceful deprecation when one upgrades to 2.x. We should recreate this prop and add it to the deprecated list.

      1. HADOOP-8727.patch
        2 kB
        Harsh J
      2. HADOOP-8727.patch
        2 kB
        Harsh J

        Issue Links

          Activity

          Harsh J created issue -
          Harsh J made changes -
          Field Original Value New Value
          Description While https://issues.apache.org/jira/browse/HADOOP-6234 added dfs.umaskmode in 0.21.0, the subsequent https://issues.apache.org/jira/browse/HADOOP-6233 simply renamed it, again in 0.21.0, without any deprecation mechanism (understandable).

          However, 1.x now carries dfs.umaskmode but there isn't a graceful deprecation when one upgrades to 2.x. We should recreate this prop and add it to the deprecated list.
          While HADOOP-6234 added dfs.umaskmode in 0.21.0, the subsequent HADOOP-6233 simply renamed it, again in 0.21.0, without any deprecation mechanism (understandable).

          However, 1.x now carries dfs.umaskmode but there isn't a graceful deprecation when one upgrades to 2.x. We should recreate this prop and add it to the deprecated list.
          Harsh J made changes -
          Link This issue relates to HADOOP-6233 [ HADOOP-6233 ]
          Harsh J made changes -
          Link This issue relates to HADOOP-6234 [ HADOOP-6234 ]
          Hide
          Harsh J added a comment -

          Attached patch adds a fallback for the dfs.umaskmode property. Since it was the new style (accepting both octal and symbolic) I added it in the appropriate place.

          Also threw in a core-default.xml doc for the new property name as it was missing.

          Show
          Harsh J added a comment - Attached patch adds a fallback for the dfs.umaskmode property. Since it was the new style (accepting both octal and symbolic) I added it in the appropriate place. Also threw in a core-default.xml doc for the new property name as it was missing.
          Harsh J made changes -
          Attachment HADOOP-8727.patch [ 12542451 ]
          Harsh J made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Assignee Harsh J [ qwertymaniac ]
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12542451/HADOOP-8727.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests in hadoop-common-project/hadoop-common:

          org.apache.hadoop.ha.TestZKFailoverController

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1361//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1361//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12542451/HADOOP-8727.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The javadoc tool did not generate any warning messages. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests in hadoop-common-project/hadoop-common: org.apache.hadoop.ha.TestZKFailoverController +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1361//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1361//console This message is automatically generated.
          Hide
          Eli Collins added a comment -

          Why not use Configuration#addDeprecation?

          Show
          Eli Collins added a comment - Why not use Configuration#addDeprecation?
          Hide
          Harsh J added a comment -

          Good point. I failed to find that addDeprecatedKeys method the first time I saw Configuration class yesterday. Fixed.

          Must've been sleepiness

          Show
          Harsh J added a comment - Good point. I failed to find that addDeprecatedKeys method the first time I saw Configuration class yesterday. Fixed. Must've been sleepiness
          Harsh J made changes -
          Attachment HADOOP-8727.patch [ 12542566 ]
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12542566/HADOOP-8727.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests in hadoop-common-project/hadoop-common:

          org.apache.hadoop.ha.TestZKFailoverController

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1362//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1362//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12542566/HADOOP-8727.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The javadoc tool did not generate any warning messages. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests in hadoop-common-project/hadoop-common: org.apache.hadoop.ha.TestZKFailoverController +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1362//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1362//console This message is automatically generated.
          Jason Lowe made changes -
          Link This issue is duplicated by HADOOP-8735 [ HADOOP-8735 ]
          Hide
          Eli Collins added a comment -

          I'd word "Configure the umask value to apply when creating new files using the FileSystem classes." instead as "The umask used when creating files and directories." since this applies to both files and directories and eg other APIs like FileContext or FsShell.

          Otherwise looks great.

          Show
          Eli Collins added a comment - I'd word "Configure the umask value to apply when creating new files using the FileSystem classes." instead as "The umask used when creating files and directories." since this applies to both files and directories and eg other APIs like FileContext or FsShell. Otherwise looks great.
          Hide
          Robert Joseph Evans added a comment -

          The patch is simple enough, and the request too, so in order to let me get this into 0.23.3 I will just make the requested doc change and check it in. +1 for the patch by the way.

          Show
          Robert Joseph Evans added a comment - The patch is simple enough, and the request too, so in order to let me get this into 0.23.3 I will just make the requested doc change and check it in. +1 for the patch by the way.
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-trunk-Commit #2660 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2660/)
          HADOOP-8727. Gracefully deprecate dfs.umaskmode in 2.x onwards (Harsh J via bobby) (Revision 1379537)

          Result = SUCCESS
          bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1379537
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/Configuration.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/resources/core-default.xml
          Show
          Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #2660 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2660/ ) HADOOP-8727 . Gracefully deprecate dfs.umaskmode in 2.x onwards (Harsh J via bobby) (Revision 1379537) Result = SUCCESS bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1379537 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/Configuration.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/resources/core-default.xml
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk-Commit #2723 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2723/)
          HADOOP-8727. Gracefully deprecate dfs.umaskmode in 2.x onwards (Harsh J via bobby) (Revision 1379537)

          Result = SUCCESS
          bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1379537
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/Configuration.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/resources/core-default.xml
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #2723 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2723/ ) HADOOP-8727 . Gracefully deprecate dfs.umaskmode in 2.x onwards (Harsh J via bobby) (Revision 1379537) Result = SUCCESS bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1379537 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/Configuration.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/resources/core-default.xml
          Hide
          Robert Joseph Evans added a comment -

          Thanks Harsh,

          I put this into trunk, branch-2, and branch-0.23

          Show
          Robert Joseph Evans added a comment - Thanks Harsh, I put this into trunk, branch-2, and branch-0.23
          Robert Joseph Evans made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Fix Version/s 0.23.3 [ 12320059 ]
          Fix Version/s 3.0.0 [ 12320357 ]
          Fix Version/s 2.2.0-alpha [ 12322473 ]
          Resolution Fixed [ 1 ]
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk-Commit #2689 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2689/)
          HADOOP-8727. Gracefully deprecate dfs.umaskmode in 2.x onwards (Harsh J via bobby) (Revision 1379537)

          Result = FAILURE
          bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1379537
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/Configuration.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/resources/core-default.xml
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #2689 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2689/ ) HADOOP-8727 . Gracefully deprecate dfs.umaskmode in 2.x onwards (Harsh J via bobby) (Revision 1379537) Result = FAILURE bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1379537 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/Configuration.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/resources/core-default.xml
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-0.23-Build #361 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/361/)
          svn merge -c 1379537 FIXES: HADOOP-8727. Gracefully deprecate dfs.umaskmode in 2.x onwards (Harsh J via bobby) (Revision 1379544)

          Result = UNSTABLE
          bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1379544
          Files :

          • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/Configuration.java
          • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/resources/core-default.xml
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-0.23-Build #361 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/361/ ) svn merge -c 1379537 FIXES: HADOOP-8727 . Gracefully deprecate dfs.umaskmode in 2.x onwards (Harsh J via bobby) (Revision 1379544) Result = UNSTABLE bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1379544 Files : /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/Configuration.java /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/resources/core-default.xml
          Hide
          Harsh J added a comment -

          Thanks Bobby! I had a busy week battling some escalations, and couldn't make this in time, sorry for having blocked progress.

          Show
          Harsh J added a comment - Thanks Bobby! I had a busy week battling some escalations, and couldn't make this in time, sorry for having blocked progress.
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk #1152 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1152/)
          HADOOP-8727. Gracefully deprecate dfs.umaskmode in 2.x onwards (Harsh J via bobby) (Revision 1379537)

          Result = SUCCESS
          bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1379537
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/Configuration.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/resources/core-default.xml
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #1152 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1152/ ) HADOOP-8727 . Gracefully deprecate dfs.umaskmode in 2.x onwards (Harsh J via bobby) (Revision 1379537) Result = SUCCESS bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1379537 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/Configuration.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/resources/core-default.xml
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk #1183 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1183/)
          HADOOP-8727. Gracefully deprecate dfs.umaskmode in 2.x onwards (Harsh J via bobby) (Revision 1379537)

          Result = SUCCESS
          bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1379537
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/Configuration.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/resources/core-default.xml
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #1183 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1183/ ) HADOOP-8727 . Gracefully deprecate dfs.umaskmode in 2.x onwards (Harsh J via bobby) (Revision 1379537) Result = SUCCESS bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1379537 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/Configuration.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/resources/core-default.xml
          Arun C Murthy made changes -
          Fix Version/s 3.0.0 [ 12320357 ]
          Arun C Murthy made changes -
          Status Resolved [ 5 ] Closed [ 6 ]

            People

            • Assignee:
              Harsh J
              Reporter:
              Harsh J
            • Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development