Hadoop Common
  1. Hadoop Common
  2. HADOOP-8727

Gracefully deprecate dfs.umaskmode in 2.x onwards

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 2.0.0-alpha
    • Fix Version/s: 0.23.3, 2.0.2-alpha
    • Component/s: conf
    • Labels:
      None

      Description

      While HADOOP-6234 added dfs.umaskmode in 0.21.0, the subsequent HADOOP-6233 simply renamed it, again in 0.21.0, without any deprecation mechanism (understandable).

      However, 1.x now carries dfs.umaskmode but there isn't a graceful deprecation when one upgrades to 2.x. We should recreate this prop and add it to the deprecated list.

      1. HADOOP-8727.patch
        2 kB
        Harsh J
      2. HADOOP-8727.patch
        2 kB
        Harsh J

        Issue Links

          Activity

          Transition Time In Source Status Execution Times Last Executer Last Execution Date
          Open Open Patch Available Patch Available
          10h 35m 1 Harsh J 26/Aug/12 15:30
          Patch Available Patch Available Resolved Resolved
          5d 4h 23m 1 Robert Joseph Evans 31/Aug/12 19:53
          Resolved Resolved Closed Closed
          40d 22h 51m 1 Arun C Murthy 11/Oct/12 18:45
          Arun C Murthy made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Arun C Murthy made changes -
          Fix Version/s 3.0.0 [ 12320357 ]
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk #1183 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1183/)
          HADOOP-8727. Gracefully deprecate dfs.umaskmode in 2.x onwards (Harsh J via bobby) (Revision 1379537)

          Result = SUCCESS
          bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1379537
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/Configuration.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/resources/core-default.xml
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #1183 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1183/ ) HADOOP-8727 . Gracefully deprecate dfs.umaskmode in 2.x onwards (Harsh J via bobby) (Revision 1379537) Result = SUCCESS bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1379537 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/Configuration.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/resources/core-default.xml
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk #1152 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1152/)
          HADOOP-8727. Gracefully deprecate dfs.umaskmode in 2.x onwards (Harsh J via bobby) (Revision 1379537)

          Result = SUCCESS
          bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1379537
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/Configuration.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/resources/core-default.xml
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #1152 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1152/ ) HADOOP-8727 . Gracefully deprecate dfs.umaskmode in 2.x onwards (Harsh J via bobby) (Revision 1379537) Result = SUCCESS bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1379537 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/Configuration.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/resources/core-default.xml
          Hide
          Harsh J added a comment -

          Thanks Bobby! I had a busy week battling some escalations, and couldn't make this in time, sorry for having blocked progress.

          Show
          Harsh J added a comment - Thanks Bobby! I had a busy week battling some escalations, and couldn't make this in time, sorry for having blocked progress.
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-0.23-Build #361 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/361/)
          svn merge -c 1379537 FIXES: HADOOP-8727. Gracefully deprecate dfs.umaskmode in 2.x onwards (Harsh J via bobby) (Revision 1379544)

          Result = UNSTABLE
          bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1379544
          Files :

          • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/Configuration.java
          • /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/resources/core-default.xml
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-0.23-Build #361 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/361/ ) svn merge -c 1379537 FIXES: HADOOP-8727 . Gracefully deprecate dfs.umaskmode in 2.x onwards (Harsh J via bobby) (Revision 1379544) Result = UNSTABLE bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1379544 Files : /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/Configuration.java /hadoop/common/branches/branch-0.23/hadoop-common-project/hadoop-common/src/main/resources/core-default.xml
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk-Commit #2689 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2689/)
          HADOOP-8727. Gracefully deprecate dfs.umaskmode in 2.x onwards (Harsh J via bobby) (Revision 1379537)

          Result = FAILURE
          bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1379537
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/Configuration.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/resources/core-default.xml
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #2689 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2689/ ) HADOOP-8727 . Gracefully deprecate dfs.umaskmode in 2.x onwards (Harsh J via bobby) (Revision 1379537) Result = FAILURE bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1379537 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/Configuration.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/resources/core-default.xml
          Robert Joseph Evans made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Fix Version/s 0.23.3 [ 12320059 ]
          Fix Version/s 3.0.0 [ 12320357 ]
          Fix Version/s 2.2.0-alpha [ 12322473 ]
          Resolution Fixed [ 1 ]
          Hide
          Robert Joseph Evans added a comment -

          Thanks Harsh,

          I put this into trunk, branch-2, and branch-0.23

          Show
          Robert Joseph Evans added a comment - Thanks Harsh, I put this into trunk, branch-2, and branch-0.23
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk-Commit #2723 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2723/)
          HADOOP-8727. Gracefully deprecate dfs.umaskmode in 2.x onwards (Harsh J via bobby) (Revision 1379537)

          Result = SUCCESS
          bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1379537
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/Configuration.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/resources/core-default.xml
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #2723 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2723/ ) HADOOP-8727 . Gracefully deprecate dfs.umaskmode in 2.x onwards (Harsh J via bobby) (Revision 1379537) Result = SUCCESS bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1379537 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/Configuration.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/resources/core-default.xml
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-trunk-Commit #2660 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2660/)
          HADOOP-8727. Gracefully deprecate dfs.umaskmode in 2.x onwards (Harsh J via bobby) (Revision 1379537)

          Result = SUCCESS
          bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1379537
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/Configuration.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/resources/core-default.xml
          Show
          Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #2660 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2660/ ) HADOOP-8727 . Gracefully deprecate dfs.umaskmode in 2.x onwards (Harsh J via bobby) (Revision 1379537) Result = SUCCESS bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1379537 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/Configuration.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/resources/core-default.xml
          Hide
          Robert Joseph Evans added a comment -

          The patch is simple enough, and the request too, so in order to let me get this into 0.23.3 I will just make the requested doc change and check it in. +1 for the patch by the way.

          Show
          Robert Joseph Evans added a comment - The patch is simple enough, and the request too, so in order to let me get this into 0.23.3 I will just make the requested doc change and check it in. +1 for the patch by the way.
          Hide
          Eli Collins added a comment -

          I'd word "Configure the umask value to apply when creating new files using the FileSystem classes." instead as "The umask used when creating files and directories." since this applies to both files and directories and eg other APIs like FileContext or FsShell.

          Otherwise looks great.

          Show
          Eli Collins added a comment - I'd word "Configure the umask value to apply when creating new files using the FileSystem classes." instead as "The umask used when creating files and directories." since this applies to both files and directories and eg other APIs like FileContext or FsShell. Otherwise looks great.
          Jason Lowe made changes -
          Link This issue is duplicated by HADOOP-8735 [ HADOOP-8735 ]
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12542566/HADOOP-8727.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests in hadoop-common-project/hadoop-common:

          org.apache.hadoop.ha.TestZKFailoverController

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1362//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1362//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12542566/HADOOP-8727.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The javadoc tool did not generate any warning messages. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests in hadoop-common-project/hadoop-common: org.apache.hadoop.ha.TestZKFailoverController +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1362//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1362//console This message is automatically generated.
          Harsh J made changes -
          Attachment HADOOP-8727.patch [ 12542566 ]
          Hide
          Harsh J added a comment -

          Good point. I failed to find that addDeprecatedKeys method the first time I saw Configuration class yesterday. Fixed.

          Must've been sleepiness

          Show
          Harsh J added a comment - Good point. I failed to find that addDeprecatedKeys method the first time I saw Configuration class yesterday. Fixed. Must've been sleepiness
          Hide
          Eli Collins added a comment -

          Why not use Configuration#addDeprecation?

          Show
          Eli Collins added a comment - Why not use Configuration#addDeprecation?
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12542451/HADOOP-8727.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests in hadoop-common-project/hadoop-common:

          org.apache.hadoop.ha.TestZKFailoverController

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1361//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1361//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12542451/HADOOP-8727.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The javadoc tool did not generate any warning messages. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests in hadoop-common-project/hadoop-common: org.apache.hadoop.ha.TestZKFailoverController +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1361//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1361//console This message is automatically generated.
          Harsh J made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Assignee Harsh J [ qwertymaniac ]
          Harsh J made changes -
          Attachment HADOOP-8727.patch [ 12542451 ]
          Hide
          Harsh J added a comment -

          Attached patch adds a fallback for the dfs.umaskmode property. Since it was the new style (accepting both octal and symbolic) I added it in the appropriate place.

          Also threw in a core-default.xml doc for the new property name as it was missing.

          Show
          Harsh J added a comment - Attached patch adds a fallback for the dfs.umaskmode property. Since it was the new style (accepting both octal and symbolic) I added it in the appropriate place. Also threw in a core-default.xml doc for the new property name as it was missing.
          Harsh J made changes -
          Link This issue relates to HADOOP-6234 [ HADOOP-6234 ]
          Harsh J made changes -
          Link This issue relates to HADOOP-6233 [ HADOOP-6233 ]
          Harsh J made changes -
          Field Original Value New Value
          Description While https://issues.apache.org/jira/browse/HADOOP-6234 added dfs.umaskmode in 0.21.0, the subsequent https://issues.apache.org/jira/browse/HADOOP-6233 simply renamed it, again in 0.21.0, without any deprecation mechanism (understandable).

          However, 1.x now carries dfs.umaskmode but there isn't a graceful deprecation when one upgrades to 2.x. We should recreate this prop and add it to the deprecated list.
          While HADOOP-6234 added dfs.umaskmode in 0.21.0, the subsequent HADOOP-6233 simply renamed it, again in 0.21.0, without any deprecation mechanism (understandable).

          However, 1.x now carries dfs.umaskmode but there isn't a graceful deprecation when one upgrades to 2.x. We should recreate this prop and add it to the deprecated list.
          Harsh J created issue -

            People

            • Assignee:
              Harsh J
              Reporter:
              Harsh J
            • Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development