Hadoop Common
  1. Hadoop Common
  2. HADOOP-8460

Document proper setting of HADOOP_PID_DIR and HADOOP_SECURE_DN_PID_DIR

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 1.0.3, 2.0.0-alpha
    • Fix Version/s: 1.2.0, 3.0.0, 2.0.2-alpha
    • Component/s: documentation
    • Labels:
      None

      Description

      We should document that in a properly setup cluster HADOOP_PID_DIR and HADOOP_SECURE_DN_PID_DIR should not point to /tmp, but should point to a directory that normal users do not have access to.

      1. HADOOP-8460-branch-1.txt
        2 kB
        Robert Joseph Evans
      2. HADOOP-8460-branch-1.txt
        1 kB
        Robert Joseph Evans
      3. HADOOP-8460-trunk.txt
        2 kB
        Robert Joseph Evans
      4. HADOOP-8460-trunk.txt
        2 kB
        Robert Joseph Evans

        Activity

        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk #1098 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1098/)
        HADOOP-8460. Document proper setting of HADOOP_PID_DIR and HADOOP_SECURE_DN_PID_DIR (bobby) (Revision 1345304)

        Result = FAILURE
        bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1345304
        Files :

        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/conf/hadoop-env.sh
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/ClusterSetup.apt.vm
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #1098 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1098/ ) HADOOP-8460 . Document proper setting of HADOOP_PID_DIR and HADOOP_SECURE_DN_PID_DIR (bobby) (Revision 1345304) Result = FAILURE bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1345304 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/conf/hadoop-env.sh /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/ClusterSetup.apt.vm
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk #1064 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1064/)
        HADOOP-8460. Document proper setting of HADOOP_PID_DIR and HADOOP_SECURE_DN_PID_DIR (bobby) (Revision 1345304)

        Result = FAILURE
        bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1345304
        Files :

        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/conf/hadoop-env.sh
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/ClusterSetup.apt.vm
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #1064 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1064/ ) HADOOP-8460 . Document proper setting of HADOOP_PID_DIR and HADOOP_SECURE_DN_PID_DIR (bobby) (Revision 1345304) Result = FAILURE bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1345304 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/conf/hadoop-env.sh /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/ClusterSetup.apt.vm
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk-Commit #2382 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2382/)
        HADOOP-8460. Document proper setting of HADOOP_PID_DIR and HADOOP_SECURE_DN_PID_DIR (bobby) (Revision 1345304)

        Result = SUCCESS
        bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1345304
        Files :

        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/conf/hadoop-env.sh
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/ClusterSetup.apt.vm
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #2382 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2382/ ) HADOOP-8460 . Document proper setting of HADOOP_PID_DIR and HADOOP_SECURE_DN_PID_DIR (bobby) (Revision 1345304) Result = SUCCESS bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1345304 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/conf/hadoop-env.sh /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/ClusterSetup.apt.vm
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Common-trunk-Commit #2310 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2310/)
        HADOOP-8460. Document proper setting of HADOOP_PID_DIR and HADOOP_SECURE_DN_PID_DIR (bobby) (Revision 1345304)

        Result = SUCCESS
        bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1345304
        Files :

        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/conf/hadoop-env.sh
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/ClusterSetup.apt.vm
        Show
        Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #2310 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2310/ ) HADOOP-8460 . Document proper setting of HADOOP_PID_DIR and HADOOP_SECURE_DN_PID_DIR (bobby) (Revision 1345304) Result = SUCCESS bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1345304 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/conf/hadoop-env.sh /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/ClusterSetup.apt.vm
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk-Commit #2327 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2327/)
        HADOOP-8460. Document proper setting of HADOOP_PID_DIR and HADOOP_SECURE_DN_PID_DIR (bobby) (Revision 1345304)

        Result = FAILURE
        bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1345304
        Files :

        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
        • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/conf/hadoop-env.sh
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/ClusterSetup.apt.vm
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #2327 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2327/ ) HADOOP-8460 . Document proper setting of HADOOP_PID_DIR and HADOOP_SECURE_DN_PID_DIR (bobby) (Revision 1345304) Result = FAILURE bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1345304 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/conf/hadoop-env.sh /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/ClusterSetup.apt.vm
        Hide
        Robert Joseph Evans added a comment -

        I put this into trunk, branch-2, and branch-1.

        Show
        Robert Joseph Evans added a comment - I put this into trunk, branch-2, and branch-1.
        Hide
        Aaron T. Myers added a comment -

        +1, the latest patches look good to me.

        Show
        Aaron T. Myers added a comment - +1, the latest patches look good to me.
        Hide
        Roman Shaposhnik added a comment -

        @Robert, I just wanted to have a fuller record of what's going on with various Hadoop deployments models on this JIRA for folks to to get a full disclosure.

        Show
        Roman Shaposhnik added a comment - @Robert, I just wanted to have a fuller record of what's going on with various Hadoop deployments models on this JIRA for folks to to get a full disclosure.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12530552/HADOOP-8460-trunk.txt
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +0 tests included. The patch appears to be a documentation patch that doesn't require tests.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these unit tests in hadoop-common-project/hadoop-common hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-site:

        org.apache.hadoop.fs.viewfs.TestViewFsTrash

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1068//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1068//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12530552/HADOOP-8460-trunk.txt against trunk revision . +1 @author. The patch does not contain any @author tags. +0 tests included. The patch appears to be a documentation patch that doesn't require tests. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The javadoc tool did not generate any warning messages. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests in hadoop-common-project/hadoop-common hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-site: org.apache.hadoop.fs.viewfs.TestViewFsTrash +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1068//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1068//console This message is automatically generated.
        Hide
        Robert Joseph Evans added a comment -

        Updated text of patches following Aaron's comments.

        Show
        Robert Joseph Evans added a comment - Updated text of patches following Aaron's comments.
        Hide
        Robert Joseph Evans added a comment -

        Roman,

        Most of Hadoop's packaging also sets the default up properly, I believe that I saw /var/run... in the setup scripts too. It is not a big issue hence it is just a major bug, and we could drop it to minor if you want. There is still a potential security hole if you are installing from the TGZ, and it is good to warn users about it so they don't misconfigure it themselves. If you want me to change the text of the warning accordingly, I am happy to do so.

        Aaron,

        Thanks for the feedback I will update the patches.

        Show
        Robert Joseph Evans added a comment - Roman, Most of Hadoop's packaging also sets the default up properly, I believe that I saw /var/run... in the setup scripts too. It is not a big issue hence it is just a major bug, and we could drop it to minor if you want. There is still a potential security hole if you are installing from the TGZ, and it is good to warn users about it so they don't misconfigure it themselves. If you want me to change the text of the warning accordingly, I am happy to do so. Aaron, Thanks for the feedback I will update the patches.
        Hide
        Roman Shaposhnik added a comment -

        This is one of the things that the packaging takes care of. Not sure what's the story with Hadoop native packages, but Bigtop puts everything in /var/run... so this is not a problem

        Show
        Roman Shaposhnik added a comment - This is one of the things that the packaging takes care of. Not sure what's the story with Hadoop native packages, but Bigtop puts everything in /var/run... so this is not a problem
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12530464/HADOOP-8460-trunk.txt
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +0 tests included. The patch appears to be a documentation patch that doesn't require tests.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these unit tests in hadoop-common-project/hadoop-common hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-site:

        org.apache.hadoop.fs.viewfs.TestViewFsTrash

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1065//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1065//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12530464/HADOOP-8460-trunk.txt against trunk revision . +1 @author. The patch does not contain any @author tags. +0 tests included. The patch appears to be a documentation patch that doesn't require tests. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The javadoc tool did not generate any warning messages. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests in hadoop-common-project/hadoop-common hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-site: org.apache.hadoop.fs.viewfs.TestViewFsTrash +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1065//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1065//console This message is automatically generated.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12530465/HADOOP-8460-branch-1.txt
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +0 tests included. The patch appears to be a documentation patch that doesn't require tests.

        -1 patch. The patch command could not apply the patch.

        Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1066//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12530465/HADOOP-8460-branch-1.txt against trunk revision . +1 @author. The patch does not contain any @author tags. +0 tests included. The patch appears to be a documentation patch that doesn't require tests. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1066//console This message is automatically generated.
        Hide
        Aaron T. Myers added a comment -

        Tiny nit:

        +# NOTE: these should be set to a directory that general users do not have write
        +# permission.

        I think you should invert the direction of this sentence, similarly to the other warning, i.e. "these should be set to a directory that can only be written to by..."

        +1 otherwise

        Show
        Aaron T. Myers added a comment - Tiny nit: +# NOTE: these should be set to a directory that general users do not have write +# permission. I think you should invert the direction of this sentence, similarly to the other warning, i.e. "these should be set to a directory that can only be written to by..." +1 otherwise
        Hide
        Robert Joseph Evans added a comment -

        This patch is intended for branch-1.

        Show
        Robert Joseph Evans added a comment - This patch is intended for branch-1.
        Hide
        Robert Joseph Evans added a comment -

        This patch is for trunk/2.0. I will look at putting up a similar patch for 1.0.

        Show
        Robert Joseph Evans added a comment - This patch is for trunk/2.0. I will look at putting up a similar patch for 1.0.

          People

          • Assignee:
            Robert Joseph Evans
            Reporter:
            Robert Joseph Evans
          • Votes:
            0 Vote for this issue
            Watchers:
            8 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development