Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-3400

DNs should be able start with jsvc even if security is disabled

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 2.0.0-alpha
    • Fix Version/s: 2.0.2-alpha
    • Component/s: datanode, scripts
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Currently if one tries to start a DN with security disabled (via hadoop.security.authentication = "simple" in the configs), but JSVC is correctly configured, the DN will refuse to start.

      1. HDFS-3400.patch
        4 kB
        Aaron T. Myers

        Activity

        Hide
        Aaron T. Myers added a comment -

        Here's a patch which makes it so that DNs will start up even if JSVC is configured properly and security is disabled via the XML confs. The only things that will cause the DN to not start are now:

        • Security is enabled but the DN is not configured to use low ports.
        • JSVC_HOME is configured, but $JSVC_HOME/jsvc is not executable.

        No tests are included since security and root access must be available to test this. I tested it manually by:

        1. Starting a DN with security disabled, and all security-related env vars set.
        2. Starting a DN with security enabled, and all security-related env vars set.
        3. Starting a DN with security disabled, and none of the security-related env vars set.
        4. Starting a DN with security enabled, and none of the security-related env vars set.
        5. Starting a DN with security enabled, all of the security-related env vars set, but not configured with low ports.

        The DN now starts properly in the first three cases. It does not start in the last two. This is the expected behavior after this patch.

        Show
        Aaron T. Myers added a comment - Here's a patch which makes it so that DNs will start up even if JSVC is configured properly and security is disabled via the XML confs. The only things that will cause the DN to not start are now: Security is enabled but the DN is not configured to use low ports. JSVC_HOME is configured, but $JSVC_HOME/jsvc is not executable. No tests are included since security and root access must be available to test this. I tested it manually by: Starting a DN with security disabled, and all security-related env vars set. Starting a DN with security enabled, and all security-related env vars set. Starting a DN with security disabled, and none of the security-related env vars set. Starting a DN with security enabled, and none of the security-related env vars set. Starting a DN with security enabled, all of the security-related env vars set, but not configured with low ports. The DN now starts properly in the first three cases. It does not start in the last two. This is the expected behavior after this patch.
        Hide
        Jakob Homan added a comment -

        +1

        Show
        Jakob Homan added a comment - +1
        Hide
        Eli Collins added a comment -

        +1 looks good to me as well

        Show
        Eli Collins added a comment - +1 looks good to me as well
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12526406/HDFS-3400.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/2413//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2413//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12526406/HDFS-3400.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/2413//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2413//console This message is automatically generated.
        Hide
        Eli Collins added a comment -

        I've committed this and merged to branch-2. Thanks ATM!

        Show
        Eli Collins added a comment - I've committed this and merged to branch-2. Thanks ATM!
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk-Commit #2302 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2302/)
        HDFS-3400. DNs should be able start with jsvc even if security is disabled. Contributed by Aaron T. Myers (Revision 1337017)

        Result = SUCCESS
        eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1337017
        Files :

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/bin/hdfs
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/SecureDataNodeStarter.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #2302 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2302/ ) HDFS-3400 . DNs should be able start with jsvc even if security is disabled. Contributed by Aaron T. Myers (Revision 1337017) Result = SUCCESS eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1337017 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/bin/hdfs /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/SecureDataNodeStarter.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Common-trunk-Commit #2228 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2228/)
        HDFS-3400. DNs should be able start with jsvc even if security is disabled. Contributed by Aaron T. Myers (Revision 1337017)

        Result = SUCCESS
        eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1337017
        Files :

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/bin/hdfs
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/SecureDataNodeStarter.java
        Show
        Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #2228 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2228/ ) HDFS-3400 . DNs should be able start with jsvc even if security is disabled. Contributed by Aaron T. Myers (Revision 1337017) Result = SUCCESS eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1337017 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/bin/hdfs /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/SecureDataNodeStarter.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk-Commit #2245 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2245/)
        HDFS-3400. DNs should be able start with jsvc even if security is disabled. Contributed by Aaron T. Myers (Revision 1337017)

        Result = SUCCESS
        eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1337017
        Files :

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/bin/hdfs
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/SecureDataNodeStarter.java
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #2245 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2245/ ) HDFS-3400 . DNs should be able start with jsvc even if security is disabled. Contributed by Aaron T. Myers (Revision 1337017) Result = SUCCESS eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1337017 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/bin/hdfs /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/SecureDataNodeStarter.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk #1040 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1040/)
        HDFS-3400. DNs should be able start with jsvc even if security is disabled. Contributed by Aaron T. Myers (Revision 1337017)

        Result = FAILURE
        eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1337017
        Files :

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/bin/hdfs
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/SecureDataNodeStarter.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #1040 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1040/ ) HDFS-3400 . DNs should be able start with jsvc even if security is disabled. Contributed by Aaron T. Myers (Revision 1337017) Result = FAILURE eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1337017 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/bin/hdfs /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/SecureDataNodeStarter.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk #1076 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1076/)
        HDFS-3400. DNs should be able start with jsvc even if security is disabled. Contributed by Aaron T. Myers (Revision 1337017)

        Result = SUCCESS
        eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1337017
        Files :

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/bin/hdfs
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/SecureDataNodeStarter.java
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #1076 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1076/ ) HDFS-3400 . DNs should be able start with jsvc even if security is disabled. Contributed by Aaron T. Myers (Revision 1337017) Result = SUCCESS eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1337017 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/bin/hdfs /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/SecureDataNodeStarter.java

          People

          • Assignee:
            Aaron T. Myers
            Reporter:
            Aaron T. Myers
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development