Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-1023

Allow http server to start as regular principal if https principal not defined.

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.22.0
    • Fix Version/s: 0.22.0
    • Component/s: namenode
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Currently limitations in Sun's KerbSSL implementation require the https server to be run as "host/[machine]@realm." and another Sun KerbSSL limitation appears to require you to store all principals in the same keytab, meaning fully functional, secured Namenodes require combined keytabs. However, it may be that one wishes to run a namenode without a secondary namenode or other utilities that require https. In this case, we should allow the http server to start and log a warning that it will not be able to accept https connections.

      1. HADOOP-1023-Y20-1.patch
        2 kB
        Jakob Homan
      2. HDFS-1023-trunk.patch
        2 kB
        Jakob Homan
      3. HDFS-1023-Y20.patch
        2 kB
        Jakob Homan
      4. HDFS-1023-Y20-Update.patch
        0.8 kB
        Jakob Homan
      5. HDFS-1023-Y20-Update-2.patch
        2 kB
        Jakob Homan

        Activity

        Hide
        hudson Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk-Commit #339 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Hdfs-trunk-Commit/339/)
        HDFS-1023. Allow http server to start as regular principal if https principal not defined.

        Show
        hudson Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #339 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Hdfs-trunk-Commit/339/ ) HDFS-1023 . Allow http server to start as regular principal if https principal not defined.
        Hide
        jghoman Jakob Homan added a comment -

        Test failures are unrelated and this is a Kerberos change, which is currently untestable. I've committed this. Resolving as fixed.

        Show
        jghoman Jakob Homan added a comment - Test failures are unrelated and this is a Kerberos change, which is currently untestable. I've committed this. Resolving as fixed.
        Hide
        hadoopqa Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12449114/HDFS-1023-trunk.patch
        against trunk revision 962678.

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed core unit tests.

        -1 contrib tests. The patch failed contrib unit tests.

        Test results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/428/testReport/
        Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/428/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/428/artifact/trunk/build/test/checkstyle-errors.html
        Console output: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/428/console

        This message is automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12449114/HDFS-1023-trunk.patch against trunk revision 962678. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/428/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/428/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/428/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/428/console This message is automatically generated.
        Hide
        devaraj Devaraj Das added a comment -

        +1

        Show
        devaraj Devaraj Das added a comment - +1
        Hide
        jghoman Jakob Homan added a comment -

        Trunk patch, straight forward port of Y20 version. Updates smooshed into this patch. No unit tests possible... manually tested on Y! clusters.

        Show
        jghoman Jakob Homan added a comment - Trunk patch, straight forward port of Y20 version. Updates smooshed into this patch. No unit tests possible... manually tested on Y! clusters.
        Hide
        jghoman Jakob Homan added a comment -

        Update to HDFS-1023 patch.

        Show
        jghoman Jakob Homan added a comment - Update to HDFS-1023 patch.
        Hide
        jghoman Jakob Homan added a comment -

        Minor update to Y20 patch.

        Show
        jghoman Jakob Homan added a comment - Minor update to Y20 patch.
        Hide
        jghoman Jakob Homan added a comment -

        Small update to avoid Findbugs warning.

        Show
        jghoman Jakob Homan added a comment - Small update to avoid Findbugs warning.
        Hide
        chris.douglas Chris Douglas added a comment -

        +1

        Show
        chris.douglas Chris Douglas added a comment - +1
        Hide
        jghoman Jakob Homan added a comment -

        It is pretty amazing/disappointing that the normal HTTP/[machine] doesn't work.

        I was pretty amazed at this too. Definitely complicates deploying a secure cluster, although only the NN and SNN need to have these combined keytabs, since they are the only https servers.
        Line 299: http://hg.openjdk.java.net/jdk7/tl/jdk/file/893034df4ec2/src/share/classes/sun/security/ssl/krb5/KerberosClientKeyExchangeImpl.java

        Show
        jghoman Jakob Homan added a comment - It is pretty amazing/disappointing that the normal HTTP/ [machine] doesn't work. I was pretty amazed at this too. Definitely complicates deploying a secure cluster, although only the NN and SNN need to have these combined keytabs, since they are the only https servers. Line 299: http://hg.openjdk.java.net/jdk7/tl/jdk/file/893034df4ec2/src/share/classes/sun/security/ssl/krb5/KerberosClientKeyExchangeImpl.java
        Hide
        aw Allen Wittenauer added a comment -

        > another Sun KerbSSL limitation appears to require you to store all principals in the same keytab

        FWIW, most of the implementations (at least that are exposed to the user) require that all principals that might get used for a given service are stored in one keytab. Even so:

        > Sun's KerbSSL implementation require the https server to be run as "host/[machine]@realm.

        It is pretty amazing/disappointing that the normal HTTP/[machine] doesn't work.

        Show
        aw Allen Wittenauer added a comment - > another Sun KerbSSL limitation appears to require you to store all principals in the same keytab FWIW, most of the implementations (at least that are exposed to the user) require that all principals that might get used for a given service are stored in one keytab. Even so: > Sun's KerbSSL implementation require the https server to be run as "host/ [machine] @realm. It is pretty amazing/disappointing that the normal HTTP/ [machine] doesn't work.
        Hide
        jghoman Jakob Homan added a comment -

        Patch implementing above, plus some better logging and extra security check, for Y20 distro. Trunk patch soon. Unit tests not applicable... sigh.

        Show
        jghoman Jakob Homan added a comment - Patch implementing above, plus some better logging and extra security check, for Y20 distro. Trunk patch soon. Unit tests not applicable... sigh.

          People

          • Assignee:
            jghoman Jakob Homan
            Reporter:
            jghoman Jakob Homan
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development