Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-1009

Support Kerberos authorization in HDFSProxy

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.22.0
    • Fix Version/s: 0.21.0
    • Component/s: contrib/hdfsproxy
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      We should add a filter to support Kerberos authorization in HDFSProxy.

      1. HDFS-1009.patch
        4 kB
        Srikanth Sundarrajan
      2. HDFS-1009.patch
        4 kB
        Srikanth Sundarrajan

        Issue Links

          Activity

          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk-Commit #233 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Hdfs-trunk-Commit/233/)

          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #233 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Hdfs-trunk-Commit/233/ )
          Hide
          Srikanth Sundarrajan added a comment -

          All the core test failures in

          http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/303/testReport/

          are failing with the folllowing message.

          Error Message
          org/apache/hadoop/conf/Configuration

          Stacktrace
          java.lang.NoClassDefFoundError: org/apache/hadoop/conf/Configuration
          at java.lang.Class.forName0(Native Method)
          at java.lang.Class.forName(Class.java:169)
          Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.conf.Configuration
          at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
          at java.security.AccessController.doPrivileged(Native Method)
          at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
          at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
          at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
          at java.lang.ClassLoader.loadClass(ClassLoader.java:252)
          at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:320)

          Note: This patch includes a new class in contrib and modifies tomcat-web.xml also under contrib

          Index: src/contrib/hdfsproxy/conf/tomcat-web.xml
          Index: src/contrib/hdfsproxy/src/java/org/apache/hadoop/hdfsproxy/KerberosAuthorizationFilter.java

          Show
          Srikanth Sundarrajan added a comment - All the core test failures in http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/303/testReport/ are failing with the folllowing message. Error Message org/apache/hadoop/conf/Configuration Stacktrace java.lang.NoClassDefFoundError: org/apache/hadoop/conf/Configuration at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:169) Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.conf.Configuration at java.net.URLClassLoader$1.run(URLClassLoader.java:200) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:188) at java.lang.ClassLoader.loadClass(ClassLoader.java:307) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) at java.lang.ClassLoader.loadClass(ClassLoader.java:252) at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:320) Note: This patch includes a new class in contrib and modifies tomcat-web.xml also under contrib Index: src/contrib/hdfsproxy/conf/tomcat-web.xml Index: src/contrib/hdfsproxy/src/java/org/apache/hadoop/hdfsproxy/KerberosAuthorizationFilter.java
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12440938/HDFS-1009.patch
          against trunk revision 931256.

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed core unit tests.

          -1 contrib tests. The patch failed contrib unit tests.

          Test results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/303/testReport/
          Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/303/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/303/artifact/trunk/build/test/checkstyle-errors.html
          Console output: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/303/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12440938/HDFS-1009.patch against trunk revision 931256. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/303/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/303/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/303/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/303/console This message is automatically generated.
          Hide
          Tsz Wo Nicholas Sze added a comment -

          I have committed this. Thanks, Srikanth!

          Show
          Tsz Wo Nicholas Sze added a comment - I have committed this. Thanks, Srikanth!
          Hide
          Srikanth Sundarrajan added a comment -

          Patch looks good. Could you add some javadoc in the header of the new class KerberosAuthorizationFilter?

          Nicholas, Thanks for taking time to review the patch. Have uploaded a revised patch which includes javadoc for the class.

          Show
          Srikanth Sundarrajan added a comment - Patch looks good. Could you add some javadoc in the header of the new class KerberosAuthorizationFilter? Nicholas, Thanks for taking time to review the patch. Have uploaded a revised patch which includes javadoc for the class.
          Hide
          Tsz Wo Nicholas Sze added a comment -

          Patch looks good. Could you add some javadoc in the header of the new class KerberosAuthorizationFilter?

          Show
          Tsz Wo Nicholas Sze added a comment - Patch looks good. Could you add some javadoc in the header of the new class KerberosAuthorizationFilter?
          Hide
          Srikanth Sundarrajan added a comment -

          Output from test-patch

          [exec] -1 overall.
          [exec]
          [exec] +1 @author. The patch does not contain any @author tags.
          [exec]
          [exec] -1 tests included. The patch doesn't appear to include any new or modified tests.
          [exec] Please justify why no new tests are needed for this patch.
          [exec] Also please list what manual steps were performed to verify this patch.
          [exec]
          [exec] +1 javadoc. The javadoc tool did not generate any warning messages.
          [exec]
          [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings.
          [exec]
          [exec] +1 findbugs. The patch does not introduce any new Findbugs warnings.
          [exec]
          [exec] +1 release audit. The applied patch does not increase the total number of release audit warnings.

          test-contrib:

          test:
          [cactus] Tomcat 5.x is stopped

          BUILD SUCCESSFUL
          Total time: 4 minutes 39 seconds

          ------------

          No new tests added with this patch, as the patch is specific to Keberos and the current unit test framework doesn't extend itself to test this. However the patch has been tested manually.

          A keytab file for the proxy user was created and the principal in keytab file is configured as proxy user in the Namenode configuration[core-site.xml] (hadoop.proxyuser.proxy.users, hadoop.proxyuser.proxy.ip-addresses). Ip address configured in Namenode core-site.xml is that of the server where hdfsproxy is setup to run and the proxy user is same as the user in the keytab file. With this, doAs requests are successful and the requests are able to retrieve files only readable by the requesting user or the users' group.

          Show
          Srikanth Sundarrajan added a comment - Output from test-patch [exec] -1 overall. [exec] [exec] +1 @author. The patch does not contain any @author tags. [exec] [exec] -1 tests included. The patch doesn't appear to include any new or modified tests. [exec] Please justify why no new tests are needed for this patch. [exec] Also please list what manual steps were performed to verify this patch. [exec] [exec] +1 javadoc. The javadoc tool did not generate any warning messages. [exec] [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings. [exec] [exec] +1 findbugs. The patch does not introduce any new Findbugs warnings. [exec] [exec] +1 release audit. The applied patch does not increase the total number of release audit warnings. test-contrib: test: [cactus] Tomcat 5.x is stopped BUILD SUCCESSFUL Total time: 4 minutes 39 seconds ------------ No new tests added with this patch, as the patch is specific to Keberos and the current unit test framework doesn't extend itself to test this. However the patch has been tested manually. A keytab file for the proxy user was created and the principal in keytab file is configured as proxy user in the Namenode configuration [core-site.xml] (hadoop.proxyuser.proxy.users, hadoop.proxyuser.proxy.ip-addresses). Ip address configured in Namenode core-site.xml is that of the server where hdfsproxy is setup to run and the proxy user is same as the user in the keytab file. With this, doAs requests are successful and the requests are able to retrieve files only readable by the requesting user or the users' group.
          Hide
          Srikanth Sundarrajan added a comment -

          Separating the KerberosAuthroization patch from HDFS-481.

          KerberosAuthorizationFilter uses a proxy user to act on behalf of the requesting user picked up from Ldap through LdapIpDirFilter.

          Show
          Srikanth Sundarrajan added a comment - Separating the KerberosAuthroization patch from HDFS-481 . KerberosAuthorizationFilter uses a proxy user to act on behalf of the requesting user picked up from Ldap through LdapIpDirFilter.
          Hide
          Srikanth Sundarrajan added a comment -

          LdapIpDirFilter sets the authorized.ugi on the request object. HDFSProxy needs to proxy as the user making the request and present it to Name node when a DFS request is made. This is implemented in the revised HDFS-481 patch.

          Show
          Srikanth Sundarrajan added a comment - LdapIpDirFilter sets the authorized.ugi on the request object. HDFSProxy needs to proxy as the user making the request and present it to Name node when a DFS request is made. This is implemented in the revised HDFS-481 patch.

            People

            • Assignee:
              Srikanth Sundarrajan
              Reporter:
              Srikanth Sundarrajan
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development