Details

    • Type: Sub-task Sub-task
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: security
    • Labels:
      None

      Description

      Follow what Hadoop is doing. Authentication via JAAS:
      http://issues.apache.org/jira/browse/HADOOP-6299
      http://java.sun.com/javase/6/docs/technotes/guides/security/jaas/JAASRefGuide.html

      Should support Kerberos, Unix, and LDAP authentication options.

      Integrate with authentication mechanisms for IPC and HDFS.

        Issue Links

          Activity

          Hide
          Mikhail Antonov added a comment -

          Thanks for opening additional Jira, we can discuss there further.

          Show
          Mikhail Antonov added a comment - Thanks for opening additional Jira, we can discuss there further.
          Hide
          Andrew Purtell added a comment -

          I'm resolving this issue. According to the OP:

          Should support Kerberos, Unix, and LDAP authentication options.

          This was done under HBASE-2742. We support Kerberos via an integration with Hadoop authentication. We support other (local) authentication methods by way of Hadoop's "simple" authentication option.

          Integrate with authentication mechanisms for IPC and HDFS.

          This was done under HBASE-2742 and HBASE-3615.

          HBase 0.92 was the first released version with these changes. Strong authentication support was provided by a separate RPC engine enabled through an optional configuration setting through 0.92 and 0.94 releases. The RPC engines were unified for the 0.96 release.

          Show
          Andrew Purtell added a comment - I'm resolving this issue. According to the OP: Should support Kerberos, Unix, and LDAP authentication options. This was done under HBASE-2742 . We support Kerberos via an integration with Hadoop authentication. We support other (local) authentication methods by way of Hadoop's "simple" authentication option. Integrate with authentication mechanisms for IPC and HDFS. This was done under HBASE-2742 and HBASE-3615 . HBase 0.92 was the first released version with these changes. Strong authentication support was provided by a separate RPC engine enabled through an optional configuration setting through 0.92 and 0.94 releases. The RPC engines were unified for the 0.96 release.
          Hide
          Andrew Purtell added a comment -

          I filed HBASE-9929 to capture this discussion. Gary Helmling you might want to repeat your above comment over there.

          Show
          Andrew Purtell added a comment - I filed HBASE-9929 to capture this discussion. Gary Helmling you might want to repeat your above comment over there.
          Hide
          Gary Helmling added a comment -

          It would certainly be possible to implement your own proxy, as Andy describes, which would need its own kerberos credentials and would perform its own authentication of clients. But that doesn't seem like core HBase functionality. Instead it's putting a proxy in place in order to circumvent security.

          I think the direction for HBase will be to support pluggable authentication of clients at the RPC layer, using the same mechanisms under development for Hadoop, but unfortunately that may be some time away.

          Show
          Gary Helmling added a comment - It would certainly be possible to implement your own proxy, as Andy describes, which would need its own kerberos credentials and would perform its own authentication of clients. But that doesn't seem like core HBase functionality. Instead it's putting a proxy in place in order to circumvent security. I think the direction for HBase will be to support pluggable authentication of clients at the RPC layer, using the same mechanisms under development for Hadoop, but unfortunately that may be some time away.
          Hide
          Mikhail Antonov added a comment -

          That is exactly what I need for my requirements, yes. I thought of trying to relay HBase shell somehow thru HBase REST server (which can impersonate) or so, but there is no obvious way which I see now to do that. Do you think it's possible, or just none ever asked to support such thing?

          Show
          Mikhail Antonov added a comment - That is exactly what I need for my requirements, yes. I thought of trying to relay HBase shell somehow thru HBase REST server (which can impersonate) or so, but there is no obvious way which I see now to do that. Do you think it's possible, or just none ever asked to support such thing?
          Hide
          Andrew Purtell added a comment -

          Various ecosystem services like Hive or Oozie do support impersonation of end users, thus bypassing that, and allow end users to be authenticated via pluggable authentication (which may authenticate users against ldap, mysql database and such). But for HBase Shell there's no impersonation possible as of now

          Hive or Oozie impersonate by utilizing a service process registered with the NN in the NN config to be afforded the elevated privilege of impersonation, and then they do their own thing. The HBase shell is a regular HBase client wrapped with an HBase DSL within the JRuby IRB, which could run anywhere, and cannot be trusted in that way. If I understand correctly, what you could use is some kind of "administration server" which would reside at a fixed location and could be trusted to impersonate, and then the shell could be modified to proxy administrative commands through it. - Yes?

          Show
          Andrew Purtell added a comment - Various ecosystem services like Hive or Oozie do support impersonation of end users, thus bypassing that, and allow end users to be authenticated via pluggable authentication (which may authenticate users against ldap, mysql database and such). But for HBase Shell there's no impersonation possible as of now Hive or Oozie impersonate by utilizing a service process registered with the NN in the NN config to be afforded the elevated privilege of impersonation, and then they do their own thing. The HBase shell is a regular HBase client wrapped with an HBase DSL within the JRuby IRB, which could run anywhere, and cannot be trusted in that way. If I understand correctly, what you could use is some kind of "administration server" which would reside at a fixed location and could be trusted to impersonate, and then the shell could be modified to proxy administrative commands through it. - Yes?
          Hide
          Mikhail Antonov added a comment -

          Yeah, I don't question whether this jira is finished or not, I just thought it may be appropriate place to ask a few questions which I couldn't answer after reading docs, JIRA and sources.

          • it's fair to say direct use of kerberos is the only option today, as with the case of Hadoop in general.*
            Various ecosystem services like Hive or Oozie do support impersonation of end users, thus bypassing that, and allow end users to be authenticated via pluggable authentication (which may authenticate users against ldap, mysql database and such). But for HBase Shell there's no impersonation possible as of now, right, and there're no developments in this direction?
          Show
          Mikhail Antonov added a comment - Yeah, I don't question whether this jira is finished or not, I just thought it may be appropriate place to ask a few questions which I couldn't answer after reading docs, JIRA and sources. it's fair to say direct use of kerberos is the only option today, as with the case of Hadoop in general.* Various ecosystem services like Hive or Oozie do support impersonation of end users, thus bypassing that, and allow end users to be authenticated via pluggable authentication (which may authenticate users against ldap, mysql database and such). But for HBase Shell there's no impersonation possible as of now, right, and there're no developments in this direction?
          Hide
          Andrew Purtell added a comment -

          can't seem to go directly thru LDAP instance without requiring user principal to be in Kerberos, right?

          Correct, it's fair to say direct use of kerberos is the only option today, as with the case of Hadoop in general.

          I still think this jira could be resolved. Additional authentication options for Hadoop are under discussion in various Hadoop JIRAs, which we can pick up when they become available.

          Show
          Andrew Purtell added a comment - can't seem to go directly thru LDAP instance without requiring user principal to be in Kerberos, right? Correct, it's fair to say direct use of kerberos is the only option today, as with the case of Hadoop in general. I still think this jira could be resolved. Additional authentication options for Hadoop are under discussion in various Hadoop JIRAs, which we can pick up when they become available.
          Hide
          Mikhail Antonov added a comment -

          I know HBase resorts to Hadoop group mapping service, which can be plugged to NN, but authentication of user itself (username/password/domain) can't seem to go directly thru LDAP instance without requiring user principal to be in Kerberos, right?

          Show
          Mikhail Antonov added a comment - I know HBase resorts to Hadoop group mapping service, which can be plugged to NN, but authentication of user itself (username/password/domain) can't seem to go directly thru LDAP instance without requiring user principal to be in Kerberos, right?
          Hide
          Mikhail Antonov added a comment -

          I see, thanks for the comment Andrew.

          I'm actually looking for the deployment picture, when I can avoid having kerberos principals for end customer of HBase Shell, but it looks like it's not supported now?

          What I'm trying to do is following:

          • Namenode/JT are secured already and have kerberos principals
          • HiveServer2 is already secured in our installation, and configured in such a way that HS itself has kerberos principals, but end users log in via LDAP and their credentials are passed to NN/JT as proxied kerberos tickets. So impersonation works just fine, like in Oozie and other "service-style" entities
          • HBase REST seems to support impersonation

          But, I don't see an option to allow end users of HBase Shell (John Smith) to authenticate via LDAP (without creating trusted bridge between Kerberos and AD, since it may be arbitrary LDAP server), and then get his credentials to be proxied via some service Kerberos principal and to be passed to HBase (something like "jsmith via hbase-shell-user/domain@REALM").

          Is there any support for that, or am I missing something?

          Show
          Mikhail Antonov added a comment - I see, thanks for the comment Andrew. I'm actually looking for the deployment picture, when I can avoid having kerberos principals for end customer of HBase Shell, but it looks like it's not supported now? What I'm trying to do is following: Namenode/JT are secured already and have kerberos principals HiveServer2 is already secured in our installation, and configured in such a way that HS itself has kerberos principals, but end users log in via LDAP and their credentials are passed to NN/JT as proxied kerberos tickets. So impersonation works just fine, like in Oozie and other "service-style" entities HBase REST seems to support impersonation But, I don't see an option to allow end users of HBase Shell (John Smith) to authenticate via LDAP (without creating trusted bridge between Kerberos and AD, since it may be arbitrary LDAP server), and then get his credentials to be proxied via some service Kerberos principal and to be passed to HBase (something like "jsmith via hbase-shell-user/domain@REALM"). Is there any support for that, or am I missing something?
          Hide
          Andrew Purtell added a comment -

          Is there any update on that, particularly around LDAP authentication for end users?

          I suppose we can close this because the scope of the OP has been addressed by a bunch of other JIRAs.

          Regarding LDAP in particular, we can use Hadoop's support for LDAP in group mapping (that was HADOOP-8121). Beyond that could be the topic of a new enhancement JIRA.

          Show
          Andrew Purtell added a comment - Is there any update on that, particularly around LDAP authentication for end users? I suppose we can close this because the scope of the OP has been addressed by a bunch of other JIRAs. Regarding LDAP in particular, we can use Hadoop's support for LDAP in group mapping (that was HADOOP-8121 ). Beyond that could be the topic of a new enhancement JIRA.
          Hide
          Mikhail Antonov added a comment -

          Is there any update on that, particularly around LDAP authentication for end users?

          Show
          Mikhail Antonov added a comment - Is there any update on that, particularly around LDAP authentication for end users?
          Hide
          stack added a comment -

          Moving out of 0.92.0. Pull it back in if you think different.

          Show
          stack added a comment - Moving out of 0.92.0. Pull it back in if you think different.
          Hide
          Andrew Purtell added a comment -
          Show
          Andrew Purtell added a comment - Hadoop will use SASL and Kerberos: https://issues.apache.org/jira/secure/attachment/12428493/security-design.pdf

            People

            • Assignee:
              Gary Helmling
              Reporter:
              Andrew Purtell
            • Votes:
              1 Vote for this issue
              Watchers:
              12 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development