Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-5837

SQL standard based secure authorization for hive

    Details

      Description

      The current default authorization is incomplete and not secure. The alternative of storage based authorization provides security but does not provide fine grained authorization.

      The proposal is to support secure fine grained authorization in hive using SQL standard based authorization model.

        Issue Links

        There are no Sub-Tasks for this issue.

          Activity

          Hide
          rajeshhadoop Rajesh Chandramohan added a comment -

          Why we are insisting on having hive.server2.enable.doAs=false along with SQLSTDAuth . We like to support Proxy user executing hive query as well right?

          Show
          rajeshhadoop Rajesh Chandramohan added a comment - Why we are insisting on having hive.server2.enable.doAs=false along with SQLSTDAuth . We like to support Proxy user executing hive query as well right?
          Hide
          PKUKILLA Premchandra Preetham Kukillaya added a comment -

          Hi Thejas,
          In the Spark Thrift Server the SQL Std Authorization does not get triggered so is this a new defect?

          --hiveconf hive.security.authenticator.manager=org.apache.hadoop.hive.ql.security.SessionStateUserAuthenticator --hiveconf hive.security.authorization.manager=org.apache.hadoop.hive.ql.security.authorization.plugin.sqlstd.SQLStdHiveAuthorizerFactory --hiveconf hive.server2.enable.doAs=false

          Show
          PKUKILLA Premchandra Preetham Kukillaya added a comment - Hi Thejas, In the Spark Thrift Server the SQL Std Authorization does not get triggered so is this a new defect? --hiveconf hive.security.authenticator.manager=org.apache.hadoop.hive.ql.security.SessionStateUserAuthenticator --hiveconf hive.security.authorization.manager=org.apache.hadoop.hive.ql.security.authorization.plugin.sqlstd.SQLStdHiveAuthorizerFactory --hiveconf hive.server2.enable.doAs=false
          Hide
          thejas Thejas M Nair added a comment -

          Closing the umbrella jira for sql standards based authorization. All the sub tasks have been resolved.

          Show
          thejas Thejas M Nair added a comment - Closing the umbrella jira for sql standards based authorization. All the sub tasks have been resolved.
          Hide
          terrasect Alex Nastetsky added a comment -
          Show
          terrasect Alex Nastetsky added a comment - Done: https://issues.apache.org/jira/browse/HIVE-6667 . Thanks.
          Hide
          thejas Thejas M Nair added a comment -

          Alex Nastetsky Please create one for 'show tables'.

          Show
          thejas Thejas M Nair added a comment - Alex Nastetsky Please create one for 'show tables'.
          Hide
          terrasect Alex Nastetsky added a comment -

          Thanks Thejas. Should I create a ticket for "show tables" or does one already exist?

          Show
          terrasect Alex Nastetsky added a comment - Thanks Thejas. Should I create a ticket for "show tables" or does one already exist?
          Hide
          thejas Thejas M Nair added a comment -

          This tasks under this ticket haven't covered 'show tables'. The focus has been on table/view/URI access. It is unlikely to be make in timeframe for 0.13 , but is likely to be part of hive 0.14 . 0.13 should be out in few weeks as we have branched for the release. 0.14 would is likely to be out in another 3-4 months, given the frequency with which hive releases have been out.
          There is some design work involved around that. There are two things to consider for 'show tables' output, privileges that the user has on the database, and the privileges the user has on the table.

          Show
          thejas Thejas M Nair added a comment - This tasks under this ticket haven't covered 'show tables'. The focus has been on table/view/URI access. It is unlikely to be make in timeframe for 0.13 , but is likely to be part of hive 0.14 . 0.13 should be out in few weeks as we have branched for the release. 0.14 would is likely to be out in another 3-4 months, given the frequency with which hive releases have been out. There is some design work involved around that. There are two things to consider for 'show tables' output, privileges that the user has on the database, and the privileges the user has on the table.
          Hide
          terrasect Alex Nastetsky added a comment -

          Hi, are there plans to secure "show tables" as part of this ticket? If so, do we have a design for it yet and what does that look like? Thanks.

          Show
          terrasect Alex Nastetsky added a comment - Hi, are there plans to secure "show tables" as part of this ticket? If so, do we have a design for it yet and what does that look like? Thanks.
          Hide
          brocknoland Brock Noland added a comment -

          I propose that we base alter and drop table privilege on ownership of the table instead.

          Ok, would this would deviate from the "SQL Standard"?

          Do have any opinion on how to deal with privilege on URI object based on your experience? What should it mean, should it mean the privilege applies to the directory and its sub dirs?

          To avoid re-implementing file system permissions I'd suggest that once a prefix to a URI is granted, that all children in that URI are also granted. Of course the file system permissions will still need to be there for the URI to be usable.

          Can things like symlinks pose security holes?

          There is no way that symlinks can be securely followed in HDFS therefore following symlinks must be disabled for this model to be secure.

          Show
          brocknoland Brock Noland added a comment - I propose that we base alter and drop table privilege on ownership of the table instead. Ok, would this would deviate from the "SQL Standard"? Do have any opinion on how to deal with privilege on URI object based on your experience? What should it mean, should it mean the privilege applies to the directory and its sub dirs? To avoid re-implementing file system permissions I'd suggest that once a prefix to a URI is granted, that all children in that URI are also granted. Of course the file system permissions will still need to be there for the URI to be usable. Can things like symlinks pose security holes? There is no way that symlinks can be securely followed in HDFS therefore following symlinks must be disabled for this model to be secure.
          Hide
          thejas Thejas M Nair added a comment -

          Brock Noland Prasad Mujumdar Shreepadma Venugopalan Do have any opinion on how to deal with privilege on URI object based on your experience? What should it mean, should it mean the privilege applies to the directory and its sub dirs ? Can things like symlinks pose security holes ? Any other issues to consider wrt URI ?

          Show
          thejas Thejas M Nair added a comment - Brock Noland Prasad Mujumdar Shreepadma Venugopalan Do have any opinion on how to deal with privilege on URI object based on your experience? What should it mean, should it mean the privilege applies to the directory and its sub dirs ? Can things like symlinks pose security holes ? Any other issues to consider wrt URI ?
          Hide
          thejas Thejas M Nair added a comment -

          The current proposal does not talk about what determines the privilege to create a view and what privileges the creator of view will have on the new view.
          Based on my reading of the standard (only looking at select access on views because of what hive supports): View has select with grant for user A creating the view, if user has select-grant on all the input columns in query-expression.
          There also seems to be rule about being able to create views without grant privileges on tables (just select), but I think we can just start with allowing on tables for which user has select-with-grant.

          The current proposal says that database ownership will determine the privileges to alter and drop table. But this would be a problem for migration, for clusters where there are many tables under a database owned by different users. I propose that we base alter and drop table privilege on ownership of the table instead.

          Show
          thejas Thejas M Nair added a comment - The current proposal does not talk about what determines the privilege to create a view and what privileges the creator of view will have on the new view. Based on my reading of the standard (only looking at select access on views because of what hive supports): View has select with grant for user A creating the view, if user has select-grant on all the input columns in query-expression. There also seems to be rule about being able to create views without grant privileges on tables (just select), but I think we can just start with allowing on tables for which user has select-with-grant. The current proposal says that database ownership will determine the privileges to alter and drop table. But this would be a problem for migration, for clusters where there are many tables under a database owned by different users. I propose that we base alter and drop table privilege on ownership of the table instead.
          Hide
          brocknoland Brock Noland added a comment -

          Should we make one of the sql standard privileges available on SERVER object?

          Privileges on the SERVER object can make sense but I feel the more important aspect is to ensure privileges are scoped to a SERVER for the reason I will outline below.

          Brock, could you give more details on the SERVER use case? I've seen people use multiple instances of HS2 for HA/scaling, but never allocating some users to some instances and others to others. What's the motivation for that?

          It's a very similar use case to federation. Enterprises often want to isolate groups of users from using the same resource. The scenario is you have group A and group B and they cannot or do not want to share the same HS2. By having server in the hierarchy you can enforce the separation amongst HS2 instances.

          Show
          brocknoland Brock Noland added a comment - Should we make one of the sql standard privileges available on SERVER object? Privileges on the SERVER object can make sense but I feel the more important aspect is to ensure privileges are scoped to a SERVER for the reason I will outline below. Brock, could you give more details on the SERVER use case? I've seen people use multiple instances of HS2 for HA/scaling, but never allocating some users to some instances and others to others. What's the motivation for that? It's a very similar use case to federation. Enterprises often want to isolate groups of users from using the same resource. The scenario is you have group A and group B and they cannot or do not want to share the same HS2. By having server in the hierarchy you can enforce the separation amongst HS2 instances.
          Hide
          alangates Alan Gates added a comment -

          Brock, could you give more details on the SERVER use case? I've seen people use multiple instances of HS2 for HA/scaling, but never allocating some users to some instances and others to others. What's the motivation for that?

          Show
          alangates Alan Gates added a comment - Brock, could you give more details on the SERVER use case? I've seen people use multiple instances of HS2 for HA/scaling, but never allocating some users to some instances and others to others. What's the motivation for that?
          Hide
          thejas Thejas M Nair added a comment -

          Brock Noland Thanks for your feedback in the jiras for SQL standard auth !

          Show
          thejas Thejas M Nair added a comment - Brock Noland Thanks for your feedback in the jiras for SQL standard auth !
          Hide
          thejas Thejas M Nair added a comment -

          Brock Noland I assume you mean URI and SERVER as objects (similar to table, views etc) on which privileges (eg, select , insert,..) can be granted. As you know, URI authorization is very essential (more than just helping with udf support), without that you cannot enforce access control (you can use 'create table' to read from any hdfs location).
          I see that SERVER object will also be useful, but not essential for a first version. Should we make one of the sql standard privileges available on SERVER object ?

          Show
          thejas Thejas M Nair added a comment - Brock Noland I assume you mean URI and SERVER as objects (similar to table, views etc) on which privileges (eg, select , insert,..) can be granted. As you know, URI authorization is very essential (more than just helping with udf support), without that you cannot enforce access control (you can use 'create table' to read from any hdfs location). I see that SERVER object will also be useful, but not essential for a first version. Should we make one of the sql standard privileges available on SERVER object ?
          Hide
          brocknoland Brock Noland added a comment -

          Thejas M Nair,

          as I mentioned here I would consider adding a URI privilege to the model described here. This allows the use of custom UDFs for users. Beyond that I think a SERVER privilege should be added as well. The reason I believe a server privilege is useful is because large deployments of Hive would like to take advantage of multiple HS2 instances while allowing users to only access a single instance. What are you thoughts on these topics?

          Show
          brocknoland Brock Noland added a comment - Thejas M Nair , as I mentioned here I would consider adding a URI privilege to the model described here. This allows the use of custom UDFs for users. Beyond that I think a SERVER privilege should be added as well. The reason I believe a server privilege is useful is because large deployments of Hive would like to take advantage of multiple HS2 instances while allowing users to only access a single instance. What are you thoughts on these topics?
          Hide
          navis Navis added a comment -

          Ah, sorry. It's issue number of internal patches applied to our product. All of them are based on hive-11 but might be rebased to trunk.

          Show
          navis Navis added a comment - Ah, sorry. It's issue number of internal patches applied to our product. All of them are based on hive-11 but might be rebased to trunk.
          Hide
          thejas Thejas M Nair added a comment -

          Navis Thanks, yes, I would certainly appreciate help with this.

          What/where are the NHIVE-* jiras that you refer to ?

          Show
          thejas Thejas M Nair added a comment - Navis Thanks, yes, I would certainly appreciate help with this. What/where are the NHIVE-* jiras that you refer to ?
          Hide
          navis Navis added a comment -

          I think I can help a little for this. (we've been using some patches for authorization)

          NHIVE-40 Check read permission for creating views
          HIVE-2818 Create table checks the current database privilege
          NHIVE-32 Check grant option for grant/revoke operation
          NHIVE-33 Support database prefix for privilege objects
          NHIVE-31 Add API for retrieving principals endowed with the specific role
          HIVE-2093 [jira] create/drop database should populate inputs/outputs and check concurrency and user permission
          NHIVE-26 Indirect roles are not reflected in authorization
          NHIVE-25 Provide error message for authorization failures
          NHIVE-24 Return show grant result in tabular format
          NHIVE-23 Implement show grant on <resource>
          NHIVE-22 implement show roles
          NHIVE-21 Show grant always return empty string via JDBC
          NHIVE-10 Authenticate SHOW_DATABASES privilege for show databases

          Show
          navis Navis added a comment - I think I can help a little for this. (we've been using some patches for authorization) NHIVE-40 Check read permission for creating views HIVE-2818 Create table checks the current database privilege NHIVE-32 Check grant option for grant/revoke operation NHIVE-33 Support database prefix for privilege objects NHIVE-31 Add API for retrieving principals endowed with the specific role HIVE-2093 [jira] create/drop database should populate inputs/outputs and check concurrency and user permission NHIVE-26 Indirect roles are not reflected in authorization NHIVE-25 Provide error message for authorization failures NHIVE-24 Return show grant result in tabular format NHIVE-23 Implement show grant on <resource> NHIVE-22 implement show roles NHIVE-21 Show grant always return empty string via JDBC NHIVE-10 Authenticate SHOW_DATABASES privilege for show databases
          Hide
          thejas Thejas M Nair added a comment -

          Functional specification authored by Alan Gates, Sushanth Sowmyan and myself.

          Show
          thejas Thejas M Nair added a comment - Functional specification authored by Alan Gates , Sushanth Sowmyan and myself.
          Hide
          thejas Thejas M Nair added a comment -

          Attaching a functional specification.

          Show
          thejas Thejas M Nair added a comment - Attaching a functional specification.

            People

            • Assignee:
              thejas Thejas M Nair
              Reporter:
              thejas Thejas M Nair
            • Votes:
              0 Vote for this issue
              Watchers:
              26 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Time Tracking

                Estimated:
                Original Estimate - 1,284h
                1,284h
                Remaining:
                Time Spent - 497.6h Remaining Estimate - 632h
                632h
                Logged:
                Time Spent - 497.6h Remaining Estimate - 632h Time Not Required
                497.6h

                  Development