Hive
  1. Hive
  2. HIVE-7209

allow metastore authorization api calls to be restricted to certain invokers

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.14.0
    • Component/s: Authentication, Metastore
    • Labels:
      None
    • Release Note:
      Hide
      With this change hive.security.metastore.authorization.manager configuration parameter allows you to specify more than one authorization manager class (comma separated).

      This patch introduces a new authorization manager for use under this configuration - org.apache.hadoop.hive.ql.security.authorization.MetaStoreAuthzAPIAuthorizerEmbedOnly. It will disallow any of the authorization api calls to be invoked in a remote metastore.
      HiveServer2 can be configured to use embedded metastore, and that will allow it to invoke metastore authorization api. Hive cli and any other remote metastore users would be denied authorization when they try to make authorization api calls. This allows restricting the authorization api use to privileged HiveServer2 process.

      Show
      With this change hive.security.metastore.authorization.manager configuration parameter allows you to specify more than one authorization manager class (comma separated). This patch introduces a new authorization manager for use under this configuration - org.apache.hadoop.hive.ql.security.authorization.MetaStoreAuthzAPIAuthorizerEmbedOnly. It will disallow any of the authorization api calls to be invoked in a remote metastore. HiveServer2 can be configured to use embedded metastore, and that will allow it to invoke metastore authorization api. Hive cli and any other remote metastore users would be denied authorization when they try to make authorization api calls. This allows restricting the authorization api use to privileged HiveServer2 process.

      Description

      Any user who has direct access to metastore can make metastore api calls that modify the authorization policy.
      The users who can make direct metastore api calls in a secure cluster configuration are usually the 'cluster insiders' such as Pig and MR users, who are not (securely) covered by the metastore based authorization policy. But it makes sense to disallow access from such users as well.

      1. HIVE-7209.4.patch
        60 kB
        Thejas M Nair
      2. HIVE-7209.3.patch
        59 kB
        Thejas M Nair
      3. HIVE-7209.2.patch
        59 kB
        Thejas M Nair
      4. HIVE-7209.1.patch
        56 kB
        Thejas M Nair

        Issue Links

          Activity

          Hide
          Thejas M Nair added a comment -

          Note that it is not recommended that user whose access is being secured using SQL standards based auth is given direct access to metastore. With that practice in place, such users cannot act maliciously to change access control policy without authorization.

          Show
          Thejas M Nair added a comment - Note that it is not recommended that user whose access is being secured using SQL standards based auth is given direct access to metastore. With that practice in place, such users cannot act maliciously to change access control policy without authorization.
          Hide
          Thejas M Nair added a comment -

          HIVE-7209.1.patch - Has changes to allow for multiple authorizers to be registered for metastore authorization.
          Also includes a new authorizer org.apache.hadoop.hive.ql.security.authorization.MetaStoreAuthzAPIAuthorizerEmbedOnly that can be added to the hive.security.metastore.authorization.manager config parameter. It will disallow any metastore api calls in remote metastore mode. If you use HS2 with embedded metastore, the HS2 can make these api calls, as the authorizer disables the calls only in remote mode.

          This approach can be extended in followup work to allow the api calls to be made to remote metastore by only certain users from certain machines.

          Show
          Thejas M Nair added a comment - HIVE-7209 .1.patch - Has changes to allow for multiple authorizers to be registered for metastore authorization. Also includes a new authorizer org.apache.hadoop.hive.ql.security.authorization.MetaStoreAuthzAPIAuthorizerEmbedOnly that can be added to the hive.security.metastore.authorization.manager config parameter. It will disallow any metastore api calls in remote metastore mode. If you use HS2 with embedded metastore, the HS2 can make these api calls, as the authorizer disables the calls only in remote mode. This approach can be extended in followup work to allow the api calls to be made to remote metastore by only certain users from certain machines.
          Hide
          Thejas M Nair added a comment -

          adding review board link

          Show
          Thejas M Nair added a comment - adding review board link
          Hide
          Thejas M Nair added a comment -

          HIVE-7209.2.patch - Addressing Ashutosh's suggestion of avoiding an additional interface.

          Show
          Thejas M Nair added a comment - HIVE-7209 .2.patch - Addressing Ashutosh's suggestion of avoiding an additional interface.
          Hide
          Ashutosh Chauhan added a comment -

          +1

          Show
          Ashutosh Chauhan added a comment - +1
          Hide
          Thejas M Nair added a comment -

          HIVE-7209.3.patch - addressing Sushanth's comment - moving the wrapped table/partition creation outside of loop.

          Show
          Thejas M Nair added a comment - HIVE-7209 .3.patch - addressing Sushanth's comment - moving the wrapped table/partition creation outside of loop.
          Hide
          Sushanth Sowmyan added a comment -

          Looks good to me. +1.

          Show
          Sushanth Sowmyan added a comment - Looks good to me. +1.
          Hide
          Thejas M Nair added a comment -

          HIVE-7209.4.patch - also updating hive-default.xml.template to mention that more than one metastore authorization manager classes can be specified under hive.security.metastore.authorization.manager .

          Show
          Thejas M Nair added a comment - HIVE-7209 .4.patch - also updating hive-default.xml.template to mention that more than one metastore authorization manager classes can be specified under hive.security.metastore.authorization.manager .
          Hide
          Hive QA added a comment -

          Overall: -1 at least one tests failed

          Here are the results of testing the latest attachment:
          https://issues.apache.org/jira/secure/attachment/12650378/HIVE-7209.4.patch

          ERROR: -1 due to 9 failed/errored test(s), 5630 tests executed
          Failed tests:

          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucket_groupby
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_parquet_columnar
          org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_dynpart_sort_optimization
          org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_insert1
          org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_load_dyn_part1
          org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_scriptfile1
          org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_root_dir_external_table
          org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_ctas
          org.apache.hive.hcatalog.pig.TestOrcHCatLoader.testReadDataPrimitiveTypes
          

          Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/459/testReport
          Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/459/console
          Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-Build-459/

          Messages:

          Executing org.apache.hive.ptest.execution.PrepPhase
          Executing org.apache.hive.ptest.execution.ExecutionPhase
          Executing org.apache.hive.ptest.execution.ReportingPhase
          Tests exited with: TestsFailedException: 9 tests failed
          

          This message is automatically generated.

          ATTACHMENT ID: 12650378

          Show
          Hive QA added a comment - Overall : -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12650378/HIVE-7209.4.patch ERROR: -1 due to 9 failed/errored test(s), 5630 tests executed Failed tests: org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucket_groupby org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_parquet_columnar org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_dynpart_sort_optimization org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_insert1 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_load_dyn_part1 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_scriptfile1 org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_root_dir_external_table org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_ctas org.apache.hive.hcatalog.pig.TestOrcHCatLoader.testReadDataPrimitiveTypes Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/459/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/459/console Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-Build-459/ Messages: Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 9 tests failed This message is automatically generated. ATTACHMENT ID: 12650378
          Hide
          Sushanth Sowmyan added a comment -

          Committed patch 4. Thanks for the patch, Thejas!

          Show
          Sushanth Sowmyan added a comment - Committed patch 4. Thanks for the patch, Thejas!
          Hide
          Sushanth Sowmyan added a comment -

          (Reopening because of the weird resolution state, intend to close again)

          Show
          Sushanth Sowmyan added a comment - (Reopening because of the weird resolution state, intend to close again)
          Hide
          Lefty Leverenz added a comment -

          For the record: The patch changes the description of hive.security.metastore.authorization.manager in hive-default.xml.template (see the release note for new functionality).

          I added a comment to HIVE-6586 so it won't get lost in the shuffle when HIVE-6037 changes HiveConf.java.

          Show
          Lefty Leverenz added a comment - For the record: The patch changes the description of hive.security.metastore.authorization.manager in hive-default.xml.template (see the release note for new functionality). I added a comment to HIVE-6586 so it won't get lost in the shuffle when HIVE-6037 changes HiveConf.java.
          Hide
          Lefty Leverenz added a comment -

          Doc notes: The description of hive.security.metastore.authorization.manager needs to be updated in the wiki (with version information, and keeping some extra information not found in HiveConf.java).

          Other than that, HIVE-7759 will add general documentation for this feature with a section in the SQL standard authorization doc about CLI behavior with SQL standard authorization turned on.

          Show
          Lefty Leverenz added a comment - Doc notes: The description of hive.security.metastore.authorization.manager needs to be updated in the wiki (with version information, and keeping some extra information not found in HiveConf.java). Configuration Properties – hive.security.metastore.authorization.manager Other than that, HIVE-7759 will add general documentation for this feature with a section in the SQL standard authorization doc about CLI behavior with SQL standard authorization turned on. SQL Standard Based Hive Authorization
          Hide
          Thejas M Nair added a comment -

          This has been fixed in 0.14 release. Please open new jira if you see any issues.

          Show
          Thejas M Nair added a comment - This has been fixed in 0.14 release. Please open new jira if you see any issues.
          Hide
          Lefty Leverenz added a comment -

          Doc: hive.security.metastore.authorization.manager has been updated in the wiki, so I'm removing the TODOC14 label. (Additional documentation will be covered with HIVE-7759, as mentioned two comments back.)

          Show
          Lefty Leverenz added a comment - Doc: hive.security.metastore.authorization.manager has been updated in the wiki, so I'm removing the TODOC14 label. (Additional documentation will be covered with HIVE-7759 , as mentioned two comments back.) Configuration Properties – hive.security.metastore.authorization.manager

            People

            • Assignee:
              Thejas M Nair
              Reporter:
              Thejas M Nair
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development