Hive
  1. Hive
  2. HIVE-6957

SQL authorization does not work with HS2 binary mode and Kerberos auth

    Details

      Description

      In HiveServer2, when Kerberos auth and binary transport modes are used, the user name that gets passed on to authorization is the long kerberos username.
      The username that is used in grant/revoke statements tend to be the short usernames.
      This also fails in authorizing statements that involve URI, as the authorization mode checks the file system permissions for given user. It does not recognize that the given long username actually owns the file or belongs to the group that owns the file.

      1. HIVE-6957.1.patch
        11 kB
        Thejas M Nair
      2. HIVE-6957.2.patch
        21 kB
        Thejas M Nair
      3. HIVE-6957.3.patch
        27 kB
        Thejas M Nair
      4. HIVE-6957.4.patch
        27 kB
        Thejas M Nair
      5. HIVE-6957.04-branch.0.13.patch
        1 kB
        Thejas M Nair

        Issue Links

          Activity

          Hide
          Thejas M Nair added a comment -

          This jira has been fixed as part of 0.13.1 release. If you find further issues, please create a new jira and link it to this one.

          Show
          Thejas M Nair added a comment - This jira has been fixed as part of 0.13.1 release. If you find further issues, please create a new jira and link it to this one.
          Hide
          Thejas M Nair added a comment -

          Patch committed to trunk. Thanks for the review Vaibhav!

          Show
          Thejas M Nair added a comment - Patch committed to trunk. Thanks for the review Vaibhav!
          Hide
          Vaibhav Gumashta added a comment -

          Thejas M Nair I don't have access to svn yet. This should be good to commit.

          Show
          Vaibhav Gumashta added a comment - Thejas M Nair I don't have access to svn yet. This should be good to commit.
          Hide
          Vaibhav Gumashta added a comment -

          +1

          Show
          Vaibhav Gumashta added a comment - +1
          Hide
          Thejas M Nair added a comment -

          HIVE-6957.04-branch.0.13.patch - patch for 0.13 branch .

          Show
          Thejas M Nair added a comment - HIVE-6957 .04-branch.0.13.patch - patch for 0.13 branch .
          Hide
          Hive QA added a comment -

          Overall: -1 at least one tests failed

          Here are the results of testing the latest attachment:
          https://issues.apache.org/jira/secure/attachment/12641843/HIVE-6957.4.patch

          ERROR: -1 due to 40 failed/errored test(s), 5420 tests executed
          Failed tests:

          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join32
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_filter_numeric
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby2_map_skew
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby_sort_1
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby_sort_skew_1
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_infer_bucket_sort_list_bucket
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_6
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_7
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_8
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_mapjoin_test_outer
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_nullgroup3
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_orc_createas1
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ppd_join4
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_select_dummy_source
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_stats_list_bucket
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_stats_partscan_1_23
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_symlink_text_input_format
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_truncate_column_list_bucket
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_udf_current_database
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_1
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_10
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_12
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_13
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_14
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_19
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_2
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_20
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_21
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_22
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_23
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_24
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_4
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_5
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_7
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_8
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_9
          org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_bucketizedhiveinputformat
          org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_root_dir_external_table
          org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_dynamic_partitions_with_whitelist
          org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_stats_partialscan_autogether
          

          Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/35/testReport
          Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/35/console

          Messages:

          Executing org.apache.hive.ptest.execution.PrepPhase
          Executing org.apache.hive.ptest.execution.ExecutionPhase
          Executing org.apache.hive.ptest.execution.ReportingPhase
          Tests exited with: TestsFailedException: 40 tests failed
          

          This message is automatically generated.

          ATTACHMENT ID: 12641843

          Show
          Hive QA added a comment - Overall : -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12641843/HIVE-6957.4.patch ERROR: -1 due to 40 failed/errored test(s), 5420 tests executed Failed tests: org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join32 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_filter_numeric org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby2_map_skew org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby_sort_1 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby_sort_skew_1 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_infer_bucket_sort_list_bucket org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_6 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_7 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_8 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_mapjoin_test_outer org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_nullgroup3 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_orc_createas1 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ppd_join4 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_select_dummy_source org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_stats_list_bucket org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_stats_partscan_1_23 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_symlink_text_input_format org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_truncate_column_list_bucket org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_udf_current_database org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_1 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_10 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_12 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_13 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_14 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_19 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_2 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_20 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_21 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_22 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_23 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_24 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_4 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_5 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_7 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_8 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_9 org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_bucketizedhiveinputformat org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_root_dir_external_table org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_dynamic_partitions_with_whitelist org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_stats_partialscan_autogether Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/35/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/35/console Messages: Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 40 tests failed This message is automatically generated. ATTACHMENT ID: 12641843
          Hide
          Thejas M Nair added a comment -

          HIVE-6957.4.patch - rename the abstract base test class to *Test.java

          Show
          Thejas M Nair added a comment - HIVE-6957 .4.patch - rename the abstract base test class to *Test.java
          Hide
          Vaibhav Gumashta added a comment -

          +1 non-binding. Latest patch looks good - tests are super useful, I think I'll use this as base to add more kerberos related tests.

          Show
          Vaibhav Gumashta added a comment - +1 non-binding. Latest patch looks good - tests are super useful, I think I'll use this as base to add more kerberos related tests.
          Hide
          Thejas M Nair added a comment -

          3.patch - fixing TestSSL failures because of change in MiniHS2

          Show
          Thejas M Nair added a comment - 3.patch - fixing TestSSL failures because of change in MiniHS2
          Hide
          Thejas M Nair added a comment -

          Addressing review comments.

          Show
          Thejas M Nair added a comment - Addressing review comments.
          Hide
          Vaibhav Gumashta added a comment -

          Thejas M Nair The patch & added tests look good. I've added some minor comments on rb. The documentation related comments are unrelated to this patch, so we can always do it later. Thanks!

          Show
          Vaibhav Gumashta added a comment - Thejas M Nair The patch & added tests look good. I've added some minor comments on rb. The documentation related comments are unrelated to this patch, so we can always do it later. Thanks!
          Hide
          Hive QA added a comment -

          Overall: -1 at least one tests failed

          Here are the results of testing the latest attachment:
          https://issues.apache.org/jira/secure/attachment/12641337/HIVE-6957.1.patch

          ERROR: -1 due to 42 failed/errored test(s), 5418 tests executed
          Failed tests:

          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join32
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_filter_numeric
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby2_map_skew
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby_sort_1
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby_sort_skew_1
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_infer_bucket_sort_list_bucket
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_6
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_7
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_8
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_mapjoin_test_outer
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_nullgroup3
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_orc_createas1
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ppd_join4
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_select_dummy_source
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_stats_list_bucket
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_stats_partscan_1_23
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_symlink_text_input_format
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_truncate_column_list_bucket
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_udf_current_database
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_1
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_10
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_12
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_13
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_14
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_17
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_19
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_2
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_20
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_21
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_22
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_23
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_24
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_4
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_5
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_7
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_8
          org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_9
          org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_bucketizedhiveinputformat
          org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_root_dir_external_table
          org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_dynamic_partitions_with_whitelist
          org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_stats_partialscan_autogether
          org.apache.hive.minikdc.TestJdbcWithMiniKdc.testConnection
          

          Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/14/testReport
          Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/14/console

          Messages:

          Executing org.apache.hive.ptest.execution.PrepPhase
          Executing org.apache.hive.ptest.execution.ExecutionPhase
          Executing org.apache.hive.ptest.execution.ReportingPhase
          Tests exited with: TestsFailedException: 42 tests failed
          

          This message is automatically generated.

          ATTACHMENT ID: 12641337

          Show
          Hive QA added a comment - Overall : -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12641337/HIVE-6957.1.patch ERROR: -1 due to 42 failed/errored test(s), 5418 tests executed Failed tests: org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join32 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_filter_numeric org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby2_map_skew org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby_sort_1 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby_sort_skew_1 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_infer_bucket_sort_list_bucket org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_6 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_7 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_8 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_mapjoin_test_outer org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_nullgroup3 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_orc_createas1 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ppd_join4 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_select_dummy_source org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_stats_list_bucket org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_stats_partscan_1_23 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_symlink_text_input_format org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_truncate_column_list_bucket org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_udf_current_database org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_1 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_10 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_12 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_13 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_14 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_17 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_19 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_2 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_20 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_21 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_22 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_23 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_24 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_4 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_5 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_7 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_8 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_9 org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_bucketizedhiveinputformat org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_root_dir_external_table org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_dynamic_partitions_with_whitelist org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_stats_partialscan_autogether org.apache.hive.minikdc.TestJdbcWithMiniKdc.testConnection Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/14/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/14/console Messages: Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 42 tests failed This message is automatically generated. ATTACHMENT ID: 12641337
          Hide
          Thejas M Nair added a comment -

          Error looks like this

          java.sql.SQLException: Error while compiling statement: FAILED: HiveAccessControlException Permission denied. Principal [name=user2@EXAMPLE.COM, type=USER] does not have following privileges on Object [type=TABLE_OR_VIEW, name=default.test_jdbc_sql_auth2] : [SELECT]
          
          Show
          Thejas M Nair added a comment - Error looks like this java.sql.SQLException: Error while compiling statement: FAILED: HiveAccessControlException Permission denied. Principal [name=user2@EXAMPLE.COM, type=USER] does not have following privileges on Object [type=TABLE_OR_VIEW, name= default .test_jdbc_sql_auth2] : [SELECT]
          Hide
          Thejas M Nair added a comment -

          The long username is not of any significance within hive. We always use the short username for all purposes including the owner in metastore.
          This patch changes the username that gets set for HS2 purposes, to the short username.

          Show
          Thejas M Nair added a comment - The long username is not of any significance within hive. We always use the short username for all purposes including the owner in metastore. This patch changes the username that gets set for HS2 purposes, to the short username.
          Hide
          Thejas M Nair added a comment -

          A workaround is to use the http transport mode for HS2.

          Show
          Thejas M Nair added a comment - A workaround is to use the http transport mode for HS2.

            People

            • Assignee:
              Thejas M Nair
              Reporter:
              Thejas M Nair
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development