Details

    • Type: Sub-task Sub-task
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.13.0
    • Component/s: Authorization
    • Labels:
      None
    • Release Note:
      A new property is added in HiveConf. {{hive.users.in.admin.role}} An admin can provide a comma separated list of users which will be added to admin role when metastore starts up. More users can still be added later on.

      Description

      SUPERUSER and PUBLIC are two roles that are always present, these need to be added automatically on metastore startup. This would be similar to creation of the "default" database from HMSHandler.init().

      A config param in hive-site.xml will be used to specify the list of users who belong to the SUPERUSER role.

      1. HIVE-5959.patch
        8 kB
        Ashutosh Chauhan
      2. HIVE-5959.2.patch
        9 kB
        Ashutosh Chauhan
      3. HIVE-5959.3.patch
        10 kB
        Ashutosh Chauhan
      4. HIVE-5959.5.patch
        11 kB
        Ashutosh Chauhan
      5. HIVE-5959.6.patch
        8 kB
        Ashutosh Chauhan
      6. HIVE-5959.7.patch
        8 kB
        Ashutosh Chauhan

        Issue Links

          Activity

          Hide
          Lefty Leverenz added a comment -

          Added a subsection to Authentication/Authorization in Configuration Properties, and documented hive.users.in.admin.role:

          But I didn't find any other config parameters for SQL standard authorization. I'll keep the separate section for now, but if no new parameters are added then it might get merged into Hive Client Security later on.

          Show
          Lefty Leverenz added a comment - Added a subsection to Authentication/Authorization in Configuration Properties, and documented hive.users.in.admin.role : Configuration Properties: SQL Standard Based Authorization But I didn't find any other config parameters for SQL standard authorization. I'll keep the separate section for now, but if no new parameters are added then it might get merged into Hive Client Security later on.
          Hide
          Ashutosh Chauhan added a comment -

          Lefty Leverenz I added the release notes. I think same text can be put on wiki also.

          Show
          Ashutosh Chauhan added a comment - Lefty Leverenz I added the release notes. I think same text can be put on wiki also.
          Hide
          Lefty Leverenz added a comment -

          This adds hive.users.in.admin.role to HiveConf.java & hive-default.xml.template, so I'll put it in the wiki with a release note. But some explanation is also needed in the Authorization wiki, and perhaps a mention in Configuration. Should that be done now, or wait for the umbrella jira (HIVE-5837)?

          Quick ref:

          Show
          Lefty Leverenz added a comment - This adds hive.users.in.admin.role to HiveConf.java & hive-default.xml.template, so I'll put it in the wiki with a release note. But some explanation is also needed in the Authorization wiki, and perhaps a mention in Configuration. Should that be done now, or wait for the umbrella jira ( HIVE-5837 )? Quick ref: HIVE-5837: SQL standard based secure authorization for hive Configuration Properties: Authentication/Authorization Hive Authorization AdminManual Configuration Configuring Hive Configuration Variables
          Hide
          Ashutosh Chauhan added a comment -

          Committed to trunk. Thanks, Thejas & Brock for review!

          Show
          Ashutosh Chauhan added a comment - Committed to trunk. Thanks, Thejas & Brock for review!
          Hide
          Hive QA added a comment -

          Overall: -1 at least one tests failed

          Here are the results of testing the latest attachment:
          https://issues.apache.org/jira/secure/attachment/12626051/HIVE-5959.7.patch

          ERROR: -1 due to 1 failed/errored test(s), 4972 tests executed
          Failed tests:

          org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_auto_sortmerge_join_16
          

          Test results: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1116/testReport
          Console output: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1116/console

          Messages:

          Executing org.apache.hive.ptest.execution.PrepPhase
          Executing org.apache.hive.ptest.execution.ExecutionPhase
          Executing org.apache.hive.ptest.execution.ReportingPhase
          Tests exited with: TestsFailedException: 1 tests failed
          

          This message is automatically generated.

          ATTACHMENT ID: 12626051

          Show
          Hive QA added a comment - Overall : -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12626051/HIVE-5959.7.patch ERROR: -1 due to 1 failed/errored test(s), 4972 tests executed Failed tests: org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_auto_sortmerge_join_16 Test results: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1116/testReport Console output: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1116/console Messages: Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 1 tests failed This message is automatically generated. ATTACHMENT ID: 12626051
          Hide
          Brock Noland added a comment -

          OK fair enough. I marked HIVE-6338 as a blocker.

          Show
          Brock Noland added a comment - OK fair enough. I marked HIVE-6338 as a blocker.
          Hide
          Ashutosh Chauhan added a comment -

          I have created HIVE-6338 and have marked it for 0.13 Reason I don't want to make these changes (which anyways don't serve purpose of this jira) is because I have set of jiras which are dependent on this one, like HIVE-5944 HIVE-5943 HIVE-5952

          I am happy to make your suggested changes in different one. It might even make sense to scrub all of Hive code to spot on those kind of incorrect exception handling and fix them.

          Show
          Ashutosh Chauhan added a comment - I have created HIVE-6338 and have marked it for 0.13 Reason I don't want to make these changes (which anyways don't serve purpose of this jira) is because I have set of jiras which are dependent on this one, like HIVE-5944 HIVE-5943 HIVE-5952 I am happy to make your suggested changes in different one. It might even make sense to scrub all of Hive code to spot on those kind of incorrect exception handling and fix them.
          Hide
          Brock Noland added a comment -

          It looks like we are moving quite a bit a code around. I don't see why we cannot do it in this jira? It's trivial...

          +      } catch (Exception e) {
          +        assert (e instanceof RuntimeException);
          +        throw (RuntimeException) e;
          +      }
          

          This code is completely bad on many levels.

          • If there is any possibility that a non-RTE can be thrown, then this code is completely wrong. Specifically if a non-RTE can be thrown then it's event worse than just ugly code because depending on if assertions are enabled or not, the type of the exception is eaten or the message and stack trace are eaten.
          • If there is no possibility that non-RTE can be thrown then the catch should be RTE in which case it's immediately re-thrown and this code is completely useless.
          Show
          Brock Noland added a comment - It looks like we are moving quite a bit a code around. I don't see why we cannot do it in this jira? It's trivial... + } catch (Exception e) { + assert (e instanceof RuntimeException); + throw (RuntimeException) e; + } This code is completely bad on many levels. If there is any possibility that a non-RTE can be thrown, then this code is completely wrong. Specifically if a non-RTE can be thrown then it's event worse than just ugly code because depending on if assertions are enabled or not, the type of the exception is eaten or the message and stack trace are eaten. If there is no possibility that non-RTE can be thrown then the catch should be RTE in which case it's immediately re-thrown and this code is completely useless.
          Hide
          Ashutosh Chauhan added a comment -

          Another rebase on trunk.

          Show
          Ashutosh Chauhan added a comment - Another rebase on trunk.
          Hide
          Hive QA added a comment -

          Overall: -1 no tests executed

          Here are the results of testing the latest attachment:
          https://issues.apache.org/jira/secure/attachment/12625894/HIVE-5959.6.patch

          Test results: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1104/testReport
          Console output: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1104/console

          Messages:

          Executing org.apache.hive.ptest.execution.PrepPhase
          Tests exited with: NonZeroExitCodeException
          Command 'bash /data/hive-ptest/working/scratch/source-prep.sh' failed with exit status 1 and output '+ [[ -n '' ]]
          + export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
          + ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
          + export 'M2_OPTS=-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost -Dhttp.proxyPort=3128'
          + M2_OPTS='-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost -Dhttp.proxyPort=3128'
          + cd /data/hive-ptest/working/
          + tee /data/hive-ptest/logs/PreCommit-HIVE-Build-1104/source-prep.txt
          + [[ false == \t\r\u\e ]]
          + mkdir -p maven ivy
          + [[ svn = \s\v\n ]]
          + [[ -n '' ]]
          + [[ -d apache-svn-trunk-source ]]
          + [[ ! -d apache-svn-trunk-source/.svn ]]
          + [[ ! -d apache-svn-trunk-source ]]
          + cd apache-svn-trunk-source
          + svn revert -R .
          ++ egrep -v '^X|^Performing status on external'
          ++ svn status --no-ignore
          ++ awk '{print $2}'
          + rm -rf target datanucleus.log ant/target shims/target shims/0.20/target shims/0.20S/target shims/0.23/target shims/aggregator/target shims/common/target shims/common-secure/target packaging/target hbase-handler/target testutils/target jdbc/target metastore/target itests/target itests/hcatalog-unit/target itests/test-serde/target itests/qtest/target itests/hive-unit/target itests/custom-serde/target itests/util/target hcatalog/target hcatalog/storage-handlers/hbase/target hcatalog/server-extensions/target hcatalog/core/target hcatalog/webhcat/svr/target hcatalog/webhcat/java-client/target hcatalog/hcatalog-pig-adapter/target hwi/target common/target common/src/gen contrib/target service/target serde/target beeline/target odbc/target cli/target ql/dependency-reduced-pom.xml ql/target
          + svn update
          
          Fetching external item into 'hcatalog/src/test/e2e/harness'
          External at revision 1562672.
          
          At revision 1562672.
          + patchCommandPath=/data/hive-ptest/working/scratch/smart-apply-patch.sh
          + patchFilePath=/data/hive-ptest/working/scratch/build.patch
          + [[ -f /data/hive-ptest/working/scratch/build.patch ]]
          + chmod +x /data/hive-ptest/working/scratch/smart-apply-patch.sh
          + /data/hive-ptest/working/scratch/smart-apply-patch.sh /data/hive-ptest/working/scratch/build.patch
          The patch does not appear to apply with p0, p1, or p2
          + exit 1
          '
          

          This message is automatically generated.

          ATTACHMENT ID: 12625894

          Show
          Hive QA added a comment - Overall : -1 no tests executed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12625894/HIVE-5959.6.patch Test results: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1104/testReport Console output: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1104/console Messages: Executing org.apache.hive.ptest.execution.PrepPhase Tests exited with: NonZeroExitCodeException Command 'bash /data/hive-ptest/working/scratch/source-prep.sh' failed with exit status 1 and output '+ [[ -n '' ]] + export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m ' + ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m ' + export 'M2_OPTS=-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost -Dhttp.proxyPort=3128' + M2_OPTS='-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost -Dhttp.proxyPort=3128' + cd /data/hive-ptest/working/ + tee /data/hive-ptest/logs/PreCommit-HIVE-Build-1104/source-prep.txt + [[ false == \t\r\u\e ]] + mkdir -p maven ivy + [[ svn = \s\v\n ]] + [[ -n '' ]] + [[ -d apache-svn-trunk-source ]] + [[ ! -d apache-svn-trunk-source/.svn ]] + [[ ! -d apache-svn-trunk-source ]] + cd apache-svn-trunk-source + svn revert -R . ++ egrep -v '^X|^Performing status on external' ++ svn status --no-ignore ++ awk '{print $2}' + rm -rf target datanucleus.log ant/target shims/target shims/0.20/target shims/0.20S/target shims/0.23/target shims/aggregator/target shims/common/target shims/common-secure/target packaging/target hbase-handler/target testutils/target jdbc/target metastore/target itests/target itests/hcatalog-unit/target itests/test-serde/target itests/qtest/target itests/hive-unit/target itests/custom-serde/target itests/util/target hcatalog/target hcatalog/storage-handlers/hbase/target hcatalog/server-extensions/target hcatalog/core/target hcatalog/webhcat/svr/target hcatalog/webhcat/java-client/target hcatalog/hcatalog-pig-adapter/target hwi/target common/target common/src/gen contrib/target service/target serde/target beeline/target odbc/target cli/target ql/dependency-reduced-pom.xml ql/target + svn update Fetching external item into 'hcatalog/src/test/e2e/harness' External at revision 1562672. At revision 1562672. + patchCommandPath=/data/hive-ptest/working/scratch/smart-apply-patch.sh + patchFilePath=/data/hive-ptest/working/scratch/build.patch + [[ -f /data/hive-ptest/working/scratch/build.patch ]] + chmod +x /data/hive-ptest/working/scratch/smart-apply-patch.sh + /data/hive-ptest/working/scratch/smart-apply-patch.sh /data/hive-ptest/working/scratch/build.patch The patch does not appear to apply with p0, p1, or p2 + exit 1 ' This message is automatically generated. ATTACHMENT ID: 12625894
          Hide
          Ashutosh Chauhan added a comment -

          I guess you are looking in createDefaultDb() method. I haven't altered logic of that method. Its existing code. Shall we try to improve that in different jira ?

          Show
          Ashutosh Chauhan added a comment - I guess you are looking in createDefaultDb() method. I haven't altered logic of that method. Its existing code. Shall we try to improve that in different jira ?
          Hide
          Brock Noland added a comment -

          This is a bad idea:

          assert (e instanceof RuntimeException);
          

          all we will know is that an assertion failed but no idea what exception caused it. If we want this to be true we should only patch runtimeexception.

          Show
          Brock Noland added a comment - This is a bad idea: assert (e instanceof RuntimeException); all we will know is that an assertion failed but no idea what exception caused it. If we want this to be true we should only patch runtimeexception.
          Hide
          Ashutosh Chauhan added a comment -

          Cannot repro above failed testcase. Anyways look unrelated. Seems like port contention issue on Hive QA box. Reuploading to get another run. Also, incoporated Swarnim's comments on improving log messages.

          Show
          Ashutosh Chauhan added a comment - Cannot repro above failed testcase. Anyways look unrelated. Seems like port contention issue on Hive QA box. Reuploading to get another run. Also, incoporated Swarnim's comments on improving log messages.
          Hide
          Hive QA added a comment -

          Overall: -1 at least one tests failed

          Here are the results of testing the latest attachment:
          https://issues.apache.org/jira/secure/attachment/12625774/HIVE-5959.5.patch

          ERROR: -1 due to 1 failed/errored test(s), 4973 tests executed
          Failed tests:

          org.apache.hive.jdbc.TestJdbcDriver2.testNewConnectionConfiguration
          

          Test results: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1093/testReport
          Console output: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1093/console

          Messages:

          Executing org.apache.hive.ptest.execution.PrepPhase
          Executing org.apache.hive.ptest.execution.ExecutionPhase
          Executing org.apache.hive.ptest.execution.ReportingPhase
          Tests exited with: TestsFailedException: 1 tests failed
          

          This message is automatically generated.

          ATTACHMENT ID: 12625774

          Show
          Hive QA added a comment - Overall : -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12625774/HIVE-5959.5.patch ERROR: -1 due to 1 failed/errored test(s), 4973 tests executed Failed tests: org.apache.hive.jdbc.TestJdbcDriver2.testNewConnectionConfiguration Test results: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1093/testReport Console output: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1093/console Messages: Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 1 tests failed This message is automatically generated. ATTACHMENT ID: 12625774
          Hide
          Thejas M Nair added a comment -

          +1

          Show
          Thejas M Nair added a comment - +1
          Hide
          Ashutosh Chauhan added a comment -

          Updated patch to incorporate feedback from RB. Also, I renamed root to admin since in sql parlance I think thats more common.

          Show
          Ashutosh Chauhan added a comment - Updated patch to incorporate feedback from RB. Also, I renamed root to admin since in sql parlance I think thats more common.
          Hide
          Brock Noland added a comment -

          The spacing of the new lines is way off? Looks like tabs and strange formatting?

          Show
          Brock Noland added a comment - The spacing of the new lines is way off? Looks like tabs and strange formatting?
          Hide
          Ashutosh Chauhan added a comment -
          Show
          Ashutosh Chauhan added a comment - RB request : https://reviews.apache.org/r/17441/
          Hide
          Hive QA added a comment -

          Overall: -1 at least one tests failed

          Here are the results of testing the latest attachment:
          https://issues.apache.org/jira/secure/attachment/12625390/HIVE-5959.patch

          ERROR: -1 due to 5 failed/errored test(s), 4962 tests executed
          Failed tests:

          org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_bucket_num_reducers
          org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_import_exported_table
          org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_load_hdfs_file_with_space_in_the_name
          org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testNegativeCliDriver_file_with_header_footer_negative
          org.apache.hadoop.hive.metastore.TestMetastoreVersion.testVersionMisMatch
          

          Test results: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1049/testReport
          Console output: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1049/console

          Messages:

          Executing org.apache.hive.ptest.execution.PrepPhase
          Executing org.apache.hive.ptest.execution.ExecutionPhase
          Executing org.apache.hive.ptest.execution.ReportingPhase
          Tests exited with: TestsFailedException: 5 tests failed
          

          This message is automatically generated.

          ATTACHMENT ID: 12625390

          Show
          Hive QA added a comment - Overall : -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12625390/HIVE-5959.patch ERROR: -1 due to 5 failed/errored test(s), 4962 tests executed Failed tests: org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_bucket_num_reducers org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_import_exported_table org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_load_hdfs_file_with_space_in_the_name org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testNegativeCliDriver_file_with_header_footer_negative org.apache.hadoop.hive.metastore.TestMetastoreVersion.testVersionMisMatch Test results: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1049/testReport Console output: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1049/console Messages: Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 5 tests failed This message is automatically generated. ATTACHMENT ID: 12625390
          Hide
          Ashutosh Chauhan added a comment -

          This patch adds root role at metastore startup time.

          Show
          Ashutosh Chauhan added a comment - This patch adds root role at metastore startup time.

            People

            • Assignee:
              Ashutosh Chauhan
              Reporter:
              Thejas M Nair
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Time Tracking

                Estimated:
                Original Estimate - 72h
                72h
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 24h Time Not Required
                24h

                  Development