Hive
  1. Hive
  2. HIVE-5635

WebHCatJTShim23 ignores security/user context

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.12.0
    • Fix Version/s: 0.13.0
    • Component/s: WebHCat
    • Labels:
      None

      Description

      WebHCatJTShim23 takes UserGroupInformation object as argument (which represents the user make the call to WebHCat or doAs user) but ignores.

      WebHCatJTShim20S uses the UserGroupInformation

      This is inconsistent and may be a security hole because in with Hadoop 2 the methods on WebHCatJTShim are likely running with 'hcat' as the user context.

      1. HIVE-5635.3.patch
        1 kB
        Eugene Koifman
      2. HIVE-5635.2.patch
        1 kB
        Eugene Koifman
      3. HIVE-5635.patch
        1 kB
        Eugene Koifman

        Activity

        Hide
        Eugene Koifman added a comment -

        ToDo: patch needs to be tested on Hadoop 2.2.0

        Show
        Eugene Koifman added a comment - ToDo: patch needs to be tested on Hadoop 2.2.0
        Hide
        Hive QA added a comment -

        Overall: -1 no tests executed

        Here are the results of testing the latest attachment:
        https://issues.apache.org/jira/secure/attachment/12610389/HIVE-5635.patch

        Test results: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/116/testReport
        Console output: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/116/console

        Messages:

        Executing org.apache.hive.ptest.execution.PrepPhase
        Tests failed with: NonZeroExitCodeException: Command 'bash /data/hive-ptest/working/scratch/source-prep.sh' failed with exit status 1 and output '+ [[ -n '' ]]
        + export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost -Dhttp.proxyPort=3128'
        + ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost -Dhttp.proxyPort=3128'
        + export 'M2_OPTS=-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost -Dhttp.proxyPort=3128'
        + M2_OPTS='-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost -Dhttp.proxyPort=3128'
        + cd /data/hive-ptest/working/
        + tee /data/hive-ptest/logs/PreCommit-HIVE-Build-116/source-prep.txt
        + [[ true == \t\r\u\e ]]
        + rm -rf ivy maven
        + mkdir -p maven ivy
        + [[ svn = \s\v\n ]]
        + [[ -n '' ]]
        + [[ -d apache-svn-trunk-source ]]
        + [[ ! -d apache-svn-trunk-source/.svn ]]
        + [[ ! -d apache-svn-trunk-source ]]
        + cd apache-svn-trunk-source
        + svn revert -R .
        Reverted 'ql/src/test/results/clientpositive/decimal_udf.q.out'
        Reverted 'ql/src/test/results/clientpositive/udf_round.q.out'
        Reverted 'ql/src/test/results/compiler/plan/udf4.q.xml'
        Reverted 'ql/src/test/org/apache/hadoop/hive/ql/exec/vector/TestVectorizationContext.java'
        Reverted 'ql/src/test/org/apache/hadoop/hive/ql/udf/TestUDFRound.java'
        Reverted 'ql/src/test/queries/clientpositive/udf_round.q'
        Reverted 'ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/Vectorizer.java'
        Reverted 'ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java'
        Reverted 'ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/RoundWithNumDigitsDoubleToDouble.java'
        Reverted 'ql/src/java/org/apache/hadoop/hive/ql/udf/UDFRound.java'
        ++ egrep -v '^X|^Performing status on external'
        ++ awk '{print $2}'
        ++ svn status --no-ignore
        + rm -rf target datanucleus.log ant/target shims/target shims/0.20/target shims/0.20S/target shims/0.23/target shims/common/target shims/common-secure/target packaging/target hbase-handler/target testutils/target jdbc/target metastore/target itests/target itests/hcatalog-unit/target itests/test-serde/target itests/qtest/target itests/hive-unit/target itests/custom-serde/target itests/util/target hcatalog/target hcatalog/storage-handlers/hbase/target hcatalog/server-extensions/target hcatalog/core/target hcatalog/webhcat/svr/target hcatalog/webhcat/java-client/target hcatalog/hcatalog-pig-adapter/target hwi/target common/target common/src/gen contrib/target service/target serde/target beeline/target odbc/target cli/target ql/dependency-reduced-pom.xml ql/target ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFRound.java ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFRound.java ql/src/java/org/apache/hadoop/hive/ql/udf/generic/RoundUtils.java
        + svn update
        U    hcatalog/webhcat/svr/src/main/config/webhcat-default.xml
        U    hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/HiveDelegator.java
        U    hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/LauncherDelegator.java
        U    hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/tool/LaunchMapper.java
        U    hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/tool/JobSubmissionConstants.java
        U    hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/tool/TempletonControllerJob.java
        U    hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/TempletonDelegator.java
        U    hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/AppConfig.java
        U    hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/Server.java
        U    hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/Main.java
        U    hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/JarDelegator.java
        U    hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/PigDelegator.java
        
        Fetching external item into 'hcatalog/src/test/e2e/harness'
        Updated external to revision 1538078.
        
        Updated to revision 1538078.
        + patchCommandPath=/data/hive-ptest/working/scratch/smart-apply-patch.sh
        + patchFilePath=/data/hive-ptest/working/scratch/build.patch
        + [[ -f /data/hive-ptest/working/scratch/build.patch ]]
        + chmod +x /data/hive-ptest/working/scratch/smart-apply-patch.sh
        + /data/hive-ptest/working/scratch/smart-apply-patch.sh /data/hive-ptest/working/scratch/build.patch
        The patch does not appear to apply with p0, p1, or p2
        + exit 1
        '
        

        This message is automatically generated.

        ATTACHMENT ID: 12610389

        Show
        Hive QA added a comment - Overall : -1 no tests executed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12610389/HIVE-5635.patch Test results: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/116/testReport Console output: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/116/console Messages: Executing org.apache.hive.ptest.execution.PrepPhase Tests failed with: NonZeroExitCodeException: Command 'bash /data/hive-ptest/working/scratch/source-prep.sh' failed with exit status 1 and output '+ [[ -n '' ]] + export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost -Dhttp.proxyPort=3128' + ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost -Dhttp.proxyPort=3128' + export 'M2_OPTS=-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost -Dhttp.proxyPort=3128' + M2_OPTS='-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost -Dhttp.proxyPort=3128' + cd /data/hive-ptest/working/ + tee /data/hive-ptest/logs/PreCommit-HIVE-Build-116/source-prep.txt + [[ true == \t\r\u\e ]] + rm -rf ivy maven + mkdir -p maven ivy + [[ svn = \s\v\n ]] + [[ -n '' ]] + [[ -d apache-svn-trunk-source ]] + [[ ! -d apache-svn-trunk-source/.svn ]] + [[ ! -d apache-svn-trunk-source ]] + cd apache-svn-trunk-source + svn revert -R . Reverted 'ql/src/test/results/clientpositive/decimal_udf.q.out' Reverted 'ql/src/test/results/clientpositive/udf_round.q.out' Reverted 'ql/src/test/results/compiler/plan/udf4.q.xml' Reverted 'ql/src/test/org/apache/hadoop/hive/ql/exec/vector/TestVectorizationContext.java' Reverted 'ql/src/test/org/apache/hadoop/hive/ql/udf/TestUDFRound.java' Reverted 'ql/src/test/queries/clientpositive/udf_round.q' Reverted 'ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/Vectorizer.java' Reverted 'ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java' Reverted 'ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/RoundWithNumDigitsDoubleToDouble.java' Reverted 'ql/src/java/org/apache/hadoop/hive/ql/udf/UDFRound.java' ++ egrep -v '^X|^Performing status on external' ++ awk '{print $2}' ++ svn status --no-ignore + rm -rf target datanucleus.log ant/target shims/target shims/0.20/target shims/0.20S/target shims/0.23/target shims/common/target shims/common-secure/target packaging/target hbase-handler/target testutils/target jdbc/target metastore/target itests/target itests/hcatalog-unit/target itests/test-serde/target itests/qtest/target itests/hive-unit/target itests/custom-serde/target itests/util/target hcatalog/target hcatalog/storage-handlers/hbase/target hcatalog/server-extensions/target hcatalog/core/target hcatalog/webhcat/svr/target hcatalog/webhcat/java-client/target hcatalog/hcatalog-pig-adapter/target hwi/target common/target common/src/gen contrib/target service/target serde/target beeline/target odbc/target cli/target ql/dependency-reduced-pom.xml ql/target ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFRound.java ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFRound.java ql/src/java/org/apache/hadoop/hive/ql/udf/generic/RoundUtils.java + svn update U hcatalog/webhcat/svr/src/main/config/webhcat-default.xml U hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/HiveDelegator.java U hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/LauncherDelegator.java U hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/tool/LaunchMapper.java U hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/tool/JobSubmissionConstants.java U hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/tool/TempletonControllerJob.java U hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/TempletonDelegator.java U hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/AppConfig.java U hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/Server.java U hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/Main.java U hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/JarDelegator.java U hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/PigDelegator.java Fetching external item into 'hcatalog/src/test/e2e/harness' Updated external to revision 1538078. Updated to revision 1538078. + patchCommandPath=/data/hive-ptest/working/scratch/smart-apply-patch.sh + patchFilePath=/data/hive-ptest/working/scratch/build.patch + [[ -f /data/hive-ptest/working/scratch/build.patch ]] + chmod +x /data/hive-ptest/working/scratch/smart-apply-patch.sh + /data/hive-ptest/working/scratch/smart-apply-patch.sh /data/hive-ptest/working/scratch/build.patch The patch does not appear to apply with p0, p1, or p2 + exit 1 ' This message is automatically generated. ATTACHMENT ID: 12610389
        Hide
        Thejas M Nair added a comment -

        Changes look good, but it needs to be rebased for current trunk.

        Show
        Thejas M Nair added a comment - Changes look good, but it needs to be rebased for current trunk.
        Hide
        Eugene Koifman added a comment -

        HIVE-5635.2.patch rebased to current trunk

        Show
        Eugene Koifman added a comment - HIVE-5635 .2.patch rebased to current trunk
        Hide
        Hive QA added a comment -

        Overall: -1 no tests executed

        Here are the results of testing the latest attachment:
        https://issues.apache.org/jira/secure/attachment/12613732/HIVE-5635.2.patch

        Test results: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/286/testReport
        Console output: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/286/console

        Messages:

        Executing org.apache.hive.ptest.execution.PrepPhase
        Executing org.apache.hive.ptest.execution.ExecutionPhase
        Tests failed with: IllegalArgumentException: null
        

        This message is automatically generated.

        ATTACHMENT ID: 12613732

        Show
        Hive QA added a comment - Overall : -1 no tests executed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12613732/HIVE-5635.2.patch Test results: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/286/testReport Console output: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/286/console Messages: Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Tests failed with: IllegalArgumentException: null This message is automatically generated. ATTACHMENT ID: 12613732
        Hide
        Hive QA added a comment -

        Overall: -1 at least one tests failed

        Here are the results of testing the latest attachment:
        https://issues.apache.org/jira/secure/attachment/12614535/HIVE-5635.3.patch

        ERROR: -1 due to 1 failed/errored test(s), 4617 tests executed
        Failed tests:

        org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_bucket_num_reducers
        

        Test results: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/360/testReport
        Console output: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/360/console

        Messages:

        Executing org.apache.hive.ptest.execution.PrepPhase
        Executing org.apache.hive.ptest.execution.ExecutionPhase
        Executing org.apache.hive.ptest.execution.ReportingPhase
        Tests failed with: TestsFailedException: 1 tests failed
        

        This message is automatically generated.

        ATTACHMENT ID: 12614535

        Show
        Hive QA added a comment - Overall : -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12614535/HIVE-5635.3.patch ERROR: -1 due to 1 failed/errored test(s), 4617 tests executed Failed tests: org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_bucket_num_reducers Test results: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/360/testReport Console output: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/360/console Messages: Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests failed with: TestsFailedException: 1 tests failed This message is automatically generated. ATTACHMENT ID: 12614535
        Hide
        Thejas M Nair added a comment -

        Patch committed to trunk.
        Thanks for the contribution Eugene!

        Show
        Thejas M Nair added a comment - Patch committed to trunk. Thanks for the contribution Eugene!
        Hide
        shanyu zhao added a comment -

        Hi Eugene Koifman, any reason you used 2 nested ugi.doAs()? Isn't 1 doAs() enough?

        Show
        shanyu zhao added a comment - Hi Eugene Koifman , any reason you used 2 nested ugi.doAs()? Isn't 1 doAs() enough?
        Hide
        Eugene Koifman added a comment -

        shanyu zhao you're right, it does seem odd. I think 1 should be enough.

        Show
        Eugene Koifman added a comment - shanyu zhao you're right, it does seem odd. I think 1 should be enough.

          People

          • Assignee:
            Eugene Koifman
            Reporter:
            Eugene Koifman
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development