Hive
  1. Hive
  2. HIVE-4669

Make username available to semantic analyzer hooks

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.10.0, 0.11.0
    • Fix Version/s: 0.13.0
    • Component/s: Query Processor
    • Labels:
      None

      Description

      Make username available to the semantic analyzer hooks.

      1. HIVE-4669.4.patch
        4 kB
        Shreepadma Venugopalan
      2. HIVE-4669.3.patch
        3 kB
        Shreepadma Venugopalan
      3. HIVE-4669.2.patch
        3 kB
        Shreepadma Venugopalan
      4. HIVE-4669.1.patch
        2 kB
        Shreepadma Venugopalan

        Activity

        Hide
        Shreepadma Venugopalan added a comment -

        Ping

        Show
        Shreepadma Venugopalan added a comment - Ping
        Hide
        Brock Noland added a comment -

        Shreepadma Venugopalan this looks good, but is there a review board item for it?

        Show
        Brock Noland added a comment - Shreepadma Venugopalan this looks good, but is there a review board item for it?
        Hide
        Ashutosh Chauhan added a comment -

        Whats the motivation for this change? Why do we want to mane username available for semantic analyzer?

        Show
        Ashutosh Chauhan added a comment - Whats the motivation for this change? Why do we want to mane username available for semantic analyzer?
        Hide
        Shreepadma Venugopalan added a comment -

        My apologies for not responding earlier. We need this for integrating Sentry with Hive.

        Show
        Shreepadma Venugopalan added a comment - My apologies for not responding earlier. We need this for integrating Sentry with Hive.
        Hide
        Shreepadma Venugopalan added a comment -

        Is there anything else needed from my side?

        Show
        Shreepadma Venugopalan added a comment - Is there anything else needed from my side?
        Hide
        Hive QA added a comment -

        Overall: -1 no tests executed

        Here are the results of testing the latest attachment:
        https://issues.apache.org/jira/secure/attachment/12586439/HIVE-4669.1.patch

        Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/971/testReport
        Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/971/console

        Messages:

        Executing org.apache.hive.ptest.execution.PrepPhase
        Tests failed with: NonZeroExitCodeException: Command 'bash /data/hive-ptest/working/scratch/source-prep.sh' failed with exit status 1 and output '+ [[ -n '' ]]
        + export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost -Dhttp.proxyPort=3128'
        + ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost -Dhttp.proxyPort=3128'
        + cd /data/hive-ptest/working/
        + tee /data/hive-ptest/logs/PreCommit-HIVE-Build-971/source-prep.txt
        + mkdir -p maven ivy
        + [[ svn = \s\v\n ]]
        + [[ -n '' ]]
        + [[ -d apache-svn-trunk-source ]]
        + [[ ! -d apache-svn-trunk-source/.svn ]]
        + [[ ! -d apache-svn-trunk-source ]]
        + cd apache-svn-trunk-source
        + svn revert -R .
        Reverted 'ql/src/java/org/apache/hadoop/hive/ql/exec/MapOperator.java'
        Reverted 'ql/src/java/org/apache/hadoop/hive/ql/exec/FetchOperator.java'
        ++ egrep -v '^X|^Performing status on external'
        ++ awk '{print $2}'
        ++ svn status --no-ignore
        + rm -rf build hcatalog/build hcatalog/core/build hcatalog/storage-handlers/hbase/build hcatalog/server-extensions/build hcatalog/webhcat/svr/build hcatalog/webhcat/java-client/build hcatalog/hcatalog-pig-adapter/build common/src/gen
        + svn update
        
        Fetching external item into 'hcatalog/src/test/e2e/harness'
        External at revision 1527832.
        
        At revision 1527832.
        + patchCommandPath=/data/hive-ptest/working/scratch/smart-apply-patch.sh
        + patchFilePath=/data/hive-ptest/working/scratch/build.patch
        + [[ -f /data/hive-ptest/working/scratch/build.patch ]]
        + chmod +x /data/hive-ptest/working/scratch/smart-apply-patch.sh
        + /data/hive-ptest/working/scratch/smart-apply-patch.sh /data/hive-ptest/working/scratch/build.patch
        The patch does not appear to apply with p0 to p2
        + exit 1
        '
        

        This message is automatically generated.

        Show
        Hive QA added a comment - Overall : -1 no tests executed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12586439/HIVE-4669.1.patch Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/971/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/971/console Messages: Executing org.apache.hive.ptest.execution.PrepPhase Tests failed with: NonZeroExitCodeException: Command 'bash /data/hive-ptest/working/scratch/source-prep.sh' failed with exit status 1 and output '+ [[ -n '' ]] + export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost -Dhttp.proxyPort=3128' + ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost -Dhttp.proxyPort=3128' + cd /data/hive-ptest/working/ + tee /data/hive-ptest/logs/PreCommit-HIVE-Build-971/source-prep.txt + mkdir -p maven ivy + [[ svn = \s\v\n ]] + [[ -n '' ]] + [[ -d apache-svn-trunk-source ]] + [[ ! -d apache-svn-trunk-source/.svn ]] + [[ ! -d apache-svn-trunk-source ]] + cd apache-svn-trunk-source + svn revert -R . Reverted 'ql/src/java/org/apache/hadoop/hive/ql/exec/MapOperator.java' Reverted 'ql/src/java/org/apache/hadoop/hive/ql/exec/FetchOperator.java' ++ egrep -v '^X|^Performing status on external' ++ awk '{print $2}' ++ svn status --no-ignore + rm -rf build hcatalog/build hcatalog/core/build hcatalog/storage-handlers/hbase/build hcatalog/server-extensions/build hcatalog/webhcat/svr/build hcatalog/webhcat/java-client/build hcatalog/hcatalog-pig-adapter/build common/src/gen + svn update Fetching external item into 'hcatalog/src/test/e2e/harness' External at revision 1527832. At revision 1527832. + patchCommandPath=/data/hive-ptest/working/scratch/smart-apply-patch.sh + patchFilePath=/data/hive-ptest/working/scratch/build.patch + [[ -f /data/hive-ptest/working/scratch/build.patch ]] + chmod +x /data/hive-ptest/working/scratch/smart-apply-patch.sh + /data/hive-ptest/working/scratch/smart-apply-patch.sh /data/hive-ptest/working/scratch/build.patch The patch does not appear to apply with p0 to p2 + exit 1 ' This message is automatically generated.
        Hide
        Shreepadma Venugopalan added a comment -

        Attached new patch rebased to the tip of trunk.

        Show
        Shreepadma Venugopalan added a comment - Attached new patch rebased to the tip of trunk.
        Hide
        Hive QA added a comment -

        Overall: -1 at least one tests failed

        Here are the results of testing the latest attachment:
        https://issues.apache.org/jira/secure/attachment/12606209/HIVE-4669.2.patch

        ERROR: -1 due to 1 failed/errored test(s), 4077 tests executed
        Failed tests:

        org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_bucket_num_reducers
        

        Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/990/testReport
        Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/990/console

        Messages:

        Executing org.apache.hive.ptest.execution.PrepPhase
        Executing org.apache.hive.ptest.execution.ExecutionPhase
        Executing org.apache.hive.ptest.execution.ReportingPhase
        Tests failed with: TestsFailedException: 1 tests failed
        

        This message is automatically generated.

        Show
        Hive QA added a comment - Overall : -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12606209/HIVE-4669.2.patch ERROR: -1 due to 1 failed/errored test(s), 4077 tests executed Failed tests: org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_bucket_num_reducers Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/990/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/990/console Messages: Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests failed with: TestsFailedException: 1 tests failed This message is automatically generated.
        Hide
        Brock Noland added a comment -

        I believe the test failure is unrelated. This is a trivial patch which doesn't impact any users of Hive so I am fine with committing it. However, I did notice that there is something funky with the patch. The first constructor which doesn't take a username argument sets the instance variable userName to what I presume is itself:

           public Driver(HiveConf conf) {
             this.conf = conf;
        +    this.userName = userName;
        +  }
        

        Since we are fixing that, this constructor can be redirected through the first constructor. I understand it's nit-picky since it's a good practice and we are going to be modifying the patch again anyways, let's do that.

        +  public Driver(HiveConf conf, String userName) {
        +    this.conf = conf;
        +    this.userName = userName;
           }
        
        Show
        Brock Noland added a comment - I believe the test failure is unrelated. This is a trivial patch which doesn't impact any users of Hive so I am fine with committing it. However, I did notice that there is something funky with the patch. The first constructor which doesn't take a username argument sets the instance variable userName to what I presume is itself: public Driver(HiveConf conf) { this.conf = conf; + this.userName = userName; + } Since we are fixing that, this constructor can be redirected through the first constructor. I understand it's nit-picky since it's a good practice and we are going to be modifying the patch again anyways, let's do that. + public Driver(HiveConf conf, String userName) { + this.conf = conf; + this.userName = userName; }
        Hide
        Shreepadma Venugopalan added a comment -

        Attached a new patch with the changes. Not sure if we'd have had to modify the patch except to remove

         this.userName = userName 

        from Driver.java.

        Show
        Shreepadma Venugopalan added a comment - Attached a new patch with the changes. Not sure if we'd have had to modify the patch except to remove this.userName = userName from Driver.java.
        Hide
        Brock Noland added a comment -

        Hi,

        Very sorry I wasn't clear, I mean the second constructor should use the first. Example below. This is pretty minor issue though and it certainly wouldn't be the worst offender in hive by any stretch of the imagination! So no need to update the patch.

          public Driver(HiveConf conf, String userName) {
             this.conf = conf;
           }
          public Driver(HiveConf conf, String userName) {
            this(conf);
            this.userName = userName;
           }
        
        Show
        Brock Noland added a comment - Hi, Very sorry I wasn't clear, I mean the second constructor should use the first. Example below. This is pretty minor issue though and it certainly wouldn't be the worst offender in hive by any stretch of the imagination! So no need to update the patch. public Driver(HiveConf conf, String userName) { this.conf = conf; } public Driver(HiveConf conf, String userName) { this(conf); this.userName = userName; }
        Hide
        Brock Noland added a comment -

        Whoops that should have been

          public Driver(HiveConf conf) {
             this.conf = conf;
           }
          public Driver(HiveConf conf, String userName) {
            this(conf);
            this.userName = userName;
           }
        
        Show
        Brock Noland added a comment - Whoops that should have been public Driver(HiveConf conf) { this.conf = conf; } public Driver(HiveConf conf, String userName) { this(conf); this.userName = userName; }
        Hide
        Shreepadma Venugopalan added a comment -

        No worries. Let's make sure the new code is clean. Uploaded a new patch.

        Show
        Shreepadma Venugopalan added a comment - No worries. Let's make sure the new code is clean. Uploaded a new patch.
        Hide
        Hive QA added a comment -

        Overall: -1 no tests executed

        Here are the results of testing the latest attachment:
        https://issues.apache.org/jira/secure/attachment/12606432/HIVE-4669.4.patch

        Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/1004/testReport
        Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/1004/console

        Messages:

        Executing org.apache.hive.ptest.execution.PrepPhase
        Executing org.apache.hive.ptest.execution.ExecutionPhase
        Tests failed with: IllegalArgumentException: null
        

        This message is automatically generated.

        Show
        Hive QA added a comment - Overall : -1 no tests executed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12606432/HIVE-4669.4.patch Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/1004/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/1004/console Messages: Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Tests failed with: IllegalArgumentException: null This message is automatically generated.
        Hide
        Hive QA added a comment -

        Overall: -1 at least one tests failed

        Here are the results of testing the latest attachment:
        https://issues.apache.org/jira/secure/attachment/12606432/HIVE-4669.4.patch

        ERROR: -1 due to 1 failed/errored test(s), 4044 tests executed
        Failed tests:

        org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_bucket_num_reducers
        

        Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/1005/testReport
        Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/1005/console

        Messages:

        Executing org.apache.hive.ptest.execution.PrepPhase
        Executing org.apache.hive.ptest.execution.ExecutionPhase
        Executing org.apache.hive.ptest.execution.ReportingPhase
        Tests failed with: TestsFailedException: 1 tests failed
        

        This message is automatically generated.

        Show
        Hive QA added a comment - Overall : -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12606432/HIVE-4669.4.patch ERROR: -1 due to 1 failed/errored test(s), 4044 tests executed Failed tests: org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_bucket_num_reducers Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/1005/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/1005/console Messages: Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests failed with: TestsFailedException: 1 tests failed This message is automatically generated.
        Hide
        Brock Noland added a comment -

        +1

        Show
        Brock Noland added a comment - +1
        Hide
        Brock Noland added a comment -

        Committed to trunk! Thank you very much for your contribution Shreepadma!

        Show
        Brock Noland added a comment - Committed to trunk! Thank you very much for your contribution Shreepadma!
        Hide
        Hudson added a comment -

        FAILURE: Integrated in Hive-trunk-hadoop2 #476 (See https://builds.apache.org/job/Hive-trunk-hadoop2/476/)
        HIVE-4669 - Make username available to semantic analyzer hooks (Shreepadma Venugopalan via Brock Noland) (brock: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1529195)

        • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/Driver.java
        • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/HiveSemanticAnalyzerHookContext.java
        • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/HiveSemanticAnalyzerHookContextImpl.java
        • /hive/trunk/service/src/java/org/apache/hive/service/cli/operation/SQLOperation.java
        Show
        Hudson added a comment - FAILURE: Integrated in Hive-trunk-hadoop2 #476 (See https://builds.apache.org/job/Hive-trunk-hadoop2/476/ ) HIVE-4669 - Make username available to semantic analyzer hooks (Shreepadma Venugopalan via Brock Noland) (brock: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1529195 ) /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/Driver.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/HiveSemanticAnalyzerHookContext.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/HiveSemanticAnalyzerHookContextImpl.java /hive/trunk/service/src/java/org/apache/hive/service/cli/operation/SQLOperation.java
        Hide
        Shreepadma Venugopalan added a comment -

        Thank you Brock!

        Show
        Shreepadma Venugopalan added a comment - Thank you Brock!
        Hide
        Hudson added a comment -

        FAILURE: Integrated in Hive-trunk-hadoop2-ptest #125 (See https://builds.apache.org/job/Hive-trunk-hadoop2-ptest/125/)
        HIVE-4669 - Make username available to semantic analyzer hooks (Shreepadma Venugopalan via Brock Noland) (brock: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1529195)

        • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/Driver.java
        • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/HiveSemanticAnalyzerHookContext.java
        • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/HiveSemanticAnalyzerHookContextImpl.java
        • /hive/trunk/service/src/java/org/apache/hive/service/cli/operation/SQLOperation.java
        Show
        Hudson added a comment - FAILURE: Integrated in Hive-trunk-hadoop2-ptest #125 (See https://builds.apache.org/job/Hive-trunk-hadoop2-ptest/125/ ) HIVE-4669 - Make username available to semantic analyzer hooks (Shreepadma Venugopalan via Brock Noland) (brock: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1529195 ) /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/Driver.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/HiveSemanticAnalyzerHookContext.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/HiveSemanticAnalyzerHookContextImpl.java /hive/trunk/service/src/java/org/apache/hive/service/cli/operation/SQLOperation.java
        Hide
        Hudson added a comment -

        FAILURE: Integrated in Hive-trunk-hadoop1-ptest #191 (See https://builds.apache.org/job/Hive-trunk-hadoop1-ptest/191/)
        HIVE-4669 - Make username available to semantic analyzer hooks (Shreepadma Venugopalan via Brock Noland) (brock: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1529195)

        • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/Driver.java
        • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/HiveSemanticAnalyzerHookContext.java
        • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/HiveSemanticAnalyzerHookContextImpl.java
        • /hive/trunk/service/src/java/org/apache/hive/service/cli/operation/SQLOperation.java
        Show
        Hudson added a comment - FAILURE: Integrated in Hive-trunk-hadoop1-ptest #191 (See https://builds.apache.org/job/Hive-trunk-hadoop1-ptest/191/ ) HIVE-4669 - Make username available to semantic analyzer hooks (Shreepadma Venugopalan via Brock Noland) (brock: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1529195 ) /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/Driver.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/HiveSemanticAnalyzerHookContext.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/HiveSemanticAnalyzerHookContextImpl.java /hive/trunk/service/src/java/org/apache/hive/service/cli/operation/SQLOperation.java
        Hide
        Hudson added a comment -

        FAILURE: Integrated in Hive-trunk-h0.21 #2381 (See https://builds.apache.org/job/Hive-trunk-h0.21/2381/)
        HIVE-4669 - Make username available to semantic analyzer hooks (Shreepadma Venugopalan via Brock Noland) (brock: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1529195)

        • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/Driver.java
        • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/HiveSemanticAnalyzerHookContext.java
        • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/HiveSemanticAnalyzerHookContextImpl.java
        • /hive/trunk/service/src/java/org/apache/hive/service/cli/operation/SQLOperation.java
        Show
        Hudson added a comment - FAILURE: Integrated in Hive-trunk-h0.21 #2381 (See https://builds.apache.org/job/Hive-trunk-h0.21/2381/ ) HIVE-4669 - Make username available to semantic analyzer hooks (Shreepadma Venugopalan via Brock Noland) (brock: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1529195 ) /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/Driver.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/HiveSemanticAnalyzerHookContext.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/HiveSemanticAnalyzerHookContextImpl.java /hive/trunk/service/src/java/org/apache/hive/service/cli/operation/SQLOperation.java

          People

          • Assignee:
            Shreepadma Venugopalan
            Reporter:
            Shreepadma Venugopalan
          • Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development