Hive
  1. Hive
  2. HIVE-2503

HiveServer should provide per session configuration

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.9.0
    • Fix Version/s: 0.9.0
    • Component/s: CLI, Server Infrastructure
    • Labels:
      None

      Description

      Currently ThriftHiveProcessorFactory returns same HiveConf instance to HiveServerHandler, making impossible to use per sesssion configuration. Just wrapping 'conf' -> 'new HiveConf(conf)' seemed to solve this problem.

        Issue Links

          Activity

          Hide
          Ashutosh Chauhan added a comment -

          @Navis,
          Can you elaborate bit more about the problem, which session you are referring to here?

          Show
          Ashutosh Chauhan added a comment - @Navis, Can you elaborate bit more about the problem, which session you are referring to here?
          Hide
          Navis added a comment -

          I'm using multiple ThriftHive.Client to single HiveServer.

          For example :

          client1 = factory.getCilent(~~~);
          client2 = factory.getCilent(~~~);
          
          client1.execute("set hiveconf:var=value1");
          client2.execute("set hiveconf:var=value2");
          
          client1.execute("set hiveconf:var"); --> returns value2 
          
          Show
          Navis added a comment - I'm using multiple ThriftHive.Client to single HiveServer. For example : client1 = factory.getCilent(~~~); client2 = factory.getCilent(~~~); client1.execute( "set hiveconf: var =value1" ); client2.execute( "set hiveconf: var =value2" ); client1.execute( "set hiveconf: var " ); --> returns value2
          Hide
          Navis added a comment -

          Removed diff-prefix and renamed patch file as supposed to be.

          Show
          Navis added a comment - Removed diff-prefix and renamed patch file as supposed to be.
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/2820/
          -----------------------------------------------------------

          Review request for hive and Carl Steinbach.

          Summary
          -------

          We uses multiple HiveClient connected to single HiveServer. After configuring connections, we've found the environment variables are different from expected.
          Current hive server uses single instance of HiveConf and this seemed to make the problem.

          This addresses bug HIVE-2503.
          https://issues.apache.org/jira/browse/HIVE-2503

          Diffs


          service/src/java/org/apache/hadoop/hive/service/HiveServer.java a2d599f

          Diff: https://reviews.apache.org/r/2820/diff

          Testing
          -------

          If we use 'standAloneServer' in TestHiveServer.class, it is very simple to make the test case. But it seemed to be 'false' which makes it more complex.

          Thanks,

          Navis

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2820/ ----------------------------------------------------------- Review request for hive and Carl Steinbach. Summary ------- We uses multiple HiveClient connected to single HiveServer. After configuring connections, we've found the environment variables are different from expected. Current hive server uses single instance of HiveConf and this seemed to make the problem. This addresses bug HIVE-2503 . https://issues.apache.org/jira/browse/HIVE-2503 Diffs service/src/java/org/apache/hadoop/hive/service/HiveServer.java a2d599f Diff: https://reviews.apache.org/r/2820/diff Testing ------- If we use 'standAloneServer' in TestHiveServer.class, it is very simple to make the test case. But it seemed to be 'false' which makes it more complex. Thanks, Navis
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/2820/#review3926
          -----------------------------------------------------------

          We should include a test case for regression purposes. You can achieve this by playing with test.service.standalone.server. If that doesn't work out, then you can always extend TestHiveServer and setup the test the way you want.

          • Ashutosh

          On 2011-11-14 08:54:00, Navis Ryu wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/2820/

          -----------------------------------------------------------

          (Updated 2011-11-14 08:54:00)

          Review request for hive and Carl Steinbach.

          Summary

          -------

          We uses multiple HiveClient connected to single HiveServer. After configuring connections, we've found the environment variables are different from expected.

          Current hive server uses single instance of HiveConf and this seemed to make the problem.

          This addresses bug HIVE-2503.

          https://issues.apache.org/jira/browse/HIVE-2503

          Diffs

          -----

          service/src/java/org/apache/hadoop/hive/service/HiveServer.java a2d599f

          Diff: https://reviews.apache.org/r/2820/diff

          Testing

          -------

          If we use 'standAloneServer' in TestHiveServer.class, it is very simple to make the test case. But it seemed to be 'false' which makes it more complex.

          Thanks,

          Navis

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2820/#review3926 ----------------------------------------------------------- We should include a test case for regression purposes. You can achieve this by playing with test.service.standalone.server. If that doesn't work out, then you can always extend TestHiveServer and setup the test the way you want. Ashutosh On 2011-11-14 08:54:00, Navis Ryu wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2820/ ----------------------------------------------------------- (Updated 2011-11-14 08:54:00) Review request for hive and Carl Steinbach. Summary ------- We uses multiple HiveClient connected to single HiveServer. After configuring connections, we've found the environment variables are different from expected. Current hive server uses single instance of HiveConf and this seemed to make the problem. This addresses bug HIVE-2503 . https://issues.apache.org/jira/browse/HIVE-2503 Diffs ----- service/src/java/org/apache/hadoop/hive/service/HiveServer.java a2d599f Diff: https://reviews.apache.org/r/2820/diff Testing ------- If we use 'standAloneServer' in TestHiveServer.class, it is very simple to make the test case. But it seemed to be 'false' which makes it more complex. Thanks, Navis
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/2820/
          -----------------------------------------------------------

          (Updated 2011-12-16 05:44:15.242903)

          Review request for hive and Carl Steinbach.

          Changes
          -------

          I know the test code is very awkward, but that was what I've expected from HiveServer.

          Summary
          -------

          We uses multiple HiveClient connected to single HiveServer. After configuring connections, we've found the environment variables are different from expected.
          Current hive server uses single instance of HiveConf and this seemed to make the problem.

          This addresses bug HIVE-2503.
          https://issues.apache.org/jira/browse/HIVE-2503

          Diffs (updated)


          service/src/java/org/apache/hadoop/hive/service/HiveServer.java 854cc99
          service/src/test/org/apache/hadoop/hive/service/TestHiveServerSessions.java PRE-CREATION

          Diff: https://reviews.apache.org/r/2820/diff

          Testing
          -------

          If we use 'standAloneServer' in TestHiveServer.class, it is very simple to make the test case. But it seemed to be 'false' which makes it more complex.

          Thanks,

          Navis

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2820/ ----------------------------------------------------------- (Updated 2011-12-16 05:44:15.242903) Review request for hive and Carl Steinbach. Changes ------- I know the test code is very awkward, but that was what I've expected from HiveServer. Summary ------- We uses multiple HiveClient connected to single HiveServer. After configuring connections, we've found the environment variables are different from expected. Current hive server uses single instance of HiveConf and this seemed to make the problem. This addresses bug HIVE-2503 . https://issues.apache.org/jira/browse/HIVE-2503 Diffs (updated) service/src/java/org/apache/hadoop/hive/service/HiveServer.java 854cc99 service/src/test/org/apache/hadoop/hive/service/TestHiveServerSessions.java PRE-CREATION Diff: https://reviews.apache.org/r/2820/diff Testing ------- If we use 'standAloneServer' in TestHiveServer.class, it is very simple to make the test case. But it seemed to be 'false' which makes it more complex. Thanks, Navis
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/2820/#review3986
          -----------------------------------------------------------

          If we use 'standAloneServer' in TestHiveServer.class, it is very simple to make the test case. But it seemed to be 'false' which makes it more complex.

          Can you supply a version of this test so I can take a look. I'm curious to know why it doesn't work. Thanks.

          service/src/test/org/apache/hadoop/hive/service/TestHiveServerSessions.java
          <https://reviews.apache.org/r/2820/#comment9025>

          Missing ASF header.

          • Carl

          On 2011-12-16 05:44:15, Navis Ryu wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/2820/

          -----------------------------------------------------------

          (Updated 2011-12-16 05:44:15)

          Review request for hive and Carl Steinbach.

          Summary

          -------

          We uses multiple HiveClient connected to single HiveServer. After configuring connections, we've found the environment variables are different from expected.

          Current hive server uses single instance of HiveConf and this seemed to make the problem.

          This addresses bug HIVE-2503.

          https://issues.apache.org/jira/browse/HIVE-2503

          Diffs

          -----

          service/src/java/org/apache/hadoop/hive/service/HiveServer.java 854cc99

          service/src/test/org/apache/hadoop/hive/service/TestHiveServerSessions.java PRE-CREATION

          Diff: https://reviews.apache.org/r/2820/diff

          Testing

          -------

          If we use 'standAloneServer' in TestHiveServer.class, it is very simple to make the test case. But it seemed to be 'false' which makes it more complex.

          Thanks,

          Navis

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2820/#review3986 ----------------------------------------------------------- If we use 'standAloneServer' in TestHiveServer.class, it is very simple to make the test case. But it seemed to be 'false' which makes it more complex. Can you supply a version of this test so I can take a look. I'm curious to know why it doesn't work. Thanks. service/src/test/org/apache/hadoop/hive/service/TestHiveServerSessions.java < https://reviews.apache.org/r/2820/#comment9025 > Missing ASF header. Carl On 2011-12-16 05:44:15, Navis Ryu wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2820/ ----------------------------------------------------------- (Updated 2011-12-16 05:44:15) Review request for hive and Carl Steinbach. Summary ------- We uses multiple HiveClient connected to single HiveServer. After configuring connections, we've found the environment variables are different from expected. Current hive server uses single instance of HiveConf and this seemed to make the problem. This addresses bug HIVE-2503 . https://issues.apache.org/jira/browse/HIVE-2503 Diffs ----- service/src/java/org/apache/hadoop/hive/service/HiveServer.java 854cc99 service/src/test/org/apache/hadoop/hive/service/TestHiveServerSessions.java PRE-CREATION Diff: https://reviews.apache.org/r/2820/diff Testing ------- If we use 'standAloneServer' in TestHiveServer.class, it is very simple to make the test case. But it seemed to be 'false' which makes it more complex. Thanks, Navis
          Hide
          jiraposter@reviews.apache.org added a comment -

          On 2011-12-19 22:35:46, Carl Steinbach wrote:

          > > If we use 'standAloneServer' in TestHiveServer.class, it is very simple to make the test case. But it seemed to be 'false' which makes it more complex.

          >

          > Can you supply a version of this test so I can take a look. I'm curious to know why it doesn't work. Thanks.

          I think it cannot be tested by making two instance of client, cause "SessionState" is a ThreadLocal thing.. Is it right?

          • Navis

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/2820/#review3986
          -----------------------------------------------------------

          On 2011-12-16 05:44:15, Navis Ryu wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/2820/

          -----------------------------------------------------------

          (Updated 2011-12-16 05:44:15)

          Review request for hive and Carl Steinbach.

          Summary

          -------

          We uses multiple HiveClient connected to single HiveServer. After configuring connections, we've found the environment variables are different from expected.

          Current hive server uses single instance of HiveConf and this seemed to make the problem.

          This addresses bug HIVE-2503.

          https://issues.apache.org/jira/browse/HIVE-2503

          Diffs

          -----

          service/src/java/org/apache/hadoop/hive/service/HiveServer.java 854cc99

          service/src/test/org/apache/hadoop/hive/service/TestHiveServerSessions.java PRE-CREATION

          Diff: https://reviews.apache.org/r/2820/diff

          Testing

          -------

          If we use 'standAloneServer' in TestHiveServer.class, it is very simple to make the test case. But it seemed to be 'false' which makes it more complex.

          Thanks,

          Navis

          Show
          jiraposter@reviews.apache.org added a comment - On 2011-12-19 22:35:46, Carl Steinbach wrote: > > If we use 'standAloneServer' in TestHiveServer.class, it is very simple to make the test case. But it seemed to be 'false' which makes it more complex. > > Can you supply a version of this test so I can take a look. I'm curious to know why it doesn't work. Thanks. I think it cannot be tested by making two instance of client, cause "SessionState" is a ThreadLocal thing.. Is it right? Navis ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2820/#review3986 ----------------------------------------------------------- On 2011-12-16 05:44:15, Navis Ryu wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2820/ ----------------------------------------------------------- (Updated 2011-12-16 05:44:15) Review request for hive and Carl Steinbach. Summary ------- We uses multiple HiveClient connected to single HiveServer. After configuring connections, we've found the environment variables are different from expected. Current hive server uses single instance of HiveConf and this seemed to make the problem. This addresses bug HIVE-2503 . https://issues.apache.org/jira/browse/HIVE-2503 Diffs ----- service/src/java/org/apache/hadoop/hive/service/HiveServer.java 854cc99 service/src/test/org/apache/hadoop/hive/service/TestHiveServerSessions.java PRE-CREATION Diff: https://reviews.apache.org/r/2820/diff Testing ------- If we use 'standAloneServer' in TestHiveServer.class, it is very simple to make the test case. But it seemed to be 'false' which makes it more complex. Thanks, Navis
          Hide
          jiraposter@reviews.apache.org added a comment -

          On 2011-12-19 22:35:46, Carl Steinbach wrote:

          > > If we use 'standAloneServer' in TestHiveServer.class, it is very simple to make the test case. But it seemed to be 'false' which makes it more complex.

          >

          > Can you supply a version of this test so I can take a look. I'm curious to know why it doesn't work. Thanks.

          Navis Ryu wrote:

          I think it cannot be tested by making two instance of client, cause "SessionState" is a ThreadLocal thing.. Is it right?

          Yup, that makes sense.

          • Carl

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/2820/#review3986
          -----------------------------------------------------------

          On 2011-12-16 05:44:15, Navis Ryu wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/2820/

          -----------------------------------------------------------

          (Updated 2011-12-16 05:44:15)

          Review request for hive and Carl Steinbach.

          Summary

          -------

          We uses multiple HiveClient connected to single HiveServer. After configuring connections, we've found the environment variables are different from expected.

          Current hive server uses single instance of HiveConf and this seemed to make the problem.

          This addresses bug HIVE-2503.

          https://issues.apache.org/jira/browse/HIVE-2503

          Diffs

          -----

          service/src/java/org/apache/hadoop/hive/service/HiveServer.java 854cc99

          service/src/test/org/apache/hadoop/hive/service/TestHiveServerSessions.java PRE-CREATION

          Diff: https://reviews.apache.org/r/2820/diff

          Testing

          -------

          If we use 'standAloneServer' in TestHiveServer.class, it is very simple to make the test case. But it seemed to be 'false' which makes it more complex.

          Thanks,

          Navis

          Show
          jiraposter@reviews.apache.org added a comment - On 2011-12-19 22:35:46, Carl Steinbach wrote: > > If we use 'standAloneServer' in TestHiveServer.class, it is very simple to make the test case. But it seemed to be 'false' which makes it more complex. > > Can you supply a version of this test so I can take a look. I'm curious to know why it doesn't work. Thanks. Navis Ryu wrote: I think it cannot be tested by making two instance of client, cause "SessionState" is a ThreadLocal thing.. Is it right? Yup, that makes sense. Carl ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2820/#review3986 ----------------------------------------------------------- On 2011-12-16 05:44:15, Navis Ryu wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2820/ ----------------------------------------------------------- (Updated 2011-12-16 05:44:15) Review request for hive and Carl Steinbach. Summary ------- We uses multiple HiveClient connected to single HiveServer. After configuring connections, we've found the environment variables are different from expected. Current hive server uses single instance of HiveConf and this seemed to make the problem. This addresses bug HIVE-2503 . https://issues.apache.org/jira/browse/HIVE-2503 Diffs ----- service/src/java/org/apache/hadoop/hive/service/HiveServer.java 854cc99 service/src/test/org/apache/hadoop/hive/service/TestHiveServerSessions.java PRE-CREATION Diff: https://reviews.apache.org/r/2820/diff Testing ------- If we use 'standAloneServer' in TestHiveServer.class, it is very simple to make the test case. But it seemed to be 'false' which makes it more complex. Thanks, Navis
          Hide
          Carl Steinbach added a comment -

          @Navis: Is this ticket ready for a final review?

          Show
          Carl Steinbach added a comment - @Navis: Is this ticket ready for a final review?
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/2820/
          -----------------------------------------------------------

          (Updated 2012-01-04 05:08:24.994913)

          Review request for hive and Carl Steinbach.

          Changes
          -------

          Added missing ASF header.

          Summary
          -------

          We uses multiple HiveClient connected to single HiveServer. After configuring connections, we've found the environment variables are different from expected.
          Current hive server uses single instance of HiveConf and this seemed to make the problem.

          This addresses bug HIVE-2503.
          https://issues.apache.org/jira/browse/HIVE-2503

          Diffs (updated)


          service/src/java/org/apache/hadoop/hive/service/HiveServer.java 854cc99
          service/src/test/org/apache/hadoop/hive/service/TestHiveServerSessions.java PRE-CREATION

          Diff: https://reviews.apache.org/r/2820/diff

          Testing
          -------

          If we use 'standAloneServer' in TestHiveServer.class, it is very simple to make the test case. But it seemed to be 'false' which makes it more complex.

          Thanks,

          Navis

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2820/ ----------------------------------------------------------- (Updated 2012-01-04 05:08:24.994913) Review request for hive and Carl Steinbach. Changes ------- Added missing ASF header. Summary ------- We uses multiple HiveClient connected to single HiveServer. After configuring connections, we've found the environment variables are different from expected. Current hive server uses single instance of HiveConf and this seemed to make the problem. This addresses bug HIVE-2503 . https://issues.apache.org/jira/browse/HIVE-2503 Diffs (updated) service/src/java/org/apache/hadoop/hive/service/HiveServer.java 854cc99 service/src/test/org/apache/hadoop/hive/service/TestHiveServerSessions.java PRE-CREATION Diff: https://reviews.apache.org/r/2820/diff Testing ------- If we use 'standAloneServer' in TestHiveServer.class, it is very simple to make the test case. But it seemed to be 'false' which makes it more complex. Thanks, Navis
          Hide
          Navis added a comment -

          @Carl: Yes, ready.

          Show
          Navis added a comment - @Carl: Yes, ready.
          Hide
          Ashutosh Chauhan added a comment -

          +1 Will commit if tests pass. Navis, can you also post the patch on jira granting license.

          Show
          Ashutosh Chauhan added a comment - +1 Will commit if tests pass. Navis, can you also post the patch on jira granting license.
          Hide
          Ashutosh Chauhan added a comment -

          Committed to trunk. Thanks, Navis!

          Show
          Ashutosh Chauhan added a comment - Committed to trunk. Thanks, Navis!
          Hide
          Hudson added a comment -

          Integrated in Hive-trunk-h0.21 #1315 (See https://builds.apache.org/job/Hive-trunk-h0.21/1315/)
          HIVE-2503: HiveServer should provide per session configuration (navis via hashutosh) (Revision 1301568)

          Result = FAILURE
          hashutosh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1301568
          Files :

          • /hive/trunk/service/src/java/org/apache/hadoop/hive/service/HiveServer.java
          • /hive/trunk/service/src/test/org/apache/hadoop/hive/service/TestHiveServerSessions.java
          Show
          Hudson added a comment - Integrated in Hive-trunk-h0.21 #1315 (See https://builds.apache.org/job/Hive-trunk-h0.21/1315/ ) HIVE-2503 : HiveServer should provide per session configuration (navis via hashutosh) (Revision 1301568) Result = FAILURE hashutosh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1301568 Files : /hive/trunk/service/src/java/org/apache/hadoop/hive/service/HiveServer.java /hive/trunk/service/src/test/org/apache/hadoop/hive/service/TestHiveServerSessions.java
          Hide
          Ashutosh Chauhan added a comment -

          This issue is closed now. It was released with the fix in 0.9.0. If there is a problem, please open a new jira and link this one with that.

          Show
          Ashutosh Chauhan added a comment - This issue is closed now. It was released with the fix in 0.9.0. If there is a problem, please open a new jira and link this one with that.
          Hide
          Hudson added a comment -

          Integrated in Hive-trunk-hadoop2 #54 (See https://builds.apache.org/job/Hive-trunk-hadoop2/54/)
          HIVE-2503: HiveServer should provide per session configuration (navis via hashutosh) (Revision 1301568)

          Result = ABORTED
          hashutosh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1301568
          Files :

          • /hive/trunk/service/src/java/org/apache/hadoop/hive/service/HiveServer.java
          • /hive/trunk/service/src/test/org/apache/hadoop/hive/service/TestHiveServerSessions.java
          Show
          Hudson added a comment - Integrated in Hive-trunk-hadoop2 #54 (See https://builds.apache.org/job/Hive-trunk-hadoop2/54/ ) HIVE-2503 : HiveServer should provide per session configuration (navis via hashutosh) (Revision 1301568) Result = ABORTED hashutosh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1301568 Files : /hive/trunk/service/src/java/org/apache/hadoop/hive/service/HiveServer.java /hive/trunk/service/src/test/org/apache/hadoop/hive/service/TestHiveServerSessions.java

            People

            • Assignee:
              Navis
              Reporter:
              Navis
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development