Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-11581

HiveServer2 should store connection params in ZK when using dynamic service discovery for simpler client connection string.

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.3.0, 2.0.0
    • Fix Version/s: 1.3.0, 2.0.0
    • Component/s: HiveServer2, JDBC
    • Labels:

      Description

      Currently, the client needs to specify several parameters based on which an appropriate connection is created with the server. In case of dynamic service discovery, when multiple HS2 instances are running, it is much more usable for the server to add its config parameters to ZK which the driver can use to configure the connection, instead of the jdbc/odbc user adding those in connection string.

      However, at minimum, client will need to specify zookeeper ensemble and that she wants the JDBC driver to use ZooKeeper:

      beeline> !connect jdbc:hive2://vgumashta.local:2181,vgumashta.local:2182,vgumashta.local:2183/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2 vgumashta vgumashta org.apache.hive.jdbc.HiveDriver
      
      1. HIVE-11581.4.patch
        34 kB
        Vaibhav Gumashta
      2. HIVE-11581.3.patch
        34 kB
        Vaibhav Gumashta
      3. HIVE-11581.3.patch
        34 kB
        Vaibhav Gumashta
      4. HIVE-11581.2.patch
        34 kB
        Vaibhav Gumashta
      5. HIVE-11581.1.patch
        34 kB
        Vaibhav Gumashta

        Issue Links

          Activity

          Hide
          thejas Thejas M Nair added a comment -

          It also helps in making the use of hive more secure. As discussed in HIVE-11555, admins can enable SSL option for not sending clear text passwords by setting connection parameters in zookeeper.

          Show
          thejas Thejas M Nair added a comment - It also helps in making the use of hive more secure. As discussed in HIVE-11555 , admins can enable SSL option for not sending clear text passwords by setting connection parameters in zookeeper.
          Hide
          vgumashta Vaibhav Gumashta added a comment - - edited

          cc Thejas M Nair

          There's some minor movement in HiveConf to organize HS2 configs in one place in the file.

          Show
          vgumashta Vaibhav Gumashta added a comment - - edited cc Thejas M Nair There's some minor movement in HiveConf to organize HS2 configs in one place in the file.
          Hide
          hiveqa Hive QA added a comment -

          Overall: -1 at least one tests failed

          Here are the results of testing the latest attachment:
          https://issues.apache.org/jira/secure/attachment/12751534/HIVE-11581.1.patch

          ERROR: -1 due to 1 failed/errored test(s), 9370 tests executed
          Failed tests:

          org.apache.hive.jdbc.TestJdbcDriver2.testDriverProperties
          

          Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/5023/testReport
          Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/5023/console
          Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-5023/

          Messages:

          Executing org.apache.hive.ptest.execution.PrepPhase
          Executing org.apache.hive.ptest.execution.ExecutionPhase
          Executing org.apache.hive.ptest.execution.ReportingPhase
          Tests exited with: TestsFailedException: 1 tests failed
          

          This message is automatically generated.

          ATTACHMENT ID: 12751534 - PreCommit-HIVE-TRUNK-Build

          Show
          hiveqa Hive QA added a comment - Overall : -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12751534/HIVE-11581.1.patch ERROR: -1 due to 1 failed/errored test(s), 9370 tests executed Failed tests: org.apache.hive.jdbc.TestJdbcDriver2.testDriverProperties Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/5023/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/5023/console Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-5023/ Messages: Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 1 tests failed This message is automatically generated. ATTACHMENT ID: 12751534 - PreCommit-HIVE-TRUNK-Build
          Hide
          vgumashta Vaibhav Gumashta added a comment -

          Some session configs were going to hive configs within the driver. Fixed in v2.

          Show
          vgumashta Vaibhav Gumashta added a comment - Some session configs were going to hive configs within the driver. Fixed in v2.
          Hide
          hiveqa Hive QA added a comment -

          Overall: -1 at least one tests failed

          Here are the results of testing the latest attachment:
          https://issues.apache.org/jira/secure/attachment/12751601/HIVE-11581.2.patch

          ERROR: -1 due to 1 failed/errored test(s), 9371 tests executed
          Failed tests:

          org.apache.hive.jdbc.TestJdbcDriver2.testDriverProperties
          

          Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/5025/testReport
          Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/5025/console
          Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-5025/

          Messages:

          Executing org.apache.hive.ptest.execution.PrepPhase
          Executing org.apache.hive.ptest.execution.ExecutionPhase
          Executing org.apache.hive.ptest.execution.ReportingPhase
          Tests exited with: TestsFailedException: 1 tests failed
          

          This message is automatically generated.

          ATTACHMENT ID: 12751601 - PreCommit-HIVE-TRUNK-Build

          Show
          hiveqa Hive QA added a comment - Overall : -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12751601/HIVE-11581.2.patch ERROR: -1 due to 1 failed/errored test(s), 9371 tests executed Failed tests: org.apache.hive.jdbc.TestJdbcDriver2.testDriverProperties Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/5025/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/5025/console Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-5025/ Messages: Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 1 tests failed This message is automatically generated. ATTACHMENT ID: 12751601 - PreCommit-HIVE-TRUNK-Build
          Hide
          thejas Thejas M Nair added a comment -

          +1
          Can you also make a minor change before commit - change pattern variable name to maybe kvPattern ?

          Show
          thejas Thejas M Nair added a comment - +1 Can you also make a minor change before commit - change pattern variable name to maybe kvPattern ?
          Hide
          vgumashta Vaibhav Gumashta added a comment -

          Patch with kvPattern variable name.

          Show
          vgumashta Vaibhav Gumashta added a comment - Patch with kvPattern variable name.
          Hide
          hiveqa Hive QA added a comment -

          Overall: -1 at least one tests failed

          Here are the results of testing the latest attachment:
          https://issues.apache.org/jira/secure/attachment/12751843/HIVE-11581.3.patch

          ERROR: -1 due to 1 failed/errored test(s), 9376 tests executed
          Failed tests:

          org.apache.hive.jdbc.TestJdbcDriver2.testDriverProperties
          

          Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/5042/testReport
          Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/5042/console
          Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-5042/

          Messages:

          Executing org.apache.hive.ptest.execution.PrepPhase
          Executing org.apache.hive.ptest.execution.ExecutionPhase
          Executing org.apache.hive.ptest.execution.ReportingPhase
          Tests exited with: TestsFailedException: 1 tests failed
          

          This message is automatically generated.

          ATTACHMENT ID: 12751843 - PreCommit-HIVE-TRUNK-Build

          Show
          hiveqa Hive QA added a comment - Overall : -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12751843/HIVE-11581.3.patch ERROR: -1 due to 1 failed/errored test(s), 9376 tests executed Failed tests: org.apache.hive.jdbc.TestJdbcDriver2.testDriverProperties Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/5042/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/5042/console Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-5042/ Messages: Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 1 tests failed This message is automatically generated. ATTACHMENT ID: 12751843 - PreCommit-HIVE-TRUNK-Build
          Hide
          hiveqa Hive QA added a comment -

          Overall: -1 at least one tests failed

          Here are the results of testing the latest attachment:
          https://issues.apache.org/jira/secure/attachment/12752074/HIVE-11581.4.patch

          ERROR: -1 due to 1 failed/errored test(s), 9377 tests executed
          Failed tests:

          org.apache.hive.hcatalog.api.TestHCatClient.testTableSchemaPropagation
          

          Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/5056/testReport
          Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/5056/console
          Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-5056/

          Messages:

          Executing org.apache.hive.ptest.execution.PrepPhase
          Executing org.apache.hive.ptest.execution.ExecutionPhase
          Executing org.apache.hive.ptest.execution.ReportingPhase
          Tests exited with: TestsFailedException: 1 tests failed
          

          This message is automatically generated.

          ATTACHMENT ID: 12752074 - PreCommit-HIVE-TRUNK-Build

          Show
          hiveqa Hive QA added a comment - Overall : -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12752074/HIVE-11581.4.patch ERROR: -1 due to 1 failed/errored test(s), 9377 tests executed Failed tests: org.apache.hive.hcatalog.api.TestHCatClient.testTableSchemaPropagation Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/5056/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/5056/console Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-5056/ Messages: Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 1 tests failed This message is automatically generated. ATTACHMENT ID: 12752074 - PreCommit-HIVE-TRUNK-Build
          Hide
          vgumashta Vaibhav Gumashta added a comment -

          Failure is unrelated.

          Show
          vgumashta Vaibhav Gumashta added a comment - Failure is unrelated.
          Hide
          thejas Thejas M Nair added a comment -

          +1 for new patch

          Show
          thejas Thejas M Nair added a comment - +1 for new patch
          Hide
          vgumashta Vaibhav Gumashta added a comment -

          Patch committed to master and branch-1. Thanks for reviewing Thejas M Nair.

          Show
          vgumashta Vaibhav Gumashta added a comment - Patch committed to master and branch-1. Thanks for reviewing Thejas M Nair .
          Hide
          leftylev Lefty Leverenz added a comment -

          Does this need documentation? If so, please add a TODOC1.3 label. (No doc needed for the HiveConf.java changes – the patch just moves some parameters around in the file.)

          Show
          leftylev Lefty Leverenz added a comment - Does this need documentation? If so, please add a TODOC1.3 label. (No doc needed for the HiveConf.java changes – the patch just moves some parameters around in the file.) HiveServer2 Clients

            People

            • Assignee:
              vgumashta Vaibhav Gumashta
              Reporter:
              vgumashta Vaibhav Gumashta
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development