Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-11100

HiveThriftServer HA issue,HiveThriftServer not registering with Zookeeper

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Won't Fix
    • Affects Version/s: 1.5.1
    • Fix Version/s: None
    • Component/s: SQL
    • Labels:
      None
    • Environment:

      Hive-1.2.1
      Hadoop-2.6.0

    • Flags:
      Patch

      Description

      hive-site.xml config:

      <property>
      <name>hive.server2.support.dynamic.service.discovery</name>
      <value>true</value>
      </property>
      <property>
      <name>hive.server2.zookeeper.namespace</name>
      <value>sparkhiveserver2</value>
      </property>
      <property>
      <name>hive.zookeeper.quorum</name>
      <value>zk1,zk2,zk3</value>
      </property>
      

      then start thrift server

      start-thriftserver.sh --master yarn
      

      In zookeeper znode "sparkhiveserver2" not found.
      hiveserver2 is working on this config!

        Issue Links

          Activity

          Hide
          apachespark Apache Spark added a comment -

          User 'xiaowangyu' has created a pull request for this issue:
          https://github.com/apache/spark/pull/9113

          Show
          apachespark Apache Spark added a comment - User 'xiaowangyu' has created a pull request for this issue: https://github.com/apache/spark/pull/9113
          Hide
          rxin Reynold Xin added a comment -

          Thanks for the patch.

          The thrift server has gone through a lot of changes lately, and we are not sure whether we'd want to keep the existing thrift server at all because it has a huge amount of technical debt. More so, I'm not sure if the zookeeper approach chosen by Hive is that great, since it is pretty difficult to operate in practice, so I'm not sure if this would become the long-term solution. For now, I'd recommend you patching your own version of Spark to get this functionality.

          Show
          rxin Reynold Xin added a comment - Thanks for the patch. The thrift server has gone through a lot of changes lately, and we are not sure whether we'd want to keep the existing thrift server at all because it has a huge amount of technical debt. More so, I'm not sure if the zookeeper approach chosen by Hive is that great, since it is pretty difficult to operate in practice, so I'm not sure if this would become the long-term solution. For now, I'd recommend you patching your own version of Spark to get this functionality.

            People

            • Assignee:
              Unassigned
              Reporter:
              wangxiaoyu Xiaoyu Wang
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development