[AMBARI-23943] Hiveserver2 fails to start on viewFS enabled cluster: {hive_server2_zookeeper_namespace} is not ready yet - ASF JIRA

XML

Word

Printable

JSON

Details

Type: Bug
Status: Resolved
Priority: Major
Resolution: Fixed
Affects Version/s: None
Fix Version/s: 2.7.0
Component/s: None
Labels:
- pull-request-available

Description

Tried to deploy a cluster with viewFS enabled via blueprint
HiveServer2 is failing to start with below error

Traceback (most recent call last):
File "/var/lib/ambari-agent/cache/stacks/HDP/3.0/services/HIVE/package/scripts/hive_server.py", line 137, in
HiveServer().execute()
File "/usr/lib/ambari-agent/lib/resource_management/libraries/script/script.py", line 353, in execute
method(env)
File "/var/lib/ambari-agent/cache/stacks/HDP/3.0/services/HIVE/package/scripts/hive_server.py", line 53, in start
hive_service('hiveserver2', action = 'start', upgrade_type=upgrade_type)
File "/var/lib/ambari-agent/cache/stacks/HDP/3.0/services/HIVE/package/scripts/hive_service.py", line 102, in hive_service
wait_for_znode()
File "/usr/lib/ambari-agent/lib/resource_management/libraries/functions/decorator.py", line 62, in wrapper
return function(*args, **kwargs)
File "/var/lib/ambari-agent/cache/stacks/HDP/3.0/services/HIVE/package/scripts/hive_service.py", line 191, in wait_for_znode
raise Fail(format("ZooKeeper node /

{hive_server2_zookeeper_namespace}

is not ready yet"))
resource_management.core.exceptions.Fail: ZooKeeper node /hiveserver2 is not ready yet

out log has a reference to hdfs:// Filesystem

2018-05-17 23:54:17,398 - Skipping fs root check as fs_root does not start with hdfs://
2018-05-17 23:54:17,399 - Execute['/var/lib/ambari-agent/tmp/start_hiveserver2_script /grid/0/log/hive/hive-server2.out /grid/0/log/hive/hive-server2.err /var/run/hive/hive-server.pid /usr/hdp/current/hive-server2/conf/ /grid/0/log/hive'] {'environment':

{'HIVE_BIN': 'hive', 'JAVA_HOME': u'/usr/lib/jvm/java-openjdk', 'HADOOP_HOME': u'/usr/hdp/current/hadoop-client'}

, 'not_if': 'ls /var/run/hive/hive-server.pid >/dev/null 2>&1 && ps -p >/dev/null 2>&1', 'user': 'hive', 'path': [u'/usr/sbin:/sbin:/usr/lib/ambari-server/*:/usr/lib64/qt-3.3/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/var/lib/ambari-agent:/usr/hdp/current/hive-server2/bin:/usr/hdp/3.0.0.0-1345/hadoop/bin']}
2018-05-17 23:54:17,507 - Execute['/usr/lib/jvm/java-openjdk/bin/java -cp /usr/lib/ambari-agent/DBConnectionVerification.jar:/usr/hdp/current/hive-server2/lib/mysql-connector-java.jar org.apache.ambari.server.DBConnectionVerification 'jdbc:mysql://ctr-e138-1518143905142-317960-01-000006.hwx.site/hivedb?createDatabaseIfNotExist=true' hiveuser [PROTECTED] com.mysql.jdbc.Driver']

{'path': ['/usr/sbin:/sbin:/usr/local/bin:/bin:/usr/bin'], 'tries': 5, 'try_sleep': 10}

2018-05-17 23:54:18,156 - call['/usr/hdp/current/zookeeper-client/bin/zkCli.sh -server ctr-e138-1518143905142-317960-01-000005.hwx.site:2181,ctr-e138-1518143905142-317960-01-000006.hwx.site:2181,ctr-e138-1518143905142-317960-01-000007.hwx.site:2181 ls /hiveserver2 | grep '\[serverUri=''] {}
2018-05-17 23:54:19,032 - call returned (1, 'Node does not exist: /hiveserver2')
2018-05-17 23:54:19,033 - Will retry 29 time(s), caught exception: ZooKeeper node /hiveserver2 is not ready yet. Sleeping for 5 sec(s)
2018-05-17 23:54:24,038 - call['/usr/hdp/current/zookeeper-client/bin/zkCli.sh -server ctr-e138-1518143905142-317960-01-000005.hwx.site:2181,ctr-e138-1518143905142-317960-01-000006.hwx.site:2181,ctr-e138-1518143905142-317960-01-000007.hwx.site:2181 ls /hiveserver2 | grep '\[serverUri=''] {}
2018-05-17 23:54:24,842 - call returned (1, 'Node does not exist: /hiveserver2')