[AMBARI-21012] Livy service check fails with wire encryption setup

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 2.5.1
• Fix Version/s: 2.5.1
• Assignee: Saisai Shao
• Reporter: Yesha Vora
• Created: 12/May/17 21:31
• Updated: 17/May/17 04:16
• Resolved: 17/May/17 02:24

Description

Livy service check fails after enabling Wire encryption. This issue exist for Livy_server and Livy2_server both.
STR:
1) Set up below properties in livy.conf to enable WE.

livy.ssl.trustStore <Truststore File>
livy.ssl.trustStorePassword <Trustore passwd>
livy.key-password <keypassword>
livy.keystore <keystore file>
livy.keystore.password <keystore passwd>

2) Run Spark Service check.

Spark service check with fail to validate Livy. It is using http port to connect to livy. When Wire encryption is enabled, it should use https protocol to connect to livy.

stderr

Traceback (most recent call last):
  File "/var/lib/ambari-agent/cache/common-services/SPARK2/2.0.0/package/scripts/service_check.py", line 62, in <module>
    SparkServiceCheck().execute()
  File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 322, in execute
    method(env)
  File "/var/lib/ambari-agent/cache/common-services/SPARK2/2.0.0/package/scripts/service_check.py", line 59, in service_check
    raise Fail(format("Connection to all Livy servers failed"))
resource_management.core.exceptions.Fail: Connection to all Livy servers failed

stdout

2017-05-12 21:21:08,531 - Using hadoop conf dir: /usr/hdp/current/hadoop-client/conf
2017-05-12 21:21:08,551 - Execute['kinit -kt /etc/security/keytabs/xxx.headless.keytab xxx@XXX; '] {'user': 'spark'}
2017-05-12 21:21:08,683 - Execute['kinit -kt /etc/security/keytabs/smokeuser.headless.keytab xxx@XXX; '] {'user': 'livy'}
2017-05-12 21:21:08,809 - Execute['curl -s -o /dev/null -w'%{http_code}' --negotiate -u: -k https://xxxx:18481 | grep 200'] {'logoutput': True, 'tries': 5, 'try_sleep': 3}
200
2017-05-12 21:21:09,010 - Execute['curl -s -o /dev/null -w'%{http_code}' --negotiate -u: -k http://xxxx:8999/sessions | grep 200'] {'logoutput': True, 'tries': 3, 'user': 'livy', 'try_sleep': 1}
2017-05-12 21:21:09,149 - Retrying after 1 seconds. Reason: Execution of 'curl -s -o /dev/null -w'%{http_code}' --negotiate -u: -k http://xxx:8999/sessions | grep 200' returned 1. 
2017-05-12 21:21:10,286 - Retrying after 1 seconds. Reason: Execution of 'curl -s -o /dev/null -w'%{http_code}' --negotiate -u: -k http://xxxx:8999/sessions | grep 200' returned 1.

Command failed after 1 tries

Issue Links

• links to: review board