[AMBARI-20065] Livy server start failed after downgrade with missing livy.server.kerberos.keytab error

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 2.5.0
• Fix Version/s: 2.5.0
• Component/s: ambari-server
• Assignee: Jonathan Hurley
• Reporter: Vivek Sharma
• Created: 17/Feb/17 15:38
• Updated: 20/Mar/17 21:29
• Resolved: 17/Feb/17 20:49

Description

STR

1. Deployed cluster with Ambari 2.4.2.0-136 and HDP version: 2.5.3.0-37
2. Upgrade Ambari to 2.5.0.0-863
3. Perform EU to 2.6.0.0 and at finalize screen, start downgrade
4. Let downgrade complete
5. Observe the status of Livy Server after downgrade is successful

Result:
Livy server reports down and log says:

17/02/10 15:10:44 INFO LivyServer: Using spark-submit version 1.6.2
17/02/10 15:10:46 INFO LivyServer: SPNEGO auth enabled (principal = HTTP/[e-mail])
Exception in thread "main" java.lang.IllegalArgumentException: requirement failed: Kerberos requires livy.server.kerberos.keytab to be provided.
	at scala.Predef$.require(Predef.scala:233)
	at com.cloudera.livy.server.LivyServer.runKinit(LivyServer.scala:173)
	at com.cloudera.livy.server.LivyServer.start(LivyServer.scala:134)
	at com.cloudera.livy.server.LivyServer$.main(LivyServer.scala:277)
	at com.cloudera.livy.server.LivyServer.main(LivyServer.scala)

On Livy host, I see the following keytab:

root@natd76-ypis-erm16tofndwngdsec-d7-6:/var/log/livy# ls -lhrt /etc/security/keytabs/livy.service.keytab
-r-------- 1 livy hadoop 543 Feb 10 00:09 /etc/security/keytabs/livy.service.keytab

Issue Links

• links to: Reviewboard