[AMBARI-24767] Error while starting Timeline v2 Reader during Move operation

• Type: Task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.7.0
• Fix Version/s: 2.7.3
• Component/s: ambari-web
• Labels: pull-request-available
• Assignee: Attila Magyar
• Reporter: Vivek Sharma
• Created: 12/Oct/18 10:45
• Updated: 26/Oct/18 15:59
• Resolved: 26/Oct/18 15:59

Description

1. Deployed HDP-3.0.3 HA cluster with Ambari-2.7.3 (Active RM and Timeline v2 Reader are on the same host)
2. Under Yarn --> Service Actions - choose to Move Timeline Service V2.0 Reader. Pick a new host and let the move operation complete

2018-10-11 13:04:33,887 ERROR reader.TimelineReaderServer (TimelineReaderServer.java:startTimelineReaderServer(236)) - Error starting TimelineReaderWebServer
org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Failed to login from keytab
	at org.apache.hadoop.yarn.server.timelineservice.reader.TimelineReaderServer.serviceInit(TimelineReaderServer.java:88)
	at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
	at org.apache.hadoop.yarn.server.timelineservice.reader.TimelineReaderServer.startTimelineReaderServer(TimelineReaderServer.java:233)
	at org.apache.hadoop.yarn.server.timelineservice.reader.TimelineReaderServer.main(TimelineReaderServer.java:246)
Caused by: org.apache.hadoop.security.KerberosAuthException: failure to login: for principal: yarn/[e-mail] from keytab /etc/security/keytabs/yarn.service.keytab javax.security.auth.login.LoginException: Unable to obtain password from user

	at org.apache.hadoop.security.UserGroupInformation.doSubjectLogin(UserGroupInformation.java:1847)
	at org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytabAndReturnUGI(UserGroupInformation.java:1215)
	at org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytab(UserGroupInformation.java:1008)
	at org.apache.hadoop.security.SecurityUtil.login(SecurityUtil.java:313)
	at org.apache.hadoop.yarn.server.timelineservice.reader.TimelineReaderServer.serviceInit(TimelineReaderServer.java:85)
	... 3 more
Caused by: javax.security.auth.login.LoginException: Unable to obtain password from user

	at com.sun.security.auth.module.Krb5LoginModule.promptForPass(Krb5LoginModule.java:897)
	at com.sun.security.auth.module.Krb5LoginModule.attemptAuthentication(Krb5LoginModule.java:760)
	at com.sun.security.auth.module.Krb5LoginModule.login(Krb5LoginModule.java:617)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at javax.security.auth.login.LoginContext.invoke(LoginContext.java:755)
	at javax.security.auth.login.LoginContext.access$000(LoginContext.java:195)
	at javax.security.auth.login.LoginContext$4.run(LoginContext.java:682)
	at javax.security.auth.login.LoginContext$4.run(LoginContext.java:680)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:680)
	at javax.security.auth.login.LoginContext.login(LoginContext.java:587)
	at org.apache.hadoop.security.UserGroupInformation$HadoopLoginContext.login(UserGroupInformation.java:1926)
	at org.apache.hadoop.security.UserGroupInformation.doSubjectLogin(UserGroupInformation.java:1837)
	... 7 more
2018-10-11 13:04:33,890 INFO  util.ExitUtil (ExitUtil.java:terminate(210)) - Exiting with status -1: Error starting TimelineReaderWebServer
2018-10-11 13:04:33,894 INFO  reader.TimelineReaderServer (LogAdapter.java:info(51)) - SHUTDOWN_MSG: 
/************************************************************
SHUTDOWN_MSG: Shutting down TimelineReaderServer at ctr-e138-1518143905142-517945-01-000002.hwx.site/172.27.74.4
************************************************************/

Issue Links

• links to: GitHub Pull Request #2447
• links to: GitHub Pull Request #2450