[HADOOP-11754] RM fails to start in non-secure mode due to authentication filter failure

• Type: Bug
• Status: Closed
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 2.7.0
• Fix Version/s: 2.7.0
• Target Version/s: 2.7.0
• Hadoop Flags: Reviewed
• Assignee: Haohui Mai
• Reporter: Sangjin Lee
• Created: 26/Mar/15 05:22
• Updated: 10/Apr/15 20:04
• Resolved: 30/Mar/15 18:46

Description

RM fails to start in the non-secure mode with the following exception:

2015-03-25 22:02:42,526 WARN org.mortbay.log: failed RMAuthenticationFilter: javax.servlet.ServletException: java.lang.RuntimeException: Could not read signature secret file: /Users/sjlee/hadoop-http-auth-signature-secret
2015-03-25 22:02:42,526 WARN org.mortbay.log: Failed startup of context org.mortbay.jetty.webapp.WebAppContext@6de50b08{/,jar:file:/Users/sjlee/hadoop-3.0.0-SNAPSHOT/share/hadoop/yarn/hadoop-yarn-common-3.0.0-SNAPSHOT.jar!/webapps/cluster}
javax.servlet.ServletException: java.lang.RuntimeException: Could not read signature secret file: /Users/sjlee/hadoop-http-auth-signature-secret
	at org.apache.hadoop.security.authentication.server.AuthenticationFilter.initializeSecretProvider(AuthenticationFilter.java:266)
	at org.apache.hadoop.security.authentication.server.AuthenticationFilter.init(AuthenticationFilter.java:225)
	at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticationFilter.init(DelegationTokenAuthenticationFilter.java:161)
	at org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilter.init(RMAuthenticationFilter.java:53)
	at org.mortbay.jetty.servlet.FilterHolder.doStart(FilterHolder.java:97)
	at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
	at org.mortbay.jetty.servlet.ServletHandler.initialize(ServletHandler.java:713)
	at org.mortbay.jetty.servlet.Context.startContext(Context.java:140)
	at org.mortbay.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1282)
	at org.mortbay.jetty.handler.ContextHandler.doStart(ContextHandler.java:518)
	at org.mortbay.jetty.webapp.WebAppContext.doStart(WebAppContext.java:499)
	at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
	at org.mortbay.jetty.handler.HandlerCollection.doStart(HandlerCollection.java:152)
	at org.mortbay.jetty.handler.ContextHandlerCollection.doStart(ContextHandlerCollection.java:156)
	at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
	at org.mortbay.jetty.handler.HandlerWrapper.doStart(HandlerWrapper.java:130)
	at org.mortbay.jetty.Server.doStart(Server.java:224)
	at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
	at org.apache.hadoop.http.HttpServer2.start(HttpServer2.java:773)
	at org.apache.hadoop.yarn.webapp.WebApps$Builder.start(WebApps.java:274)
	at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.startWepApp(ResourceManager.java:974)
	at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceStart(ResourceManager.java:1074)
	at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
	at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(ResourceManager.java:1208)
Caused by: java.lang.RuntimeException: Could not read signature secret file: /Users/sjlee/hadoop-http-auth-signature-secret
	at org.apache.hadoop.security.authentication.util.FileSignerSecretProvider.init(FileSignerSecretProvider.java:59)
	at org.apache.hadoop.security.authentication.server.AuthenticationFilter.initializeSecretProvider(AuthenticationFilter.java:264)
	... 23 more
...
2015-03-25 22:02:42,538 FATAL org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Error starting ResourceManager
org.apache.hadoop.yarn.webapp.WebAppException: Error starting http server
	at org.apache.hadoop.yarn.webapp.WebApps$Builder.start(WebApps.java:279)
	at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.startWepApp(ResourceManager.java:974)
	at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceStart(ResourceManager.java:1074)
	at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
	at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(ResourceManager.java:1208)
Caused by: java.io.IOException: Problem in starting http server. Server handlers failed
	at org.apache.hadoop.http.HttpServer2.start(HttpServer2.java:785)
	at org.apache.hadoop.yarn.webapp.WebApps$Builder.start(WebApps.java:274)
	... 4 more

This is likely a regression introduced by HADOOP-10670.

Issue Links

• breaks: HADOOP-11815 HttpServer2 should destroy SignerSecretProvider when it stops
• is broken by: HADOOP-10670 Allow AuthenticationFilters to load secret from signature secret files
• is duplicated by: HADOOP-11763 RM in insecure model get start failure after HADOOP-10670.
• supercedes: HDFS-5796 The file system browser in the namenode UI requires SPNEGO.