[HDFS-3813] Log error message if security and WebHDFS are enabled but principal/keytab are not configured

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.0.0-alpha
• Fix Version/s: 2.0.3-alpha
• Component/s: security, webhdfs
• Labels: newbie
• Hadoop Flags: Reviewed
• Assignee: Stephen Chu
• Reporter: Stephen Chu
• Created: 17/Aug/12 01:05
• Updated: 15/Feb/13 13:12
• Resolved: 05/Oct/12 00:57

Description

I configured a secure HDFS cluster, but failed to start the NameNode because I had enabled WebHDFS without specifying dfs.web.authentication.kerberos.principal in hdfs-site.xml.

In the NN logs, I saw:

2012-05-28 17:50:13,021 INFO org.apache.hadoop.security.authentication.server.KerberosAuthenticationHandler: Login using keytab /etc/hdfs.keytab, for principal HTTP/[e-mail]
2012-05-28 17:50:13,030 INFO org.apache.hadoop.security.authentication.server.KerberosAuthenticationHandler: Initialized, principal [HTTP/[e-mail]] from keytab [/etc/hdfs.keytab]
2012-05-28 17:50:13,031 WARN org.apache.hadoop.security.authentication.server.AuthenticationFilter: 'signature.secret' configuration not set, using a random value as secret
2012-05-28 17:50:13,032 WARN org.mortbay.log: failed SPNEGO: javax.servlet.ServletException: javax.servlet.ServletException: Principal not defined in configuration
2012-05-28 17:50:13,033 WARN org.mortbay.log: Failed startup of context org.mortbay.jetty.webapp.WebAppContext@21453d72{/,file:/usr/lib/hadoop-hdfs/webapps/hdfs}
javax.servlet.ServletException: javax.servlet.ServletException: Principal not defined in configuration
	at org.apache.hadoop.security.authentication.server.KerberosAuthenticationHandler.init(KerberosAuthenticationHandler.java:185)
	at org.apache.hadoop.security.authentication.server.AuthenticationFilter.init(AuthenticationFilter.java:146)
	at org.mortbay.jetty.servlet.FilterHolder.doStart(FilterHolder.java:97)
	at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
	at org.mortbay.jetty.servlet.ServletHandler.initialize(ServletHandler.java:713)
	at org.mortbay.jetty.servlet.Context.startContext(Context.java:140)
	at org.mortbay.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1282)
	at org.mortbay.jetty.handler.ContextHandler.doStart(ContextHandler.java:518)
	at org.mortbay.jetty.webapp.WebAppContext.doStart(WebAppContext.java:499)
	at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
	at org.mortbay.jetty.handler.HandlerCollection.doStart(HandlerCollection.java:152)
	at org.mortbay.jetty.handler.ContextHandlerCollection.doStart(ContextHandlerCollection.java:156)
	at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
	at org.mortbay.jetty.handler.HandlerWrapper.doStart(HandlerWrapper.java:130)
	at org.mortbay.jetty.Server.doStart(Server.java:224)
	at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
	at org.apache.hadoop.http.HttpServer.start(HttpServer.java:617)
	at org.apache.hadoop.hdfs.server.namenode.NameNodeHttpServer.start(NameNodeHttpServer.java:173)
	at org.apache.hadoop.hdfs.server.namenode.NameNode.startHttpServer(NameNode.java:529)
	at org.apache.hadoop.hdfs.server.namenode.NameNode.startCommonServices(NameNode.java:471)
	at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:434)
	at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:590)
	at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:571)
	at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1134)
	at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1193)
Caused by: javax.servlet.ServletException: Principal not defined in configuration
	at org.apache.hadoop.security.authentication.server.KerberosAuthenticationHandler.init(KerberosAuthenticationHandler.java:146)
	... 24 more
2012-05-28 17:50:13,034 WARN org.mortbay.log: Nested in javax.servlet.ServletException: javax.servlet.ServletException: Principal not defined in configuration:
javax.servlet.ServletException: Principal not defined in configuration
	at org.apache.hadoop.security.authentication.server.KerberosAuthenticationHandler.init(KerberosAuthenticationHandler.java:146)
	at org.apache.hadoop.security.authentication.server.AuthenticationFilter.init(AuthenticationFilter.java:146)
	at org.mortbay.jetty.servlet.FilterHolder.doStart(FilterHolder.java:97)
	at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
	at org.mortbay.jetty.servlet.ServletHandler.initialize(ServletHandler.java:713)
	at org.mortbay.jetty.servlet.Context.startContext(Context.java:140)
	at org.mortbay.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1282)
	at org.mortbay.jetty.handler.ContextHandler.doStart(ContextHandler.java:518)
	at org.mortbay.jetty.webapp.WebAppContext.doStart(WebAppContext.java:499)
	at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
	at org.mortbay.jetty.handler.HandlerCollection.doStart(HandlerCollection.java:152)
	at org.mortbay.jetty.handler.ContextHandlerCollection.doStart(ContextHandlerCollection.java:156)
	at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
	at org.mortbay.jetty.handler.HandlerWrapper.doStart(HandlerWrapper.java:130)
	at org.mortbay.jetty.Server.doStart(Server.java:224)
	at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
	at org.apache.hadoop.http.HttpServer.start(HttpServer.java:617)
	at org.apache.hadoop.hdfs.server.namenode.NameNodeHttpServer.start(NameNodeHttpServer.java:173)
	at org.apache.hadoop.hdfs.server.namenode.NameNode.startHttpServer(NameNode.java:529)
	at org.apache.hadoop.hdfs.server.namenode.NameNode.startCommonServices(NameNode.java:471)
	at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:434)
	at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:590)
	at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:571)
	at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1134)
	at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1193)
2012-05-28 17:50:13,041 INFO org.mortbay.log: Started [e-mail]:50070
2012-05-28 17:50:13,041 INFO org.apache.hadoop.hdfs.server.namenode.NameNode: Web-server up at: c1225.hal.cloudera.com:50070
2012-05-28 17:50:13,042 INFO org.apache.hadoop.ipc.Server: IPC Server Responder: starting
2012-05-28 17:50:13,042 INFO org.apache.hadoop.ipc.Server: IPC Server listener on 17020: starting
2012-05-28 17:50:13,045 INFO org.apache.hadoop.hdfs.server.namenode.NameNode: NameNode up at: c1225.hal.cloudera.com/172.29.98.216:17020
2012-05-28 17:50:13,045 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Starting services required for standby state
2012-05-28 17:50:13,048 INFO org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer: Will roll logs on active node at c1226.hal.cloudera.com/172.29.98.217:17020 every 120 seconds.
2012-05-28 17:50:13,058 INFO org.apache.hadoop.hdfs.server.namenode.ha.StandbyCheckpointer: Starting standby checkpoint thread...
Checkpointing active NN at c1226.hal.cloudera.com:50070
Serving checkpoints at c1225.hal.cloudera.com/172.29.98.216:50070

I couldn't figure out what I had misconfigured, but ATM found that I was missing dfs.web.authentication.kerberos.principal.

Logging an error if this property is not configured when WebHDFS and security are enabled would be useful for future users running into the same problem.

Issue Links

• is related to: HADOOP-8410 SPNEGO filter should have better error messages when not fully configured