[HADOOP-16457] Hadoop does not work with Kerberos config in hdfs-site.xml for simple security

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 3.3.0
• Fix Version/s: 3.3.0
• Target Version/s: 3.3.0
• Assignee: Prabhu Joseph
• Reporter: Eric Yang
• Created: 24/Jul/19 16:11
• Updated: 23/Oct/19 16:14
• Resolved: 06/Aug/19 21:09

Description

When http filter initializers is setup to use StaticUserWebFilter, AuthFilter is still setup. This prevents datanode to talk to namenode.

Error message in namenode logs:

2019-07-24 15:47:38,038 INFO org.apache.hadoop.hdfs.DFSUtil: Filter initializers set : org.apache.hadoop.http.lib.StaticUserWebFilter,org.apache.hadoop.hdfs.web.AuthFilterInitializer
2019-07-24 16:06:26,212 WARN SecurityLogger.org.apache.hadoop.security.authorize.ServiceAuthorizationManager: Authorization failed for hdfs (auth:SIMPLE) for protocol=interface org.apache.hadoop.hdfs.server.protocol.DatanodeProtocol: this service is only accessible by dn/[e-mail]

Errors in datanode log:

2019-07-24 16:07:01,253 WARN org.apache.hadoop.hdfs.server.datanode.DataNode: Problem connecting to server: eyang-1.openstacklocal/172.26.111.17:9000

The logic in HADOOP-16354 always added AuthFilter regardless security is enabled or not. This is incorrect. When simple security is chosen and using StaticUserWebFilter. AutheFilter check should not be required for datanode to communicate with namenode.