[HDFS-14375] DataNode cannot serve BlockPool to multiple NameNodes in the different realm

• Type: Bug
• Status: Patch Available
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 3.1.1
• Component/s: security
• Assignee: Jihyun
• Reporter: Jihyun Cho
• Created: 18/Mar/19 10:55
• Updated: 15/Aug/19 18:28

Description

Let me explain the environment for a description.

KDC(TEST1.COM) <-- Cross-realm trust -->  KDC(TEST2.COM)
   |                                         |
NameNode1                                 NameNode2
   |                                         |
   ---------- DataNodes (federated) ----------

We configured the secure clusters and federated them.

• Principal
  • NameNode1 : nn/[e-mail]
  • NameNode2 : nn/[e-mail]
  • DataNodes : dn/[e-mail]

But DataNodes could not connect to NameNode1 with below error.

WARN SecurityLogger.org.apache.hadoop.security.authorize.ServiceAuthorizationManager: Authorization failed for dn/[e-mail] (auth:KERBEROS) for protocol=interface org.apache.hadoop.hdfs.server.protocol.DatanodeProtocol: this service is only accessible by dn/[e-mail]

We have avoided the error with attached patch.
The patch checks only using username and hostname except realm.
I think there is no problem. Because if realms are different and no cross-realm setting, they cannot communication each other. If you are worried about this, please let me know.

In the long run, it would be better if I could set multiple realms for authorize. Like this;

<property>
  <name>dfs.namenode.kerberos.trust-realms</name>
  <value>TEST1.COM,TEST2.COM</value>
</property>