[HDFS-16218] RBF: Use HdfsConfiguration for passing in Router principal

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.4.0
• Fix Version/s: 3.4.0
• Component/s: rbf
• Labels: newbie pull-request-available

Environment

Hadoop 3.3.0 + patches, Kerberos authentication is enabled

• Target Version/s: 3.4.0
• Hadoop Flags: Reviewed
• Assignee: Fengnan Li
• Reporter: Akira Ajisaka
• Created: 10/Sep/21 05:04
• Updated: 27/Jan/24 02:36
• Resolved: 16/Sep/21 05:58

Description

RouterFedBalance fails to connect to DFSRouter when Kerberos is enabled because "dfs.federation.router.kerberos.principal" in hdfs-site.xml is not loaded.

21/09/08 17:21:38 ERROR rbfbalance.RouterFedBalance: Submit balance job failed.
java.io.IOException: DestHost:destPort 0.0.0.0:8111 , LocalHost:localPort <host>/<IP>:0. Failed on local exception: java.io.IOException: Couldn't set up IO streams: java.lang.IllegalArgumentException: Failed to specify server's Kerberos principal name
at org.apache.hadoop.hdfs.protocolPB.RouterAdminProtocolTranslatorPB.getMountTableEntries(RouterAdminProtocolTranslatorPB.java:198)
at org.apache.hadoop.hdfs.rbfbalance.MountTableProcedure.getMountEntry(MountTableProcedure.java:140)
at org.apache.hadoop.hdfs.rbfbalance.RouterFedBalance.getSrcPath(RouterFedBalance.java:326)
at org.apache.hadoop.hdfs.rbfbalance.RouterFedBalance.access$000(RouterFedBalance.java:68)
at org.apache.hadoop.hdfs.rbfbalance.RouterFedBalance$Builder.build(RouterFedBalance.java:168)
at org.apache.hadoop.hdfs.rbfbalance.RouterFedBalance.submit(RouterFedBalance.java:302)
at org.apache.hadoop.hdfs.rbfbalance.RouterFedBalance.run(RouterFedBalance.java:216)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:90)
at org.apache.hadoop.hdfs.rbfbalance.RouterFedBalance.main(RouterFedBalance.java:376)

When adding the property specifically by "-D" option, the command worked.

Issue Links

• links to: GitHub Pull Request #3444