[HDFS-16217] RBF: Set default value of hdfs.fedbalance.procedure.scheduler.journal.uri by adding appropriate config resources

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.4.0
• Fix Version/s: 3.4.0
• Component/s: rbf
• Labels: pull-request-available

Environment

Hadoop 3.3.0 with patches

• Target Version/s: 3.4.0
• Hadoop Flags: Reviewed
• Assignee: Viraj Jasani Assign to me
• Reporter: Akira Ajisaka
• Created: 10/Sep/21 04:56
• Updated: 27/Jan/24 02:36
• Resolved: 17/Sep/21 07:41

Description

When dfs.federation.router.federation.rename.option is set to DISTCP and hdfs.fedbalance.procedure.scheduler.journal.uri is not set, DFSRouter fails to launch.

2021-09-08 15:39:11,818 ERROR org.apache.hadoop.hdfs.server.federation.router.DFSRouter: Failed to start router
java.lang.NullPointerException
at java.base/java.net.URI$Parser.parse(URI.java:3104)
at java.base/java.net.URI.<init>(URI.java:600)
at org.apache.hadoop.hdfs.server.federation.router.RouterRpcServer.initRouterFedRename(RouterRpcServer.java:444)
at org.apache.hadoop.hdfs.server.federation.router.RouterRpcServer.<init>(RouterRpcServer.java:419)
at org.apache.hadoop.hdfs.server.federation.router.Router.createRpcServer(Router.java:391)
at org.apache.hadoop.hdfs.server.federation.router.Router.serviceInit(Router.java:188)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
at org.apache.hadoop.hdfs.server.federation.router.DFSRouter.main(DFSRouter.java:69)

hdfs.fedbalance.procedure.scheduler.journal.uri is hdfs://localhost:8020/tmp/procedure by default, however, the default value is not used in DFSRouter.

Issue Links

• links to: GitHub Pull Request #3434