Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-15747 RBF: Rename across sub-namespaces.
  3. HDFS-16218

RBF: Use HdfsConfiguration for passing in Router principal

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 3.4.0
    • rbf
    • Hadoop 3.3.0 + patches, Kerberos authentication is enabled

    Description

      RouterFedBalance fails to connect to DFSRouter when Kerberos is enabled because "dfs.federation.router.kerberos.principal" in hdfs-site.xml is not loaded.

      21/09/08 17:21:38 ERROR rbfbalance.RouterFedBalance: Submit balance job failed.
      java.io.IOException: DestHost:destPort 0.0.0.0:8111 , LocalHost:localPort <host>/<IP>:0. Failed on local exception: java.io.IOException: Couldn't set up IO streams: java.lang.IllegalArgumentException: Failed to specify server's Kerberos principal name
      at org.apache.hadoop.hdfs.protocolPB.RouterAdminProtocolTranslatorPB.getMountTableEntries(RouterAdminProtocolTranslatorPB.java:198)
      at org.apache.hadoop.hdfs.rbfbalance.MountTableProcedure.getMountEntry(MountTableProcedure.java:140)
      at org.apache.hadoop.hdfs.rbfbalance.RouterFedBalance.getSrcPath(RouterFedBalance.java:326)
      at org.apache.hadoop.hdfs.rbfbalance.RouterFedBalance.access$000(RouterFedBalance.java:68)
      at org.apache.hadoop.hdfs.rbfbalance.RouterFedBalance$Builder.build(RouterFedBalance.java:168)
      at org.apache.hadoop.hdfs.rbfbalance.RouterFedBalance.submit(RouterFedBalance.java:302)
      at org.apache.hadoop.hdfs.rbfbalance.RouterFedBalance.run(RouterFedBalance.java:216)
      at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
      at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:90)
      at org.apache.hadoop.hdfs.rbfbalance.RouterFedBalance.main(RouterFedBalance.java:376)

      When adding the property specifically by "-D" option, the command worked.

      Attachments

        Issue Links

          Activity

            People

              fengnanli Fengnan Li
              aajisaka Akira Ajisaka
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 0.5h
                  0.5h