Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.9.0
    • Fix Version/s: 3.1.0, 2.10.0, 2.9.1, 3.0.3
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      In a typical router-based federation cluster, hdfs-site.xml is supposed to be:

      <property>
          <name>dfs.nameservices</name>
          <value>ns1,ns2,ns-fed</value>
        </property>
        <property>
          <name>dfs.ha.namenodes.ns-fed</name>
          <value>r1,r2</value>
        </property>
        <property>
          <name>dfs.namenode.rpc-address.ns1</name>
          <value>host1:8020</value>
        </property>
        <property>
          <name>dfs.namenode.rpc-address.ns2</name>
          <value>host2:8020</value>
        </property>
        <property>
          <name>dfs.namenode.rpc-address.ns-fed.r1</name>
          <value>host1:8888</value>
        </property>
        <property>
          <name>dfs.namenode.rpc-address.ns-fed.r2</name>
          <value>host2:8888</value>
        </property>
      

      dfs.ha.namenodes.ns-fed here is used for client to access the Router. However with this configuration on server node, Router fails to start with error:

      org.apache.hadoop.HadoopIllegalArgumentException: Configuration has multiple addresses that match local node's address. Please configure the system with dfs.nameservice.id and dfs.ha.namenode.id
              at org.apache.hadoop.hdfs.DFSUtil.getSuffixIDs(DFSUtil.java:1198)
              at org.apache.hadoop.hdfs.DFSUtil.getNameServiceId(DFSUtil.java:1131)
              at org.apache.hadoop.hdfs.DFSUtil.getNamenodeNameServiceId(DFSUtil.java:1086)
              at org.apache.hadoop.hdfs.server.federation.router.Router.createLocalNamenodeHearbeatService(Router.java:466)
              at org.apache.hadoop.hdfs.server.federation.router.Router.createNamenodeHearbeatServices(Router.java:423)
              at org.apache.hadoop.hdfs.server.federation.router.Router.serviceInit(Router.java:199)
              at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
              at org.apache.hadoop.hdfs.server.federation.router.DFSRouter.main(DFSRouter.java:69)
      2018-03-01 18:05:56,208 ERROR org.apache.hadoop.hdfs.server.federation.router.DFSRouter: Failed to start router
      

      Then the router tries to find the local namenode, multiple properties: dfs.namenode.rpc-address.ns1, dfs.namenode.rpc-address.ns-fed.r1 match the local address.

        Attachments

        1. HDFS-13214.004.patch
          9 kB
          Yiqun Lin
        2. HDFS-13214.003.patch
          9 kB
          Yiqun Lin
        3. HDFS-13214.002.patch
          7 kB
          Yiqun Lin
        4. HDFS-13214.001.patch
          1 kB
          Yiqun Lin

          Activity

            People

            • Assignee:
              linyiqun Yiqun Lin
              Reporter:
              Tao Jie Tao Jie
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: