Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-2595

misleading message when fs.default.name not set and running sbin/start-dfs.sh

    Details

    • Type: Improvement Improvement
    • Status: Open
    • Priority: Minor Minor
    • Resolution: Unresolved
    • Affects Version/s: 0.23.0
    • Fix Version/s: None
    • Component/s: namenode, scripts
    • Labels:
      None

      Description

      sbin/start-dfs.sh calls bin/hdfs getconf -namenodes which reports the following error if the default value of fs.default.name from core-default.xml is used:

      Incorrect configuration: namenode address dfs.namenode.servicerpc-address or dfs.namenode.rpc-address is not configured.

      In previous versions of hadoop, I would see a slightly more helpful message:

      org.apache.hadoop.hdfs.server.namenode.NameNode: java.lang.IllegalArgumentException: Does not contain a valid host:port authority: file:///

        Activity

        Hide
        Joe Crobak added a comment -

        One possible solution is to change the text of the exception to include a message about fs.default.name in DfsUtils#getNNServiceRpcAddresses(). The catch clause for the IAE when computing default address could log a warning about fs.default.name not being set correctly (similar to previous versions of hadoop).

        Show
        Joe Crobak added a comment - One possible solution is to change the text of the exception to include a message about fs.default.name in DfsUtils#getNNServiceRpcAddresses(). The catch clause for the IAE when computing default address could log a warning about fs.default.name not being set correctly (similar to previous versions of hadoop).
        Hide
        Baranenko Nikolay added a comment -

        Today I tryed to install hadoop-2.7.3.
        When I tried to start start-all.sh got error

        hduser@hadoop02:~/hadoop/sbin$ start-all.sh
        This script is Deprecated. Instead use start-dfs.sh and start-yarn.sh
        16/12/18 23:36:51 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
        Incorrect configuration: namenode address dfs.namenode.servicerpc-address or dfs.namenode.rpc-address is not configured.
        Starting namenodes on []
        localhost: starting namenode, logging to /home/hduser/hadoop/logs/hadoop-hduser-namenode-hadoop02.out
        localhost: starting datanode, logging to /home/hduser/hadoop/logs/hadoop-hduser-datanode-hadoop02.out
        Starting secondary namenodes [0.0.0.0]
        0.0.0.0: starting secondarynamenode, logging to /home/hduser/hadoop/logs/hadoop-hduser-secondarynamenode-hadoop02.out
        16/12/18 23:37:10 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
        starting yarn daemons
        resourcemanager running as process 2198. Stop it first.
        localhost: nodemanager running as process 2292. Stop it first.

        hdfs-site.xml

        <configuration>

        <property>
        <name>dfs.replication</name>
        <value>1</value>
        </property>
        <property>
        <name>dfs.name.dir</name>
        <value>file:///home/hduser/hadoopdata/hdfs/namenode </value>
        </property>
        <property>
        <name>dfs.data.dir</name>
        <value>file:///home/hduser/hadoopdata/hdfs/datanode </value >
        </property>

        </configuration>

        Show
        Baranenko Nikolay added a comment - Today I tryed to install hadoop-2.7.3. When I tried to start start-all.sh got error hduser@hadoop02:~/hadoop/sbin$ start-all.sh This script is Deprecated. Instead use start-dfs.sh and start-yarn.sh 16/12/18 23:36:51 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable Incorrect configuration: namenode address dfs.namenode.servicerpc-address or dfs.namenode.rpc-address is not configured. Starting namenodes on [] localhost: starting namenode, logging to /home/hduser/hadoop/logs/hadoop-hduser-namenode-hadoop02.out localhost: starting datanode, logging to /home/hduser/hadoop/logs/hadoop-hduser-datanode-hadoop02.out Starting secondary namenodes [0.0.0.0] 0.0.0.0: starting secondarynamenode, logging to /home/hduser/hadoop/logs/hadoop-hduser-secondarynamenode-hadoop02.out 16/12/18 23:37:10 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable starting yarn daemons resourcemanager running as process 2198. Stop it first. localhost: nodemanager running as process 2292. Stop it first. hdfs-site.xml <configuration> <property> <name>dfs.replication</name> <value>1</value> </property> <property> <name>dfs.name.dir</name> <value> file:///home/hduser/hadoopdata/hdfs/namenode </value> </property> <property> <name>dfs.data.dir</name> <value> file:///home/hduser/hadoopdata/hdfs/datanode </value > </property> </configuration>

          People

          • Assignee:
            Unassigned
            Reporter:
            Joe Crobak
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:

              Development