Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-3426

Datanode does not start up if the local machines DNS isnt working right and dfs.datanode.dns.interface==default

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Minor
    • Resolution: Fixed
    • 0.21.0
    • 0.21.0
    • None
    • None
    • Ubuntu 8.04, at home, no reverse DNS

    • Reviewed

    Description

      This is the third Java project I've been involved in that doesnt work on my home network, due to implementation issues with java.net.InetAddress.getLocalHost(), issues that only show up on an unamanged network. Fortunately my home network exists to find these problems early.

      In hadoop, if the local hostname doesnt resolve, the datanode does not start up:

      Caused by: java.net.UnknownHostException: k2: k2
      at java.net.InetAddress.getLocalHost(InetAddress.java:1353)
      at org.apache.hadoop.net.DNS.getDefaultHost(DNS.java:185)
      at org.apache.hadoop.dfs.DataNode.startDataNode(DataNode.java:184)
      at org.apache.hadoop.dfs.DataNode.(DataNode.java:162)
      at org.apache.hadoop.dfs.ExtDataNode.(ExtDataNode.java:55)
      at org.smartfrog.services.hadoop.components.datanode.DatanodeImpl.sfStart(DatanodeImpl.java:60)

      While this is a valid option in a production (non-virtual) cluster, if you are playing with VMWare/Xen private networks or on a home network, you can't rely on DNS.

      1. In these situations, its usually better to fall back to using "localhost" or 127.0.0.1 as a hostname if Java can't work it out for itself,
      2. Its often good to cache this if used in lots of parts of the system, otherwise the 30s timeouts can cause problems of their own.

      Attachments

        1. hadoop-3426.patch
          12 kB
          Steve Loughran
        2. hadoop-3426.patch
          19 kB
          Steve Loughran
        3. hadoop-3426.patch
          19 kB
          Steve Loughran
        4. hadoop-3426.patch
          14 kB
          Steve Loughran
        5. hadoop-3426.patch
          14 kB
          Steve Loughran

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            stevel@apache.org Steve Loughran
            stevel@apache.org Steve Loughran
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Issue deployment