Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-4870

periodically re-resolve hostnames in included and excluded datanodes list

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Patch Available
    • Minor
    • Resolution: Unresolved
    • 2.0.5-alpha
    • None
    • None

    Description

      We currently only resolve the hostnames in the included and excluded datanodes list once-- when the list is read. The rationale for this is that in big clusters, DNS resolution for thousands of nodes can take a long time (when generating a datanode list in getDatanodeListForReport, for example). However, if the DNS information changes for one of these hosts, we should reflect that. A background thread could do these DNS resolutions every few minutes without blocking any foreground operations.

      Attachments

        1. HDFS-4870.001.patch
          13 kB
          Colin McCabe

        Issue Links

          relates to

          Bug - A problem which impairs or prevents the functions of the product. HDFS-3934 duplicative dfs_hosts entries handled wrong

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Closed

          Activity

            People

              cmccabe Colin McCabe
              cmccabe Colin McCabe
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated: