Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-3446

HostsFileReader silently ignores bad includes/excludes

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 1.0.0, 2.0.0-alpha
    • Fix Version/s: 2.0.2-alpha
    • Component/s: namenode
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Incompatible change, Reviewed
    • Release Note:
      Hide
      HDFS no longer silently ignores missing or unreadable host files specified by dfs.hosts or dfs.hosts.exclude. In order to specify that no hosts should be included or excluded, administrators should either refrain from setting the relevant config properties, or create an empty file in order to represent an empty list.
      Show
      HDFS no longer silently ignores missing or unreadable host files specified by dfs.hosts or dfs.hosts.exclude. In order to specify that no hosts should be included or excluded, administrators should either refrain from setting the relevant config properties, or create an empty file in order to represent an empty list.

      Description

      The HostsFileReader silently fails if the includes or excludes files do
      not exist or are not readable. The current behavior is to overwrite the
      existing set of hosts or excluded hosts, regardless of whether or not
      the includes/excludes file exists.

      This behavior was introduced in HADOOP-5643 to support updating the job
      tracker's node lists. The HostsFileReader is no longer used by the job
      tracker. If this behavior was intentional, it no longer seems necessary
      for any reason. The HostsFileReader is still used by NodeListManager as
      well as the DatanodeManager, and in both cases, throwing an exception
      when the include/exclude files aren't found/readable is desirable.

      We should validate the given includes and excludes files before using
      them.

      1. hdfs-3446.txt
        13 kB
        Matthew Jacobs
      2. hdfs-3446.txt
        11 kB
        Matthew Jacobs

        Issue Links

          Activity

            People

            • Assignee:
              Matthew Jacobs
              Reporter:
              Matthew Jacobs
            • Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development