[HDFS-3446] HostsFileReader silently ignores bad includes/excludes

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.0.0, 2.0.0-alpha
• Fix Version/s: 2.0.2-alpha
• Component/s: namenode
• Hadoop Flags: Incompatible change, Reviewed

Release Note

HDFS no longer silently ignores missing or unreadable host files specified by dfs.hosts or dfs.hosts.exclude. In order to specify that no hosts should be included or excluded, administrators should either refrain from setting the relevant config properties, or create an empty file in order to represent an empty list.

• Assignee: Matthew Jacobs
• Reporter: Matthew Jacobs
• Created: 19/May/12 00:06
• Updated: 11/Oct/12 17:46
• Resolved: 29/Jun/12 22:24

Description

The HostsFileReader silently fails if the includes or excludes files do
not exist or are not readable. The current behavior is to overwrite the
existing set of hosts or excluded hosts, regardless of whether or not
the includes/excludes file exists.

This behavior was introduced in HADOOP-5643 to support updating the job
tracker's node lists. The HostsFileReader is no longer used by the job
tracker. If this behavior was intentional, it no longer seems necessary
for any reason. The HostsFileReader is still used by NodeListManager as
well as the DatanodeManager, and in both cases, throwing an exception
when the include/exclude files aren't found/readable is desirable.

We should validate the given includes and excludes files before using
them.

Issue Links

• relates to: HADOOP-8854 Document backward incompatible changes between hadoop-1.x and 2.x