[YARN-1741] XInclude support broken for YARN ResourceManager

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Won't Fix
• Affects Version/s: 2.4.0
• Component/s: resourcemanager
• Labels: regression
• Assignee: Xuan Gong
• Reporter: Eric Sirianni
• Created: 20/Feb/14 21:30
• Updated: 21/Jul/20 20:10
• Resolved: 21/Jul/20 20:10

Description

The XInclude support in Hadoop configuration files (introduced via HADOOP-4944) was broken by the recent ConfigurationProvider changes to YARN ResourceManager. Specifically, YARN-1459 and, more generally, the YARN-1611 family of JIRAs for ResourceManager HA.

The issue is that ConfigurationProvider provides a raw InputStream as a Configuration resource for what was previously a Path-based resource.

For Path resources, the absolute file path is used as the systemId for the DocumentBuilder.parse() call:

      } else if (resource instanceof Path) {          // a file resource
...
          doc = parse(builder, new BufferedInputStream(
              new FileInputStream(file)), ((Path)resource).toString());
        }

The systemId is used to resolve XIncludes (among other things):

    /**
     * Parse the content of the given <code>InputStream</code> as an
     * XML document and return a new DOM Document object.
...
     * @param systemId Provide a base for resolving relative URIs.
...
     */
    public Document parse(InputStream is, String systemId)

However, for loading raw InputStream resources, the systemId is set to null:

      } else if (resource instanceof InputStream) {
        doc = parse(builder, (InputStream) resource, null);

causing XInclude resolution to fail.

In our particular environment, we make extensive use of XIncludes to standardize common configuration parameters across multiple Hadoop clusters.

Issue Links

• is duplicated by: YARN-2210 resource manager fails to start if core-site.xml contains an xi:include