Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-1741

XInclude support broken for YARN ResourceManager



    • Type: Bug
    • Status: Resolved
    • Priority: Critical
    • Resolution: Won't Fix
    • Affects Version/s: 2.4.0
    • Fix Version/s: None
    • Component/s: resourcemanager
    • Labels:


      The XInclude support in Hadoop configuration files (introduced via HADOOP-4944) was broken by the recent ConfigurationProvider changes to YARN ResourceManager. Specifically, YARN-1459 and, more generally, the YARN-1611 family of JIRAs for ResourceManager HA.

      The issue is that ConfigurationProvider provides a raw InputStream as a Configuration resource for what was previously a Path-based resource.

      For Path resources, the absolute file path is used as the systemId for the DocumentBuilder.parse() call:

            } else if (resource instanceof Path) {          // a file resource
                doc = parse(builder, new BufferedInputStream(
                    new FileInputStream(file)), ((Path)resource).toString());

      The systemId is used to resolve XIncludes (among other things):

           * Parse the content of the given <code>InputStream</code> as an
           * XML document and return a new DOM Document object.
           * @param systemId Provide a base for resolving relative URIs.
          public Document parse(InputStream is, String systemId)

      However, for loading raw InputStream resources, the systemId is set to null:

            } else if (resource instanceof InputStream) {
              doc = parse(builder, (InputStream) resource, null);

      causing XInclude resolution to fail.

      In our particular environment, we make extensive use of XIncludes to standardize common configuration parameters across multiple Hadoop clusters.


          Issue Links



              • Assignee:
                xgong Xuan Gong
                sirianni Eric Sirianni
              • Votes:
                0 Vote for this issue
                11 Start watching this issue


                • Created: