Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-1741

XInclude support broken for YARN ResourceManager

    Details

    • Type: Bug
    • Status: Open
    • Priority: Critical
    • Resolution: Unresolved
    • Affects Version/s: 2.4.0
    • Fix Version/s: None
    • Component/s: resourcemanager
    • Labels:

      Description

      The XInclude support in Hadoop configuration files (introduced via HADOOP-4944) was broken by the recent ConfigurationProvider changes to YARN ResourceManager. Specifically, YARN-1459 and, more generally, the YARN-1611 family of JIRAs for ResourceManager HA.

      The issue is that ConfigurationProvider provides a raw InputStream as a Configuration resource for what was previously a Path-based resource.

      For Path resources, the absolute file path is used as the systemId for the DocumentBuilder.parse() call:

            } else if (resource instanceof Path) {          // a file resource
      ...
                doc = parse(builder, new BufferedInputStream(
                    new FileInputStream(file)), ((Path)resource).toString());
              }
      

      The systemId is used to resolve XIncludes (among other things):

          /**
           * Parse the content of the given <code>InputStream</code> as an
           * XML document and return a new DOM Document object.
      ...
           * @param systemId Provide a base for resolving relative URIs.
      ...
           */
          public Document parse(InputStream is, String systemId)
      

      However, for loading raw InputStream resources, the systemId is set to null:

            } else if (resource instanceof InputStream) {
              doc = parse(builder, (InputStream) resource, null);
      

      causing XInclude resolution to fail.

      In our particular environment, we make extensive use of XIncludes to standardize common configuration parameters across multiple Hadoop clusters.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                xgong Xuan Gong
                Reporter:
                sirianni Eric Sirianni
              • Votes:
                0 Vote for this issue
                Watchers:
                9 Start watching this issue

                Dates

                • Created:
                  Updated: