Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-5194

Avoid adding yarn-site to all Configuration instances created by the JVM

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None
    • Target Version/s:

      Description

      static {
          addDeprecatedKeys();
          Configuration.addDefaultResource(YARN_DEFAULT_CONFIGURATION_FILE);
          Configuration.addDefaultResource(YARN_SITE_CONFIGURATION_FILE);
        }
      

      This puts the contents of yarn-default and yarn-site into every configuration instance created in the VM after YarnConfiguration has been initialized.

      This should be changed to a local addResource for the specific YarnConfiguration instance, instead of polluting every Configuration instance.

      Incompatible change. Have set the target version to 3.x.

      The same applies to HdfsConfiguration (hdfs-site.xml), and Configuration (core-site.xml etc).
      core-site may be worth including everywhere, however it would be better to expect users to explicitly add the relevant resources.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              sseth Siddharth Seth
            • Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated: