Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-3835

hadoop-yarn-server-resourcemanager test package bundles core-site.xml, yarn-site.xml

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • 2.6.0
    • 2.8.0, 3.0.0-alpha1
    • resourcemanager
    • None
    • Reviewed

    Description

      It looks like by default yarn is bundling core-site.xml, yarn-site.xml in test artifact of hadoop-yarn-server-resourcemanager which means that any downstream project which uses this a dependency can have a problem in picking up the user supplied/environment supplied core-site.xml, yarn-site.xml

      So we should ideally exclude these .xml files from being bundled into the test-jar. (Similar to YARN-1748)

      I also proactively looked at other YARN modules where this might be happening.

      vamsee-MBP:hadoop-yarn-project vamsee$ find . -name "*-site.xml"
      ./hadoop-yarn/conf/yarn-site.xml
      ./hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/test/resources/yarn-site.xml
      ./hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-unmanaged-am-launcher/src/test/resources/yarn-site.xml
      ./hadoop-yarn/hadoop-yarn-client/src/test/resources/core-site.xml
      ./hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/core-site.xml
      ./hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/resources/core-site.xml
      ./hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/resources/yarn-site.xml
      ./hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/target/test-classes/core-site.xml
      ./hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/target/test-classes/yarn-site.xml
      ./hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests/src/test/resources/core-site.xml
      

      And out of these only two modules (hadoop-yarn-server-resourcemanager, hadoop-yarn-server-tests) are building test-jars. In future, if we start building test-jar of other modules, we should exclude these xml files from being bundled.

      Attachments

        1. YARN-3835.patch
          0.9 kB
          Vamsee K. Yarlagadda

        Issue Links

          Activity

            People

              vamsee Vamsee K. Yarlagadda
              vamsee Vamsee K. Yarlagadda
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: