Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-3835

hadoop-yarn-server-resourcemanager test package bundles core-site.xml, yarn-site.xml

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 2.6.0
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: resourcemanager
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      It looks like by default yarn is bundling core-site.xml, yarn-site.xml in test artifact of hadoop-yarn-server-resourcemanager which means that any downstream project which uses this a dependency can have a problem in picking up the user supplied/environment supplied core-site.xml, yarn-site.xml

      So we should ideally exclude these .xml files from being bundled into the test-jar. (Similar to YARN-1748)

      I also proactively looked at other YARN modules where this might be happening.

      vamsee-MBP:hadoop-yarn-project vamsee$ find . -name "*-site.xml"
      ./hadoop-yarn/conf/yarn-site.xml
      ./hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/test/resources/yarn-site.xml
      ./hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-unmanaged-am-launcher/src/test/resources/yarn-site.xml
      ./hadoop-yarn/hadoop-yarn-client/src/test/resources/core-site.xml
      ./hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/core-site.xml
      ./hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/resources/core-site.xml
      ./hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/resources/yarn-site.xml
      ./hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/target/test-classes/core-site.xml
      ./hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/target/test-classes/yarn-site.xml
      ./hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests/src/test/resources/core-site.xml
      

      And out of these only two modules (hadoop-yarn-server-resourcemanager, hadoop-yarn-server-tests) are building test-jars. In future, if we start building test-jar of other modules, we should exclude these xml files from being bundled.

        Attachments

        1. YARN-3835.patch
          0.9 kB
          Vamsee Yarlagadda

          Issue Links

            Activity

              People

              • Assignee:
                vamsee Vamsee Yarlagadda
                Reporter:
                vamsee Vamsee Yarlagadda
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: