Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-5813

YarnChild does not load job.xml with mapreduce.job.classloader=true

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Blocker
    • Resolution: Fixed
    • 2.3.0
    • 2.4.0
    • mrv2, task
    • None

    Description

      YarnChild.main uses JobConf.addResource(String) to load job.xml that relies on class loading. When mapreduce.job.classloader=true the job-speicific part of the class path is separated from CLASSPATH into APP_CLASSPATH. Therefore job.xml is inaccessible for the default class loader. Later writeLocalJobFile overwrites the correct localized job.xml on disk as well.

      This problem is easily avoided by using JobConf.addResource(Path) to read the localized job.xml without relying on class loading.

      Attachments

        1. MAPREDUCE-5813.v01.patch
          1.0 kB
          Gera Shegalov
        2. MAPREDUCE-5813.v02.patch
          5 kB
          Gera Shegalov

        Issue Links

          Activity

            People

              jira.shegalov Gera Shegalov
              jira.shegalov Gera Shegalov
              Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: