Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-5813

YarnChild does not load job.xml with mapreduce.job.classloader=true

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: 2.3.0
    • Fix Version/s: 2.4.0
    • Component/s: mrv2, task
    • Labels:
      None
    • Target Version/s:

      Description

      YarnChild.main uses JobConf.addResource(String) to load job.xml that relies on class loading. When mapreduce.job.classloader=true the job-speicific part of the class path is separated from CLASSPATH into APP_CLASSPATH. Therefore job.xml is inaccessible for the default class loader. Later writeLocalJobFile overwrites the correct localized job.xml on disk as well.

      This problem is easily avoided by using JobConf.addResource(Path) to read the localized job.xml without relying on class loading.

        Attachments

        1. MAPREDUCE-5813.v01.patch
          1.0 kB
          Gera Shegalov
        2. MAPREDUCE-5813.v02.patch
          5 kB
          Gera Shegalov

          Issue Links

            Activity

              People

              • Assignee:
                jira.shegalov Gera Shegalov
                Reporter:
                jira.shegalov Gera Shegalov
              • Votes:
                0 Vote for this issue
                Watchers:
                8 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: