Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-5814

fat jar with *-default.xml may fail when mapreduce.job.classloader=true.

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 2.3.0
    • 2.5.0
    • mrv2
    • None
    • Reviewed

    Description

      We faced a failure when a job.jar compiled against 0.20+ hadoop artifacts had to run with mapreduce.job.classloader=true because it needed a more recent guava as a dependency. The job failed because the cluster's *-default.xml files were overshadowed by the ones in the fat jar. We propose to treat these default config files like the system packages org.apache.hadoop. to avoid a counterintuitivie behavior as if we had mapreduce.job.user.classpath.first set.

      Attachments

        1. MAPREDUCE-5814.v01.patch
          1 kB
          Gera Shegalov
        2. MAPREDUCE-5814.v02.patch
          7 kB
          Gera Shegalov
        3. MAPREDUCE-5814.v03.patch
          6 kB
          Gera Shegalov

        Issue Links

          Activity

            People

              jira.shegalov Gera Shegalov
              jira.shegalov Gera Shegalov
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: