Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-1641

Job submission should fail if same uri is added for mapred.cache.files and mapred.cache.archives


    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: distributed-cache
    • Labels:


      The behavior of mapred.cache.files and mapred.cache.archives is different during localization in the following way:

      If a jar file is added to mapred.cache.files, it will be localized under TaskTracker under a unique path.
      If a jar file is added to mapred.cache.archives, it will be localized under a unique path in a directory named the jar file name, and will be unarchived under the same directory.

      If same jar file is passed for both the configurations, the behavior undefined. Thus the job submission should fail.
      Currently, since distributed cache processes files before archives, the jar file will be just localized and not unarchived.

      1. mapreduce-1641--2010-05-21.patch
        21 kB
        Dick King
      2. mapreduce-1641--2010-05-19.patch
        21 kB
        Al Thompson
      3. mapreduce-1641--2010-04-27.patch
        21 kB
        Dick King
      4. patch-1641-ydist-bugfix.txt
        4 kB
        Amareshwari Sriramadasu
      5. duped-files-archives--off-0-20-101--2010-04-23--1819.patch
        26 kB
        Dick King
      6. duped-files-archives--off-0-20-101--2010-04-21.patch
        21 kB
        Dick King
      7. BZ-3539321--off-0-20-101--2010-04-20.patch
        11 kB
        Dick King

        Issue Links


          No work has yet been logged on this issue.


            • Assignee:
              Dick King
              Amareshwari Sriramadasu
            • Votes:
              0 Vote for this issue
              3 Start watching this issue


              • Created: