Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-1641

Job submission should fail if same uri is added for mapred.cache.files and mapred.cache.archives

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: distributed-cache
    • Labels:
      None

      Description

      The behavior of mapred.cache.files and mapred.cache.archives is different during localization in the following way:

      If a jar file is added to mapred.cache.files, it will be localized under TaskTracker under a unique path.
      If a jar file is added to mapred.cache.archives, it will be localized under a unique path in a directory named the jar file name, and will be unarchived under the same directory.

      If same jar file is passed for both the configurations, the behavior undefined. Thus the job submission should fail.
      Currently, since distributed cache processes files before archives, the jar file will be just localized and not unarchived.

        Attachments

        1. BZ-3539321--off-0-20-101--2010-04-20.patch
          11 kB
          Dick King
        2. duped-files-archives--off-0-20-101--2010-04-21.patch
          21 kB
          Dick King
        3. duped-files-archives--off-0-20-101--2010-04-23--1819.patch
          26 kB
          Dick King
        4. patch-1641-ydist-bugfix.txt
          4 kB
          Amareshwari Sriramadasu
        5. mapreduce-1641--2010-04-27.patch
          21 kB
          Dick King
        6. mapreduce-1641--2010-05-19.patch
          21 kB
          Al Thompson
        7. mapreduce-1641--2010-05-21.patch
          21 kB
          Dick King

        Issue Links

          Activity

            People

            • Assignee:
              dking Dick King
              Reporter:
              amareshwari Amareshwari Sriramadasu

              Dates

              • Created:
                Updated:
                Resolved:

                Issue deployment