Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-1098

Incorrect synchronization in DistributedCache causes TaskTrackers to freeze up during localization of Cache for tasks.

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.21.0
    • Component/s: tasktracker
    • Labels:
      None
    • Release Note:
      Fixed the distributed cache's localizeCache to lock only the uri it is localizing.

      Description

      Currently org.apache.hadoop.filecache.DistributedCache.getLocalCache(URI, Configuration, Path, FileStatus, boolean, long, Path, boolean) allows only one TaskRunner thread in TT to localize DistributedCache across jobs. Current way of synchronization is across baseDir this has to be changed to lock on the same baseDir.

        Attachments

        1. patch-1098-ydist.txt
          4 kB
          Amareshwari Sriramadasu
        2. patch-1098-ydist.txt
          25 kB
          Amareshwari Sriramadasu
        3. patch-1098-ydist.txt
          26 kB
          Arun C Murthy
        4. patch-1098-7-y20.txt
          32 kB
          Hemanth Yamijala
        5. patch-1098-7.txt
          31 kB
          Amareshwari Sriramadasu
        6. patch-1098-7.txt
          32 kB
          Arun C Murthy
        7. patch-1098-6.txt
          31 kB
          Amareshwari Sriramadasu
        8. patch-1098-5.txt
          31 kB
          Amareshwari Sriramadasu
        9. patch-1098-4.txt
          31 kB
          Amareshwari Sriramadasu
        10. patch-1098-3.txt
          29 kB
          Amareshwari Sriramadasu
        11. patch-1098-2.txt
          4 kB
          Amareshwari Sriramadasu
        12. patch-1098-1.txt
          4 kB
          Amareshwari Sriramadasu
        13. patch-1098-0.20.txt
          3 kB
          Amareshwari Sriramadasu
        14. patch-1098.txt
          1.0 kB
          Amareshwari Sriramadasu
        15. MAPREDUCE-1098.patch
          5 kB
          Arun C Murthy
        16. MAPREDUCE-1098.patch
          5 kB
          Arun C Murthy
        17. MAPREDUCE-1098.patch
          16 kB
          Arun C Murthy

          Issue Links

            Activity

              People

              • Assignee:
                amareshwari Amareshwari Sriramadasu
                Reporter:
                sreekanth Sreekanth Ramakrishnan
              • Votes:
                0 Vote for this issue
                Watchers:
                10 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: