Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-1098

Incorrect synchronization in DistributedCache causes TaskTrackers to freeze up during localization of Cache for tasks.

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.21.0
    • Component/s: tasktracker
    • Labels:
      None
    • Release Note:
      Fixed the distributed cache's localizeCache to lock only the uri it is localizing.

      Description

      Currently org.apache.hadoop.filecache.DistributedCache.getLocalCache(URI, Configuration, Path, FileStatus, boolean, long, Path, boolean) allows only one TaskRunner thread in TT to localize DistributedCache across jobs. Current way of synchronization is across baseDir this has to be changed to lock on the same baseDir.

        Attachments

        1. patch-1098-7-y20.txt
          32 kB
          Hemanth Yamijala
        2. patch-1098-ydist.txt
          26 kB
          Arun C Murthy
        3. patch-1098-7.txt
          32 kB
          Arun C Murthy
        4. patch-1098-7.txt
          31 kB
          Amareshwari Sriramadasu
        5. patch-1098-ydist.txt
          25 kB
          Amareshwari Sriramadasu
        6. patch-1098-6.txt
          31 kB
          Amareshwari Sriramadasu
        7. patch-1098-5.txt
          31 kB
          Amareshwari Sriramadasu
        8. patch-1098-4.txt
          31 kB
          Amareshwari Sriramadasu
        9. patch-1098-3.txt
          29 kB
          Amareshwari Sriramadasu
        10. MAPREDUCE-1098.patch
          16 kB
          Arun C Murthy
        11. MAPREDUCE-1098.patch
          5 kB
          Arun C Murthy
        12. MAPREDUCE-1098.patch
          5 kB
          Arun C Murthy
        13. patch-1098-ydist.txt
          4 kB
          Amareshwari Sriramadasu
        14. patch-1098-2.txt
          4 kB
          Amareshwari Sriramadasu
        15. patch-1098-0.20.txt
          3 kB
          Amareshwari Sriramadasu
        16. patch-1098-1.txt
          4 kB
          Amareshwari Sriramadasu
        17. patch-1098.txt
          1.0 kB
          Amareshwari Sriramadasu

          Issue Links

            Activity

              People

              • Assignee:
                amareshwari Amareshwari Sriramadasu
                Reporter:
                sreekanth Sreekanth Ramakrishnan
              • Votes:
                0 Vote for this issue
                Watchers:
                10 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: