Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-1098

Incorrect synchronization in DistributedCache causes TaskTrackers to freeze up during localization of Cache for tasks.

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.21.0
    • Component/s: tasktracker
    • Labels:
      None
    • Release Note:
      Fixed the distributed cache's localizeCache to lock only the uri it is localizing.

      Description

      Currently org.apache.hadoop.filecache.DistributedCache.getLocalCache(URI, Configuration, Path, FileStatus, boolean, long, Path, boolean) allows only one TaskRunner thread in TT to localize DistributedCache across jobs. Current way of synchronization is across baseDir this has to be changed to lock on the same baseDir.

        Attachments

        1. MAPREDUCE-1098.patch
          16 kB
          Arun C Murthy
        2. MAPREDUCE-1098.patch
          5 kB
          Arun C Murthy
        3. MAPREDUCE-1098.patch
          5 kB
          Arun C Murthy
        4. patch-1098.txt
          1.0 kB
          Amareshwari Sriramadasu
        5. patch-1098-0.20.txt
          3 kB
          Amareshwari Sriramadasu
        6. patch-1098-1.txt
          4 kB
          Amareshwari Sriramadasu
        7. patch-1098-2.txt
          4 kB
          Amareshwari Sriramadasu
        8. patch-1098-3.txt
          29 kB
          Amareshwari Sriramadasu
        9. patch-1098-4.txt
          31 kB
          Amareshwari Sriramadasu
        10. patch-1098-5.txt
          31 kB
          Amareshwari Sriramadasu
        11. patch-1098-6.txt
          31 kB
          Amareshwari Sriramadasu
        12. patch-1098-7.txt
          32 kB
          Arun C Murthy
        13. patch-1098-7.txt
          31 kB
          Amareshwari Sriramadasu
        14. patch-1098-7-y20.txt
          32 kB
          Hemanth Yamijala
        15. patch-1098-ydist.txt
          26 kB
          Arun C Murthy
        16. patch-1098-ydist.txt
          25 kB
          Amareshwari Sriramadasu
        17. patch-1098-ydist.txt
          4 kB
          Amareshwari Sriramadasu

          Issue Links

            Activity

              People

              • Assignee:
                amareshwari Amareshwari Sriramadasu
                Reporter:
                sreekanth Sreekanth Ramakrishnan
              • Votes:
                0 Vote for this issue
                Watchers:
                10 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: