Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-1098

Incorrect synchronization in DistributedCache causes TaskTrackers to freeze up during localization of Cache for tasks.

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.21.0
    • Component/s: tasktracker
    • Labels:
      None
    • Release Note:
      Fixed the distributed cache's localizeCache to lock only the uri it is localizing.

      Description

      Currently org.apache.hadoop.filecache.DistributedCache.getLocalCache(URI, Configuration, Path, FileStatus, boolean, long, Path, boolean) allows only one TaskRunner thread in TT to localize DistributedCache across jobs. Current way of synchronization is across baseDir this has to be changed to lock on the same baseDir.

      1. patch-1098-ydist.txt
        4 kB
        Amareshwari Sriramadasu
      2. patch-1098-ydist.txt
        25 kB
        Amareshwari Sriramadasu
      3. patch-1098-ydist.txt
        26 kB
        Arun C Murthy
      4. patch-1098-7-y20.txt
        32 kB
        Hemanth Yamijala
      5. patch-1098-7.txt
        31 kB
        Amareshwari Sriramadasu
      6. patch-1098-7.txt
        32 kB
        Arun C Murthy
      7. patch-1098-6.txt
        31 kB
        Amareshwari Sriramadasu
      8. patch-1098-5.txt
        31 kB
        Amareshwari Sriramadasu
      9. patch-1098-4.txt
        31 kB
        Amareshwari Sriramadasu
      10. patch-1098-3.txt
        29 kB
        Amareshwari Sriramadasu
      11. patch-1098-2.txt
        4 kB
        Amareshwari Sriramadasu
      12. patch-1098-1.txt
        4 kB
        Amareshwari Sriramadasu
      13. patch-1098-0.20.txt
        3 kB
        Amareshwari Sriramadasu
      14. patch-1098.txt
        1.0 kB
        Amareshwari Sriramadasu
      15. MAPREDUCE-1098.patch
        5 kB
        Arun C Murthy
      16. MAPREDUCE-1098.patch
        5 kB
        Arun C Murthy
      17. MAPREDUCE-1098.patch
        16 kB
        Arun C Murthy

        Issue Links

          Activity

            People

            • Assignee:
              Amareshwari Sriramadasu
              Reporter:
              Sreekanth Ramakrishnan
            • Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development