Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-1098

Incorrect synchronization in DistributedCache causes TaskTrackers to freeze up during localization of Cache for tasks.

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 0.21.0
    • tasktracker
    • None
    • Fixed the distributed cache's localizeCache to lock only the uri it is localizing.

    Description

      Currently org.apache.hadoop.filecache.DistributedCache.getLocalCache(URI, Configuration, Path, FileStatus, boolean, long, Path, boolean) allows only one TaskRunner thread in TT to localize DistributedCache across jobs. Current way of synchronization is across baseDir this has to be changed to lock on the same baseDir.

      Attachments

        1. patch-1098-ydist.txt
          4 kB
          Amareshwari Sriramadasu
        2. patch-1098-ydist.txt
          25 kB
          Amareshwari Sriramadasu
        3. patch-1098-ydist.txt
          26 kB
          Arun Murthy
        4. patch-1098-7-y20.txt
          32 kB
          Hemanth Yamijala
        5. patch-1098-7.txt
          31 kB
          Amareshwari Sriramadasu
        6. patch-1098-7.txt
          32 kB
          Arun Murthy
        7. patch-1098-6.txt
          31 kB
          Amareshwari Sriramadasu
        8. patch-1098-5.txt
          31 kB
          Amareshwari Sriramadasu
        9. patch-1098-4.txt
          31 kB
          Amareshwari Sriramadasu
        10. patch-1098-3.txt
          29 kB
          Amareshwari Sriramadasu
        11. patch-1098-2.txt
          4 kB
          Amareshwari Sriramadasu
        12. patch-1098-1.txt
          4 kB
          Amareshwari Sriramadasu
        13. patch-1098-0.20.txt
          3 kB
          Amareshwari Sriramadasu
        14. patch-1098.txt
          1.0 kB
          Amareshwari Sriramadasu
        15. MAPREDUCE-1098.patch
          5 kB
          Arun Murthy
        16. MAPREDUCE-1098.patch
          5 kB
          Arun Murthy
        17. MAPREDUCE-1098.patch
          16 kB
          Arun Murthy

        Issue Links

          relates to

          Bug - A problem which impairs or prevents the functions of the product. MAPREDUCE-1186 While localizing a DistributedCache file, TT sets permissions recursively on the whole base-dir

          • Major - Major loss of function.
          • Closed

          Activity

            People

              amareshwari Amareshwari Sriramadasu
              sreekanth Sreekanth Ramakrishnan
              Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: