Details

    • Type: Sub-task Sub-task
    • Status: Closed
    • Priority: Blocker Blocker
    • Resolution: Fixed
    • Affects Version/s: 0.23.1, 2.0.0-alpha
    • Fix Version/s: 0.23.1
    • Component/s: mrv2, nodemanager
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Fixed a deadlock in NodeManager LocalDirectories's handling service.

      Description

      MAPREDUCE-3121 cloned Configuration object in LocalDirsHandlerService.init() to avoid others to access that configuration object. But since it is used in local FileSystem object creation in LocalDirAllocator.AllocatorPerContext and the same FileSystem object is used in ShuffleHandler.Shuffle.localDirAllocator, this is causing a deadlock when accessing this configuration object from LocalDirsHandlerService and ShuffleHandler along with AllocatorPerContext object.

      1. 3519.patch
        4 kB
        Ravi Gummadi
      2. deadlock.txt
        7 kB
        Ravi Gummadi
      3. 3519.v1.patch
        3 kB
        Ravi Gummadi

        Activity

        Allen Wittenauer made changes -
        Affects Version/s 2.0.0-alpha [ 12320354 ]
        Affects Version/s 0.24.0 [ 12317654 ]
        Arun C Murthy made changes -
        Status Resolved [ 5 ] Closed [ 6 ]
        Vinod Kumar Vavilapalli made changes -
        Link This issue is related to MAPREDUCE-3121 [ MAPREDUCE-3121 ]
        Vinod Kumar Vavilapalli made changes -
        Parent MAPREDUCE-3121 [ 12525190 ]
        Issue Type Bug [ 1 ] Sub-task [ 7 ]
        Vinod Kumar Vavilapalli made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Hadoop Flags Reviewed [ 10343 ]
        Release Note Fixed a deadlock in NodeManager LocalDirectories's handling service.
        Target Version/s 0.24.0, 0.23.1 [ 12317654, 12318883 ] 0.23.1, 0.24.0 [ 12318883, 12317654 ]
        Resolution Fixed [ 1 ]
        Robert Joseph Evans made changes -
        Target Version/s 0.23.1, 0.24.0 [ 12318883, 12317654 ]
        Priority Major [ 3 ] Blocker [ 1 ]
        Ravi Gummadi made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Ravi Gummadi made changes -
        Attachment 3519.v1.patch [ 12506731 ]
        Ravi Gummadi made changes -
        Description MAPREDUCE-3121 cloned Configuration object in LocalDirsHandlerService.init() to avoid others to access that configuration object. But since it is set in the base class of LocalDirsHandlerService using super.init(conf), it is reflected and is accessible to some other services. This is causing a deadlock when accessing this configuration object from LocalDirsHandlerService and ShuffleHandler along with AllocatorPerContext object. MAPREDUCE-3121 cloned Configuration object in LocalDirsHandlerService.init() to avoid others to access that configuration object. But since it is used in local FileSystem object creation in LocalDirAllocator.AllocatorPerContext and the same FileSystem object is used in ShuffleHandler.Shuffle.localDirAllocator, this is causing a deadlock when accessing this configuration object from LocalDirsHandlerService and ShuffleHandler along with AllocatorPerContext object.
        Mahadev konar made changes -
        Component/s mrv2 [ 12314301 ]
        Ravi Gummadi made changes -
        Attachment deadlock.txt [ 12506573 ]
        Ravi Gummadi made changes -
        Attachment 3519.patch [ 12506572 ]
        Ravi Gummadi made changes -
        Field Original Value New Value
        Link This issue is related to MAPREDUCE-3121 [ MAPREDUCE-3121 ]
        Ravi Gummadi created issue -

          People

          • Assignee:
            Ravi Gummadi
            Reporter:
            Ravi Gummadi
          • Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development