Hadoop Common
  1. Hadoop Common
  2. HADOOP-8325

Add a ShutdownHookManager to be used by different components instead of the JVM shutdownhook

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Critical Critical
    • Resolution: Fixed
    • Affects Version/s: 2.0.0-alpha
    • Fix Version/s: 0.23.3, 2.0.2-alpha
    • Component/s: fs
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      FileSystem adds a JVM shutdown hook when a filesystem instance is cached.

      MRAppMaster also uses a JVM shutdown hook, among other things, the MRAppMaster JVM shutdown hook is used to ensure state are written to HDFS.

      This creates a race condition because each JVM shutdown hook is a separate thread and if there are multiple JVM shutdown hooks there is not assurance of order of execution, they could even run in parallel.

      1. HADOOP-8325.patch
        10 kB
        Alejandro Abdelnur
      2. HADOOP-8325.patch
        15 kB
        Alejandro Abdelnur
      3. HADOOP-8325.patch
        16 kB
        Alejandro Abdelnur
      4. HADOOP-8325.patch
        16 kB
        Alejandro Abdelnur
      5. HADOOP-8325.patch
        16 kB
        Alejandro Abdelnur
      6. HADOOP-8325.patch
        16 kB
        Alejandro Abdelnur
      7. HADOOP-8325.patch
        16 kB
        Alejandro Abdelnur
      8. HADOOP-8325.patch
        17 kB
        Alejandro Abdelnur

        Issue Links

          Activity

          Alejandro Abdelnur created issue -
          Alejandro Abdelnur made changes -
          Field Original Value New Value
          Summary remove FileSystem JVM shutdown hook that does closeAll() Add a ShutdownHookManager to be used by different components instead of the JVM shutdownhook
          Alejandro Abdelnur made changes -
          Attachment HADOOP-8325.patch [ 12524908 ]
          Alejandro Abdelnur made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Alejandro Abdelnur made changes -
          Link This issue blocks MAPREDUCE-4135 [ MAPREDUCE-4135 ]
          Alejandro Abdelnur made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Alejandro Abdelnur made changes -
          Attachment HADOOP-8325.patch [ 12524911 ]
          Alejandro Abdelnur made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Alejandro Abdelnur made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Alejandro Abdelnur made changes -
          Attachment HADOOP-8325.patch [ 12524912 ]
          Alejandro Abdelnur made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Alejandro Abdelnur made changes -
          Attachment HADOOP-8325.patch [ 12524914 ]
          Alejandro Abdelnur made changes -
          Link This issue blocks MAPREDUCE-4205 [ MAPREDUCE-4205 ]
          Alejandro Abdelnur made changes -
          Attachment HADOOP-8325.patch [ 12524937 ]
          Alejandro Abdelnur made changes -
          Attachment HADOOP-8325.patch [ 12524963 ]
          Alejandro Abdelnur made changes -
          Attachment HADOOP-8325.patch [ 12524980 ]
          Alejandro Abdelnur made changes -
          Attachment HADOOP-8325.patch [ 12525091 ]
          Alejandro Abdelnur made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Hadoop Flags Reviewed [ 10343 ]
          Resolution Fixed [ 1 ]
          Arun C Murthy made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Robert Joseph Evans made changes -
          Fix Version/s 0.23.3 [ 12320059 ]
          Fix Version/s 2.0.0-alpha [ 12320352 ]
          Arun C Murthy made changes -
          Fix Version/s 2.0.2-alpha [ 12322473 ]
          Gavin made changes -
          Link This issue blocks MAPREDUCE-4135 [ MAPREDUCE-4135 ]
          Gavin made changes -
          Link This issue is depended upon by MAPREDUCE-4135 [ MAPREDUCE-4135 ]
          Gavin made changes -
          Link This issue blocks MAPREDUCE-4205 [ MAPREDUCE-4205 ]
          Gavin made changes -
          Link This issue is depended upon by MAPREDUCE-4205 [ MAPREDUCE-4205 ]

            People

            • Assignee:
              Alejandro Abdelnur
              Reporter:
              Alejandro Abdelnur
            • Votes:
              0 Vote for this issue
              Watchers:
              14 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development