Hadoop Common
  1. Hadoop Common
  2. HADOOP-2721

Use job control for tasks (and therefore for pipes and streaming)

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.21.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      We should use the setsid command when the task is launched to create a new session. We should be able to use the setsid program when we launch the bash process to create a new session. That will allow us to kill the entire session with a single signal and remove the need for the ping methods in both TaskTracker.Child and pipes.

      The patch uses setsid when creating new tasks sothat subprocesses of this process will be with in this new session(and this process will be the process leader for all the subprocesses). Thus killing the subprocesses becomes easy(just by killing all the processes in this process group) when killing the task.

      1. HADOOP-2721-v9.patch
        51 kB
        Ravi Gummadi
      2. HADOOP-2721-v3.patch
        34 kB
        Ravi Gummadi
      3. HADOOP-2721-v2.patch
        24 kB
        Ravi Gummadi
      4. HADOOP-2721-v11.patch
        56 kB
        Ravi Gummadi
      5. HADOOP-2721-v11.4.patch
        57 kB
        Ravi Gummadi
      6. HADOOP-2721-v11.3.patch
        58 kB
        Ravi Gummadi
      7. HADOOP-2721-v11.1.patch
        58 kB
        Ravi Gummadi
      8. HADOOP-2721-v10.patch
        53 kB
        Ravi Gummadi
      9. HADOOP-2721.patch
        5 kB
        Ravi Gummadi
      10. HADOOP-2721.patch
        6 kB
        Ravi Gummadi
      11. HADOOP-2721.patch
        6 kB
        Ravi Gummadi

        Issue Links

          Activity

          Owen O'Malley created issue -
          Owen O'Malley made changes -
          Field Original Value New Value
          Link This issue is duplicated by HADOOP-2719 [ HADOOP-2719 ]
          Robert Chansler made changes -
          Fix Version/s 0.17.0 [ 12312913 ]
          Amareshwari Sriramadasu made changes -
          Link This issue relates to HADOOP-1380 [ HADOOP-1380 ]
          Ravi Gummadi made changes -
          Attachment HADOOP-2721.patch [ 12395935 ]
          Hemanth Yamijala made changes -
          Link This issue blocks HADOOP-4490 [ HADOOP-4490 ]
          Ravi Gummadi made changes -
          Attachment HADOOP-2721.patch [ 12396163 ]
          Ravi Gummadi made changes -
          Attachment HADOOP-2721.patch [ 12396174 ]
          Ravi Gummadi made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Assignee Devaraj Das [ devaraj ] Ravi Gummadi [ ravidotg ]
          Release Note The patch uses setsid when creating new tasks sothat subprocesses of this process will be with in this new session(and this process will be the process leader for all the subprocesses). Thus killing the subprocesses becomes easy(just by killing all the processes in this process group) when killing the task.
          Ravi Gummadi made changes -
          Attachment HADOOP-2721-v2.patch [ 12396393 ]
          Ravi Gummadi made changes -
          Attachment HADOOP-2721-v3.patch [ 12396660 ]
          Ravi Gummadi made changes -
          Attachment HADOOP-2721-v9.patch [ 12397425 ]
          Ravi Gummadi made changes -
          Attachment HADOOP-2721-v10.patch [ 12397752 ]
          Ravi Gummadi made changes -
          Attachment HADOOP-2721-v11.patch [ 12398304 ]
          Ravi Gummadi made changes -
          Attachment HADOOP-2721-v11.1.patch [ 12398398 ]
          Ravi Gummadi made changes -
          Attachment HADOOP-2721-v11.3.patch [ 12398462 ]
          Ravi Gummadi made changes -
          Attachment HADOOP-2721-v11.4.patch [ 12399112 ]
          Devaraj Das made changes -
          Fix Version/s 0.21.0 [ 12313563 ]
          Resolution Fixed [ 1 ]
          Hadoop Flags [Reviewed]
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Owen O'Malley made changes -
          Component/s pipes [ 12311773 ]
          Owen O'Malley made changes -
          Component/s mapred [ 12310690 ]
          Owen O'Malley made changes -
          Component/s contrib/streaming [ 12310972 ]
          Robert Chansler made changes -
          Release Note The patch uses setsid when creating new tasks sothat subprocesses of this process will be with in this new session(and this process will be the process leader for all the subprocesses). Thus killing the subprocesses becomes easy(just by killing all the processes in this process group) when killing the task.
          Description We should use the setsid command when the task is launched to create a new session. We should be able to use the setsid program when we launch the bash process to create a new session. That will allow us to kill the entire session with a single signal and remove the need for the ping methods in both TaskTracker.Child and pipes. We should use the setsid command when the task is launched to create a new session. We should be able to use the setsid program when we launch the bash process to create a new session. That will allow us to kill the entire session with a single signal and remove the need for the ping methods in both TaskTracker.Child and pipes.

          The patch uses setsid when creating new tasks sothat subprocesses of this process will be with in this new session(and this process will be the process leader for all the subprocesses). Thus killing the subprocesses becomes easy(just by killing all the processes in this process group) when killing the task.
          Tom White made changes -
          Status Resolved [ 5 ] Closed [ 6 ]

            People

            • Assignee:
              Ravi Gummadi
              Reporter:
              Owen O'Malley
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development