Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-390

Corner case exists in detecting Java process deaths that might lead to orphan pipes processes lying around in memory

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      In HADOOP-2092, the child pipes process periodically pings the parent Java process to find out whether it is alive. The ping cycle is 5 seconds. Consider the following scenario:
      1) The Java task dies at the beginning of the ping cycle
      2) A new Java task starts and binds to the same port as the earlier Java task's port
      3) The pipes process wakes up and does a ping - it will still be successful since the port number hasn't changed
      This will lead to orphan processes lying around in memory. The detection of parent process deaths can be made more reliable at least on Unix'ish platforms by checking whether the parent process ID is 1, and if so exit. This will take care of the most common platform that hadoop is run on. For non-unix platforms, the existing ping mechanism can be retained. Thoughts?

        Issue Links

          Activity

          Hide
          owen.omalley Owen O'Malley added a comment -

          Since we use random ports, this isn't critical. I would rather move the whole system to use job control like the shells do.

          Show
          owen.omalley Owen O'Malley added a comment - Since we use random ports, this isn't critical. I would rather move the whole system to use job control like the shells do.
          Hide
          aw Allen Wittenauer added a comment -

          I believe this one has actually been resolved with the rewrite of the pipes interface.

          Show
          aw Allen Wittenauer added a comment - I believe this one has actually been resolved with the rewrite of the pipes interface.

            People

            • Assignee:
              Unassigned
              Reporter:
              devaraj Devaraj Das
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development