Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-229

Provide a command line option to check if a Hadoop jobtracker is idle

    Details

    • Type: New Feature New Feature
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      This is an RFE for providing a way to determine from the hadoop command line whether a jobtracker is idle. One possibility is to have something like hadoop jobtracker -idle <time>. Hadoop would return true (maybe via some stdout output) if the jobtracker had no work to do (jobs running / prepared) since <time> seconds, false otherwise.

      This would be useful for management / provisioning systems like Hadoop-On-Demand HADOOP-1301, which can then deallocate the idle, provisioned clusters automatically, and release resources.

        Activity

        Hide
        Owen O'Malley added a comment -

        Another approach to this would be to have an admin command for the job tracker that would have the job tracker stop accepting new jobs and shut itself down when the last job completed. That way the external process wouldn't need to continually poll the job tracker to see if it was done yet. Thoughts?

        Show
        Owen O'Malley added a comment - Another approach to this would be to have an admin command for the job tracker that would have the job tracker stop accepting new jobs and shut itself down when the last job completed. That way the external process wouldn't need to continually poll the job tracker to see if it was done yet. Thoughts?
        Hide
        Enis Soztutar added a comment -

        Another approach to this would be to have an admin command for the job tracker that would have the job tracker stop accepting new jobs and shut itself down when the last job completed.

        I remember this was discussed before but never implemented. I think this and a command to gracefully stop the jt would be great.

        Show
        Enis Soztutar added a comment - Another approach to this would be to have an admin command for the job tracker that would have the job tracker stop accepting new jobs and shut itself down when the last job completed. I remember this was discussed before but never implemented. I think this and a command to gracefully stop the jt would be great.
        Hide
        Michael Bieniosek added a comment -

        It's possible to do this through the JobClient API currently.

        Show
        Michael Bieniosek added a comment - It's possible to do this through the JobClient API currently.
        Hide
        Hemanth Yamijala added a comment -

        Another approach to this would be to have an admin command for the job tracker that would have the job tracker stop accepting new jobs and shut itself down when the last job completed. That way the external process wouldn't need to continually poll the job tracker to see if it was done yet. Thoughts?

        The requirement we have in Hadoop On Demand is to shutdown a job tracker when it has not run any jobs for a period of time. This is to help in releasing unused resources, and reclaim them. The approach suggested above actually makes sense if we want to run the job tracker for a period of time and then shutdown. This seems to be different from what we need, right ?

        The problem of continous polling does exist. But I feel it may not be too intensive. Any alternative that does not have this problem, but solves the use-case would also be great.

        Show
        Hemanth Yamijala added a comment - Another approach to this would be to have an admin command for the job tracker that would have the job tracker stop accepting new jobs and shut itself down when the last job completed. That way the external process wouldn't need to continually poll the job tracker to see if it was done yet. Thoughts? The requirement we have in Hadoop On Demand is to shutdown a job tracker when it has not run any jobs for a period of time. This is to help in releasing unused resources, and reclaim them. The approach suggested above actually makes sense if we want to run the job tracker for a period of time and then shutdown. This seems to be different from what we need, right ? The problem of continous polling does exist. But I feel it may not be too intensive. Any alternative that does not have this problem, but solves the use-case would also be great.
        Hide
        Arun C Murthy added a comment -

        The other other issue is see with the JT shutting itself down is that the TTs (and HoD) don't know why that happened. If we just implement the isidle <for_this_long>, we could let HoD make the decision to shut down the entire cluster via the usual scripts... thoughts?

        Show
        Arun C Murthy added a comment - The other other issue is see with the JT shutting itself down is that the TTs (and HoD) don't know why that happened. If we just implement the isidle <for_this_long> , we could let HoD make the decision to shut down the entire cluster via the usual scripts... thoughts?
        Hide
        Arun C Murthy added a comment -

        Let me clarify, I agree 'graceful shutdown' initiated by the JT is the right solution.

        However, the command, as described here, is a good interim solution IMHO.

        Show
        Arun C Murthy added a comment - Let me clarify, I agree 'graceful shutdown' initiated by the JT is the right solution. However, the command, as described here, is a good interim solution IMHO.
        Hide
        Allen Wittenauer added a comment -

        I'm going to close this as fixed.

        Clients can ask the JT if they are currently busy. By doing this periodically, they can build an idle time.

        Show
        Allen Wittenauer added a comment - I'm going to close this as fixed. Clients can ask the JT if they are currently busy. By doing this periodically, they can build an idle time.

          People

          • Assignee:
            Unassigned
            Reporter:
            Hemanth Yamijala
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development