[HDFS-326] Add a lifecycle interface for Hadoop components: namenodes, job clients, etc.

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Won't Fix
• Assignee: Steve Loughran
• Reporter: Steve Loughran
• Created: 24/Jun/08 10:35
• Updated: 02/May/13 02:29
• Resolved: 18/Feb/12 16:19

Description

I'd like to propose we have a standard interface for hadoop components, the things that get started or stopped when you bring up a namenode. currently, some of these classes have a stop() or shutdown() method, with no standard name/interface, but no way of seeing if they are live, checking their health of shutting them down reliably. Indeed, there is a tendency for the spawned threads to not want to die; to require the entire process to be killed to stop the workers.

Having a standard interface would make it easier for

• management tools to manage the different things
• monitoring the state of things
• subclassing

The latter is interesting as right now TaskTracker and JobTracker start up threads in their constructor; that's very dangerous as subclasses may have their methods called before they are full initialised. Adding this interface would be the right time to clean up the startup process so that subclassing is less risky.

Issue Links

• blocks: MAPREDUCE-213 Provide a way to query job tracker about its daemon thread's status
• depends upon: HADOOP-5348 Create a ThrowableWritable for serializing exceptions robustly
• depends upon: HADOOP-3642 add a HadoopIOException that can be thrown in any method that has IOException on its signature
• depends upon: HADOOP-4059 Add tests that try starting the various hadoop command line scripts
• incorporates: HADOOP-3415 JobEndNotifier isnt synchronized, doesnt check state before acting
• incorporates: HADOOP-6194 Add service base class and tests to hadoop-common/util
• is depended upon by: HDFS-239 Brief, baseline namenode health check
• is depended upon by: HADOOP-3969 Provide Mechanism to optionally expose public org.apache.hadoop.util.Services APIs
• is depended upon by: MAPREDUCE-233 Integrate TaskTracker with the Service base class
• relates to: HADOOP-6915 Add daemon restart option