[FLUME-2894] Flume components should stop in the correct order (graceful shutdown)

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.6.0, 1.7.0
• Fix Version/s: 1.9.0
• Component/s: Channel, Node, Sinks+Sources
• Assignee: Laxman
• Reporter: Piotr Wiecek
• Created: 21/Mar/16 10:00
• Updated: 11/Mar/22 03:13
• Resolved: 03/May/18 11:31

Description

Flume components should be stopped in the right way:

• stop the sources (in order to not receiving further notifications),
• wait until all events within the channels are consumed by the sinks,
• stop the channels and the sinks.

Currently, the shutdown hook stops the components in a random manner.
E.g.: SINK, CHANNEL, SOURCE.

Components are stored in the HashMap:

Map<LifecycleAware, Supervisoree> supervisedProcesses;
...
supervisedProcesses = new HashMap<LifecycleAware, Supervisoree>();
...
@Override
  public synchronized void stop() {
  ...
  for (final Entry<LifecycleAware, Supervisoree> entry : supervisedProcesses
        .entrySet()) {

      if (entry.getKey().getLifecycleState().equals(LifecycleState.START)) {
        entry.getValue().status.desiredState = LifecycleState.STOP;
        entry.getKey().stop();
      }
    }
....

The problems which we can have:

• not all Events will be consumed (Sink will be stopped first)
• Source will continue to accept messages even though other components are stopped

Issue Links

• links to: GitHub Pull Request #101
• links to: GitHub Pull Request #296