Uploaded image for project: 'Flume'
  1. Flume
  2. FLUME-2894

Flume components should stop in the correct order (graceful shutdown)

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.6.0, 1.7.0
    • Fix Version/s: 1.9.0
    • Component/s: Channel, Node, Sinks+Sources
    • Labels:
      None

      Description

      Flume components should be stopped in the right way:

      • stop the sources (in order to not receiving further notifications),
      • wait until all events within the channels are consumed by the sinks,
      • stop the channels and the sinks.

      Currently, the shutdown hook stops the components in a random manner.
      E.g.: SINK, CHANNEL, SOURCE.

      Components are stored in the HashMap:

      Map<LifecycleAware, Supervisoree> supervisedProcesses;
      ...
      supervisedProcesses = new HashMap<LifecycleAware, Supervisoree>();
      ...
      @Override
        public synchronized void stop() {
        ...
        for (final Entry<LifecycleAware, Supervisoree> entry : supervisedProcesses
              .entrySet()) {
      
            if (entry.getKey().getLifecycleState().equals(LifecycleState.START)) {
              entry.getValue().status.desiredState = LifecycleState.STOP;
              entry.getKey().stop();
            }
          }
      ....
      

      The problems which we can have:

      • not all Events will be consumed (Sink will be stopped first)
      • Source will continue to accept messages even though other components are stopped

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                lakshman Laxman
                Reporter:
                pwiecek Piotr Wiecek
              • Votes:
                0 Vote for this issue
                Watchers:
                8 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: