Uploaded image for project: 'Flume'
  1. Flume
  2. FLUME-2894

Flume components should stop in the correct order (graceful shutdown)

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 1.6.0, 1.7.0
    • 1.9.0
    • Channel, Node, Sinks+Sources
    • None

    Description

      Flume components should be stopped in the right way:

      • stop the sources (in order to not receiving further notifications),
      • wait until all events within the channels are consumed by the sinks,
      • stop the channels and the sinks.

      Currently, the shutdown hook stops the components in a random manner.
      E.g.: SINK, CHANNEL, SOURCE.

      Components are stored in the HashMap:

      Map<LifecycleAware, Supervisoree> supervisedProcesses;
      ...
      supervisedProcesses = new HashMap<LifecycleAware, Supervisoree>();
      ...
      @Override
        public synchronized void stop() {
        ...
        for (final Entry<LifecycleAware, Supervisoree> entry : supervisedProcesses
              .entrySet()) {
      
            if (entry.getKey().getLifecycleState().equals(LifecycleState.START)) {
              entry.getValue().status.desiredState = LifecycleState.STOP;
              entry.getKey().stop();
            }
          }
      ....
      

      The problems which we can have:

      • not all Events will be consumed (Sink will be stopped first)
      • Source will continue to accept messages even though other components are stopped

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            lakshman Laxman
            pwiecek Piotr Wiecek
            Votes:
            0 Vote for this issue
            Watchers:
            10 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 0.5h
                0.5h

                Slack

                  Issue deployment