Flume
  1. Flume
  2. FLUME-1257

Components that fail to start can put flume into a state which it can't shutdown from

    Details

    • Type: Bug Bug
    • Status: Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: v1.1.0, v1.2.0
    • Fix Version/s: None
    • Component/s: Configuration, Node
    • Labels:
      None

      Description

      Clean shutdown of a flume agent where a component fails to start doesn't work.

      One way of confirming this is to try and use a FileChannel without hadoop IO jars on the classpath.

      My understanding of this is that the first Ctrl+C will try to stop the supervisor, which in turn should take down everything, but AbstractFileConfigurationProvider#stop will try to gently stop the local executor, which in turn is in an endless loop trying to start up a channel(DefaultLogicalNodeManager#startAllComponents). This loop can only be broken by an interrupt, but none ever comes to it, or the higher level(which would try shutdownNow on the executors.

      The interrupts will never come, since they are all relying on something further up the chain for them, but it doesn't exist.

      One solution for this is just to be less merciful in AbstractFileConfiguration#stop() and give it a moderate timeout, then do executorService.shutdownNow()

        Issue Links

          Activity

          Juhani Connolly made changes -
          Link This issue is related to FLUME-966 [ FLUME-966 ]
          Juhani Connolly made changes -
          Field Original Value New Value
          Affects Version/s v1.1.0 [ 12319284 ]
          Affects Version/s v1.2.0 [ 12320243 ]
          Component/s Configuration [ 12316615 ]
          Component/s Node [ 12315315 ]
          Juhani Connolly created issue -

            People

            • Assignee:
              Unassigned
              Reporter:
              Juhani Connolly
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:

                Development