Flume
  1. Flume
  2. FLUME-1230

Sink gets initialized even when not active

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: v1.2.0
    • Component/s: None
    • Labels:
      None

      Description

      Flume configuration requires that components that are not declared in the active lists (agent.sources, agent.sinks, agent.channels, and agent.sinkgroups) be disabled by default. However, in some cases, sinks that are not in the active list will get initialized and started.

      Following configuration can reproduce this scenario:

      agent.sources = s
      agent.channels = c
      #agent.sinks = k1 k2
      
      agent.sources.s.type = netcat
      agent.sources.s.port = 12121
      agent.sources.s.bind = localhost
      agent.sources.s.channels = c
      
      agent.channels.c.type = memory
      
      agent.sinks.k1.type = logger
      agent.sinks.k1.channel = c
      
      agent.sinks.k2.type = null
      agent.sinks.k2.channel = c
      
      1. FLUME-1230-refactor.diff
        125 kB
        Arvind Prabhakar
      2. FLUME-1230.patch
        15 kB
        Hari Shreedharan

        Issue Links

          Activity

          Hide
          Hari Shreedharan added a comment -

          Arvind - if you don't mind, I can look into this issue.

          Show
          Hari Shreedharan added a comment - Arvind - if you don't mind, I can look into this issue.
          Hide
          Arvind Prabhakar added a comment -

          Hari - thanks, go for it.

          Show
          Arvind Prabhakar added a comment - Hari - thanks, go for it.
          Hide
          Arvind Prabhakar added a comment -

          @Hari - here is a minor refactoring of the configuration code that removes some duplication of logic. Also, simplifies it to a certain extent. I have added some tests, existing tests pass and done basic manual verification. All seems to be ok. But I am sure that there are bugs in it, so take it cautiously if you would like to.

          Patch generated on revision 1342958

          Show
          Arvind Prabhakar added a comment - @Hari - here is a minor refactoring of the configuration code that removes some duplication of logic. Also, simplifies it to a certain extent. I have added some tests, existing tests pass and done basic manual verification. All seems to be ok. But I am sure that there are bugs in it, so take it cautiously if you would like to. Patch generated on revision 1342958
          Hide
          Mike Percy added a comment -

          Hey guys, let's push this fix in and continue the needed refactoring in another JIRA.

          Show
          Mike Percy added a comment - Hey guys, let's push this fix in and continue the needed refactoring in another JIRA.
          Hide
          Mike Percy added a comment -

          +1. Patch committed. Thanks Hari!

          Show
          Mike Percy added a comment - +1. Patch committed. Thanks Hari!
          Hide
          Hari Shreedharan added a comment -

          Thanks Mike. I have filed FLUME-1237 to track the refactoring Arvind suggested above.

          Show
          Hari Shreedharan added a comment - Thanks Mike. I have filed FLUME-1237 to track the refactoring Arvind suggested above.
          Hide
          Hudson added a comment -

          Integrated in flume-trunk #216 (See https://builds.apache.org/job/flume-trunk/216/)
          FLUME-1230. Components can get initialized even when not active.

          (Hari Shreedharan via Mike Percy) (Revision 1344126)

          Result = SUCCESS
          mpercy : http://svn.apache.org/viewvc/?view=rev&rev=1344126
          Files :

          • /incubator/flume/trunk/flume-ng-configuration/src/main/java/org/apache/flume/conf/FlumeConfiguration.java
          • /incubator/flume/trunk/flume-ng-node/src/main/java/org/apache/flume/conf/properties/PropertiesFileConfigurationProvider.java
          Show
          Hudson added a comment - Integrated in flume-trunk #216 (See https://builds.apache.org/job/flume-trunk/216/ ) FLUME-1230 . Components can get initialized even when not active. (Hari Shreedharan via Mike Percy) (Revision 1344126) Result = SUCCESS mpercy : http://svn.apache.org/viewvc/?view=rev&rev=1344126 Files : /incubator/flume/trunk/flume-ng-configuration/src/main/java/org/apache/flume/conf/FlumeConfiguration.java /incubator/flume/trunk/flume-ng-node/src/main/java/org/apache/flume/conf/properties/PropertiesFileConfigurationProvider.java

            People

            • Assignee:
              Hari Shreedharan
              Reporter:
              Arvind Prabhakar
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development