Flume
  1. Flume
  2. FLUME-1036

Reconfiguration of AVRO or NETCAT source causes port bind exception

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: v1.2.0
    • Fix Version/s: v1.2.0
    • Component/s: Sinks+Sources
    • Labels:
      None
    • Environment:

      [CentOS 6.2 64-bit]

      Description

      This issue occurs with a source that has binds to a port, such as NETCAT or AVRO. In the steps below, I use AVRO as an example.

      Steps:

      1. Run with a props file that includes an avro source, such as a props file with these contents:

      1. a = agent
      2. r = source
      3. c = channel
      4. k = sink
        a1.sources = r1
        a1.channels = c1
        a1.sinks = k1
      5. ===SOURCES===
        a1.sources.r1.type = AVRO
        a1.sources.r1.bind = localhost
        a1.sources.r1.port = 1473
        a1.sources.r1.channels = c1
      6. ===CHANNELS===
        a1.channels.c1.type = MEMORY
      7. ===SINKS===
        a1.sinks.k1.type = LOGGER
        a1.sinks.k1.channel = c1

      2. Run the flume node
      [will@localhost flume-1.2.0-incubating-SNAPSHOT]$ pwd
      /home/will/git/apache/flume/flume-1.2.0-incubating-SNAPSHOT
      [will@localhost flume-1.2.0-incubating-SNAPSHOT]$ bin/flume-ng node --conf conf --conf-file conf/a1.properties --name a1

      3. Simulate a reconfig of a file by just touching the props file:
      [will@localhost flume-1.2.0-incubating-SNAPSHOT]$ touch conf/a1.properties

      At this point, an exception is thrown because an attempt is made to bind to the same port, even though the port number specified in config file didn't change (actually, nothing changed but the last modification time of the file).

      2012-03-19 02:17:48,573 (lifecycleSupervisor-1-1) [ERROR - org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:210)] Unable to start EventDrivenSourceRunner: { source:AvroSource:

      { bindAddress:localhost port:1473 }

      } - Exception follows.
      org.jboss.netty.channel.ChannelException: Failed to bind to: localhost/127.0.0.1:1473
      at org.jboss.netty.bootstrap.ServerBootstrap.bind(ServerBootstrap.java:303)
      at org.apache.avro.ipc.NettyServer.<init>(NettyServer.java:85)
      at org.apache.avro.ipc.NettyServer.<init>(NettyServer.java:66)
      at org.apache.flume.source.AvroSource.start(AvroSource.java:124)
      at org.apache.flume.source.EventDrivenSourceRunner.start(EventDrivenSourceRunner.java:39)
      at org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:208)

      1. FLUME-1036-fix-1.patch
        1 kB
        Hari Shreedharan
      2. FLUME-1036-2.patch
        9 kB
        Hari Shreedharan
      3. FLUME-1036-1.patch
        9 kB
        Hari Shreedharan

        Issue Links

          Activity

          Hide
          Hari Shreedharan added a comment -

          This seems to be a slightly serious problem. The reason for this is that when the configuration changes, we are probably not cleaning up system resources. At this point, when the conf changes we try to load new components but the issue is that we have not cleaned up the old ones. As a result the old components' sockets etc are not cleaned up immediately and we try to create a new socket to bind to the same port(in this scenario).

          Show
          Hari Shreedharan added a comment - This seems to be a slightly serious problem. The reason for this is that when the configuration changes, we are probably not cleaning up system resources. At this point, when the conf changes we try to load new components but the issue is that we have not cleaned up the old ones. As a result the old components' sockets etc are not cleaned up immediately and we try to create a new socket to bind to the same port(in this scenario).
          Hide
          Hari Shreedharan added a comment -

          This seems to be what is happening:

          • initial configuration is loaded, node starts fine. All sources and sinks are bound and started.
          • config file is changed: AbstractFileConfigurationProvider.run() finds that the update time is > lastChangedTime, calls doLoad()
          • doLoad is simply a synchronized wrapper around load(). load() creates a new NodeConfiguration object and passes it to DefaultLogicalNodeManager(actually an implementation of NodeConfigurationAware), and starts all of the SourceRunners and SinkRunners, leaving the old node dangling until hopefully, at some point it is cleaned up by GC.

          We should keep track of the NodeConfiguration currently active and make sure we kill it before attempting to start a new one.

          Show
          Hari Shreedharan added a comment - This seems to be what is happening: initial configuration is loaded, node starts fine. All sources and sinks are bound and started. config file is changed: AbstractFileConfigurationProvider.run() finds that the update time is > lastChangedTime, calls doLoad() doLoad is simply a synchronized wrapper around load(). load() creates a new NodeConfiguration object and passes it to DefaultLogicalNodeManager(actually an implementation of NodeConfigurationAware), and starts all of the SourceRunners and SinkRunners, leaving the old node dangling until hopefully, at some point it is cleaned up by GC. We should keep track of the NodeConfiguration currently active and make sure we kill it before attempting to start a new one.
          Hide
          Hari Shreedharan added a comment -

          As a side effect of the above problem, Will and I just confirmed that if we reconfigure, the old components are still active, even though new ones are created. I think we should make sure all components of the old configuration are stopped before new ones are created.

          Show
          Hari Shreedharan added a comment - As a side effect of the above problem, Will and I just confirmed that if we reconfigure, the old components are still active, even though new ones are created. I think we should make sure all components of the old configuration are stopped before new ones are created.
          Hide
          Will McQueen added a comment -

          Just to add to that, side effects are similar to what was observed in FLUME-1004 when testing the SEQ source.

          Show
          Will McQueen added a comment - Just to add to that, side effects are similar to what was observed in FLUME-1004 when testing the SEQ source.
          Hide
          Jarek Jarcec Cecho added a comment -

          I believe that I've already reported this issue in FLUME-862.

          Show
          Jarek Jarcec Cecho added a comment - I believe that I've already reported this issue in FLUME-862 .
          Hide
          Hari Shreedharan added a comment - - edited

          Jarek - yes, this seems to be the same issue. For now just updating my investigation here:

          • I updated the PropertiesFileConfigurationProvider/DefaultLogicalNodeManager to stop the old components. Some issues:
            LifecycleSupervisor allows you to only switch to a desiredstate and wait for MonitorRunnable to check the state within 3 seconds, which runs as a different thread. So any stop/interrupt/join calls on any server-like process will block only the MonitorRunnable thread, not the thread which changed the state. So it is possible that the config provider thread will start the new components before the old ones are dead - this is not the desired behavior. We absolutely must wait for the old components to die before the new ones start up. So my proposal:
          • Add a method to LifecycleSupervisor - changeStateImmediately which will run the monitorRunnable immediately and wait for it to die before returning. This lets us wait till the state is changed and makes sure the component died.
          Show
          Hari Shreedharan added a comment - - edited Jarek - yes, this seems to be the same issue. For now just updating my investigation here: I updated the PropertiesFileConfigurationProvider/DefaultLogicalNodeManager to stop the old components. Some issues: LifecycleSupervisor allows you to only switch to a desiredstate and wait for MonitorRunnable to check the state within 3 seconds, which runs as a different thread. So any stop/interrupt/join calls on any server-like process will block only the MonitorRunnable thread, not the thread which changed the state. So it is possible that the config provider thread will start the new components before the old ones are dead - this is not the desired behavior. We absolutely must wait for the old components to die before the new ones start up. So my proposal: Add a method to LifecycleSupervisor - changeStateImmediately which will run the monitorRunnable immediately and wait for it to die before returning. This lets us wait till the state is changed and makes sure the component died.
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/4464/
          -----------------------------------------------------------

          Review request for Flume.

          Summary
          -------

          Added functionality to support stopping of components if configuration changes.

          This addresses bug FLUME-1036.
          https://issues.apache.org/jira/browse/FLUME-1036

          Diffs


          flume-ng-core/src/main/java/org/apache/flume/lifecycle/LifecycleSupervisor.java dd757fc
          flume-ng-core/src/main/java/org/apache/flume/sink/LoggerSink.java f6f9d33
          flume-ng-core/src/main/java/org/apache/flume/source/NetcatSource.java 94245ac
          flume-ng-node/src/main/java/org/apache/flume/conf/properties/PropertiesFileConfigurationProvider.java 97f72e1
          flume-ng-node/src/main/java/org/apache/flume/node/nodemanager/DefaultLogicalNodeManager.java c150097
          flume-ng-node/src/main/java/org/apache/flume/node/nodemanager/NodeConfigurationAware.java 47249e9

          Diff: https://reviews.apache.org/r/4464/diff

          Testing
          -------

          Tested change of configuration - no bind exception when port remains the same. When port changes, old port is no longer accessible, new port is.

          Thanks,

          Hari

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4464/ ----------------------------------------------------------- Review request for Flume. Summary ------- Added functionality to support stopping of components if configuration changes. This addresses bug FLUME-1036 . https://issues.apache.org/jira/browse/FLUME-1036 Diffs flume-ng-core/src/main/java/org/apache/flume/lifecycle/LifecycleSupervisor.java dd757fc flume-ng-core/src/main/java/org/apache/flume/sink/LoggerSink.java f6f9d33 flume-ng-core/src/main/java/org/apache/flume/source/NetcatSource.java 94245ac flume-ng-node/src/main/java/org/apache/flume/conf/properties/PropertiesFileConfigurationProvider.java 97f72e1 flume-ng-node/src/main/java/org/apache/flume/node/nodemanager/DefaultLogicalNodeManager.java c150097 flume-ng-node/src/main/java/org/apache/flume/node/nodemanager/NodeConfigurationAware.java 47249e9 Diff: https://reviews.apache.org/r/4464/diff Testing ------- Tested change of configuration - no bind exception when port remains the same. When port changes, old port is no longer accessible, new port is. Thanks, Hari
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/4464/
          -----------------------------------------------------------

          (Updated 2012-03-23 05:21:27.505890)

          Review request for Flume.

          Changes
          -------

          removing some whitespace changes.

          Summary
          -------

          Added functionality to support stopping of components if configuration changes.

          This addresses bug FLUME-1036.
          https://issues.apache.org/jira/browse/FLUME-1036

          Diffs (updated)


          flume-ng-core/src/main/java/org/apache/flume/lifecycle/LifecycleSupervisor.java dd757fc
          flume-ng-core/src/main/java/org/apache/flume/source/NetcatSource.java 94245ac
          flume-ng-node/src/main/java/org/apache/flume/conf/properties/PropertiesFileConfigurationProvider.java 97f72e1
          flume-ng-node/src/main/java/org/apache/flume/node/nodemanager/DefaultLogicalNodeManager.java c150097
          flume-ng-node/src/main/java/org/apache/flume/node/nodemanager/NodeConfigurationAware.java 47249e9

          Diff: https://reviews.apache.org/r/4464/diff

          Testing
          -------

          Tested change of configuration - no bind exception when port remains the same. When port changes, old port is no longer accessible, new port is.

          Thanks,

          Hari

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4464/ ----------------------------------------------------------- (Updated 2012-03-23 05:21:27.505890) Review request for Flume. Changes ------- removing some whitespace changes. Summary ------- Added functionality to support stopping of components if configuration changes. This addresses bug FLUME-1036 . https://issues.apache.org/jira/browse/FLUME-1036 Diffs (updated) flume-ng-core/src/main/java/org/apache/flume/lifecycle/LifecycleSupervisor.java dd757fc flume-ng-core/src/main/java/org/apache/flume/source/NetcatSource.java 94245ac flume-ng-node/src/main/java/org/apache/flume/conf/properties/PropertiesFileConfigurationProvider.java 97f72e1 flume-ng-node/src/main/java/org/apache/flume/node/nodemanager/DefaultLogicalNodeManager.java c150097 flume-ng-node/src/main/java/org/apache/flume/node/nodemanager/NodeConfigurationAware.java 47249e9 Diff: https://reviews.apache.org/r/4464/diff Testing ------- Tested change of configuration - no bind exception when port remains the same. When port changes, old port is no longer accessible, new port is. Thanks, Hari
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/4464/#review6272
          -----------------------------------------------------------

          flume-ng-core/src/main/java/org/apache/flume/sink/LoggerSink.java
          <https://reviews.apache.org/r/4464/#comment13597>

          Is this what you want?

          • Brock

          On 2012-03-23 05:21:27, Hari Shreedharan wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/4464/

          -----------------------------------------------------------

          (Updated 2012-03-23 05:21:27)

          Review request for Flume.

          Summary

          -------

          Added functionality to support stopping of components if configuration changes.

          This addresses bug FLUME-1036.

          https://issues.apache.org/jira/browse/FLUME-1036

          Diffs

          -----

          flume-ng-core/src/main/java/org/apache/flume/lifecycle/LifecycleSupervisor.java dd757fc

          flume-ng-core/src/main/java/org/apache/flume/source/NetcatSource.java 94245ac

          flume-ng-node/src/main/java/org/apache/flume/conf/properties/PropertiesFileConfigurationProvider.java 97f72e1

          flume-ng-node/src/main/java/org/apache/flume/node/nodemanager/DefaultLogicalNodeManager.java c150097

          flume-ng-node/src/main/java/org/apache/flume/node/nodemanager/NodeConfigurationAware.java 47249e9

          Diff: https://reviews.apache.org/r/4464/diff

          Testing

          -------

          Tested change of configuration - no bind exception when port remains the same. When port changes, old port is no longer accessible, new port is.

          Thanks,

          Hari

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4464/#review6272 ----------------------------------------------------------- flume-ng-core/src/main/java/org/apache/flume/sink/LoggerSink.java < https://reviews.apache.org/r/4464/#comment13597 > Is this what you want? Brock On 2012-03-23 05:21:27, Hari Shreedharan wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4464/ ----------------------------------------------------------- (Updated 2012-03-23 05:21:27) Review request for Flume. Summary ------- Added functionality to support stopping of components if configuration changes. This addresses bug FLUME-1036 . https://issues.apache.org/jira/browse/FLUME-1036 Diffs ----- flume-ng-core/src/main/java/org/apache/flume/lifecycle/LifecycleSupervisor.java dd757fc flume-ng-core/src/main/java/org/apache/flume/source/NetcatSource.java 94245ac flume-ng-node/src/main/java/org/apache/flume/conf/properties/PropertiesFileConfigurationProvider.java 97f72e1 flume-ng-node/src/main/java/org/apache/flume/node/nodemanager/DefaultLogicalNodeManager.java c150097 flume-ng-node/src/main/java/org/apache/flume/node/nodemanager/NodeConfigurationAware.java 47249e9 Diff: https://reviews.apache.org/r/4464/diff Testing ------- Tested change of configuration - no bind exception when port remains the same. When port changes, old port is no longer accessible, new port is. Thanks, Hari
          Hide
          jiraposter@reviews.apache.org added a comment -

          On 2012-03-23 05:22:51, Brock Noland wrote:

          > flume-ng-core/src/main/java/org/apache/flume/sink/LoggerSink.java, line 67

          > <https://reviews.apache.org/r/4464/diff/1/?file=94851#file94851line67>

          >

          > Is this what you want?

          I think you are looking at an older diff. I removed this in r2.

          • Hari

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/4464/#review6272
          -----------------------------------------------------------

          On 2012-03-23 05:21:27, Hari Shreedharan wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/4464/

          -----------------------------------------------------------

          (Updated 2012-03-23 05:21:27)

          Review request for Flume.

          Summary

          -------

          Added functionality to support stopping of components if configuration changes.

          This addresses bug FLUME-1036.

          https://issues.apache.org/jira/browse/FLUME-1036

          Diffs

          -----

          flume-ng-core/src/main/java/org/apache/flume/lifecycle/LifecycleSupervisor.java dd757fc

          flume-ng-core/src/main/java/org/apache/flume/source/NetcatSource.java 94245ac

          flume-ng-node/src/main/java/org/apache/flume/conf/properties/PropertiesFileConfigurationProvider.java 97f72e1

          flume-ng-node/src/main/java/org/apache/flume/node/nodemanager/DefaultLogicalNodeManager.java c150097

          flume-ng-node/src/main/java/org/apache/flume/node/nodemanager/NodeConfigurationAware.java 47249e9

          Diff: https://reviews.apache.org/r/4464/diff

          Testing

          -------

          Tested change of configuration - no bind exception when port remains the same. When port changes, old port is no longer accessible, new port is.

          Thanks,

          Hari

          Show
          jiraposter@reviews.apache.org added a comment - On 2012-03-23 05:22:51, Brock Noland wrote: > flume-ng-core/src/main/java/org/apache/flume/sink/LoggerSink.java, line 67 > < https://reviews.apache.org/r/4464/diff/1/?file=94851#file94851line67 > > > Is this what you want? I think you are looking at an older diff. I removed this in r2. Hari ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4464/#review6272 ----------------------------------------------------------- On 2012-03-23 05:21:27, Hari Shreedharan wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4464/ ----------------------------------------------------------- (Updated 2012-03-23 05:21:27) Review request for Flume. Summary ------- Added functionality to support stopping of components if configuration changes. This addresses bug FLUME-1036 . https://issues.apache.org/jira/browse/FLUME-1036 Diffs ----- flume-ng-core/src/main/java/org/apache/flume/lifecycle/LifecycleSupervisor.java dd757fc flume-ng-core/src/main/java/org/apache/flume/source/NetcatSource.java 94245ac flume-ng-node/src/main/java/org/apache/flume/conf/properties/PropertiesFileConfigurationProvider.java 97f72e1 flume-ng-node/src/main/java/org/apache/flume/node/nodemanager/DefaultLogicalNodeManager.java c150097 flume-ng-node/src/main/java/org/apache/flume/node/nodemanager/NodeConfigurationAware.java 47249e9 Diff: https://reviews.apache.org/r/4464/diff Testing ------- Tested change of configuration - no bind exception when port remains the same. When port changes, old port is no longer accessible, new port is. Thanks, Hari
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/4464/#review6277
          -----------------------------------------------------------

          Thanks for the patch Hari. A couple of suggestions to consider:

          • Rather than adding another method in NodeConfigurationAware which is invoked by the PropertiesFIleConfigurationProvider - it will be better to instead keep a track of the old configuration within the DefaultLogicalNodeManager which onNodeConfigurationChanged() invocation first shuts down and removes the old components from last configuration before initializing the new configuration based components.
          • Similarly, rather than introducing setStopStateImmediately(), we should instead fix the implementation of unsupervise() to do the necessary shutdown and cleanup.

          flume-ng-core/src/main/java/org/apache/flume/lifecycle/LifecycleSupervisor.java
          <https://reviews.apache.org/r/4464/#comment13618>

          this block needs to be indented.

          • Arvind

          On 2012-03-23 05:21:27, Hari Shreedharan wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/4464/

          -----------------------------------------------------------

          (Updated 2012-03-23 05:21:27)

          Review request for Flume.

          Summary

          -------

          Added functionality to support stopping of components if configuration changes.

          This addresses bug FLUME-1036.

          https://issues.apache.org/jira/browse/FLUME-1036

          Diffs

          -----

          flume-ng-core/src/main/java/org/apache/flume/lifecycle/LifecycleSupervisor.java dd757fc

          flume-ng-core/src/main/java/org/apache/flume/source/NetcatSource.java 94245ac

          flume-ng-node/src/main/java/org/apache/flume/conf/properties/PropertiesFileConfigurationProvider.java 97f72e1

          flume-ng-node/src/main/java/org/apache/flume/node/nodemanager/DefaultLogicalNodeManager.java c150097

          flume-ng-node/src/main/java/org/apache/flume/node/nodemanager/NodeConfigurationAware.java 47249e9

          Diff: https://reviews.apache.org/r/4464/diff

          Testing

          -------

          Tested change of configuration - no bind exception when port remains the same. When port changes, old port is no longer accessible, new port is.

          Thanks,

          Hari

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4464/#review6277 ----------------------------------------------------------- Thanks for the patch Hari. A couple of suggestions to consider: Rather than adding another method in NodeConfigurationAware which is invoked by the PropertiesFIleConfigurationProvider - it will be better to instead keep a track of the old configuration within the DefaultLogicalNodeManager which onNodeConfigurationChanged() invocation first shuts down and removes the old components from last configuration before initializing the new configuration based components. Similarly, rather than introducing setStopStateImmediately(), we should instead fix the implementation of unsupervise() to do the necessary shutdown and cleanup. flume-ng-core/src/main/java/org/apache/flume/lifecycle/LifecycleSupervisor.java < https://reviews.apache.org/r/4464/#comment13618 > this block needs to be indented. Arvind On 2012-03-23 05:21:27, Hari Shreedharan wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4464/ ----------------------------------------------------------- (Updated 2012-03-23 05:21:27) Review request for Flume. Summary ------- Added functionality to support stopping of components if configuration changes. This addresses bug FLUME-1036 . https://issues.apache.org/jira/browse/FLUME-1036 Diffs ----- flume-ng-core/src/main/java/org/apache/flume/lifecycle/LifecycleSupervisor.java dd757fc flume-ng-core/src/main/java/org/apache/flume/source/NetcatSource.java 94245ac flume-ng-node/src/main/java/org/apache/flume/conf/properties/PropertiesFileConfigurationProvider.java 97f72e1 flume-ng-node/src/main/java/org/apache/flume/node/nodemanager/DefaultLogicalNodeManager.java c150097 flume-ng-node/src/main/java/org/apache/flume/node/nodemanager/NodeConfigurationAware.java 47249e9 Diff: https://reviews.apache.org/r/4464/diff Testing ------- Tested change of configuration - no bind exception when port remains the same. When port changes, old port is no longer accessible, new port is. Thanks, Hari
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/4464/
          -----------------------------------------------------------

          (Updated 2012-03-23 17:58:44.921477)

          Review request for Flume.

          Changes
          -------

          Incorporated Arvind's feedback.

          Summary
          -------

          Added functionality to support stopping of components if configuration changes.

          This addresses bug FLUME-1036.
          https://issues.apache.org/jira/browse/FLUME-1036

          Diffs (updated)


          flume-ng-core/src/main/java/org/apache/flume/lifecycle/LifecycleSupervisor.java dd757fc
          flume-ng-core/src/main/java/org/apache/flume/source/NetcatSource.java 94245ac
          flume-ng-node/src/main/java/org/apache/flume/conf/properties/PropertiesFileConfigurationProvider.java 97f72e1
          flume-ng-node/src/main/java/org/apache/flume/node/nodemanager/DefaultLogicalNodeManager.java c150097

          Diff: https://reviews.apache.org/r/4464/diff

          Testing
          -------

          Tested change of configuration - no bind exception when port remains the same. When port changes, old port is no longer accessible, new port is.

          Thanks,

          Hari

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4464/ ----------------------------------------------------------- (Updated 2012-03-23 17:58:44.921477) Review request for Flume. Changes ------- Incorporated Arvind's feedback. Summary ------- Added functionality to support stopping of components if configuration changes. This addresses bug FLUME-1036 . https://issues.apache.org/jira/browse/FLUME-1036 Diffs (updated) flume-ng-core/src/main/java/org/apache/flume/lifecycle/LifecycleSupervisor.java dd757fc flume-ng-core/src/main/java/org/apache/flume/source/NetcatSource.java 94245ac flume-ng-node/src/main/java/org/apache/flume/conf/properties/PropertiesFileConfigurationProvider.java 97f72e1 flume-ng-node/src/main/java/org/apache/flume/node/nodemanager/DefaultLogicalNodeManager.java c150097 Diff: https://reviews.apache.org/r/4464/diff Testing ------- Tested change of configuration - no bind exception when port remains the same. When port changes, old port is no longer accessible, new port is. Thanks, Hari
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/4464/#review6308
          -----------------------------------------------------------

          Ship it!

          +1. Please attach the patch to the Jira.

          • Arvind

          On 2012-03-23 17:58:44, Hari Shreedharan wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/4464/

          -----------------------------------------------------------

          (Updated 2012-03-23 17:58:44)

          Review request for Flume.

          Summary

          -------

          Added functionality to support stopping of components if configuration changes.

          This addresses bug FLUME-1036.

          https://issues.apache.org/jira/browse/FLUME-1036

          Diffs

          -----

          flume-ng-core/src/main/java/org/apache/flume/lifecycle/LifecycleSupervisor.java dd757fc

          flume-ng-core/src/main/java/org/apache/flume/source/NetcatSource.java 94245ac

          flume-ng-node/src/main/java/org/apache/flume/conf/properties/PropertiesFileConfigurationProvider.java 97f72e1

          flume-ng-node/src/main/java/org/apache/flume/node/nodemanager/DefaultLogicalNodeManager.java c150097

          Diff: https://reviews.apache.org/r/4464/diff

          Testing

          -------

          Tested change of configuration - no bind exception when port remains the same. When port changes, old port is no longer accessible, new port is.

          Thanks,

          Hari

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4464/#review6308 ----------------------------------------------------------- Ship it! +1. Please attach the patch to the Jira. Arvind On 2012-03-23 17:58:44, Hari Shreedharan wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4464/ ----------------------------------------------------------- (Updated 2012-03-23 17:58:44) Review request for Flume. Summary ------- Added functionality to support stopping of components if configuration changes. This addresses bug FLUME-1036 . https://issues.apache.org/jira/browse/FLUME-1036 Diffs ----- flume-ng-core/src/main/java/org/apache/flume/lifecycle/LifecycleSupervisor.java dd757fc flume-ng-core/src/main/java/org/apache/flume/source/NetcatSource.java 94245ac flume-ng-node/src/main/java/org/apache/flume/conf/properties/PropertiesFileConfigurationProvider.java 97f72e1 flume-ng-node/src/main/java/org/apache/flume/node/nodemanager/DefaultLogicalNodeManager.java c150097 Diff: https://reviews.apache.org/r/4464/diff Testing ------- Tested change of configuration - no bind exception when port remains the same. When port changes, old port is no longer accessible, new port is. Thanks, Hari
          Hide
          Arvind Prabhakar added a comment -

          Patch committed. Thanks Hari!

          Show
          Arvind Prabhakar added a comment - Patch committed. Thanks Hari!
          Hide
          Hudson added a comment -

          Integrated in flume-trunk #142 (See https://builds.apache.org/job/flume-trunk/142/)
          FLUME-1036. Reconfiguration of AVRO or NETCAT source causes port bind exception

          (Hari Shreedharan via Arvind Prabhakar) (Revision 1304595)

          Result = SUCCESS
          arvind : http://svn.apache.org/viewvc/?view=rev&rev=1304595
          Files :

          • /incubator/flume/trunk/flume-ng-core/src/main/java/org/apache/flume/lifecycle/LifecycleSupervisor.java
          • /incubator/flume/trunk/flume-ng-core/src/main/java/org/apache/flume/source/NetcatSource.java
          • /incubator/flume/trunk/flume-ng-node/src/main/java/org/apache/flume/conf/properties/PropertiesFileConfigurationProvider.java
          • /incubator/flume/trunk/flume-ng-node/src/main/java/org/apache/flume/node/nodemanager/DefaultLogicalNodeManager.java
          Show
          Hudson added a comment - Integrated in flume-trunk #142 (See https://builds.apache.org/job/flume-trunk/142/ ) FLUME-1036 . Reconfiguration of AVRO or NETCAT source causes port bind exception (Hari Shreedharan via Arvind Prabhakar) (Revision 1304595) Result = SUCCESS arvind : http://svn.apache.org/viewvc/?view=rev&rev=1304595 Files : /incubator/flume/trunk/flume-ng-core/src/main/java/org/apache/flume/lifecycle/LifecycleSupervisor.java /incubator/flume/trunk/flume-ng-core/src/main/java/org/apache/flume/source/NetcatSource.java /incubator/flume/trunk/flume-ng-node/src/main/java/org/apache/flume/conf/properties/PropertiesFileConfigurationProvider.java /incubator/flume/trunk/flume-ng-node/src/main/java/org/apache/flume/node/nodemanager/DefaultLogicalNodeManager.java
          Hide
          Hari Shreedharan added a comment -

          Bug found

          Show
          Hari Shreedharan added a comment - Bug found
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/4470/
          -----------------------------------------------------------

          Review request for Flume.

          Summary
          -------

          Change caused a bug triggered on reconfiguration. Fixing it.

          This addresses bug FLUME-1036.
          https://issues.apache.org/jira/browse/FLUME-1036

          Diffs


          flume-ng-node/src/main/java/org/apache/flume/node/nodemanager/DefaultLogicalNodeManager.java cdd08e7

          Diff: https://reviews.apache.org/r/4470/diff

          Testing
          -------

          Tested by reconfiguring multiple times. Looks ok.

          Thanks,

          Hari

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4470/ ----------------------------------------------------------- Review request for Flume. Summary ------- Change caused a bug triggered on reconfiguration. Fixing it. This addresses bug FLUME-1036 . https://issues.apache.org/jira/browse/FLUME-1036 Diffs flume-ng-node/src/main/java/org/apache/flume/node/nodemanager/DefaultLogicalNodeManager.java cdd08e7 Diff: https://reviews.apache.org/r/4470/diff Testing ------- Tested by reconfiguring multiple times. Looks ok. Thanks, Hari
          Hide
          Hari Shreedharan added a comment -

          Fixed a bug that caused reconfig to fail

          Show
          Hari Shreedharan added a comment - Fixed a bug that caused reconfig to fail
          Hide
          jiraposter@reviews.apache.org added a comment -

          -----------------------------------------------------------
          This is an automatically generated e-mail. To reply, visit:
          https://reviews.apache.org/r/4470/#review6313
          -----------------------------------------------------------

          Ship it!

          +1

          • Arvind

          On 2012-03-23 22:28:46, Hari Shreedharan wrote:

          -----------------------------------------------------------

          This is an automatically generated e-mail. To reply, visit:

          https://reviews.apache.org/r/4470/

          -----------------------------------------------------------

          (Updated 2012-03-23 22:28:46)

          Review request for Flume.

          Summary

          -------

          Change caused a bug triggered on reconfiguration. Fixing it.

          This addresses bug FLUME-1036.

          https://issues.apache.org/jira/browse/FLUME-1036

          Diffs

          -----

          flume-ng-node/src/main/java/org/apache/flume/node/nodemanager/DefaultLogicalNodeManager.java cdd08e7

          Diff: https://reviews.apache.org/r/4470/diff

          Testing

          -------

          Tested by reconfiguring multiple times. Looks ok.

          Thanks,

          Hari

          Show
          jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4470/#review6313 ----------------------------------------------------------- Ship it! +1 Arvind On 2012-03-23 22:28:46, Hari Shreedharan wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4470/ ----------------------------------------------------------- (Updated 2012-03-23 22:28:46) Review request for Flume. Summary ------- Change caused a bug triggered on reconfiguration. Fixing it. This addresses bug FLUME-1036 . https://issues.apache.org/jira/browse/FLUME-1036 Diffs ----- flume-ng-node/src/main/java/org/apache/flume/node/nodemanager/DefaultLogicalNodeManager.java cdd08e7 Diff: https://reviews.apache.org/r/4470/diff Testing ------- Tested by reconfiguring multiple times. Looks ok. Thanks, Hari
          Hide
          Arvind Prabhakar added a comment -

          Fix patch committed. Thanks Hari for the prompt follow-up and the patch!

          Show
          Arvind Prabhakar added a comment - Fix patch committed. Thanks Hari for the prompt follow-up and the patch!
          Hide
          Hudson added a comment -

          Integrated in flume-trunk #144 (See https://builds.apache.org/job/flume-trunk/144/)
          FLUME-1036. Reconfiguration of AVRO or NETCAT source causes port bind exception.

          (Hari Shreedharan via Arvind Prabhakar) (Revision 1304637)

          Result = SUCCESS
          arvind : http://svn.apache.org/viewvc/?view=rev&rev=1304637
          Files :

          • /incubator/flume/trunk/flume-ng-node/src/main/java/org/apache/flume/node/nodemanager/DefaultLogicalNodeManager.java
          Show
          Hudson added a comment - Integrated in flume-trunk #144 (See https://builds.apache.org/job/flume-trunk/144/ ) FLUME-1036 . Reconfiguration of AVRO or NETCAT source causes port bind exception. (Hari Shreedharan via Arvind Prabhakar) (Revision 1304637) Result = SUCCESS arvind : http://svn.apache.org/viewvc/?view=rev&rev=1304637 Files : /incubator/flume/trunk/flume-ng-node/src/main/java/org/apache/flume/node/nodemanager/DefaultLogicalNodeManager.java

            People

            • Assignee:
              Hari Shreedharan
              Reporter:
              Will McQueen
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development