Uploaded image for project: 'Flume'
  1. Flume
  2. FLUME-2752

Flume AvroSource will leak the memory and the OOM will be happened.

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.6.0
    • Fix Version/s: 1.8.0
    • Component/s: Sinks+Sources
    • Labels:
      None

      Description

      If the flume agent config the nonexist IP for the avro source,the exception will be happened as follow:
      2015-07-21 19:57:47,054 | ERROR | [lifecycleSupervisor-1-2] | Unable to start EventDrivenSourceRunner: { source:Avro source avro_source_21155:

      { bindAddress: 51.196.27.32, port: 21155 }

      } - Exception follows. | org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:253)
      org.jboss.netty.channel.ChannelException: Failed to bind to: /51.196.27.32:21155
      at org.jboss.netty.bootstrap.ServerBootstrap.bind(ServerBootstrap.java:297)
      at org.apache.avro.ipc.NettyServer.<init>(NettyServer.java:106)
      at org.apache.flume.source.AvroSource.start(AvroSource.java:294)
      at org.apache.flume.source.EventDrivenSourceRunner.start(EventDrivenSourceRunner.java:44)
      at org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:251)
      at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
      at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
      at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
      at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
      at java.lang.Thread.run(Thread.java:745)
      Caused by: java.net.BindException: Cannot assign requested address
      at sun.nio.ch.Net.bind0(Native Method)
      at sun.nio.ch.Net.bind(Net.java:437)
      at sun.nio.ch.Net.bind(Net.java:429)
      at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223)
      at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
      at org.jboss.netty.channel.socket.nio.NioServerSocketPipelineSink.bind(NioServerSocketPipelineSink.java:140)
      at org.jboss.netty.channel.socket.nio.NioServerSocketPipelineSink.handleServerSocket(NioServerSocketPipelineSink.java:90)
      at org.jboss.netty.channel.socket.nio.NioServerSocketPipelineSink.eventSunk(NioServerSocketPipelineSink.java:64)
      at org.jboss.netty.channel.Channels.bind(Channels.java:569)
      at org.jboss.netty.channel.AbstractChannel.bind(AbstractChannel.java:189)
      at org.jboss.netty.bootstrap.ServerBootstrap$Binder.channelOpen(ServerBootstrap.java:342)
      at org.jboss.netty.channel.Channels.fireChannelOpen(Channels.java:170)
      at org.jboss.netty.channel.socket.nio.NioServerSocketChannel.<init>(NioServerSocketChannel.java:80)
      at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.newChannel(NioServerSocketChannelFactory.java:158)
      at org.jboss.netty.channel.socket.nio.NioServerSocketChannelFactory.newChannel(NioServerSocketChannelFactory.java:86)
      at org.jboss.netty.bootstrap.ServerBootstrap.bind(ServerBootstrap.java:276)

      if the above exception happened for 2 hours,and the agent JVM -Xxx is 4G,the OutOfMemory will be happened.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                sati Attila Simon
                Reporter:
                yinghua_zh yinghua_zh
              • Votes:
                0 Vote for this issue
                Watchers:
                7 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: