Details
-
Bug
-
Status: Open
-
Major
-
Resolution: Unresolved
-
1.7.5, 1.7.6
-
None
-
None
-
openSuse 13.1 64 bit, jdk1.7.0_51 64 bit
Description
NettyTransceiver created without channel factory results in file descriptors leak while connecting with IOException (java.io.IOException: Error connecting to, NettyTransceiver.java:280).
It seems very similar to https://issues.apache.org/jira/browse/FLUME-2221. But I am not flume user.
To reproduce - run the test below (IP:9999 was nobody listened.)
In my enviroment (run from IDEA, openSuse 13.1 64bit, all system settings are by default) stable fails after 146 cycles (count = 146).
@Test public void testAvroNettyTest() { for (int count = 0; count < 4500; count++) { try { NettyTransceiver apiClient = new NettyTransceiver(new InetSocketAddress("<IP>", 9999), 5000L); } catch (Exception exc) { System.out.println(String.format("=========================== %d ===========================", count)); exc.printStackTrace(); if ( exc.getMessage().contains("Too many open files") || exc.getCause().getMessage().contains("Too many open files") ) { System.out.println("Found [Too many open files]."); break; } } } }
And test that OK
@Test public void testAvroCustomNettyTest() throws Exception { for (int count = 0; count < 4500; count++) { NioClientSocketChannelFactory channelFactory = null; ExecutorService bossExecutor = null, workerExecutor = null; try { bossExecutor = Executors.newCachedThreadPool(new NettyTransceiverThreadFactory( "Avro " + NettyTransceiver.class.getSimpleName() + " Boss")); workerExecutor = Executors.newCachedThreadPool(new NettyTransceiverThreadFactory( "Avro " + NettyTransceiver.class.getSimpleName() + " I/O Worker")); channelFactory = new NioClientSocketChannelFactory( bossExecutor, workerExecutor ); NettyTransceiver apiClient = new NettyTransceiver( new InetSocketAddress("<IP>", 9999), channelFactory, 5000L); } catch (Exception exc) { System.out.println(String.format("=========================== %d ===========================", count)); exc.printStackTrace(); if ( channelFactory != null ) { channelFactory.shutdown(); } if ( exc.getMessage().contains("Too many open files") || exc.getCause().getMessage().contains("Too many open files") ) { System.out.println("Found [Too many open files]."); break; } } } }
So workaround exists it would be cool if NettyTransceiver in next release will be more "friendly" (release all internal resources if initialization failed).