[DIRAPI-52] Connection seems to remain opened, leading to 'Too many open files' on some servers

• Type: Bug
• Status: Closed
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 1.0.0-M4, 1.0.0-M5
• Fix Version/s: 1.0.0-M6
• Environment: various, see description
• Assignee: Unassigned
• Reporter: Martin Zdila
• Created: 27/Jul/11 11:38
• Updated: 20/Jan/12 08:16
• Resolved: 27/Jul/11 16:32

Description

Testcase:

final LdapConnectionConfig lcc = new LdapConnectionConfig();
lcc.setLdapHost("ad.example.com");
lcc.setUseSsl(true);
lcc.setLdapPort(LdapConnectionConfig.DEFAULT_LDAPS_PORT);
lcc.setTrustManagers(new NoVerificationTrustManager());

for (int i = 0; i < 100; i++) {
System.out.println;
final LdapNetworkConnection connection = new LdapNetworkConnection(lcc);
connection.bind();
final EntryCursor cursor = connection.search("ou=something,dc=example.com", "(&(cn=comeCommonName)(objectClass=person)(sn=)(givenName=))", SearchScope.SUBTREE, "cn");
while (cursor.next())

{ cursor.get(); }

connection.unBind();
connection.close();
Thread.sleep(1000);
}

Running this on some servers causes many open files and they are growing on each iteration:
lsof -p <pid>:
...
java 18719 gofer 37u 0000 0,7 0 13 anon_inode
java 18719 gofer 38r FIFO 0,6 0t0 9155974 pipe
java 18719 gofer 39w FIFO 0,6 0t0 9155974 pipe
java 18719 gofer 40u 0000 0,7 0 13 anon_inode
java 18719 gofer 41r FIFO 0,6 0t0 9155976 pipe
java 18719 gofer 42w FIFO 0,6 0t0 9155976 pipe
java 18719 gofer 43u 0000 0,7 0 13 anon_inode
java 18719 gofer 44r FIFO 0,6 0t0 9155977 pipe
java 18719 gofer 45w FIFO 0,6 0t0 9155977 pipe
java 18719 gofer 46u 0000 0,7 0 13 anon_inode
java 18719 gofer 47r FIFO 0,6 0t0 9155978 pipe
java 18719 gofer 48w FIFO 0,6 0t0 9155978 pipe
java 18719 gofer 49u 0000 0,7 0 13 anon_inode
java 18719 gofer 50r FIFO 0,6 0t0 9155980 pipe
java 18719 gofer 51w FIFO 0,6 0t0 9155980 pipe
java 18719 gofer 52u 0000 0,7 0 13 anon_inode
java 18719 gofer 53r FIFO 0,6 0t0 9155981 pipe
java 18719 gofer 54w FIFO 0,6 0t0 9155981 pipe
java 18719 gofer 55u 0000 0,7 0 13 anon_inode
java 18719 gofer 56r FIFO 0,6 0t0 9155982 pipe
java 18719 gofer 57w FIFO 0,6 0t0 9155982 pipe
java 18719 gofer 58u 0000 0,7 0 13 anon_inode
java 18719 gofer 59r FIFO 0,6 0t0 9155984 pipe
java 18719 gofer 60w FIFO 0,6 0t0 9155984 pipe
java 18719 gofer 61u 0000 0,7 0 13 anon_inode
java 18719 gofer 62r FIFO 0,6 0t0 9155985 pipe
java 18719 gofer 63w FIFO 0,6 0t0 9155985 pipe
java 18719 gofer 64u 0000 0,7 0 13 anon_inode
java 18719 gofer 65r FIFO 0,6 0t0 9155986 pipe
java 18719 gofer 66w FIFO 0,6 0t0 9155986 pipe
... and many many more

This happens on:
Linux somedns1 2.6.26-2-xen-686 #1 SMP Thu Jan 27 05:44:37 UTC 2011 i686 GNU/Linux
java version "1.6.0_26"
Java(TM) SE Runtime Environment (build 1.6.0_26-b03)
Java HotSpot(TM) Client VM (build 20.1-b02, mixed mode, sharing)

Also on another platform, but there are slightly less open pipes and anon_inodes:

Linux somedns2 2.6.26-2-686-bigmem #1 SMP Wed Nov 4 21:12:12 UTC 2009 i686 GNU/Linux
java version "1.6.0_12"
Java(TM) SE Runtime Environment (build 1.6.0_12-b04)
Java HotSpot(TM) Server VM (build 11.2-b01, mixed mode)

On my local computer I see only couple of open pipes and anon_inodes and they don't grow:
Linux bono 2.6.38-10-generic #46-Ubuntu SMP Tue Jun 28 15:07:17 UTC 2011 x86_64 x86_64 x86_64 GNU/Linux
java version "1.6.0_26"
Java(TM) SE Runtime Environment (build 1.6.0_26-b03)
Java HotSpot(TM) 64-Bit Server VM (build 20.1-b02, mixed mode)

I've found slightly similar isse here: http://issues.opennms.org/browse/NMS-4631
New file handlers are being opened by Apache Mina. I am not sure who is responsible for closing it. In my testcase the connection is properly closed.