Details
-
Improvement
-
Status: Closed
-
Minor
-
Resolution: Cannot Reproduce
-
1.0.0-RC42
-
None
-
None
Description
Problem
The first in the day to log into fortress-commander or FM will experience a waiting time up to five minutes
Cause
LDAP connections are created in a pool. After a certain idle time they are closed from the server (ApacheDS) unoticed by the client (the connections in the pool). When a stale connection is tried used, it has a 30 second wait until it is determined that it is stale. The pool then tries the next connection until all connections has been tried. New connections are only inititated when all connections in the pool have been tried and timed out.
The test case to repeat the issue, roughly;
Set up a commander connected to ApacheDS. Log in and refresh a lot to activate a couple of connections. Log out and let it run over night. When you log in the next day, you will see the delay.
Solution
The pools are initiated in
org.apache.directory.fortress.core.ldap.ApacheDsDataProvider
We can take a look at which configuration options there are for connections and pools.
We have applied the following in ApacheDsDataProvider:
adminPool.setTestWhileIdle(true);
adminPool.setTimeBetweenEvictionRunsMillis(1000 * 60 * 30);
...
userPool.setTestWhileIdle(true);
userPool.setTimeBetweenEvictionRunsMillis(1000*60*30);
...
logPool.setTestWhileIdle(true);
logPool.setTimeBetweenEvictionRunsMillis(1000 * 60 * 30);
Every 30 minutes this will test three (3 is default) idle threads and evict them if they fail an attempt of a simple lookup on the rootDSE.
I have not yet established how long time it takes before a connection is stale
I hope 30 minutes is enough - I suspect that the test will keep the connections alive if the time is too short.
NOTE
A patch is appended with the above solution. We actively use it in production and all seems happy about it => meaning that empirical test seems to indicate that the patch is a suitable fix with no perceived downsides.
Earlier analysis, theory
I guess that it should be fixed in
org.apache.directory.fortress.core.ldap.ApacheDsDataProvider ?
It uses
org.apache.directory.ldap.client.api.LdapConnectionConfig where a timeout can be set.
We also use
https://commons.apache.org/proper/commons-pool/api-1.6/org/apache/commons/pool/impl/GenericObjectPool.html
We currently sets maxIdle to infinite
userPool.setMaxIdle( -1 );
Setting a default as a percent of max connections could help?
We can take a look at testOnBorrow :
"When testOnBorrow is set, the pool will attempt to validate each object before it is returned from the borrowObject() method. (Using the provided factory's PoolableObjectFactory.validateObject(T) method.) Objects that fail to validate will be dropped from the pool, and a different object will be borrowed. The default setting for this parameter is false."
Optionally, one may configure the pool to examine and possibly evict objects as they sit idle in the pool and to ensure that a minimum number of idle objects are available. This is performed by an "idle object eviction" thread, which runs asynchronously. Caution should be used when configuring this optional feature. Eviction runs contend with client threads for access to objects in the pool, so if they run too frequently performance issues may result. The idle object eviction thread may be configured using the following attributes:
- timeBetweenEvictionRunsMillis indicates how long the eviction thread should sleep before "runs" of examining idle objects. When non-positive, no eviction thread will be launched. The default setting for this parameter is -1 (i.e., idle object eviction is disabled by default).
- minEvictableIdleTimeMillis specifies the minimum amount of time that an object may sit idle in the pool before it is eligible for eviction due to idle time. When non-positive, no object will be dropped from the pool due to idle time alone. This setting has no effect unless timeBetweenEvictionRunsMillis > 0. The default setting for this parameter is 30 minutes.
- testWhileIdle indicates whether or not idle objects should be validated using the factory's PoolableObjectFactory.validateObject(T) method. Objects that fail to validate will be dropped from the pool. This setting has no effect unless timeBetweenEvictionRunsMillis > 0. The default setting for this parameter is false.
- softMinEvictableIdleTimeMillis specifies the minimum amount of time an object may sit idle in the pool before it is eligible for eviction by the idle object evictor (if any), with the extra condition that at least "minIdle" object instances remain in the pool. When non-positive, no objects will be evicted from the pool due to idle time alone. This setting has no effect unless timeBetweenEvictionRunsMillis > 0. and it is superceded by minEvictableIdleTimeMillis (that is, if minEvictableIdleTimeMillis is positive, then softMinEvictableIdleTimeMillis is ignored). The default setting for this parameter is -1 (disabled).
- numTestsPerEvictionRun determines the number of objects examined in each run of the idle object evictor. This setting has no effect unless timeBetweenEvictionRunsMillis > 0. The default setting for this parameter is 3.
The LdapConnection has a method "isConnected()" - maybe that could be used in testOnBorrow, if it does as hoped for.
https://directory.apache.org/api/gen-docs/latest/apidocs/org/apache/directory/ldap/client/api/LdapConnection
The LdapConnection also has a method setTimeOut - timeout for responses - which might or might not have an effect.
Some test using isConnected() are already implemented in
Org.apache.directory.ldap.client.api.ValidatingPoolableLdapConnectionFactory.
We can add a validator with setValidator - it seem that there already is a default LookupLdapConnectionValidator set if just validation has been configured:
public void setValidator(LdapConnectionValidator validator)
Sets the validator to use when validation occurs. Note that validation will only occur if the connection pool was configured to validate. This means one of:
- setTestOnBorrow
- setTestWhileIdle
- setTestOnReturn
must have been set to true on the pool. The default validator is LookupLdapConnectionValidator.
LdapConnectionValidator
- An LdapConnection validator intended to be used by a GenericObjectPool to determine whether or not a conneciton is still usable.
There are already two validators available:
DefaultLdapConnectionValidator
- An implementation of LdapConnectionValidator that checks to see that the connection isConnected() and isAuthenticated().
LookupLdapConnectionValidator (The default) - An implementation of LdapConnectionValidator that attempts a simple lookup on the rootDSE.
Summary:
If we configure "testWhileIdle" and " timeBetweenEvictionRunsMillis" it should activate the LookupLdapConnectionValidator and reduce the issue. We should as well consider the other pool-settings for appropriate defaults.
Attachments
Attachments
Issue Links
- is duplicated by
-
-
- Resolved
-