Launching ADS when it is full of entry is very long

• Key: DIRSERVER-183
• Type: Bug
• Status: Closed
• Resolution: Fixed
• Priority: Minor
• Assignee: Unassigned
• Reporter: Emmanuel Lecharny
• Fix Version/s: 1.0-RC1

Description

When you launch ApacheDS after having loaded it with thousands of entries, it takes a while before it can be accessed.

It seems that all the entries are fetched at launch (all the DN are parsed).

That does not seems to be reasonnable, as if we have more than a few thousands entries, it can take ages to start !

for instance, it takes 21 secondes to start my server with 8155 entries.

Comments

[Alex Karasulu]

This is very serious!

[Trustin Lee]

Emmanuel, can you attach a YourKit profiler data?

[Emmanuel Lecharny]

This is due to the method
org.apache.ldap.server.DefaultDirectoryService.createTestEntries()

I don't get the point raised by the comment :

        /*
         * Unfortunately to test non-root user startup of the core and make sure
         * all the appropriate functionality is there we need to load more user
         * entries at startup due to a chicken and egg like problem. The value
         * of this property is a list of attributes to be added.
         */

but it seems that the solution is not the best one.

[Emmanuel Lecharny]

The previous analyze was plain wrong.

The responsible is org.apache.ldap.server.authz.TupleCache.initialize() which loops through all the entries :

...
            Name baseDn = new LdapName( suffix );
            ExprNode filter = new SimpleNode( OC_ATTR, ACSUBENTRY_OC, SimpleNode.EQUALITY );
            SearchControls ctls = new SearchControls();
            ctls.setSearchScope( SearchControls.SUBTREE_SCOPE );
            NamingEnumeration results = nexus.search( baseDn, env, filter, ctls );
            while ( results.hasMore() )
            {
...

This is obviously not a good solution. The TupleCache might be built on the fly, or be serialized when quitting the server correctly.

There is a bigger problem here which I will describe after this ...

This code works off of an index so it is fine. It will simply search the server for all (objectClass=accessControlSubentry). This will not or should not search everything if there is an index built on objectClass. (We should make sure this is always present otherwise we do a full scan). I kind of presume this index is always there which is not good to do but it is natural I think to conduct these kinds of searches.

Do a little experiment. Delete the db first, add the objectClass index, then add all your entries. Test startup time again. It should be much faster. I think we need to make sure objectClass is always indexed.

Ok the bigger problem is still the fact that we do full index scans on DNs which uses a DN parser to do comparisons. I'll have to look into this one carefull.

Please don't make any presumptions to change code in the trunk at this point in time (JUST WITH THIS HAIRY PROBLEM). There is a lot of undocumented functionality here (my fault) but if you do make changes do it in a branch and let's review it together. Until we have regression tests in place we need to be extra careful.

Thanks!

[Emmanuel Lecharny]

Ok, the ObjectClass index didn't exist.

I created it, and the server was launched very quickly.

So we can change this issue to Improvment : the server should *always* create an index for ObjectClass, and it should not be necessary to declare this attribute in the configuration file.

[Emmanuel Lecharny]

Not a blocker any more

[Alex Karasulu]

Added objectClass index to be present by default ... still need ot make it a specialized system index though but this is another issue.

[Emmanuel Lecharny]

closed