Improvment in the way DN are treated

• Key: DIRSERVER-600
• Type: Improvement
• Status: Closed
• Resolution: Fixed
• Priority: Major
• Assignee: Unassigned
• Reporter: Emmanuel Lecharny

Description

We have mesured that DN parsing is the hot spot in the server : we spend around 80 % of server time to parse DNs.

After having studied the way BindRequest does handle DN, we can say that for a single bind for the user "uid=admin,ou=system", we have more than 40 parsing done. Here is a list of all the parsing in the DnComparator.compare() method :
(The '*' in front of each line shows where a parsing of both DN is done)
* Compare 'ou=groups,ou=system' and 'uid=admin,ou=system'
* Compare 'prefnodename=sysPrefRoot,ou=system' and 'uid=admin,ou=system'
Compare 'uid=admin,ou=system' and 'uid=admin,ou=system'
Compare 'uid=admin,ou=system' and 'uid=admin,ou=system'
* Compare 'ou=groups,ou=system' and 'uid=admin,ou=system'
* Compare 'prefnodename=sysPrefRoot,ou=system' and 'uid=admin,ou=system'
Compare 'uid=admin,ou=system' and 'uid=admin,ou=system'
Compare 'uid=admin,ou=system' and 'uid=admin,ou=system'
* Compare 'ou=groups,ou=system' and 'ou=system'
* Compare 'ou=interceptors,ou=configuration,ou=system' and 'ou=system'
* Compare 'ou=configuration,ou=system' and 'ou=system'
Compare 'ou=system' and 'ou=system'
Compare 'ou=system' and 'ou=system'
* Compare 'ou=groups,ou=system' and 'ou=system'
* Compare 'ou=interceptors,ou=configuration,ou=system' and 'ou=system'
* Compare 'ou=configuration,ou=system' and 'ou=system'
Compare 'ou=system' and 'ou=system'
Compare 'ou=system' and 'ou=system'
* Compare 'ou=groups,ou=system' and 'ou=system'
* Compare 'ou=interceptors,ou=configuration,ou=system' and 'ou=system'
* Compare 'ou=configuration,ou=system' and 'ou=system'
Compare 'ou=system' and 'ou=system'
Compare 'ou=system' and 'ou=system'
* Compare 'ou=groups,ou=system' and 'ou=system'
* Compare 'ou=interceptors,ou=configuration,ou=system' and 'ou=system'
* Compare 'ou=configuration,ou=system' and 'ou=system'
Compare 'ou=system' and 'ou=system'
Compare 'ou=system' and 'ou=system'
* Compare 'ou=groups,ou=system' and 'ou=system'
* Compare 'ou=interceptors,ou=configuration,ou=system' and 'ou=system'
* Compare 'ou=configuration,ou=system' and 'ou=system'
Compare 'ou=system' and 'ou=system'

If we can change this compare function to work with Name instead of String we could have a huge improvment : more that 35x faster
(I wrote a micro-bench to mesure the difference : around 36 times faster).

As this is not the single place where we can gain time (we have a lot or String -> DN and DN -> String conversions during the Bind process), we would be able to handle Bind Request much much faster, too (being aware that we have a lot of other costly operaions that will nor allow a x40 improvment).

My personnal guess is that we should be able to handle around 5000 binds per second against the current 300, but this is just a guess :)

Comments

[Emmanuel Lecharny]

A better solution would be to do a String comparizon into the DnComparator. It makes sense as the DN has already been normalized before being stored into the backend.

The performance increase is dramatic : for a bind request, the comparizon cost will be done around 670 times faster !

This approach has been already partially implemented month ago and is effective when two DN are equals : the server speed has been increased and is now 4 times faster than it was. In the previous list, all the DN that does not have a '*' are compared as strings. We just have to do the same thing for other elements.

[Emmanuel Lecharny]

To be checked, but replacing :
...
            if ( ( ( String ) obj1 ).compareTo( ( String ) obj2 ) == 0 )
            {
                return 0;
            }
            else
            {
                try
                {
                    synchronized ( parserMutex )
                    {
                        dn1 = parser.parse( ( String ) obj1 );
                    }
                }
                catch ( NamingException ne )
                {
                    throw new IllegalArgumentException( "first argument (" + obj1 + ") was not a distinguished name" );
                }
            }
...


by

...
            return -( ( String ) obj1 ).compareTo( ( String ) obj2 );
...


in org.apache.directory.shared.ldap.schema.DnComparator.compare( Object o1, Object o2) method seems to be enough to really boost ApachedDS performance ! (Note the negation in front of the string comparizon, because DN are compared in the reverse order)

It has te be seriously checked though...

Has been modified and fixed a while ago !