Incorrect SearchResult name and "compare" failure using CoreContextFactory

• Key: DIRSERVER-169
• Type: Bug
• Status: Closed
• Resolution: Later
• Priority: Major
• Assignee: Alex Karasulu
• Reporter: Luke Taylor
• Component/s: core
• Affects Version/s: 1.5.0, 1.0.2, 1.0.1, 1.0, 1.0-RC4, 1.0-RC3, 1.0-RC2, 1.0-RC1, pre-1.0
• Fix Version/s: 1.5.1

Environment

OS X,
java version "1.5.0_05"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_05-83)
Java HotSpot(TM) Client VM (build 1.5.0_05-48, mixed mode, sharing)

Description

Attached is a test case following on from my post a while back to the mailing list, viz:

My setup is like this:

I have a simple DIT with a root "dc=acegisecurity,dc=org". This has two subcontexts "ou=people" and "ou=groups" for my users and roles respectively. When the test base class instantiated, I create a
MutableStartupConfiguration and add a partition to it with the suffix "dc=acegisecurity,dc=org". I then create a context with this configuration as follows:

    env.setProperty( Context.PROVIDER_URL, "dc=acegisecurity,dc=org" );
    env.setProperty( Context.INITIAL_CONTEXT_FACTORY,
             CoreContextFactory.class.getName());
    env.putAll( cfg.toJndiEnvironment() );

    serverContext = new InitialDirContext( env );

When I need a context in my tests it is created the same way.

Bind authentication works fine in both scenarios. I have problems with two things when trying to use CoreContextFactory :

1. The name returned by a search. When I do a search for a user in the directory, I get back the full DN rather than the name relative to the context I search in. So if I call

   ctx.search("ou=people", "(uid={0})", new String[] {"bob"}, ctls);

on a context obtained as above, I get back a SearchResult with name

"uid=bob,ou=people,dc=acegisecurity,dc=org"

whereas with the full server (or OpenLDAP) I get

"uid=bob"

as expected. This then unfortunately leads to an attempt to bind with an an unknown DN which causes the infinite recursion problem.

2. Performing "compare" operations. I had problems with this before, as reported in

http://issues.apache.org/jira/browse/DIRLDAP-77

but this now works with the full server, thanks to Emmanuel's speedy response. Running the same search code against a context obtained from CoreContextFactory fails however. A compare is never performed and the search returns an empty enumeration. Is there some way I can get my client code (as posted in JIRA):

     SearchControls ctls = new SearchControls();
     ctls.setReturningAttributes(new String[0]);
     ctls.setSearchScope(SearchControls.OBJECT_SCOPE);

     String filter = "(userPassword={0})";
     NamingEnumeration results = ctx.search(dn, filter, new
           Object[]{password.getBytes()}, ctls);

to trigger a compare call on the context? The compare/search also fails for non-binary attributes.

Comments

[Emmanuel Lecharny]

Hi Luke,

The first point seems to be just the normal behavior of the SearchResult.getName() method. You have a method isRelative() that should tell you if the getName() returns a name relative to the current context, or not.

This is a point that should be investigated a littlmle bit more.

I did not had time to dig yur second point.

[Luke Taylor]

Hi Emmanuel,

Thanks for the response. My understanding of isRelative() is that it should return true unless the name in the result is from a referral and is a full URL, otherwise it
should be relative to the "target context" (the context the search is performed in). There is some info on this in the Java tutorial:

http://java.sun.com/products/jndi/tutorial/ldap/search/result.html

Though that may not be correct :).

In any case, the behaviour doesn't appear to be consistent. If I modify the test to use

        if(searchResult.isRelative()) {
            if(searchBase.length() > 0) {
                userDn.append(",");
                userDn.append(searchBase);
            }

            userDn.append(",");
            userDn.append(ctx.getNameInNamespace());
        }


The isRelative() method returns "true". In this case the name should be relative to "ou=people,dc=acegisecurity,dc=org", since I'm passing "ou=people" to the search method. However the name of the search result is "uid=bob,ou=people,dc=acegisecurity,dc=org".

[Emmanuel Lecharny]

Luke,

you are totally right. Excerpt from the tutorial :

"If you need to get the entry's full DN, then you can either do some bookkeeping to keep track of the ancestors of the SearchResult or use Context.getNameInNamespace()(in the API reference documentation)."

So I bet we will have to propose a fix for this, too :(

Ok, don't worry. We will try to debug this and give you a reasonnable timeframe for a working patch. Let say a few days ;)

[Luke Taylor]

Thanks a lot Emmanuel. Let me know if I can do anything to help.

Anyone know if these points are still an issue in the latest codebase? Still waiting for Emmanuel's patch timeframe estimate ;-) .

[Emmanuel Lecharny]

No time frame available right now :(

We are busy on very time consuming tasks atm, and it's very difficult for us to find some spare time out of our spare time we spend on ADS:)

However, I think that as soon as we have a fix for the concurrence problem we have on ADS, we will be able to spend some more extra time on a few bugs in the bug list, like your one. Another possibility is that we just get fed of working hard on hard stuff, and try to clear our mind buffers by fixing your problem, just to feel that we are usefull :)

Don't be afraid to do some polling on us, that's just cool.

[Luke Taylor]

Thanks for the update :). I'll try and have a browse through the latest source sometime and check the tests are still applicable.

[Alex Karasulu]

Is there any way I can get a test case that is a patch which isolates this problem without all the ACEGI specifics? Any status on this issue?

[Luke Taylor]

I don't think there are any Acegi specifics, apart from the DIT name. There's a JUnit class which contains two test methods, demonstrating each of the two issues, and a separate class which contains the embedded server. Did you have a problem running it? I haven't checked that it still compiles against the latest apache-ds code. The report was pre-1.0, but we're currently using 1.0-RC1 as a dependency and still have problems with that.

[Luke Taylor]

I've attached the latest version of the LdapTestServer we are running against apache-ds-rc1, in case the API has changed.

I'll see if I can update our dependencies to the latest version and try runnng against that. Let me know if there's anything else I can do to help.

[Alex Karasulu]

Luke if you could write a unit test case which starts ApacheDS and does some operations against it to exactly isolate your problem I can get you a fix quickly. Take a look at how to write a unit test case from here as an example:

Here's one that does not startup ApacheDS networking but uses the core JNDI provider directly:

http://svn.apache.org/viewvc/directory/branches/apacheds/1.0/core-unit/src/test/java/org/apache/directory/server/core/jndi/CreateContextITest.java?revision=428079&view=markup

Here's a test case that starts up networking so you can hit the embedded apacheds through SUN JNDI:

http://svn.apache.org/viewvc/directory/branches/apacheds/1.0/server-unit/src/test/java/org/apache/directory/server/ModifyRemoveTest.java?revision=430261&view=markup

If you could be so kind as to set up a test case in either of these areas depending on whether or not you use ApacheDS JNDI or SUN JNDI providers I can fix this issue of yours very quickly. Plus I can add the test case to our codebase if you attach it as a patch (JIRA attachment) using svn diff.

And yes if you worked against the current 1.0-RC4-SNAPSHOT'd branch that would be best for us. I want to make sure we're up to date and kill this bug of yours.

Thanks very much!

[Luke Taylor]

Ok. Here's an updated test.

I can't build the RC4 code since I'm running into issues with tools.jar as a dependency (a maven problem with OS X, I believe), but I ran it against the RC3 jars.

The password comparison now appears to work if a string is used for the attribute, but fails for a byte array.

The SearchResult name still appears to be wrong.

Thanks,

Luke.

[Alex Karasulu]

On commit revision 437255 I made it so SearchResult.isRelative() returns the correct value. This will allow you to detect whether or not the search results returned are relative names or absolute names. The core JNDI provider will always return absolute names. Here's a link to svn for your convenience:

    http://svn.apache.org/viewvc?view=rev&revision=437255

This fixes 1/2 of your issue. The second problem you're facing is the same as DIRSERVER-715. Basically binary attributes in search filters are causing big problems within ApacheDS. I'm going to try to fix this tomorrow night with Emmanuel. Thanks for the solid test case Luke we really appreciate it.

[Alex Karasulu]

Binary attributes in search filters are causing issues in ApacheDS. This problem is what makes "(userPassword={0})" fail as well as "(userCertificate={0})" etc fail. Fixing the binary attribute problem in search filters will fix both DIRSERVER-169 and DIRSERVER-715.

[Alex Karasulu]

For 1.0 branch:

    http://svn.apache.org/viewvc?view=rev&revision=437312 for shared LDAP
    http://svn.apache.org/viewvc?view=rev&revision=437314 for apacheds

For 1.1 branch:

    http://svn.apache.org/viewvc?view=rev&revision=437313 for shared LDAP
    http://svn.apache.org/viewvc?view=rev&revision=437315 for apacheds

[Luke Taylor]

Great! Thanks for the fix(es). Look forward to the next release.

[Luke Taylor]

"On commit revision 437255 I made it so SearchResult.isRelative() returns the correct value. This will allow you to detect whether or not the search results returned are relative names or absolute names. The core JNDI provider will always return absolute names."

This still seems potentially inconsistent with the discussion above and the quotations from the Java tutorial - that the search result should be relative to the target context the search is performed in and isRelative() should return true unless the name is a URL.

[Emmanuel Lecharny]

Reopened to be sure we have investigated all the consequences of the fix.

[Emmanuel Lecharny]

Has to be clearly fixed for 1.0.1 and 1.5.0

[Emmanuel Lecharny]

From JNDI tutorial :

"LDAP names as they are used in the protocol are always fully qualified names that identify entries that start from the root of the LDAP namespace (as defined by the server). Following are some examples of fully qualified LDAP names.

    cn=John Smith, ou=Marketing, o=Some Corporation, c=gb
    cn=Ted Geisel, ou=People, o=JNDITutorial

In the JNDI, however, names are always relative; that is, you always name an object relative to a context. For example, you can name the entry "cn=Ted Geisel" relative to the context named "ou=People, o=JNDITutorial". Or you can name the entry "cn=Ted Geisel, ou=People" relative to the context named "o=JNDITutorial". Or, you can create an initial context that points at the root of the LDAP server's namespace and name the entry "cn=Ted Geisel, ou=People, o=JNDITutorial". "

I guess that if someone is using JNDI to use a Ldap Server, then it's the responsibility of the JNDI layer to strip the absolute part before returning the response.

We have to check that our implementation of JNDI does respect this contract.

[Emmanuel Lecharny]

Ok, we need to take a decision about this issue.

First, the second part of the issue (binary value used as a filter) seems to be fixed. We have a unit test for it : http://svn.apache.org/viewvc/directory/branches/apacheds/1.0/core-unit/src/test/java/org/apache/directory/server/core/jndi/DIRSERVER169ITest.java?view=markup&pathrev=437314

On the other issue, I do think that JNDI approach might not fit our need. With Alex patch, we can just check if the returned entry name is absolute and relative, so I guess we are ok. In my mind, returning a relative name (which is, in fact, the DN) seems a nonsense : to get back the DN, you will have to concatenate the context path to the relative returned result. Why do a user should bother about it? Anyway, this is JNDI spec ...

In my mind, we should just let it be. Sorry or the spec breakage, but it's quite a reasonnable position regarding LDAP specs - we are not really writing a JNDI implementation, but a LDAP server.

wdyt ?

[Alex Karasulu]

Emmanuel I agree with you on this one but I want to find a reasonable solution for
Luke that fits the expected behavior recommended by JNDI.

However there are some trade off with doing this when dealing with the LDAP
service that sits on top of the server-side JNDI LDAP provider. Namely for efficiency
the LDAP service uses absolute names in the search results to minimize the overhead
of tracking separate contexts and having to assemble the DN. The LDAP service uses
the root context for all operations so DNs and not relative names can be used for search
requests.

Now Luke is using the embedded provider and his issue is with this behavior. What
we can do is make the embedded provider follow the spec yet have the LDAP server
provide an additional environment parameter to always request the absolute DN. However
I don't want to try this without some good amount of testing and we don't have time for
1.0.1. What I will do is put this issue off for 1.0.2 and we can really try to find a reasonable
approach that will make everyone happy.

[Alex Karasulu]

Pushing a fix for this to 1.0.2 and 1.5.1.

[Alex Karasulu]

I don't think it's worth making this change for 1.0.x considering the possible destablization that may result. It's best that we fix this issue in the 1.5.x branch where we have the freedom to refactor for it.

[Emmanuel Lecharny]

Should we postpone again ?