escaping problem with custom partition search results

• Key: DIRSERVER-374
• Type: Bug
• Status: Closed
• Resolution: Invalid
• Priority: Major
• Assignee: Alex Karasulu
• Reporter: Norval Hope
• Component/s: core

Description

I have observed a strange problem in implementing a custom partition that proxies to another remote LDAP server: the results of search() operations have blanks replaced with "%20" so that JXplorer is unable to explore them. The temporary solution I have in place is to wrap the original search results returned by the remote server using the following class:

============================
    /**
     * ApacheDS seems to have a bug where SearchResult s with relative DNs
     * have URL encoding applied twice, so blanks come out as %20.
     */
    public static final class AvoidEscapingNamingEnumeration
            implements NamingEnumeration
    {
        private final String baseDN;
        private final NamingEnumeration ne;

        public AvoidEscapingNamingEnumeration(final String baseDN,
                final NamingEnumeration ne)
        {
            this.baseDN = baseDN;
            this.ne = ne;
        }

        public void close() throws NamingException
        {
            ne.close();
        }

        public boolean hasMore() throws NamingException
        {
            return ne.hasMore();
        }

        public Object next() throws NamingException
        {
            final SearchResult sr = (SearchResult)ne.next();
            final String fullDN;
            final SearchResult sr2;
            final String name = sr.getName();

            if (!sr.isRelative() || (name == null) || "".equals(name))
                return sr;
            fullDN = name + "," + baseDN;
            sr.setName(fullDN);
            sr.setRelative(false);
            return sr;
        }

        public boolean hasMoreElements()
        {
            try
            {
                return hasMore();
            }
            catch (NamingException e)
            {
                log.error(this.getClass().getName()
                        + ": error in hasMoreElements", e);
                return false;
            }
        }

        public Object nextElement()
        {
            try
            {
                return next();
            }
            catch (NamingException e)
            {
                log.error(this.getClass().getName()
                        + ": error in nextElement", e);
                return null;
            }
        }
    }
==========================

where the search method itself looks like this:


==========================
    public NamingEnumeration search(Name base, final Map env,
            final ExprNode filter, final SearchControls searchControls)
            throws NamingException
    {
        final String deref = (String)env.get("java.naming.ldap.derefAliases");
        final int scope = searchControls.getSearchScope();
        String attrIds[] = searchControls.getReturningAttributes();
        final String newFilter;
        final StringBuffer sb;
        final String baseDn;
        final String[] attrNames;
        final String last;

        if (attrIds == null)
            attrIds = BLANK_ATTRS;
        sb = new StringBuffer();
        filter.printToBuffer(sb);
        newFilter = sb.toString();
        baseDn = base.toString();

        last = base.get(0);
        if (! "dc=etadb".equals(last))
        {
                // don't want to change name seen by outside world
            base = (Name)base.clone();
            base.add("dc=etadb");
        }

        attrNames = normaliseAttrNames(attrIds);
        final SearchControls sc = new SearchControls();
        sc.setSearchScope(scope);
        sc.setReturningAttributes(attrNames);
           sc.setDerefLinkFlag(Boolean.valueOf(deref).booleanValue());
        final NamingEnumeration ne = _ctx.search(base, newFilter, sc);
        return new AvoidEscapingNamingEnumeration(baseDn, ne);
    }

==========================

so it seems whatever is doing the escaping leaves results with full DNs alone (note that just setting sr.setRelative(false) has no effect by itself). I'm not familiar enough with the DS architecture yet to work out where the escaping is occurring and hence come up with a better fix.

Comments

[Norval Hope]

I have reduced the problem down to an extremely simple dummy custom partition and I'm attaching this partition class and associated XML config file to activate it.

You then need to connect to Apache DS using JXplorer and the following parameters:
host: localhost
port: 20391
ldap version: 3
base dn: dc=dummy
login dn: uid=admin,ou=system
password: <DS admin password>

Initially you will see an entry under dc=dummy with the correct name "test space". After two refreshes (not sure why two rather then one) you will instead see "test%20space" (as the dummy partition toggles its behaviour between interfering with search results so they work (broken=false) and leaving them alone so that the escaped name appears in JXplorer.

Further debugging (inside JXplorer) has shown that in the broken case the NameClassPair returned in the results has name="ldap://localhost:20391/cn=test%20space" and isRelative=false, where as in the working case it has name "cn=test space" and isRelative=true.

If I understood the MINA stack better I might be able to contribute more then just this test case, but despite attempting to debug with a number of different approaches I'm afraid I only got this far.

[Norval Hope]

the dummy partition

DS config file to activate dummy partition

[Alex Karasulu]

This is really odd because ldapsearch (openLDAP command line tool) returns the correct result:

=================================
mac:~/Projects/apache/directory/apacheds/trunk akarasulu$ ldapsearch -h localhost -p 20391 -s one -b 'dc=dummy' -x -w secret -D 'uid=admin,ou=system'
# extended LDIF
#
# LDAPv3
# base <dc=dummy> with scope one
# filter: (objectclass=*)
# requesting: ALL
#

# test space
dn: cn=test space

# search result
search: 2
result: 0 Success
matchedDN: dc=dummy

# numResponses: 2
# numEntries: 1
=============================================

JXPlorer on the otherhand does not. It handles the DN replacing the space with a '%20'. Just in case I went through all the code to see the encoding of the result was done properly. It all looked just right. I don't know what could be making JXPlorer treat this differently. Just in case I will try another GUI based browser.
  

[Norval Hope]

I tried Softerra's LDAP Browser 2.6 and it didn't have a problem either.

based on the following input (which I wasn't aware of):

" Marc Boorshtein <[e-mail]> to Apache, me
  More options Sep 23 (4 days ago)

JNDI will return an LDAP URL as the name of an entry if it falls outside of the the search base (ie if a referral to another server was followed that uses a different namespace)."

I have talked to the author of JXplorer and determined that I will provide him a patch to deal with the problem on the client side.

Three points of confusion remain:
    1. How/where the LDAP URLs come in to play: I presume the Sun JNDI library somehow determines that a SearchResults object from one namespace is being passed back through a different LDAP connection and decides that LDAP URLs must be used.
   2. How a proxy partition can convince the offending party not to convert the names of NameClassPairs returned to LDAP URLs, because the results look fine when the proxy partition sees them and even when they leave ApacheDS but seem to be adjusted by the JNDI layer on the client.
   3. Even when LDAP URLs come in to play, it seems strange that they are presented URL-encoded in the client. Is it really a requirement on JNDI clients that they need to scan search results and possibly perform URL-decoding?

[Emmanuel Lecharny]

Norbert, is this still considered has a ADS bug? I have had exactly the same behavior lately with JExplorer, and I'm just wondering if it should be considered as a JXplorer problem or not.

wdyt ?

[Norval Hope]

I think it is fair to say that this problem should no longer be viewed as an ADS bug.

I'm leaning to the opinion that it is either a problem in Sun's client JNDI library (for not unencoding the name part of the URL in this case) or in JXplorer proper. I have a patch to JXplorer which I need to chase up and submit to that project (at least I know how to fix the problem there).

[Emmanuel Lecharny]

Has Norbert stated, it seems to be a JXplorer bug, or a SUN JNDI pb, not an ADS one.

We will keep an eye ont it ;)

[Norval Hope]

Came across this listing http://mail-archives.apache.org/mod_mbox/directory-commits/200602.mbox/[e-mail]%3E (included in case link goes away) which explains how the problem occurs as a result of com.sun.jndi.ldap.LdapSearchEnumeration's behaviour:

=============================Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Directory Wiki" for change notification.

The following page has been changed by MarcDexet:
http://wiki.apache.org/directory/LdapSearch

New page:
Describe LdapSearch here.

Stuff about LpadSearch and SearchResult
== Some little things you could use ==
=== DN from my search are looking like URL with true escape code inside ===
Oh my god I get search result with ugly ldap url "ldap://127.0.0.1:389/" in front of.
Is it a bug ? No, it's normal and written in RFC 2251.
An LDAP URL is returned for an alias that falls outside of the
search base.

You haven't aliasing in your context ? Maybe you use the Sun Ldap factory ?
Let's see what happens: The URL format is set as name to the SearchResult if this one is NOT
relative.

cf. http://java.sun.com/products/jndi/tutorial/ldap/referral/follow.html

And how could JNDI guess SearchResult is unrelative ?

If you take a look [http://www.jsourcery.com/output/sun/j2se/j2sdk/1.4.2_04/com/sun/jndi/ldap/LdapSearchEnumeration.source.html
to sun LdapSearchEnumeration source] who creates SearchResult, relativity is set to false
if the DN doesn't contain the search base DN.

So assume you get such a search
{{{
ldapUrl=ldap://dir.acme.com
baseDn ="dc=acme,dc=com"
filter="(&(sn=foo)(employeetype=dreamer))";
attrs =[];
}}}



If a LdapName is
{{{
cn=John FOO,ou=dreams
}}}
it's deducted as unrelative and you will get
{{{ dn=ldap://dir.acme.com/cn=John FOO,ou=dreams
}}}

But if LdapName is
{{{
cn=John FOO,ou=dreams,dc=acme,dc=com
}}}
is deducted as relative and you will get
{{{
dn=cn=John FOO,ou=dreams
}}}

Don't forget LdapName checking is what happen ''inside'' LdapSearchEnumeration, not what you
could see at result enumeration.
=============================

[Emmanuel Lecharny]

Closing all issues created in 2005 and before which are marked resolved