query incorrectly parsed if first part contains wild-cards (asterisk) - most prominently for gpg/gnupg

• Key: DIRSERVER-651
• Type: Bug
• Status: Closed
• Resolution: Fixed
• Priority: Major
• Assignee: Emmanuel Lecharny
• Reporter: Ralf Hauser

Description

As reported by Valdimir (http://mail-archives.apache.org/mod_mbox/directory-dev/200606.mbox/ajax/[e-mail]%3e) this query is not handled correctly.

In short:
  ldapsearch -x -H ldap://localhost:11500 -D "dn=bugs" -w bunny -b "dc=pgpkeys" "(&(pgpuserid=test*)(pgpdisabled=0))"
only brings up a SimpleNode instead of a BranchNode.

Some further insights:
-----------------
1) a unit test on the query with the parser in shared-ldap-0.9.5.jar appears to work:
            FilterParserImpl parser = new FilterParserImpl();
            ExprNode node = parser
                    .parse("(&(pgpuserid=*@test*)(pgpdisabled=0))");
  ==> a BranchNode is returned here, but not when using apacheDS
2) when switching the order of the sub-queries, I do see the BranchNode even when using apacheDS with both parts:
     ldapsearch -x -H ldap://localhost:2389 -d5 -D "dn=bugs" -w bunny -b "dc=pgpkeys" "(&(pgpdisabled=0)(pgpuserid=@test*))"
3) increasing the debug level to "ldapsearch -d10" hints that the full query is sent to apacheDS and not only the "pgpdisabled=0" part
4) when setting a break-point in org.apache.directory.shared.ldap.filter.FilterParserImpl, it appears that when doing my tests, the parse() is never called??

Comments

[Ralf Hauser]

Interestingly, already very early when doing the request with ldapsearch, the server only sees the "disabled=0" part:
Thread [IoThreadPool-1] (Suspended (breakpoint at line 148 in org.apache.directory.shared.ldap.message.MessageDecoder$1))
org.apache.directory.shared.ldap.message.MessageDecoder$1.decodeOccurred(org.apache.directory.shared.asn1.codec.stateful.StatefulDecoder, java.lang.Object) line: 148
org.apache.directory.shared.ldap.codec.TwixDecoder.decode(java.lang.Object) line: 114
org.apache.directory.shared.ldap.message.MessageDecoder.decode(java.lang.Object) line: 206
org.apache.mina.filter.codec.asn1.Asn1CodecDecoder.decode(org.apache.mina.common.IoSession, org.apache.mina.common.ByteBuffer, org.apache.mina.filter.codec.ProtocolDecoderOutput) line: 37
org.apache.mina.filter.codec.ProtocolCodecFilter.messageReceived(org.apache.mina.common.IoFilter$NextFilter, org.apache.mina.common.IoSession, java.lang.Object) line: 56
org.apache.mina.transport.socket.nio.support.SocketFilterChain(org.apache.mina.common.support.AbstractIoFilterChain).callNextMessageReceived(org.apache.mina.common.IoFilterChain$Entry, org.apache.mina.common.IoSession, java.lang.Object) line: 494
org.apache.mina.common.support.AbstractIoFilterChain.access$1000(org.apache.mina.common.support.AbstractIoFilterChain, org.apache.mina.common.IoFilterChain$Entry, org.apache.mina.common.IoSession, java.lang.Object) line: 52
org.apache.mina.common.support.AbstractIoFilterChain$EntryImpl$1.messageReceived(org.apache.mina.common.IoSession, java.lang.Object) line: 761
org.apache.mina.filter.ThreadPoolFilter.processEvent(org.apache.mina.common.IoFilter$NextFilter, org.apache.mina.common.IoSession, org.apache.mina.filter.ThreadPoolFilter$EventType, java.lang.Object) line: 665
org.apache.mina.filter.ThreadPoolFilter$Worker.processEvents(org.apache.mina.filter.ThreadPoolFilter$SessionBuffer) line: 421
org.apache.mina.filter.ThreadPoolFilter$Worker.run() line: 376

LdapMessage
    message Id : 2
    Search Request
        Base Object : 'dc=pgpkeys'
        Scope : whole subtree
        Deref Aliases : never Deref Aliases
        Size Limit : no limit
        Time Limit : no limit
        Types Only : false
        Filter : '(pgpdisabled=0)'

and then, in ExprNode org.apache.directory.shared.ldap.codec.TwixTransformer.transformFilter(Filter twixFilter), only an Filter twixFilter= AttributeValueAssertionFilter (id=3647) is seen.

When doing the same without the asterisk in the query, already in decodeOccurred(), the LdapMessage contains both parts of the query
LdapMessage
    message Id : 2
    Search Request
        Base Object : 'dc=pgpkeys'
        Scope : whole subtree
        Deref Aliases : never Deref Aliases
        Size Limit : no limit
        Time Limit : no limit
        Types Only : false
        Filter : '(&(pgpuserid=[e-mail])(pgpdisabled=0))'

So either already the org.apache.directory.shared.ldap.message.MessageDecoder doesn't decode the queries with asterisks correctly or it is the clients that do not send the combined query as soon as there is the asterisk.

I'll attach two ldapsearch logs with debug output.
My intuition tells me that since gpg and ldapsearch behave the same way, that this isn't a client issue (and seeing what the client sends by changing "-d5" to "-d999" hints at that too), but this cold be easily verifed with an etheral sniffing session.

[Emmanuel Lecharny]

I can't download the attachment, I get an error :
The image "http://issues.apache.org/jira/secure/attachment/12336540/ldapAsterisk.txt" cannot be displayed, because it contains errors.

Are they images or full text ?

Another question : have you extended the schema to add a pgpuserid attribute?

Last point, if you are not afraid by logs, you can set the server to debug mode (including the codec) modifying the log4j configuration file, and attach a tar.gz of those logs. It will help to check if the client send the filter correctly, because we are dumping the PDU bytes into the logs.

[Emmanuel Lecharny]

I confirm that this is a problem in the ASN.1 decoder.

I gonna fix it asap.

[Emmanuel Lecharny]

To inform people who are wainting for a fix :

Working search ilters so far (junit tests created) :
- (a=b)
- (& (a=b) )
- (& (a=b) (c=d) )
- (& (& (a=b ) )
- (& (& (a=b) (c=d) ) )
- (& (& (a=b) ) (c=d) )
- (& (& (a=b) (c=d) ) (e=f) )
- (& (& (a=b) ) (& (c=d) ) )
- (& (& (a=b) (c=d) ) (& (e=f) ) )

The only test failing is a test with something like :
(&(|(objectclass=*)(ou=*))(!(objectclass>=ttt))), which has pretty much the same structure as (& (& (a=b) (c=d) ) (& (e=f) ) ), so I guess the problem is not in the structure construction now (this has been fixed), but in the handling of NOT filters (the OR has been replaced by a AND, and it does not change anything).

More comming tomorrow...

[Emmanuel Lecharny]

I have a fix !!!

It was a nasty little bug in PresentFilter which prevented the stack to be correctly popped...

I'm not closing the issue right now because more work need to be done and I don't have currently the time to finish the needed extra tests. Will commit it tonite. Whatever, the 815 tests are running ok in shared-ldap.

Champagne ! (4 days to kill this bug :(

[Emmanuel Lecharny]

Ok, the code has been committed into the optimization-trunks branch.

   [[ Old comment, sent by email on Fri, 14 Jul 2006 09:35:39 +0200 ]]

Cool! What is the ETA for the next release?

[Emmanuel Lecharny]

closed