Directory ApacheDS
  1. Directory ApacheDS
  2. DIRSERVER-1651

rfc 4533 implementation differences between openldap and apacheDS

    Details

    • Type: Bug Bug
    • Status: Reopened
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: 2.0.0-M2
    • Fix Version/s: 2.0.0-RC1
    • Component/s: ldap

      Description

      Tthere is an incompatibility between the RFC 4533 implementation of apacheDS and openldap.
      openldap uses the cookie structure "rid=<replicaId>" (initial) or "rid=<replicaId>,csn=<Csn value>" (update) while apacheDS is using NULL for the initial state and the structure "<replicaId>;<Csn value>" for the update state. in the RFC its said:

      The absence of a cookie or an initialized synchronization state in a cookie indicates a request for initial content.....

      first is apacheDS like, second is openldap like

      It should be possible to adapt the structure or the behavior.

        Activity

        Hide
        Pierre-Arnaud Marcelot added a comment -

        Version 2.0.0-M3 has been released a couple months ago.

        Assigned the remaining opened JIRA to the next iteration (2.0.0-M4).

        Show
        Pierre-Arnaud Marcelot added a comment - Version 2.0.0-M3 has been released a couple months ago. Assigned the remaining opened JIRA to the next iteration (2.0.0-M4).
        Hide
        Emmanuel Lecharny added a comment -

        Damn right.

        Let's fix that.

        Show
        Emmanuel Lecharny added a comment - Damn right. Let's fix that.
        Hide
        Hajo Kliemeck added a comment -

        Yeah, but if openldap sends an initial cookie without csn ("rid=001"), it is not equal to "1"

        Show
        Hajo Kliemeck added a comment - Yeah, but if openldap sends an initial cookie without csn ("rid=001"), it is not equal to "1"
        Hide
        Emmanuel Lecharny added a comment -

        As far as I know, OpenLDAP uses a numeric value for the rid :

        "The <rid> must have no more than 3 decimal digits" (http://www.openldap.org/doc/admin24/replication.html), chap 18.3.1.4.

        Show
        Emmanuel Lecharny added a comment - As far as I know, OpenLDAP uses a numeric value for the rid : "The <rid> must have no more than 3 decimal digits" ( http://www.openldap.org/doc/admin24/replication.html ), chap 18.3.1.4.
        Hide
        Hajo Kliemeck added a comment -

        hey,

        another issue. openldap is using a string value for the replicaId, apacheDS is using an int. any ideas?

        greets

        Show
        Hajo Kliemeck added a comment - hey, another issue. openldap is using a string value for the replicaId, apacheDS is using an int. any ideas? greets
        Hide
        Kiran Ayyagari added a comment -

        Thanks for reporting.
        Fixed here http://svn.apache.org/viewvc?rev=1164848&view=rev

        Show
        Kiran Ayyagari added a comment - Thanks for reporting. Fixed here http://svn.apache.org/viewvc?rev=1164848&view=rev
        Hide
        Emmanuel Lecharny added a comment -

        well, as soon as you need to authenticate to be able to send the first request, I assume it's quite unlikely to have somebody spoofing a cookie...

        Show
        Emmanuel Lecharny added a comment - well, as soon as you need to authenticate to be able to send the first request, I assume it's quite unlikely to have somebody spoofing a cookie...
        Hide
        Kiran Ayyagari added a comment -

        Absolutely, but (at least in theory ) a changed replicaId in a cookie can trick the provider to send different entries which are not otherwise allowed for this consumer

        Show
        Kiran Ayyagari added a comment - Absolutely, but (at least in theory ) a changed replicaId in a cookie can trick the provider to send different entries which are not otherwise allowed for this consumer
        Hide
        Howard Chu added a comment -

        There is nothing to be gained from maliciously spoofing the cookie, since the
        operation is part of a regular Search request. I.e., the client can only ever
        retrieve any information that server authorizations would already allow the
        client to see.

        Indeed, slapd's -c option allows a sysadmin to set any cookie value at all;
        this is intended to be used to force a consumer to re-pull data from an older
        point in time, in case more recent data was lost/curropted/whatever.


        – Howard Chu
        CTO, Symas Corp. http://www.symas.com
        Director, Highland Sun http://highlandsun.com/hyc/
        Chief Architect, OpenLDAP http://www.openldap.org/project/

        Show
        Howard Chu added a comment - There is nothing to be gained from maliciously spoofing the cookie, since the operation is part of a regular Search request. I.e., the client can only ever retrieve any information that server authorizations would already allow the client to see. Indeed, slapd's -c option allows a sysadmin to set any cookie value at all; this is intended to be used to force a consumer to re-pull data from an older point in time, in case more recent data was lost/curropted/whatever. – – Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
        Hide
        Howard Chu added a comment -

        "rid=<replicaId>,csn=<Csn value>" (update) while apacheDS is using NULL for
        the initial state and the structure"<replicaId>;<Csn value>" for the update
        state. in the RFC its said:
        indicates a request for initial content.....

        Note that if you don't send <replicaId> in your initial query, the provider
        cannot possibly know what replicaId to use in the first cookie it sends back
        to you.

        There's a desire to ensure that a cookie unambiguously defines a specific
        provider-consumer relationship, and that a given cookie cannot be used with a
        different provider or a different consumer. (In practice this doesn't matter
        at all to the replication protocol, but keeping replicaIDs straight aids in
        debugging when you're looking at hex dumps of packet traces.)

        – Howard Chu
        CTO, Symas Corp. http://www.symas.com
        Director, Highland Sun http://highlandsun.com/hyc/
        Chief Architect, OpenLDAP http://www.openldap.org/project/

        Show
        Howard Chu added a comment - "rid=<replicaId>,csn=<Csn value>" (update) while apacheDS is using NULL for the initial state and the structure"<replicaId>;<Csn value>" for the update state. in the RFC its said: indicates a request for initial content..... Note that if you don't send <replicaId> in your initial query, the provider cannot possibly know what replicaId to use in the first cookie it sends back to you. There's a desire to ensure that a cookie unambiguously defines a specific provider-consumer relationship, and that a given cookie cannot be used with a different provider or a different consumer. (In practice this doesn't matter at all to the replication protocol, but keeping replicaIDs straight aids in debugging when you're looking at hex dumps of packet traces.) – – Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
        Hide
        Kiran Ayyagari added a comment -

        This is still susceptible for spoofing unless cryptographically signed, IMHO the solution is to encrypt the whole cookie

        Show
        Kiran Ayyagari added a comment - This is still susceptible for spoofing unless cryptographically signed, IMHO the solution is to encrypt the whole cookie
        Hide
        Emmanuel Lecharny added a comment -

        We can simply follow OpenLDAP syntax for the cookie.

        Now, there are some interesting chapter (7) in RFC 4533 :

        " Implementors should take precautions against malicious cookie
        content, including malformed cookies or valid cookies used with
        different security associations and/or protections in an attempt to
        obtain unauthorized access to information. Servers may include a
        digital signature in the cookie to detect tampering."

        I just wonder if we should not add some additional information at the end of the cookie, an opaque one.

        if we assume the cookie format is :
        rid=<replicaId>[,csn=<Csn value>]

        we can also add :
        rid=<replicaId>[,csn=<Csn value>][,signature=<signature>]

        Show
        Emmanuel Lecharny added a comment - We can simply follow OpenLDAP syntax for the cookie. Now, there are some interesting chapter (7) in RFC 4533 : " Implementors should take precautions against malicious cookie content, including malformed cookies or valid cookies used with different security associations and/or protections in an attempt to obtain unauthorized access to information. Servers may include a digital signature in the cookie to detect tampering." I just wonder if we should not add some additional information at the end of the cookie, an opaque one. if we assume the cookie format is : rid=<replicaId> [,csn=<Csn value>] we can also add : rid=<replicaId> [,csn=<Csn value>] [,signature=<signature>]
        Hide
        Howard Chu added a comment -

        That's probably the easiest thing to do. Sorry for this; OpenLDAP clearly
        deviated from the intention of RFC4533 here. I.e., the cookie was meant to be
        an opaque value that is not interpreted by the consumers. I believe in the
        usual case, a consumer that needed to "know" the replication state was
        supposed to generate its own private state, independent of the cookie that the
        provider sent.

        Unfortunately this intention simply doesn't work in the real world. The view
        of entities as being only providers or only consumers is too simplistic. Even
        when syncrepl was first being developed, we had the notion of cascaded
        replication, where a consumer was itself a provider to other downstream
        consumers. When you have multiple entities participating in replication, it's
        important to be able to see that each one's notion of the replication state
        agrees with everyone else's. It's also important to be able to determine this
        without knowing in advance whether an entity is a provider or a consumer.
        Particularly since in cascading or multi-master replication, the entity is both.

        So in OpenLDAP we abandoned the notion that consumers could just stash the
        provider's cookie without peeking inside, and could generate their own cookies
        independently if they were going to serve as providers to other servers.

        I believe we could have made it work correctly, with each server generating
        its own state and leaving received cookies as opaque values, but then it would
        have been impossible to do a simple query to see if two entities are in sync.


        – Howard Chu
        CTO, Symas Corp. http://www.symas.com
        Director, Highland Sun http://highlandsun.com/hyc/
        Chief Architect, OpenLDAP http://www.openldap.org/project/

        Show
        Howard Chu added a comment - That's probably the easiest thing to do. Sorry for this; OpenLDAP clearly deviated from the intention of RFC4533 here. I.e., the cookie was meant to be an opaque value that is not interpreted by the consumers. I believe in the usual case, a consumer that needed to "know" the replication state was supposed to generate its own private state, independent of the cookie that the provider sent. Unfortunately this intention simply doesn't work in the real world. The view of entities as being only providers or only consumers is too simplistic. Even when syncrepl was first being developed, we had the notion of cascaded replication, where a consumer was itself a provider to other downstream consumers. When you have multiple entities participating in replication, it's important to be able to see that each one's notion of the replication state agrees with everyone else's. It's also important to be able to determine this without knowing in advance whether an entity is a provider or a consumer. Particularly since in cascading or multi-master replication, the entity is both. So in OpenLDAP we abandoned the notion that consumers could just stash the provider's cookie without peeking inside, and could generate their own cookies independently if they were going to serve as providers to other servers. I believe we could have made it work correctly, with each server generating its own state and leaving received cookies as opaque values, but then it would have been impossible to do a simple query to see if two entities are in sync. – – Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
        Hide
        Emmanuel Lecharny added a comment -

        Absolutely.

        As we are currently working on this part, I think we are going to make the cookie compatinle.

        Thanks for the JIRA !

        Show
        Emmanuel Lecharny added a comment - Absolutely. As we are currently working on this part, I think we are going to make the cookie compatinle. Thanks for the JIRA !

          People

          • Assignee:
            Kiran Ayyagari
            Reporter:
            Hajo Kliemeck
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:

              Development