Uploaded image for project: 'Directory Client API'
  1. Directory Client API
  2. DIRAPI-154

Implement "online" OpenLDAP schema parsing in DefaultSchemaLoader

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.0.0-M20
    • Fix Version/s: 1.0.0-RC2
    • Labels:
      None

      Description

      The constructor in DefaultSchemaLoader has the following comment:

        // TODO Handle schema loading on other LDAP servers
      

      For OpenLDAP this would require searching subtree under cn=schema,cn=config and handling attributes like:

        olcAttributeTypes: {1}( 2.5.4.4 NAME ( 'sn' 'surname' ) DESC 'RFC2256: last (family) name(s) for which the entity is known by' SUP name )
      
        olcObjectClasses: {4}( 2.5.6.6 NAME 'person' DESC 'RFC2256: a person' SUP top STRUCTURAL MUST ( sn $ cn ) MAY ( userPassword $ telephoneNumber $ seeAlso $ description ) )
      

      Note that there is also an 'olcObjectIdentifier' attribute type that seems to be used to create macros for interpolation in OID's used in other attributes:

        olcObjectIdentifier: OLcfg 1.3.6.1.4.1.4203.1.12.2
        olcObjectIdentifier: OLcfgAt OLcfg:3
        olcObjectIdentifier: OLcfgOvAt OLcfgAt:3
        olcObjectIdentifier: OMsyn 1.3.6.1.4.1.1466.115.121.1
        olcObjectIdentifier: OMsBoolean OMsyn:7
      
        olcAttributeTypes: ( OLcfgOvAt:10.4 NAME 'olcUniqueStrict' DESC 'Enforce uniqueness of null values' EQUALITY booleanMatch SYNTAX OMsBoolean SINGLE-VALUE )
      

      However this additional complexity seems to only be used for the internal/vendor-specific schema itself (not 'core', 'nis', etc.).

      Attached is LDIF output of from searching cn=schema,cn=config.

      Mailing List reference: https://mail-archives.apache.org/mod_mbox/directory-api/201308.mbox/%3CCABzFU-eOfmwiFpR1w0Fd-JhNFBVLO0NkHKa5w61ReFBEq%3DWkaw%40mail.gmail.com%3E

      1. loadSchema.log
        14 kB
        Gerald Turner
      2. openldap-schema.ldif
        89 kB
        Gerald Turner

        Activity

        Hide
        semancik Radovan Semancik added a comment -

        It should work. Although the schema is not read from cn=config suffix. That would be troublesome as the cn=config is not usually accessible. However OpenLDAP correctly publishes schema DN in the root DN entry and the API is using that to locate the schema definition (which is cn=subschema if I remember correctly). I have tested with several OpenLDAP versions and the schema seems to work fine.

        Sorry, I completely forgot to resolve this issue. It was fixed few months ago already.

        Show
        semancik Radovan Semancik added a comment - It should work. Although the schema is not read from cn=config suffix. That would be troublesome as the cn=config is not usually accessible. However OpenLDAP correctly publishes schema DN in the root DN entry and the API is using that to locate the schema definition (which is cn=subschema if I remember correctly). I have tested with several OpenLDAP versions and the schema seems to work fine. Sorry, I completely forgot to resolve this issue. It was fixed few months ago already.
        Hide
        elecharny Emmanuel Lecharny added a comment -

        Radovan, do you have a status for this one ?

        Show
        elecharny Emmanuel Lecharny added a comment - Radovan, do you have a status for this one ?
        Hide
        elecharny Emmanuel Lecharny added a comment -

        Radovan has started to work on this feature

        Show
        elecharny Emmanuel Lecharny added a comment - Radovan has started to work on this feature
        Hide
        elecharny Emmanuel Lecharny added a comment -

        I think that offering a way to load the OpenLDAP schema even if we are not connected to a server is useful.

        There is more than just loading the schema from a server : we have some missing meta-data that we use in the loadSchema, like the fact that the schema is disabled or enabled, etc.

        I'm also expecting to have to implement some specific operations for SyntaxCheckers, Comparators, Normalizers.

        At some point, I suspect we have to provide some packages having all those elements gathered (a OpenLDAP schema package, another one for AD, etc...)

        Show
        elecharny Emmanuel Lecharny added a comment - I think that offering a way to load the OpenLDAP schema even if we are not connected to a server is useful. There is more than just loading the schema from a server : we have some missing meta-data that we use in the loadSchema, like the fact that the schema is disabled or enabled, etc. I'm also expecting to have to implement some specific operations for SyntaxCheckers, Comparators, Normalizers. At some point, I suspect we have to provide some packages having all those elements gathered (a OpenLDAP schema package, another one for AD, etc...)
        Hide
        gturner Gerald Turner added a comment -

        I progressed a little further by disabling the throw on errors and changing from loadAllEnabled() to loadAllEnabledRelaxed():

        public void loadSchema( SchemaLoader loader ) throws LdapException
        {
        try
        {
        SchemaManager tmp = new DefaultSchemaManager( loader );

        tmp.loadAllEnabledRelaxed();

        if ( !tmp.getErrors().isEmpty() )
        {
        String msg = "there are errors while loading the schema";
        LOG.error( msg + " {}", tmp.getErrors() );
        // Ignore: throw new LdapException( msg );
        }
        ...

        Now the schema is loaded and the registries are full of schema objects, however a new kind of error happens when retrieving attributes from searches:

        10:03:11,226 ERROR [org.apache.directory.api.ldap.model.entry.DefaultAttribute] (main) ERR_04131 The value is expected to be a String

        ...an attribute such as 'cn' is now stored as binary, toString:

        cn: '0x4A 0x42 0x6F 0x73 0x73 0x41 0x64 0x6D 0x69 0x6E '

        BTW, I created a fresh/empty OpenLDAP installation (using Debian packaging), the messy duplicates and other brokeness (reported in my previous comment, loadSchema.log attachment) is pretty much the same - seems that OpenLDAP is distributed with dirty schemas. Perhaps keep the 'isApacheDs' and if false then use relaxed schema loading?

        Show
        gturner Gerald Turner added a comment - I progressed a little further by disabling the throw on errors and changing from loadAllEnabled() to loadAllEnabledRelaxed(): public void loadSchema( SchemaLoader loader ) throws LdapException { try { SchemaManager tmp = new DefaultSchemaManager( loader ); tmp.loadAllEnabledRelaxed(); if ( !tmp.getErrors().isEmpty() ) { String msg = "there are errors while loading the schema"; LOG.error( msg + " {}", tmp.getErrors() ); // Ignore: throw new LdapException( msg ); } ... Now the schema is loaded and the registries are full of schema objects, however a new kind of error happens when retrieving attributes from searches: 10:03:11,226 ERROR [org.apache.directory.api.ldap.model.entry.DefaultAttribute] (main) ERR_04131 The value is expected to be a String ...an attribute such as 'cn' is now stored as binary, toString: cn: '0x4A 0x42 0x6F 0x73 0x73 0x41 0x64 0x6D 0x69 0x6E ' BTW, I created a fresh/empty OpenLDAP installation (using Debian packaging), the messy duplicates and other brokeness (reported in my previous comment, loadSchema.log attachment) is pretty much the same - seems that OpenLDAP is distributed with dirty schemas. Perhaps keep the 'isApacheDs' and if false then use relaxed schema loading?
        Hide
        gturner Gerald Turner added a comment -

        Interesting!

        I built the project with the 'if (isApacheDs...' removed and it's now loading the schema from the OpenLDAP server.

        However loadSchema() throws an exception and I see a bunch of log output about duplicates and other brokeness.

        org.apache.directory.api.ldap.model.exception.LdapException: there are errors while loading the schema
        at org.apache.directory.ldap.client.api.LdapNetworkConnection.loadSchema(LdapNetworkConnection.java:3378)
        at org.apache.directory.ldap.client.api.LdapNetworkConnection.loadSchema(LdapNetworkConnection.java:3356)
        at net.xoint.usermanager.UserManagerTest.test2(UserManagerTest.java:127)

        Show
        gturner Gerald Turner added a comment - Interesting! I built the project with the 'if (isApacheDs...' removed and it's now loading the schema from the OpenLDAP server. However loadSchema() throws an exception and I see a bunch of log output about duplicates and other brokeness. org.apache.directory.api.ldap.model.exception.LdapException: there are errors while loading the schema at org.apache.directory.ldap.client.api.LdapNetworkConnection.loadSchema(LdapNetworkConnection.java:3378) at org.apache.directory.ldap.client.api.LdapNetworkConnection.loadSchema(LdapNetworkConnection.java:3356) at net.xoint.usermanager.UserManagerTest.test2(UserManagerTest.java:127)
        Hide
        seelmann Stefan Seelmann added a comment -

        I don't think this is required. The same mechanism done for ApacheDS (which is defined in http://tools.ietf.org/html/rfc4512#section-4.4) can also be done for most other LDAP servers, supposed they only provide a single schema:
        1. Get the 'subschemaSubentry' from RootDSE, this points to the schema entry. In case of OpenLDAP that is 'cn=subschema'.
        2. The schema entry contains all the schema attributes (objectClasses, attributesTypes, etc.)

        So I think we can just remove the 'if ( isApacheDs( rootDse ) )' condition.

        Show
        seelmann Stefan Seelmann added a comment - I don't think this is required. The same mechanism done for ApacheDS (which is defined in http://tools.ietf.org/html/rfc4512#section-4.4 ) can also be done for most other LDAP servers, supposed they only provide a single schema: 1. Get the 'subschemaSubentry' from RootDSE, this points to the schema entry. In case of OpenLDAP that is 'cn=subschema'. 2. The schema entry contains all the schema attributes (objectClasses, attributesTypes, etc.) So I think we can just remove the 'if ( isApacheDs( rootDse ) )' condition.
        Hide
        gturner Gerald Turner added a comment -

        LDIF output from searching cn=schema,cn=config

        Show
        gturner Gerald Turner added a comment - LDIF output from searching cn=schema,cn=config

          People

          • Assignee:
            elecharny Emmanuel Lecharny
            Reporter:
            gturner Gerald Turner
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development