Directory ApacheDS
  1. Directory ApacheDS
  2. DIRSERVER-1979

Adding child entries can't be found after restarting server

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 2.0.0-M16, 2.0.0-M17
    • Fix Version/s: 2.0.0-M17
    • Component/s: ldap
    • Labels:
      None
    • Environment:
      windows seven.

      Description

      Suppose to have the following situation on the Server (created with ldif):

      dc=xxx,dc=com
      ........ou=groups
      .............. cn=aim
      .............. cn=ais
      .................... cn=aisOperator
      .............. cn=admin

      If you stop and restart the server and then create a subentry (named for example "child") under any leaf on the tree (in this case under "aim" or "aisOperator" or "admin"), happen the following:
      1) the new entry is not visible with a search that start
      from ou=groups,dc=xxx,dc=com
      2) the new entry is visible only if the search start from dc=xxx,dc=com
      or starting from the leaf: ou=aim,ou=groups,dc=xxx,dc=com
      3) If you perform an ldif export on ou=groups,dc=xxx,dc=com, you can't find the new added subentry.
      4) if the subentry is created under cn=ais (that is not a leaf because has already a subentry) the new "child" entry is founded by the search

      I was thinking that the problem was related to DIRSERVER-1976 (solved on M17) but we have the same problems also on this version.

        Activity

        Fortunato Margariti created issue -
        Hide
        Kiran Ayyagari added a comment -

        Are you sure the subentries control is added in each of those search requests?

        Show
        Kiran Ayyagari added a comment - Are you sure the subentries control is added in each of those search requests?
        Hide
        Fortunato Margariti added a comment -

        Yes, we have tried in both case, with subentries control enabled or disabled the result is the same.

        Show
        Fortunato Margariti added a comment - Yes, we have tried in both case, with subentries control enabled or disabled the result is the same.
        Hide
        Fortunato Margariti added a comment -

        the problem happen only when restart the server. if i create the mentioned tree structure with ldif, and add subentries without restarting the server, that subentries are founded with the search also if they are added on a leaf. We have found the problem after same days when we restarted the server. The problem seems to happen only on subentries that we want to add on entry created before the restarting. (this was the reason because i thought that the problem was related on DIRSERVER-1976), but doesn't happen if we create before a NEW entry and then the subentry.

        Show
        Fortunato Margariti added a comment - the problem happen only when restart the server. if i create the mentioned tree structure with ldif, and add subentries without restarting the server, that subentries are founded with the search also if they are added on a leaf. We have found the problem after same days when we restarted the server. The problem seems to happen only on subentries that we want to add on entry created before the restarting. (this was the reason because i thought that the problem was related on DIRSERVER-1976 ), but doesn't happen if we create before a NEW entry and then the subentry.
        Hide
        Emmanuel Lecharny added a comment -

        I had fixed an issue related to AdministrativePoint a year ago. It seems to be the very same kind of problem : the subentries are stored into a cache, and should be reloaded at startup, otherwise the cache will be emptied. I think this is what happens.

        If so, this is quite simple to fix : just browse the backends for subentries in the init() phase.

        Show
        Emmanuel Lecharny added a comment - I had fixed an issue related to AdministrativePoint a year ago. It seems to be the very same kind of problem : the subentries are stored into a cache, and should be reloaded at startup, otherwise the cache will be emptied. I think this is what happens. If so, this is quite simple to fix : just browse the backends for subentries in the init() phase.
        Hide
        Emmanuel Lecharny added a comment -

        Just to be sure : when you say "subentry", you mean a normal entry added under another one, or a real LDAP "subentry" ? (ie, an entry with the "objectClass: subentry" attributeType in it)

        Show
        Emmanuel Lecharny added a comment - Just to be sure : when you say "subentry", you mean a normal entry added under another one, or a real LDAP "subentry" ? (ie, an entry with the "objectClass: subentry" attributeType in it)
        Hide
        Fortunato Margariti added a comment -

        Ok for subentry I mean a normal entry, added under a leaft entry that didn't already contains childs (because otherwise it works) . Just to be clear, you can reproduce this problem following these step:
        1) Create an entry with this portion of LDIF -
        dn: cn=imadmin,ou=groups,dc=xxx,dc=com
        objectClass: groupOfUniqueNames
        objectClass: top
        cn: imadmin
        description: AdministrationGroup
        uniqueMember: uid=dummy
        2) Stop and restart the server.
        3) Add a "subentry" (or child) with DN for example cn=child,cn=imadmin,ou=groups,dc=xxx,dc=com of type groupOfUniqueNames.
        4) if you perform a query starting from groups , the result doesn't contain the entry "child".
        5) if you perform a query starting from cn=imadmin,ou=groups,dc=xxx,dc=com or from the root partition the query works.
        6) if you perform an export the result is the same of the previous two points.

        Show
        Fortunato Margariti added a comment - Ok for subentry I mean a normal entry, added under a leaft entry that didn't already contains childs (because otherwise it works) . Just to be clear, you can reproduce this problem following these step: 1) Create an entry with this portion of LDIF - dn: cn=imadmin,ou=groups,dc=xxx,dc=com objectClass: groupOfUniqueNames objectClass: top cn: imadmin description: AdministrationGroup uniqueMember: uid=dummy 2) Stop and restart the server. 3) Add a "subentry" (or child) with DN for example cn=child,cn=imadmin,ou=groups,dc=xxx,dc=com of type groupOfUniqueNames. 4) if you perform a query starting from groups , the result doesn't contain the entry "child". 5) if you perform a query starting from cn=imadmin,ou=groups,dc=xxx,dc=com or from the root partition the query works. 6) if you perform an export the result is the same of the previous two points.
        Hide
        Emmanuel Lecharny added a comment -

        Thanks for the clarification... I was testing the server with real 'subentries' and that was working fine, even after a restart.

        I'll do a test right away. I suspect that the RDN index is not correctly updated in this case (that would be the reasonnable explanation).

        Show
        Emmanuel Lecharny added a comment - Thanks for the clarification... I was testing the server with real 'subentries' and that was working fine, even after a restart. I'll do a test right away. I suspect that the RDN index is not correctly updated in this case (that would be the reasonnable explanation).
        Hide
        Emmanuel Lecharny added a comment -

        Also if you can provide the exact search requests you are doing (scope, etc) that could help.

        Show
        Emmanuel Lecharny added a comment - Also if you can provide the exact search requests you are doing (scope, etc) that could help.
        Hide
        Fortunato Margariti added a comment -

        Yes of course. To fast my test i'm using the quick search panel of apache directory studio now, but i have the same result using java spring ldaptemplate. The search request in the quick search of directory studio has the following parameter:
        1) Search base: ou=groups,dc=xxx,dc=com
        2) filter: (objectClass=*)
        3) returning attributes: empty
        4)Controls: no flags (i've tried also to flag subentries but no object are founded with the search)
        5)Scope: subtree
        6) Aliases Dereferencing: default values
        7) Referrals Handling: default values

        Show
        Fortunato Margariti added a comment - Yes of course. To fast my test i'm using the quick search panel of apache directory studio now, but i have the same result using java spring ldaptemplate. The search request in the quick search of directory studio has the following parameter: 1) Search base: ou=groups,dc=xxx,dc=com 2) filter: (objectClass=*) 3) returning attributes: empty 4)Controls: no flags (i've tried also to flag subentries but no object are founded with the search) 5)Scope: subtree 6) Aliases Dereferencing: default values 7) Referrals Handling: default values
        Emmanuel Lecharny made changes -
        Field Original Value New Value
        Summary Adding subentries after restarting server, aren't found. Adding child entries can't be found after restarting server
        Hide
        Emmanuel Lecharny added a comment -

        I renamed the tittle for clarity

        Show
        Emmanuel Lecharny added a comment - I renamed the tittle for clarity
        Hide
        Emmanuel Lecharny added a comment -

        I confirm the pb. I wrote a test that demonstrate it. The first time, I added the child before shutting down the server, and in this case, we correctly see it when we restart the server. Then if we create a second child at the same level, it is seen too.

        Adding child1 : cn=child1,cn=imadmin,ou=groups,dc=test,ou=system
        Entries found :
        --------------
        ou=groups,dc=test,ou=system
        cn=child1,cn=imadmin,ou=groups,dc=test,ou=system
        dc=test,ou=system
        cn=imadmin,ou=groups,dc=test,ou=system
        Stopping the service---------------------------------
        
        Starting the service---------------------------------
        Adding child2 : cn=child2,cn=imadmin,ou=groups,dc=test,ou=system
        Enties found :
        --------------
        ou=groups,dc=test,ou=system
        cn=child1,cn=imadmin,ou=groups,dc=test,ou=system
        cn=child2,cn=imadmin,ou=groups,dc=test,ou=system
        dc=test,ou=system
        cn=imadmin,ou=groups,dc=test,ou=system
        Stopping the service---------------------------------
        
        Starting the service---------------------------------
        
        Entries found :
        --------------
        ou=groups,dc=test,ou=system
        cn=child1,cn=imadmin,ou=groups,dc=test,ou=system
        cn=child2,cn=imadmin,ou=groups,dc=test,ou=system
        dc=test,ou=system
        cn=imadmin,ou=groups,dc=test,ou=system
        

        The same test, but this time, we shutdown the server before adding the child entry :

        Entries found :
        --------------
        ou=groups,dc=test,ou=system
        dc=test,ou=system
        cn=imadmin,ou=groups,dc=test,ou=system
        Stopping the service---------------------------------
        
        Starting the service---------------------------------
        Adding child2 : cn=child2,cn=imadmin,ou=groups,dc=test,ou=system
        Child2 exists :
        Entry
            dn[n]: cn=child2,cn=imadmin,ou=groups,dc=test,ou=system
            objectclass: top
            objectclass: groupOfUniqueNames
            uniquemember: uid=dummy2
            cn: child2
            description: child2
        
        Entries found :
        --------------
        ou=groups,dc=test,ou=system
        dc=test,ou=system
        cn=imadmin,ou=groups,dc=test,ou=system
        Stopping the service---------------------------------
        
        Starting the service---------------------------------
        Child2 STILL exists :
        Entry
            dn[n]: cn=child2,cn=imadmin,ou=groups,dc=test,ou=system
            objectclass: top
            objectclass: groupOfUniqueNames
            uniquemember: uid=dummy2
            cn: child2
            description: child2
        
        
        Entries found :
        --------------
        ou=groups,dc=test,ou=system
        dc=test,ou=system
        cn=imadmin,ou=groups,dc=test,ou=system
        

        As we can see, child2 is created, it exists, but it's not found by a subtree scope search...

        Show
        Emmanuel Lecharny added a comment - I confirm the pb. I wrote a test that demonstrate it. The first time, I added the child before shutting down the server, and in this case, we correctly see it when we restart the server. Then if we create a second child at the same level, it is seen too. Adding child1 : cn=child1,cn=imadmin,ou=groups,dc=test,ou=system Entries found : -------------- ou=groups,dc=test,ou=system cn=child1,cn=imadmin,ou=groups,dc=test,ou=system dc=test,ou=system cn=imadmin,ou=groups,dc=test,ou=system Stopping the service--------------------------------- Starting the service--------------------------------- Adding child2 : cn=child2,cn=imadmin,ou=groups,dc=test,ou=system Enties found : -------------- ou=groups,dc=test,ou=system cn=child1,cn=imadmin,ou=groups,dc=test,ou=system cn=child2,cn=imadmin,ou=groups,dc=test,ou=system dc=test,ou=system cn=imadmin,ou=groups,dc=test,ou=system Stopping the service--------------------------------- Starting the service--------------------------------- Entries found : -------------- ou=groups,dc=test,ou=system cn=child1,cn=imadmin,ou=groups,dc=test,ou=system cn=child2,cn=imadmin,ou=groups,dc=test,ou=system dc=test,ou=system cn=imadmin,ou=groups,dc=test,ou=system The same test, but this time, we shutdown the server before adding the child entry : Entries found : -------------- ou=groups,dc=test,ou=system dc=test,ou=system cn=imadmin,ou=groups,dc=test,ou=system Stopping the service--------------------------------- Starting the service--------------------------------- Adding child2 : cn=child2,cn=imadmin,ou=groups,dc=test,ou=system Child2 exists : Entry dn[n]: cn=child2,cn=imadmin,ou=groups,dc=test,ou=system objectclass: top objectclass: groupOfUniqueNames uniquemember: uid=dummy2 cn: child2 description: child2 Entries found : -------------- ou=groups,dc=test,ou=system dc=test,ou=system cn=imadmin,ou=groups,dc=test,ou=system Stopping the service--------------------------------- Starting the service--------------------------------- Child2 STILL exists : Entry dn[n]: cn=child2,cn=imadmin,ou=groups,dc=test,ou=system objectclass: top objectclass: groupOfUniqueNames uniquemember: uid=dummy2 cn: child2 description: child2 Entries found : -------------- ou=groups,dc=test,ou=system dc=test,ou=system cn=imadmin,ou=groups,dc=test,ou=system As we can see, child2 is created, it exists, but it's not found by a subtree scope search...
        Hide
        Emmanuel Lecharny added a comment -

        I have added a test to demonstrate the issue (the test is @Ignored atm):

        http://svn.apache.org/r1601810

        Show
        Emmanuel Lecharny added a comment - I have added a test to demonstrate the issue (the test is @Ignored atm): http://svn.apache.org/r1601810
        Hide
        Kiran Ayyagari added a comment -

        The issue is in the RDN index, where the ParentIdAndRdn keys are not replaced after updating the counts of children and descendants.
        I suspect some JDBM cache related issue here, cause for the first 3 entries injected into the server these counts are properly updated
        in the RDN index entry of root context.

        As a workaround if I place "rdnIdx.drop( parentId );" at line 938 in updateRdnIdx() of AbstractBTreePartition the test passes.

        Show
        Kiran Ayyagari added a comment - The issue is in the RDN index, where the ParentIdAndRdn keys are not replaced after updating the counts of children and descendants. I suspect some JDBM cache related issue here, cause for the first 3 entries injected into the server these counts are properly updated in the RDN index entry of root context. As a workaround if I place "rdnIdx.drop( parentId );" at line 938 in updateRdnIdx() of AbstractBTreePartition the test passes.
        Hide
        Emmanuel Lecharny added a comment -

        Question :is the test pasding when uding mavibot ?

        Show
        Emmanuel Lecharny added a comment - Question :is the test pasding when uding mavibot ?
        Hide
        Kiran Ayyagari added a comment -

        No, it is failing with Mavibot too (tested with M4 and trunk). However it passes with Mavibot when the above said change is applied in updateRdnIdx().

        I think now we must add a addOrReplace() to Mavibot and JDBM.

        Show
        Kiran Ayyagari added a comment - No, it is failing with Mavibot too (tested with M4 and trunk). However it passes with Mavibot when the above said change is applied in updateRdnIdx(). I think now we must add a addOrReplace() to Mavibot and JDBM.
        Hide
        Kiran Ayyagari added a comment -
        Show
        Kiran Ayyagari added a comment - Fixed here http://svn.apache.org/r1601876 .
        Kiran Ayyagari made changes -
        Status Open [ 1 ] Resolved [ 5 ]
        Assignee Kiran Ayyagari [ akiran ]
        Fix Version/s 2.0.0-M17 [ 12326507 ]
        Resolution Fixed [ 1 ]

          People

          • Assignee:
            Kiran Ayyagari
            Reporter:
            Fortunato Margariti
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development