Uploaded image for project: 'Directory ApacheDS'
  1. Directory ApacheDS
  2. DIRSERVER-2047

Some data can be lost when using ldapadd command to insert data into apacheds

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: 2.0.0-M19
    • Fix Version/s: 2.0.0-M20
    • Component/s: ldap
    • Labels:
    • Environment:
      centOS 6.5 with openldap and apacheds installed

      Description

      In our system, we need to do data backup and restore for apacheds. For now, we use the ldapsearch and ldapadd command to do BR function. We use ldapsearch to backup apacheds data to be a ldif file and use ldapadd to restore the data. But when the ldif is a little big, I always found that the data can't be restored successfully, but the ladpadd command showed that the data can be added successfully. No exceptions for ldapadd command. But the restored data didn't exist in the node. This bug only happened when the ldif file is a little big. I mean if the data entry greater than 500 entries. But the node for backup and restore is using mavibot partition. Because I found so many problems for jdbm. So I change it to mavibot partition. Another question is that do you know is there some good way to do the data backup and restore for apacheds?
      May be my mavibot partition has some problems, so someone can tell me how to create a mavibot partition in APACHEDS? Because the mavibot parttion I created can't be viewed when click the "Open Configuration" in apacheds studio. Also I used unboundid JDK to insert many entries apacheds and the same problem happened. So someone can tell me how to config the mavibot partition on APACHEDS? That's may be very helpful.
      Below is the exceptions from apacheds when problems happened:
      [16:21:41] INFO [org.apache.directory.server.ldap.LdapServer] - Ldap service stopped.
      [16:21:41] WARN [org.apache.directory.server.core.shared.partition.DefaultPartitionNexus] - Failed to flush partition data out.
      org.apache.directory.api.ldap.model.exception.LdapOperationErrorException
      at org.apache.directory.server.core.partition.impl.btree.AbstractBTreePartition.saveContextCsn(AbstractBTreePartition.java:3364)
      at org.apache.directory.server.core.shared.partition.DefaultPartitionNexus.sync(DefaultPartitionNexus.java:319)
      at org.apache.directory.server.core.DefaultDirectoryService.shutdown(DefaultDirectoryService.java:1283)
      at org.apache.directory.server.ApacheDsService.stop(ApacheDsService.java:600)
      at org.apache.directory.server.wrapper.ApacheDsTanukiWrapper.stop(ApacheDsTanukiWrapper.java:97)
      at org.tanukisoftware.wrapper.WrapperManager$13.run(WrapperManager.java:3134)
      Caused by: java.lang.NullPointerException
      at org.apache.directory.server.core.partition.impl.btree.AbstractBTreePartition.saveContextCsn(AbstractBTreePartition.java:3350)
      ... 5 more
      [16:21:41] ERROR [org.apache.directory.server.wrapper.ApacheDsTanukiWrapper] - Failed to stop the service.
      org.apache.directory.api.util.exception.MultiException: ERR_265 Grouping many exceptions on root nexus sync()
      at org.apache.directory.server.core.shared.partition.DefaultPartitionNexus.sync(DefaultPartitionNexus.java:328)
      at org.apache.directory.server.core.DefaultDirectoryService.shutdown(DefaultDirectoryService.java:1283)
      at org.apache.directory.server.ApacheDsService.stop(ApacheDsService.java:600)
      at org.apache.directory.server.wrapper.ApacheDsTanukiWrapper.stop(ApacheDsTanukiWrapper.java:97)
      at org.tanukisoftware.wrapper.WrapperManager$13.run(WrapperManager.java:3134)
      Nested exceptions to follow:
      INFO | jvm 1 | 2015/02/02 17:11:06 | [17:11:06] WARN [org.apache.directory.server.ldap.LdapProtocolHandler] - Unexpected exception forcing session to close: sending disconnect notice to client.
      INFO | jvm 1 | 2015/02/02 17:11:06 | java.io.IOException: Connection reset by peer
      INFO | jvm 1 | 2015/02/02 17:11:06 | at sun.nio.ch.FileDispatcherImpl.read0(Native Method)
      INFO | jvm 1 | 2015/02/02 17:11:06 | at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
      INFO | jvm 1 | 2015/02/02 17:11:06 | at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223)
      INFO | jvm 1 | 2015/02/02 17:11:06 | at sun.nio.ch.IOUtil.read(IOUtil.java:197)
      INFO | jvm 1 | 2015/02/02 17:11:06 | at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:379)
      INFO | jvm 1 | 2015/02/02 17:11:06 | at org.apache.mina.transport.socket.nio.NioProcessor.read(NioProcessor.java:311)
      INFO | jvm 1 | 2015/02/02 17:11:06 | at org.apache.mina.transport.socket.nio.NioProcessor.read(NioProcessor.java:45)
      INFO | jvm 1 | 2015/02/02 17:11:06 | at org.apache.mina.core.polling.AbstractPollingIoProcessor.read(AbstractPollingIoProcessor.java:694)
      INFO | jvm 1 | 2015/02/02 17:11:06 | at org.apache.mina.core.polling.AbstractPollingIoProcessor.process(AbstractPollingIoProcessor.java:668)
      INFO | jvm 1 | 2015/02/02 17:11:06 | at org.apache.mina.core.polling.AbstractPollingIoProcessor.process(AbstractPollingIoProcessor.java:657)
      INFO | jvm 1 | 2015/02/02 17:11:06 | at org.apache.mina.core.polling.AbstractPollingIoProcessor.access$600(AbstractPollingIoProcessor.java:67)
      INFO | jvm 1 | 2015/02/02 17:11:06 | at org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.run(AbstractPollingIoProcessor.java:1121)
      INFO | jvm 1 | 2015/02/02 17:11:06 | at org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:64)
      INFO | jvm 1 | 2015/02/02 17:11:06 | at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
      INFO | jvm 1 | 2015/02/02 17:11:06 | at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
      INFO | jvm 1 | 2015/02/02 17:11:06 | at java.lang.Thread.run(Thread.java:745)

      1. site_topology_schema.ldif
        57 kB
        linzhao
      2. supercluster_partition.ldif
        5 kB
        linzhao
      3. site.ldif
        337 kB
        linzhao
      4. ordered-data.ldif
        5 kB
        Emmanuel Lecharny

        Activity

        Hide
        elecharny Emmanuel Lecharny added a comment -

        Hmmm, weird bug... We definitively have to do some test.

        You get this error while stopping the server, right ?

        Otherwise, we are working on a bulk load system that will be way faster than using ldapadd, it's just not finished yet.

        Show
        elecharny Emmanuel Lecharny added a comment - Hmmm, weird bug... We definitively have to do some test. You get this error while stopping the server, right ? Otherwise, we are working on a bulk load system that will be way faster than using ldapadd, it's just not finished yet.
        Hide
        wayswssb linzhao added a comment -

        Hi Emmanuel,

        I also used unboundid JDK to do the back and restore function. But I found that sometimes apacheds still doesn't work for this function. My thought is that the mavibot partition "ou=supercluster" I created has some problems. I just want to do the backup and restore work for this partition. But after I create mavibot "ou=supercluster" partition, I can't read the "ou=supercluster" partition from apacheds studio. It shows that Unable to load the configuration ERR_04274 Can't find an OID for the name ads-mavibotPartition.

        Do you have some documents about how to create the mavibot partition? I will upload my mavibot partition ldif file into this bug, could you help to check it?

        One thing I want to say is that this bug happened randomly, but it is easily to reproduce. When insert many entries into apacheds at one time, some data can be lost.

        Show
        wayswssb linzhao added a comment - Hi Emmanuel, I also used unboundid JDK to do the back and restore function. But I found that sometimes apacheds still doesn't work for this function. My thought is that the mavibot partition "ou=supercluster" I created has some problems. I just want to do the backup and restore work for this partition. But after I create mavibot "ou=supercluster" partition, I can't read the "ou=supercluster" partition from apacheds studio. It shows that Unable to load the configuration ERR_04274 Can't find an OID for the name ads-mavibotPartition. Do you have some documents about how to create the mavibot partition? I will upload my mavibot partition ldif file into this bug, could you help to check it? One thing I want to say is that this bug happened randomly, but it is easily to reproduce. When insert many entries into apacheds at one time, some data can be lost.
        Hide
        wayswssb linzhao added a comment -

        Yes, I got this error while stopping the service. But when the server is running, sometimes the backup and restore function doesn't work. Please see the comments above.

        Show
        wayswssb linzhao added a comment - Yes, I got this error while stopping the service. But when the server is running, sometimes the backup and restore function doesn't work. Please see the comments above.
        Hide
        wayswssb linzhao added a comment -

        The mavibot partition ldif file I attached.

        Show
        wayswssb linzhao added a comment - The mavibot partition ldif file I attached.
        Hide
        wayswssb linzhao added a comment -

        I found someone find the same problems with avlpartition. Please see this link https://issues.apache.org/jira/browse/DIRSERVER.

        Show
        wayswssb linzhao added a comment - I found someone find the same problems with avlpartition. Please see this link https://issues.apache.org/jira/browse/DIRSERVER .
        Hide
        wayswssb linzhao added a comment -

        Also please see this issue DIRSERVER-1991, I can reproduce this issue with mavibot partition.

        Show
        wayswssb linzhao added a comment - Also please see this issue DIRSERVER-1991 , I can reproduce this issue with mavibot partition.
        Hide
        elecharny Emmanuel Lecharny added a comment -

        Ok, many thanks for your input. I'm currently away for one week, with little time on my computer, but I'll have a loo, as soon I'm back.

        Show
        elecharny Emmanuel Lecharny added a comment - Ok, many thanks for your input. I'm currently away for one week, with little time on my computer, but I'll have a loo, as soon I'm back.
        Hide
        elecharny Emmanuel Lecharny added a comment -

        I have tested DIRSERVER-1991 with a Mavibot Partition, doing :

        • add 10 000 entries
        • lookup 400 000 entries randomly
        • delete the 10 000 entries
        • add 10 000 entries

        No problem at all.

        I suspect that he real problem is that the LDIF file is not ordered, thus reinjecting it into the server fails due to some missing parents.

        Show
        elecharny Emmanuel Lecharny added a comment - I have tested DIRSERVER-1991 with a Mavibot Partition, doing : add 10 000 entries lookup 400 000 entries randomly delete the 10 000 entries add 10 000 entries No problem at all. I suspect that he real problem is that the LDIF file is not ordered, thus reinjecting it into the server fails due to some missing parents.
        Hide
        elecharny Emmanuel Lecharny added a comment -

        Actually, your data.ldif file is unordred : it contains entries which parent is declared later in the file.

        Show
        elecharny Emmanuel Lecharny added a comment - Actually, your data.ldif file is unordred : it contains entries which parent is declared later in the file.
        Hide
        elecharny Emmanuel Lecharny added a comment -

        Can you give the ordered-data.ldif file I just attached a try ?

        Show
        elecharny Emmanuel Lecharny added a comment - Can you give the ordered-data.ldif file I just attached a try ?
        Hide
        wayswssb linzhao added a comment -

        I have tried the ordered-data.ldif, but the problem still happens. This bug can't be reproduced every time, but it can be reproduced easily. Any problem with my mavibot partition? Also I found that the lost data can be found in mavibot.db file. But it can't be read out from the backend database. Any suggestion for this?

        Show
        wayswssb linzhao added a comment - I have tried the ordered-data.ldif, but the problem still happens. This bug can't be reproduced every time, but it can be reproduced easily. Any problem with my mavibot partition? Also I found that the lost data can be found in mavibot.db file. But it can't be read out from the backend database. Any suggestion for this?
        Hide
        wayswssb linzhao added a comment -

        Another question is that how to debug apacheds? Now I can compile and build the apacheds successfully.

        Show
        wayswssb linzhao added a comment - Another question is that how to debug apacheds? Now I can compile and build the apacheds successfully.
        Hide
        elecharny Emmanuel Lecharny added a comment - - edited

        We usually launch ApacheDS inside Eclipse, in debug mode. The starting point is the UberjarMain class, in the apacheds-service module.

        Show
        elecharny Emmanuel Lecharny added a comment - - edited We usually launch ApacheDS inside Eclipse, in debug mode. The starting point is the UberjarMain class, in the apacheds-service module.
        Hide
        akiran Kiran Ayyagari added a comment -

        Another question is that how to debug apacheds?

        you can also run the server in debug mode from command line
        1. go to <apacheds-root>/apacheds/service
        2. run ./apacheds.sh -debug
        this will run JDWP at port 8008.

        Show
        akiran Kiran Ayyagari added a comment - Another question is that how to debug apacheds? you can also run the server in debug mode from command line 1. go to <apacheds-root>/apacheds/service 2. run ./apacheds.sh -debug this will run JDWP at port 8008.
        Hide
        wayswssb linzhao added a comment -

        I can debug the apacheds now. I also found that the problem happened at MavibotCursor class next() method. When the problem happened, the next() method always return false. Also I use apacheds studio to see it, the node still can't be seen. But this problem only happened when the data is huge, for me, I test more than 500 entries and the problem can be reproduced.

        Show
        wayswssb linzhao added a comment - I can debug the apacheds now. I also found that the problem happened at MavibotCursor class next() method. When the problem happened, the next() method always return false. Also I use apacheds studio to see it, the node still can't be seen. But this problem only happened when the data is huge, for me, I test more than 500 entries and the problem can be reproduced.
        Hide
        wayswssb linzhao added a comment -

        Please see this pictures, the ou=nodes should have some data. But it can't be browse from the apacheds studio, but I am sure it existed in mavibot.db file.

        Show
        wayswssb linzhao added a comment - Please see this pictures, the ou=nodes should have some data. But it can't be browse from the apacheds studio, but I am sure it existed in mavibot.db file.
        Hide
        wayswssb linzhao added a comment -

        Also sometimes I can only see single node ou=Supercluster under apacheds studio, but there is lots of data under ou=supercluster node.

        Show
        wayswssb linzhao added a comment - Also sometimes I can only see single node ou=Supercluster under apacheds studio, but there is lots of data under ou=supercluster node.
        Hide
        wayswssb linzhao added a comment -

        I have more findings for this bug, I debug the mavibot M6 and found that when the problem happened, hasNext in TupleCursor class always return false. "return false" happened at line 197 parentPos.pos == AFTER_LAST. It seems that parentPos.pos always equals to -2.

        Show
        wayswssb linzhao added a comment - I have more findings for this bug, I debug the mavibot M6 and found that when the problem happened, hasNext in TupleCursor class always return false. "return false" happened at line 197 parentPos.pos == AFTER_LAST. It seems that parentPos.pos always equals to -2.
        Hide
        elecharny Emmanuel Lecharny added a comment -

        Have you set any limit on the number of entries that can be returned by the server ?

        Show
        elecharny Emmanuel Lecharny added a comment - Have you set any limit on the number of entries that can be returned by the server ?
        Hide
        elecharny Emmanuel Lecharny added a comment -

        More specifcially, check the ads-maxSizeLimit parameter.

        Show
        elecharny Emmanuel Lecharny added a comment - More specifcially, check the ads-maxSizeLimit parameter.
        Hide
        wayswssb linzhao added a comment -

        I didn't set ads-maxSizeLimit in my program, so do I need to do this?

        Show
        wayswssb linzhao added a comment - I didn't set ads-maxSizeLimit in my program, so do I need to do this?
        Hide
        wayswssb linzhao added a comment -

        This bug can randomly reproduced, but it can be reproduced easily, so is it related to the ads-maxSizeLimit?

        Show
        wayswssb linzhao added a comment - This bug can randomly reproduced, but it can be reproduced easily, so is it related to the ads-maxSizeLimit?
        Hide
        wayswssb linzhao added a comment -

        I found that maxSizeLimit is 1000 on my apacheds, but the entries I insert is less than 1000.

        Show
        wayswssb linzhao added a comment - I found that maxSizeLimit is 1000 on my apacheds, but the entries I insert is less than 1000.
        Hide
        wayswssb linzhao added a comment -

        Any updates for this case? I can provide the apacheds log for this case.

        Show
        wayswssb linzhao added a comment - Any updates for this case? I can provide the apacheds log for this case.
        Hide
        wayswssb linzhao added a comment -

        I can find that when the problem happened, findPos method AbstractPage can't find the position for this key, so it always move the cursor to the last. Array keys in AbstractPage page has some problems.

        Show
        wayswssb linzhao added a comment - I can find that when the problem happened, findPos method AbstractPage can't find the position for this key, so it always move the cursor to the last. Array keys in AbstractPage page has some problems.
        Hide
        elecharny Emmanuel Lecharny added a comment -

        I wasn't able to reproduce the pb on my laptop. I have written a test that inject 10 000 entries into a server, it works like a charm, as I said.

        At this point, any data you have can help :

        • attach the logs you have
        • if you have a ldif file that make the server choke, please attach it too. If it contains private informations, can you strip them from the file ? Or you can mail me directly (elecharny at apache.org) so that we can discuss about the pb and how to reproduce it on my machine
        • can you also provide info about the hardware you are running the code on (CPU, memory, disk, etc) and JVM being used

        Thanks !

        Show
        elecharny Emmanuel Lecharny added a comment - I wasn't able to reproduce the pb on my laptop. I have written a test that inject 10 000 entries into a server, it works like a charm, as I said. At this point, any data you have can help : attach the logs you have if you have a ldif file that make the server choke, please attach it too. If it contains private informations, can you strip them from the file ? Or you can mail me directly (elecharny at apache.org) so that we can discuss about the pb and how to reproduce it on my machine can you also provide info about the hardware you are running the code on (CPU, memory, disk, etc) and JVM being used Thanks !
        Hide
        wayswssb linzhao added a comment -

        this is the ldif file I used. If you delete all the data and remove them again, it will reproduce the issue easily.

        Show
        wayswssb linzhao added a comment - this is the ldif file I used. If you delete all the data and remove them again, it will reproduce the issue easily.
        Hide
        elecharny Emmanuel Lecharny added a comment -

        Thanks ! I'll give it a try and come back to you...

        Show
        elecharny Emmanuel Lecharny added a comment - Thanks ! I'll give it a try and come back to you...
        Hide
        elecharny Emmanuel Lecharny added a comment -

        Ah, damn... I will also need the schema you are using, otherwise I can't import the ldif.

        Show
        elecharny Emmanuel Lecharny added a comment - Ah, damn... I will also need the schema you are using, otherwise I can't import the ldif.
        Hide
        wayswssb linzhao added a comment -

        Sorry, I should say that you can try this file, this file can reproduce this bug more easily. When you import your data done, please make sure "ou=Configuration", "ou=mediaPathExclusion", "ou=networks", "ou=nodeds", "ou=sitelinks", "ou=sites", "ou=territories", "ou=territoryOwners" has data. And I also attaced the ou=supercluster partition and site_topology_schema.ldif.

        Show
        wayswssb linzhao added a comment - Sorry, I should say that you can try this file, this file can reproduce this bug more easily. When you import your data done, please make sure "ou=Configuration", "ou=mediaPathExclusion", "ou=networks", "ou=nodeds", "ou=sitelinks", "ou=sites", "ou=territories", "ou=territoryOwners" has data. And I also attaced the ou=supercluster partition and site_topology_schema.ldif.
        Hide
        elecharny Emmanuel Lecharny added a comment -

        Weird...

        I can import all the entries without any problem, but the server does not show ma any entry under ou=sites ...

        Show
        elecharny Emmanuel Lecharny added a comment - Weird... I can import all the entries without any problem, but the server does not show ma any entry under ou=sites ...
        Hide
        elecharny Emmanuel Lecharny added a comment -

        There is definitively something very broken.

        I can see the entries when I do a full search from the ou=SuperCluster, searching for (siteUid=*), search that does not work when I do the same search from ou=sites,ou=superCluster.

        Although all the entries are present in the server.

        I'm debugging now.

        Show
        elecharny Emmanuel Lecharny added a comment - There is definitively something very broken. I can see the entries when I do a full search from the ou=SuperCluster, searching for (siteUid=*), search that does not work when I do the same search from ou=sites,ou=superCluster. Although all the entries are present in the server. I'm debugging now.
        Hide
        elecharny Emmanuel Lecharny added a comment -

        It seems we are hitting a very specific and rare use case : the key we are looking for in a BTree is the first one in a page, and we want to browse from this element. What happens is that we select the leaf just before the one containing the element, and move the cursor AFTER_LAST for this page. And now, as it's on the last position of the previous leaf, the cursor just returns immediately, instead of checking if there are other following leaves.

        Show
        elecharny Emmanuel Lecharny added a comment - It seems we are hitting a very specific and rare use case : the key we are looking for in a BTree is the first one in a page, and we want to browse from this element. What happens is that we select the leaf just before the one containing the element, and move the cursor AFTER_LAST for this page. And now, as it's on the last position of the previous leaf, the cursor just returns immediately, instead of checking if there are other following leaves.
        Hide
        elecharny Emmanuel Lecharny added a comment -

        Ok, the clear guilty is the TupleCursor.hasNext() method :

            public boolean hasNext() throws EndOfFileExceededException, IOException
            {
                // First check that we have elements in the BTree
                if ( ( stack == null ) || ( stack.length == 0 ) )
                {
                    return false;
                }
        
                // Take the leaf and check if we have no mare values
                ParentPos<K, V> parentPos = stack[depth];
        
                if ( parentPos.page == null )
                {
                    // Empty BTree, get out
                    return false;
                }
        
                if ( parentPos.pos == AFTER_LAST )
                {
                    return false; <---- DAMN WRONG !!!
                }
        ...
        

        If the current parentPos.pos is AFTER_LAST, we have to check that we don't have some more pages in the stack.

        Show
        elecharny Emmanuel Lecharny added a comment - Ok, the clear guilty is the TupleCursor.hasNext() method : public boolean hasNext() throws EndOfFileExceededException, IOException { // First check that we have elements in the BTree if ( ( stack == null ) || ( stack.length == 0 ) ) { return false ; } // Take the leaf and check if we have no mare values ParentPos<K, V> parentPos = stack[depth]; if ( parentPos.page == null ) { // Empty BTree, get out return false ; } if ( parentPos.pos == AFTER_LAST ) { return false ; <---- DAMN WRONG !!! } ... If the current parentPos.pos is AFTER_LAST , we have to check that we don't have some more pages in the stack.
        Hide
        elecharny Emmanuel Lecharny added a comment -

        wgthdxt!%&!!!

        This is worse than that. We don't set the position correctly at the very beginning when we create the browser...

        Show
        elecharny Emmanuel Lecharny added a comment - wgthdxt!%&!!! This is worse than that. We don't set the position correctly at the very beginning when we create the browser...
        Hide
        elecharny Emmanuel Lecharny added a comment -

        I have created a test that fails consistently when we do browse a BTree : PersistedBTreeBrowseTest.testBrowseBTree500. That proves we have and issue in the way we build the cursor.

        Show
        elecharny Emmanuel Lecharny added a comment - I have created a test that fails consistently when we do browse a BTree : PersistedBTreeBrowseTest.testBrowseBTree500 . That proves we have and issue in the way we build the cursor.
        Hide
        wayswssb linzhao added a comment -

        Do you have the quick solution for this issue? I need a quick update for this issue. Thanks.

        Show
        wayswssb linzhao added a comment - Do you have the quick solution for this issue? I need a quick update for this issue. Thanks.
        Hide
        wayswssb linzhao added a comment -

        Can you share your test case with me? I can help you to debug the test case.

        Show
        wayswssb linzhao added a comment - Can you share your test case with me? I can help you to debug the test case.
        Hide
        elecharny Emmanuel Lecharny added a comment -

        The test case has been committed into Mavibot, you just have to check out the code.

        And, no, I don't have a quick solution atm, just because I don't know yet what is wrong. But as soon I understand what is broken, then the fix will be easy.

        Show
        elecharny Emmanuel Lecharny added a comment - The test case has been committed into Mavibot, you just have to check out the code. And, no, I don't have a quick solution atm, just because I don't know yet what is wrong. But as soon I understand what is broken, then the fix will be easy.
        Hide
        elecharny Emmanuel Lecharny added a comment -

        sigh... The test case is valid. the browseFrom() method works well...

        Show
        elecharny Emmanuel Lecharny added a comment - sigh... The test case is valid. the browseFrom() method works well...
        Hide
        wayswssb linzhao added a comment -

        ...Do you have a test case can reproduce the issue? I need it to debug the problem. It is very urgent for me to debug this problem.

        Show
        wayswssb linzhao added a comment - ...Do you have a test case can reproduce the issue? I need it to debug the problem. It is very urgent for me to debug this problem.
        Hide
        elecharny Emmanuel Lecharny added a comment -

        I yhink I have a fix. Running tests atm.

        Show
        elecharny Emmanuel Lecharny added a comment - I yhink I have a fix. Running tests atm.
        Hide
        wayswssb linzhao added a comment -

        You mean you have fixed the problem? You already commited the code? Let me check out the code and run the tests again.

        Show
        wayswssb linzhao added a comment - You mean you have fixed the problem? You already commited the code? Let me check out the code and run the tests again.
        Hide
        wayswssb linzhao added a comment - - edited

        I saw you only change the test code and I can run the test case on my VM. The test case can be passed. But since you didn't change the source code, so my thought is that for my site case will still fail. So you will change the source code later?

        Show
        wayswssb linzhao added a comment - - edited I saw you only change the test code and I can run the test case on my VM. The test case can be passed. But since you didn't change the source code, so my thought is that for my site case will still fail. So you will change the source code later?
        Hide
        elecharny Emmanuel Lecharny added a comment -

        Tests were not conclusive. It's harder than I thought, and I'm still fighting to get a fix cut.

        Basicallly, it all boils down to a wrong cursor construction, as we don't set the initial position at the right place when browsing from a given key (which is what we do when we do a search using a specific position in the DIT). What happen is that we use the RdnIndex, and try to find the RDN corresponding to the BaseDN used. When found, we are supposed to take all the descendant of this RDN (that will be all the entries below this baseDN), which has the same parent ID - the ID for this BaseDN -. Depending on the B-tree layout, we may hit some corner cases where we have to move across a page, and it's not computed correctly.

        The fix is not simple as we have to move up and down in the BTree.

        Show
        elecharny Emmanuel Lecharny added a comment - Tests were not conclusive. It's harder than I thought, and I'm still fighting to get a fix cut. Basicallly, it all boils down to a wrong cursor construction, as we don't set the initial position at the right place when browsing from a given key (which is what we do when we do a search using a specific position in the DIT). What happen is that we use the RdnIndex, and try to find the RDN corresponding to the BaseDN used. When found, we are supposed to take all the descendant of this RDN (that will be all the entries below this baseDN), which has the same parent ID - the ID for this BaseDN -. Depending on the B-tree layout, we may hit some corner cases where we have to move across a page, and it's not computed correctly. The fix is not simple as we have to move up and down in the BTree.
        Hide
        wayswssb linzhao added a comment -

        If you can have a test case can reproduce this issue, I can help you with this issue.

        Show
        wayswssb linzhao added a comment - If you can have a test case can reproduce this issue, I can help you with this issue.
        Hide
        wayswssb linzhao added a comment -

        So this issue is related to the process of construct or the process of query tree, if the problem is related to the process of construct tree, it may be different to fix.

        Show
        wayswssb linzhao added a comment - So this issue is related to the process of construct or the process of query tree, if the problem is related to the process of construct tree, it may be different to fix.
        Hide
        wayswssb linzhao added a comment - - edited

        My code change from your thought:
        public TupleCursor<K, V> browse( K key, ReadTransaction<K, V> transaction, ParentPos<K, V>[] stack, int depth )
        throws IOException
        {
        int pos = findPos( key );

        if ( pos < 0 )

        { pos = -pos; }

        // We first stack the current page
        stack[depth++] = new ParentPos<K, V>( this, pos );

        Page<K, V> page = children[pos].getValue();

        TupleCursor<K, V> oldTupleCursor = page.browse( key, transaction, stack, depth );
        TupleCursor<K, V> tupleCursor = oldTupleCursor;
        List<Integer> integers = new ArrayList<Integer>();
        integers.add(Integer.valueOf(pos));
        while (!tupleCursor.hasNext())
        {
        pos = findPos(integers, key);
        if (pos == keys.length)

        { break; }

        stack[depth] = new ParentPos<K, V>( this, pos );
        page = children[pos].getValue();
        tupleCursor = page.browse( key, transaction, stack, depth );
        }
        return oldTupleCursor;
        }

        public int findPos(List<Integer> integers, K key)
        {
        for (int i = 0;i < keys.length;i++)
        {
        if (compare( keys[i].getKey(), key ) == 0 && !integers.contains(Integer.valueOf))

        { integers.add(Integer.valueOf(i)); return i; }

        }
        return keys.length;
        }

        BTW, the code change happened in AbstartPage,java. Please help to review it. Thanks.

        Show
        wayswssb linzhao added a comment - - edited My code change from your thought: public TupleCursor<K, V> browse( K key, ReadTransaction<K, V> transaction, ParentPos<K, V>[] stack, int depth ) throws IOException { int pos = findPos( key ); if ( pos < 0 ) { pos = -pos; } // We first stack the current page stack [depth++] = new ParentPos<K, V>( this, pos ); Page<K, V> page = children [pos] .getValue(); TupleCursor<K, V> oldTupleCursor = page.browse( key, transaction, stack, depth ); TupleCursor<K, V> tupleCursor = oldTupleCursor; List<Integer> integers = new ArrayList<Integer>(); integers.add(Integer.valueOf(pos)); while (!tupleCursor.hasNext()) { pos = findPos(integers, key); if (pos == keys.length) { break; } stack [depth] = new ParentPos<K, V>( this, pos ); page = children [pos] .getValue(); tupleCursor = page.browse( key, transaction, stack, depth ); } return oldTupleCursor; } public int findPos(List<Integer> integers, K key) { for (int i = 0;i < keys.length;i++) { if (compare( keys [i] .getKey(), key ) == 0 && !integers.contains(Integer.valueOf )) { integers.add(Integer.valueOf(i)); return i; } } return keys.length; } BTW, the code change happened in AbstartPage,java. Please help to review it. Thanks.
        Hide
        wayswssb linzhao added a comment -

        Since this is mavibot bug, and mavibot is jar in apacheds project, so it is not very easy for me to test the site.ldif case. Please help to test it. Thanks.

        Show
        wayswssb linzhao added a comment - Since this is mavibot bug, and mavibot is jar in apacheds project, so it is not very easy for me to test the site.ldif case. Please help to test it. Thanks.
        Hide
        elecharny Emmanuel Lecharny added a comment -

        Ok, I think I have a real fix for the pb now. It's slightly more complex that what you are proposing, but at least the mavibot tests are passing, and I'm currently running ApacheDS tests. I was also able to see all the entries using a mavibot partition with your data.

        I'm currently in a train for 3 hours, so due to the less than perfect internet connection I do have, I may be able to commit only later.

        In any case, if this fix flies, I have to thank you for you support and the effective informations you provided !

        Also keep in mind that it's really still highly experimental, but we will definitively switch to mavibot asap for the whole server.

        Show
        elecharny Emmanuel Lecharny added a comment - Ok, I think I have a real fix for the pb now. It's slightly more complex that what you are proposing, but at least the mavibot tests are passing, and I'm currently running ApacheDS tests. I was also able to see all the entries using a mavibot partition with your data. I'm currently in a train for 3 hours, so due to the less than perfect internet connection I do have, I may be able to commit only later. In any case, if this fix flies, I have to thank you for you support and the effective informations you provided ! Also keep in mind that it's really still highly experimental, but we will definitively switch to mavibot asap for the whole server.
        Hide
        elecharny Emmanuel Lecharny added a comment -

        I just pushed a fix : http://svn.apache.org/r1659557 and http://svn.apache.org/r1659558.

        Feel free to test it. You will need to build mavibot first, then to build ApacheDS.

        Show
        elecharny Emmanuel Lecharny added a comment - I just pushed a fix : http://svn.apache.org/r1659557 and http://svn.apache.org/r1659558 . Feel free to test it. You will need to build mavibot first, then to build ApacheDS.
        Hide
        wayswssb linzhao added a comment -

        OK, thanks for your support, I will test it and let you know the result.

        Show
        wayswssb linzhao added a comment - OK, thanks for your support, I will test it and let you know the result.
        Hide
        wayswssb linzhao added a comment -

        BTW, you may need t o test my case for many times, the problem can't be reproduced every time. Anyhow, I will test my case with your code either.

        Show
        wayswssb linzhao added a comment - BTW, you may need t o test my case for many times, the problem can't be reproduced every time. Anyhow, I will test my case with your code either.
        Hide
        wayswssb linzhao added a comment -

        I will tell you that the problem still existed, NullPointerException happened in org.apache.directory.server.core.partition.impl.btree.AbstractBTreePartition.java line 1154, but it is very strange, the NullPointerException has no stack trace at all. The problem is very easy to happen.

        Show
        wayswssb linzhao added a comment - I will tell you that the problem still existed, NullPointerException happened in org.apache.directory.server.core.partition.impl.btree.AbstractBTreePartition.java line 1154, but it is very strange, the NullPointerException has no stack trace at all. The problem is very easy to happen.
        Hide
        wayswssb linzhao added a comment -

        in TupleCursor.java line 229
        if ( parentPos.valueCursor != null && parentPos.valueCursor.hasNext() )
        {
        return true;
        }

        Sometimes, valueCurors in parentPos is null. I fixed NPE and will retest it.

        Show
        wayswssb linzhao added a comment - in TupleCursor.java line 229 if ( parentPos.valueCursor != null && parentPos.valueCursor.hasNext() ) { return true; } Sometimes, valueCurors in parentPos is null. I fixed NPE and will retest it.
        Hide
        wayswssb linzhao added a comment - - edited

        But I didn't think your code change affects this line, why does your change cause the NullPointerException? What I did is just put your fix into mavibot M6 code, I only change the TupleCursor and PersistedLeaf class.

        Show
        wayswssb linzhao added a comment - - edited But I didn't think your code change affects this line, why does your change cause the NullPointerException? What I did is just put your fix into mavibot M6 code, I only change the TupleCursor and PersistedLeaf class.
        Hide
        wayswssb linzhao added a comment -

        I saw that you create the valueCursor in PersistentedLeaf.java, so some problems happened in your browser method?

        Show
        wayswssb linzhao added a comment - I saw that you create the valueCursor in PersistentedLeaf.java, so some problems happened in your browser method?
        Hide
        wayswssb linzhao added a comment -

        I just change the code to prevent NullPointerException, but I should say the problem can still be reproduced. The hasNext method in TupleCursor.java still return false. This issue is very strange.

        Show
        wayswssb linzhao added a comment - I just change the code to prevent NullPointerException, but I should say the problem can still be reproduced. The hasNext method in TupleCursor.java still return false. This issue is very strange.
        Hide
        wayswssb linzhao added a comment - - edited

        My thought is that if the position is wrong, sometimes the parent position of the leaf is still wrong, so the problem happened.

        Show
        wayswssb linzhao added a comment - - edited My thought is that if the position is wrong, sometimes the parent position of the leaf is still wrong, so the problem happened.
        Hide
        elecharny Emmanuel Lecharny added a comment -

        There is defintively a pb when we have multiple values associated with a key. I have a test case reproducing the pb, debugging it atm.

        Show
        elecharny Emmanuel Lecharny added a comment - There is defintively a pb when we have multiple values associated with a key. I have a test case reproducing the pb, debugging it atm.
        Hide
        wayswssb linzhao added a comment - - edited

        OK, I will also build a pb with multiple values associated with a key. If key associated with multiple values, how to find the correct cursor for this key. I am confused with the algorithm for constructing mavibot btree. Could you explain it?

        Show
        wayswssb linzhao added a comment - - edited OK, I will also build a pb with multiple values associated with a key. If key associated with multiple values, how to find the correct cursor for this key. I am confused with the algorithm for constructing mavibot btree. Could you explain it?
        Hide
        wayswssb linzhao added a comment -

        I found another question, the length of the children and keys array are different in AbstractPage, is this correct?

        Show
        wayswssb linzhao added a comment - I found another question, the length of the children and keys array are different in AbstractPage, is this correct?
        Hide
        elecharny Emmanuel Lecharny added a comment -

        This test is wrong.

        When you do a browseFrom( K ), you just set the cursor position before K, then you will iterate up to the end of the btree. In your case, the first loop set the key to uid1, which is by no means guaranteed to be the first key (its value is randomely generated), so you can have a loop across 50, 100 or 150 elements.

        Here is a fixed test :

            @Test
            public void testBrowseBTreeMV() throws IOException, BTreeAlreadyManagedException, KeyNotFoundException
            {
                String uid1 = "uid1";
                String uid2 = "uid2";
                String uid3 = "uid3";
                System.out.println( "uid1: " + uid1 );
                System.out.println( "uid2: " + uid2 );
                System.out.println( "uid3: " + uid3 );
        
                // Inject some data
                for ( int i = 0; i < 50; i++ )
                {
                    btreeStr.insert( uid1, UUID.randomUUID().toString() );
                }
        
                for ( int i = 0; i < 50; i++ )
                {
                    btreeStr.insert( uid2, UUID.randomUUID().toString() );
                }
        
                for ( int i = 0; i < 50; i++ )
                {
                    btreeStr.insert( uid3, UUID.randomUUID().toString() );
                }
        
                // Now, browse the BTree starting from 0 to the end
                TupleCursor<String, String> cursor = btreeStr.browseFrom( uid1 );
                assertTrue( cursor.hasNext() );
                String key = null;
                int count = 0;
        
                while ( cursor.hasNext() )
                {
                    Tuple<String, String> tuple = cursor.next();
                    key = tuple.getKey();
                    System.out.println( count );
        
                    if ( count < 50 )
                    {
                        assertEquals( uid1, key );
                    }
                    else if ( count < 100 )
                    {
                        assertEquals( uid2, key );
                    }
                    else
                    {
                        assertEquals( uid3, key );
                    }
        
                    count++;
                }
        
                assertEquals( 150, count );
        
                cursor = btreeStr.browseFrom( uid2 );
                assertTrue( cursor.hasNext() );
                key = null;
                count = 0;
        
                while ( cursor.hasNext() )
                {
                    Tuple<String, String> tuple = cursor.next();
                    key = tuple.getKey();
        
                    if ( count < 50 )
                    {
                        assertEquals( uid2, key );
                    }
                    else
                    {
                        assertEquals( uid3, key );
                    }
        
                    count++;
                }
        
                assertEquals( 100, count );
        
                cursor = btreeStr.browseFrom( uid3 );
                assertTrue( cursor.hasNext() );
                key = null;
                count = 0;
        
                while ( cursor.hasNext() )
                {
                    Tuple<String, String> tuple = cursor.next();
                    key = tuple.getKey();
                    assertEquals( uid3, key );
                    count++;
                }
        
                assertEquals( 50, count );
            }
        
        Show
        elecharny Emmanuel Lecharny added a comment - This test is wrong. When you do a browseFrom( K ) , you just set the cursor position before K, then you will iterate up to the end of the btree. In your case, the first loop set the key to uid1, which is by no means guaranteed to be the first key (its value is randomely generated), so you can have a loop across 50, 100 or 150 elements. Here is a fixed test : @Test public void testBrowseBTreeMV() throws IOException, BTreeAlreadyManagedException, KeyNotFoundException { String uid1 = "uid1" ; String uid2 = "uid2" ; String uid3 = "uid3" ; System .out.println( "uid1: " + uid1 ); System .out.println( "uid2: " + uid2 ); System .out.println( "uid3: " + uid3 ); // Inject some data for ( int i = 0; i < 50; i++ ) { btreeStr.insert( uid1, UUID.randomUUID().toString() ); } for ( int i = 0; i < 50; i++ ) { btreeStr.insert( uid2, UUID.randomUUID().toString() ); } for ( int i = 0; i < 50; i++ ) { btreeStr.insert( uid3, UUID.randomUUID().toString() ); } // Now, browse the BTree starting from 0 to the end TupleCursor< String , String > cursor = btreeStr.browseFrom( uid1 ); assertTrue( cursor.hasNext() ); String key = null ; int count = 0; while ( cursor.hasNext() ) { Tuple< String , String > tuple = cursor.next(); key = tuple.getKey(); System .out.println( count ); if ( count < 50 ) { assertEquals( uid1, key ); } else if ( count < 100 ) { assertEquals( uid2, key ); } else { assertEquals( uid3, key ); } count++; } assertEquals( 150, count ); cursor = btreeStr.browseFrom( uid2 ); assertTrue( cursor.hasNext() ); key = null ; count = 0; while ( cursor.hasNext() ) { Tuple< String , String > tuple = cursor.next(); key = tuple.getKey(); if ( count < 50 ) { assertEquals( uid2, key ); } else { assertEquals( uid3, key ); } count++; } assertEquals( 100, count ); cursor = btreeStr.browseFrom( uid3 ); assertTrue( cursor.hasNext() ); key = null ; count = 0; while ( cursor.hasNext() ) { Tuple< String , String > tuple = cursor.next(); key = tuple.getKey(); assertEquals( uid3, key ); count++; } assertEquals( 50, count ); }
        Hide
        elecharny Emmanuel Lecharny added a comment -

        Actually a more valid test would be one where the key you pick is at the border of a page (assuming a page contains 16 keys, It would be a key at the beginning of a page or at the end of a page). the testBrowseBTree500 creates a random tree containing 500 elements, and then test that browsing it from the very first key to the very last key works well. This cover the use case I just mentionned, except that each key is associated with only one value. The test should be modified so that each key has from 1 to N values (where N > threshold. Threshold is the number of values we can store before the array keeping the values is transformed to a sub-btree). Working on that.

        Show
        elecharny Emmanuel Lecharny added a comment - Actually a more valid test would be one where the key you pick is at the border of a page (assuming a page contains 16 keys, It would be a key at the beginning of a page or at the end of a page). the testBrowseBTree500 creates a random tree containing 500 elements, and then test that browsing it from the very first key to the very last key works well. This cover the use case I just mentionned, except that each key is associated with only one value. The test should be modified so that each key has from 1 to N values (where N > threshold. Threshold is the number of values we can store before the array keeping the values is transformed to a sub-btree). Working on that.
        Hide
        elecharny Emmanuel Lecharny added a comment -

        We have two different kind of pages : Leaf and Node. A Node has N keys, and N+1 children. A Leaf has N keys and N children. A key in a Node is just used as a marker, helping the btree to go down to the correct leaf, we just keep the higher key from the right subtree in a node, so that you can know if you have to go down the btree using the left child or the right child, by comparing your key with the key stored in the node.

        Show
        elecharny Emmanuel Lecharny added a comment - We have two different kind of pages : Leaf and Node. A Node has N keys, and N+1 children. A Leaf has N keys and N children. A key in a Node is just used as a marker, helping the btree to go down to the correct leaf, we just keep the higher key from the right subtree in a node, so that you can know if you have to go down the btree using the left child or the right child, by comparing your key with the key stored in the node.
        Hide
        elecharny Emmanuel Lecharny added a comment - - edited

        Ok, there is definitively a bug with multi-values. When I inject 2 values per key, and start browsing the btree, at some point, the cursor does not return the right tuple :

        <0,0>
        <0,1>
        <1,1>
        <1,2>
        <2,2>
        <2,3>
        <3,3>
        <3,4>
        <4,4>
        <4,5>
        <5,5>
        <5,6>
        <6,6>
        <6,7>
        <7,7>
        <7,8>
        <8,8>
        <8,9>
        <9,10>  <<------ Wrong !!!
        

        The test is simple, I inject 500 keys, each key has 2 values, which are (Key, Key+1), then I browse the btree starting at the beginning. It works well for the 9 first values. The last tuple should have been <9,9>, not <9,10>.

        Show
        elecharny Emmanuel Lecharny added a comment - - edited Ok, there is definitively a bug with multi-values. When I inject 2 values per key, and start browsing the btree, at some point, the cursor does not return the right tuple : <0,0> <0,1> <1,1> <1,2> <2,2> <2,3> <3,3> <3,4> <4,4> <4,5> <5,5> <5,6> <6,6> <6,7> <7,7> <7,8> <8,8> <8,9> <9,10> <<------ Wrong !!! The test is simple, I inject 500 keys, each key has 2 values, which are (Key, Key+1), then I browse the btree starting at the beginning. It works well for the 9 first values. The last tuple should have been <9,9>, not <9,10>.
        Hide
        elecharny Emmanuel Lecharny added a comment -

        Ah, my bad. This is not a bug. The stored tuples are <9,"9"> and <9,"10">, where "10" is lexicographically lower that "9". I have to fix the test.

        Show
        elecharny Emmanuel Lecharny added a comment - Ah, my bad. This is not a bug. The stored tuples are <9,"9"> and <9,"10">, where "10" is lexicographically lower that "9". I have to fix the test.
        Hide
        elecharny Emmanuel Lecharny added a comment -

        I changed the test, and it works just fine, whatever is the number of values for each key (tested with 1, 2, 3, 4, 5, 17).

        Show
        elecharny Emmanuel Lecharny added a comment - I changed the test, and it works just fine, whatever is the number of values for each key (tested with 1, 2, 3, 4, 5, 17).
        Hide
        wayswssb linzhao added a comment -

        The bug is very difficult to fix. So I need to write some logs in the code to identify the issue. So you think we should change the way when browser the Btree? The algorithm for construct Btree is corrent? Please give me some advice. Thanks for your support. BTW, Chinese new year is coming, I will take a big holiday for 7 days. But I will do some work for this bug during the holiday.

        Show
        wayswssb linzhao added a comment - The bug is very difficult to fix. So I need to write some logs in the code to identify the issue. So you think we should change the way when browser the Btree? The algorithm for construct Btree is corrent? Please give me some advice. Thanks for your support. BTW, Chinese new year is coming, I will take a big holiday for 7 days. But I will do some work for this bug during the holiday.
        Hide
        elecharny Emmanuel Lecharny added a comment -

        The browser seems to work, the test proves it. There is one more thing that needs to be tested is the same test when using a key which is a composite one, like what we use for teh RDN index. Typically, we use a key which is a composition of a RDN and a UUID.

        Happy new chinese year !

        Show
        elecharny Emmanuel Lecharny added a comment - The browser seems to work, the test proves it. There is one more thing that needs to be tested is the same test when using a key which is a composite one, like what we use for teh RDN index. Typically, we use a key which is a composition of a RDN and a UUID. Happy new chinese year !
        Hide
        wayswssb linzhao added a comment -

        Thank you. The last working day before new year, continue to fix this bug.

        Show
        wayswssb linzhao added a comment - Thank you. The last working day before new year, continue to fix this bug.
        Hide
        wayswssb linzhao added a comment -

        Hi Emmanuel,

        Please continue to debug this issue, I just reproduce this bug so many times with your new changed code. Also I change some code to dump the Btree when each time it delete or add a value.

        Thanks,
        Lin

        Show
        wayswssb linzhao added a comment - Hi Emmanuel, Please continue to debug this issue, I just reproduce this bug so many times with your new changed code. Also I change some code to dump the Btree when each time it delete or add a value. Thanks, Lin
        Hide
        wayswssb linzhao added a comment -

        I uploaded the site topology data again, but this time I simplify the data, could you try the new file again to see that happens again or not? I am not sure the file work or not. Please test it first. I can't do the test now since the Chinese new year holiday is coming. I will do the test later. My thought is that if the data is simple, may be we can find the root cause easily.

        Show
        wayswssb linzhao added a comment - I uploaded the site topology data again, but this time I simplify the data, could you try the new file again to see that happens again or not? I am not sure the file work or not. Please test it first. I can't do the test now since the Chinese new year holiday is coming. I will do the test later. My thought is that if the data is simple, may be we can find the root cause easily.
        Hide
        wayswssb linzhao added a comment - - edited

        Any updates for this issue?

        Show
        wayswssb linzhao added a comment - - edited Any updates for this issue?
        Hide
        elecharny Emmanuel Lecharny added a comment -

        Sorry, nothing new on this front. I was busy on another nasty problem that forbid us to release Mavibot (a test failure) and I'm still stuck on this for a few more hours.

        Actually, I have sadly many other things on my plate, including day job :/

        Show
        elecharny Emmanuel Lecharny added a comment - Sorry, nothing new on this front. I was busy on another nasty problem that forbid us to release Mavibot (a test failure) and I'm still stuck on this for a few more hours. Actually, I have sadly many other things on my plate, including day job :/
        Hide
        wayswssb linzhao added a comment -

        Hi Lecharny,

        Do you have spare time to see this issue again? I came back from holiday now.

        Show
        wayswssb linzhao added a comment - Hi Lecharny, Do you have spare time to see this issue again? I came back from holiday now.
        Hide
        elecharny Emmanuel Lecharny added a comment -

        I just committed a bunch of fixes in Mavibot this afternoon, and we are almost ready for a release. I do think that the server works properly with this new version.

        Show
        elecharny Emmanuel Lecharny added a comment - I just committed a bunch of fixes in Mavibot this afternoon, and we are almost ready for a release. I do think that the server works properly with this new version.
        Hide
        wayswssb linzhao added a comment -

        I am sorry to say that the problem still existed. I check out the newest code and found that at line 229 in TupleCustor, the parentPos.valueCursor is null, so sometimes NullPointerException happened. Please help to check this.

        Show
        wayswssb linzhao added a comment - I am sorry to say that the problem still existed. I check out the newest code and found that at line 229 in TupleCustor, the parentPos.valueCursor is null, so sometimes NullPointerException happened. Please help to check this.
        Hide
        wayswssb linzhao added a comment -

        The problem can be easily reproduced.

        Show
        wayswssb linzhao added a comment - The problem can be easily reproduced.
        Hide
        elecharny Emmanuel Lecharny added a comment -

        Hmmm.

        With your data set ?

        /me reviewing the code again...

        Show
        elecharny Emmanuel Lecharny added a comment - Hmmm. With your data set ? /me reviewing the code again...
        Hide
        akiran Kiran Ayyagari added a comment -

        linzhao Did you build the Mavibot partition as well after building the latest Mavibot?

        Show
        akiran Kiran Ayyagari added a comment - linzhao Did you build the Mavibot partition as well after building the latest Mavibot?
        Hide
        wayswssb linzhao added a comment -

        After I fixed NullPointerException, the issue still can be reproduced.

        Show
        wayswssb linzhao added a comment - After I fixed NullPointerException, the issue still can be reproduced.
        Hide
        wayswssb linzhao added a comment -

        I am sure I use the newest mavibot code. What I did is build mavibot project first and build apacheds later.

        Show
        wayswssb linzhao added a comment - I am sure I use the newest mavibot code. What I did is build mavibot project first and build apacheds later.
        Hide
        wayswssb linzhao added a comment - - edited

        I am confused with findPos in AbstractPage.java. There may be many same keys in the keys array, so how do we find the correct position? One more thing is that after I fixed NullPointerException the problem can't be easily reproduced with the newest mavibot code.

        Show
        wayswssb linzhao added a comment - - edited I am confused with findPos in AbstractPage.java. There may be many same keys in the keys array, so how do we find the correct position? One more thing is that after I fixed NullPointerException the problem can't be easily reproduced with the newest mavibot code.
        Hide
        wayswssb linzhao added a comment -

        I just found that there are too many btree in the project, 1.3.6.1.4.1.18060.0.4.1.2.50_reverse, 1.3.6.1.4.1.18060.0.4.1.2.50_forward and master, what are they used for?

        Show
        wayswssb linzhao added a comment - I just found that there are too many btree in the project, 1.3.6.1.4.1.18060.0.4.1.2.50_reverse, 1.3.6.1.4.1.18060.0.4.1.2.50_forward and master, what are they used for?
        Hide
        elecharny Emmanuel Lecharny added a comment -

        It's explain in the findPos() javadoc.

            /**
             * Finds the position of the given key in the page. If we have found the key,
             * we will return its position as a negative value.
             * <p/>
             * Assuming that the array is zero-indexed, the returned value will be : <br/>
             *   position = - ( position + 1)
             * <br/>
             * So for the following table of keys : <br/>
             * <pre>
             * +---+---+---+---+
             * | b | d | f | h |
             * +---+---+---+---+
             *   0   1   2   3
             * </pre>
             * looking for 'b' will return -1 (-(0+1)) and looking for 'f' will return -3 (-(2+1)).<br/>
             * Computing the real position is just a matter to get -(position++).
             * <p/>
             * If we don't find the key in the table, we will return the position of the key
             * immediately above the key we are looking for. <br/>
             * For instance, looking for :
             * <ul>
             * <li>'a' will return 0</li>
             * <li>'b' will return -1</li>
             * <li>'c' will return 1</li>
             * <li>'d' will return -2</li>
             * <li>'e' will return 2</li>
             * <li>'f' will return -3</li>
             * <li>'g' will return 3</li>
             * <li>'h' will return -4</li>
             * <li>'i' will return 4</li>
             * </ul>
             *
             *
             * @param key The key to find
             * @return The position in the page.
             */
            public int findPos( K key )
                ...
        

        How did you fix the NPE ?

        Show
        elecharny Emmanuel Lecharny added a comment - It's explain in the findPos() javadoc. /** * Finds the position of the given key in the page. If we have found the key, * we will return its position as a negative value. * <p/> * Assuming that the array is zero-indexed, the returned value will be : <br/> * position = - ( position + 1) * <br/> * So for the following table of keys : <br/> * <pre> * +---+---+---+---+ * | b | d | f | h | * +---+---+---+---+ * 0 1 2 3 * </pre> * looking for 'b' will return -1 (-(0+1)) and looking for 'f' will return -3 (-(2+1)).<br/> * Computing the real position is just a matter to get -(position++). * <p/> * If we don't find the key in the table, we will return the position of the key * immediately above the key we are looking for . <br/> * For instance, looking for : * <ul> * <li>'a' will return 0</li> * <li>'b' will return -1</li> * <li>'c' will return 1</li> * <li>'d' will return -2</li> * <li>'e' will return 2</li> * <li>'f' will return -3</li> * <li>'g' will return 3</li> * <li>'h' will return -4</li> * <li>'i' will return 4</li> * </ul> * * * @param key The key to find * @ return The position in the page. */ public int findPos( K key ) ... How did you fix the NPE ?
        Hide
        elecharny Emmanuel Lecharny added a comment -

        So, let me give you some quick explaination about what those B-trees are good for.

        Storing entries into a LDAP server is not only about storing the entries themselves in a B-tree. We of course do that, they are stored in what we call the MasterTable. This table associates a unique UUID to a serialized entry. The B-tree is ordered using the UUID.

        Ok, now when doing a search you don't want to fetch all the entries from the Mastertable, so we have external indexes, which are stored into other B-trees. Typically, if you have indexed the 'cn' attribute, you will have a B-tree associating each 'cn' with the list of entry UUID having this attribute.

        There are a few mandatory indexes :

        • objectClass
        • entryCSN
        • alias
        • oneAlias
        • present (used to index the attributeTypes that are used in ever entries)
        • RDN
          and a few others

        An intersting index is the RDN index. In fact, this index uses 2 B-trees. Each entry has a DN, which is a list of RDNs. We want to be able to retrieve an entry using it's DN, and we use the RDN index for that purpose.
        OTOH, the RDN index just have RDNs, not full DN. Add to that a RDN can be shared between many entries. This index is a bit specific in that we store the RDN plus it's parent's ID as a key. We have a <ParentIdAndRDN, UUID> B-tree index for that.
        The second B-TRree for the RDN index is a revert B-tree, ie a B-tree where the UUID is the key and the ParentIdAndRDN the value. That allos us to build back the DN which is not stored within the entry.

        One could argue that the B-tree<UUID, ParentIdAndRdn> index is useless, because we already have it : it's the MasterTable. That's true, except that we would have to deserialize many Entries in order to rebuild the DN, which would be a costly operation.

        Hope it's helps...

        Show
        elecharny Emmanuel Lecharny added a comment - So, let me give you some quick explaination about what those B-trees are good for. Storing entries into a LDAP server is not only about storing the entries themselves in a B-tree. We of course do that, they are stored in what we call the MasterTable. This table associates a unique UUID to a serialized entry. The B-tree is ordered using the UUID. Ok, now when doing a search you don't want to fetch all the entries from the Mastertable, so we have external indexes, which are stored into other B-trees. Typically, if you have indexed the 'cn' attribute, you will have a B-tree associating each 'cn' with the list of entry UUID having this attribute. There are a few mandatory indexes : objectClass entryCSN alias oneAlias present (used to index the attributeTypes that are used in ever entries) RDN and a few others An intersting index is the RDN index. In fact, this index uses 2 B-trees. Each entry has a DN, which is a list of RDNs. We want to be able to retrieve an entry using it's DN, and we use the RDN index for that purpose. OTOH, the RDN index just have RDNs, not full DN. Add to that a RDN can be shared between many entries. This index is a bit specific in that we store the RDN plus it's parent's ID as a key. We have a <ParentIdAndRDN, UUID> B-tree index for that. The second B-TRree for the RDN index is a revert B-tree, ie a B-tree where the UUID is the key and the ParentIdAndRDN the value. That allos us to build back the DN which is not stored within the entry. One could argue that the B-tree<UUID, ParentIdAndRdn> index is useless, because we already have it : it's the MasterTable. That's true, except that we would have to deserialize many Entries in order to rebuild the DN, which would be a costly operation. Hope it's helps...
        Hide
        wayswssb linzhao added a comment - - edited

        Also, could you give me your build and let me test your build? You can add some debug info in your build. But first please fix the NullPointerException, you can give a simple apacheds rpm. This is my fix at line 229 if (parentPos.valueCursor != null && parentPos.valueCursor.hasNext() ). BTW, my email is lin.zhao@polycom.com.

        Show
        wayswssb linzhao added a comment - - edited Also, could you give me your build and let me test your build? You can add some debug info in your build. But first please fix the NullPointerException, you can give a simple apacheds rpm. This is my fix at line 229 if (parentPos.valueCursor != null && parentPos.valueCursor.hasNext() ). BTW, my email is lin.zhao@polycom.com.
        Hide
        elecharny Emmanuel Lecharny added a comment -

        You are right. There are many use cases where the valueCursor can be null at some point, all legit. The most annoying one is when the key we are looking for is at the end of a leaf, and it has some following keys. In this case, we keep the cursor where it is, but we don't set the valueCursor, because we haven't yet moved to the next leaf.

        Checking the code atm.

        Show
        elecharny Emmanuel Lecharny added a comment - You are right. There are many use cases where the valueCursor can be null at some point, all legit. The most annoying one is when the key we are looking for is at the end of a leaf, and it has some following keys. In this case, we keep the cursor where it is, but we don't set the valueCursor, because we haven't yet moved to the next leaf. Checking the code atm.
        Hide
        elecharny Emmanuel Lecharny added a comment -

        Btw, I think your proposed patch is correct ( if (parentPos.valueCursor != null && parentPos.valueCursor.hasNext() ) )

        Show
        elecharny Emmanuel Lecharny added a comment - Btw, I think your proposed patch is correct ( if (parentPos.valueCursor != null && parentPos.valueCursor.hasNext() ) )
        Hide
        elecharny Emmanuel Lecharny added a comment -

        Patch applied : http://svn.apache.org/r1662242

        Can you test it ?

        Show
        elecharny Emmanuel Lecharny added a comment - Patch applied : http://svn.apache.org/r1662242 Can you test it ?
        Hide
        ali.yang Lifan Yang added a comment -

        Hi Emmanuel,

        I am Lin's colleague.

        PersistedBTree will persist data into a file. So I think Mavibot unit test can use the data file from ApacheDS to reproduce this problem quicker. Right?

        Show
        ali.yang Lifan Yang added a comment - Hi Emmanuel, I am Lin's colleague. PersistedBTree will persist data into a file. So I think Mavibot unit test can use the data file from ApacheDS to reproduce this problem quicker. Right?
        Hide
        elecharny Emmanuel Lecharny added a comment -

        Right. We just have to move the serializer code to Mavibot.

        Show
        elecharny Emmanuel Lecharny added a comment - Right. We just have to move the serializer code to Mavibot.
        Hide
        ali.yang Lifan Yang added a comment - - edited

        Could you provide the concrete class name of this serializer. MavibotEntrySerializer.java?

        Show
        ali.yang Lifan Yang added a comment - - edited Could you provide the concrete class name of this serializer. MavibotEntrySerializer.java?
        Hide
        wayswssb linzhao added a comment -

        I have tested it and found the problem still existed. The condition is like that all the parent pos reached the last of the node, so the next method in tuple cursor always return false. The error logic happened at TupleCursor line 197 to 218. Please help to check it.

        Show
        wayswssb linzhao added a comment - I have tested it and found the problem still existed. The condition is like that all the parent pos reached the last of the node, so the next method in tuple cursor always return false. The error logic happened at TupleCursor line 197 to 218. Please help to check it.
        Hide
        wayswssb linzhao added a comment -

        May a logic error here, at line 1354 in PersistedLeaf.java.
        pivot = rightLeaf.keys[0].getKey();
        if ( pivot == null )
        {
        pivot = rightLeaf.keys[0].getKey();
        }

        I changed it this way:
        K pivot = leftLeaf.keys[leftLeaf.keys.length -1].getKey();

        if ( pivot == null )

        { pivot = rightLeaf.keys[0].getKey(); }

        Correct?

        Show
        wayswssb linzhao added a comment - May a logic error here, at line 1354 in PersistedLeaf.java. pivot = rightLeaf.keys [0] .getKey(); if ( pivot == null ) { pivot = rightLeaf.keys [0] .getKey(); } I changed it this way: K pivot = leftLeaf.keys [leftLeaf.keys.length -1] .getKey(); if ( pivot == null ) { pivot = rightLeaf.keys[0].getKey(); } Correct?
        Hide
        elecharny Emmanuel Lecharny added a comment - - edited

        Ah !

        This is a good catch, although the code is correct. The idea is that a page is split, and we get the new key that has to be moved to the parent from the right page. Let me explain how it works :

        Before the split, the page is full, and we have to inject a new value in it :

            +---+---+---+
            | A | B | C |
            +---+---+---+
                |   |   |
         ...<---+   |   +--->...
                    V
            +---+---+---+---+
            | 1 | 3 | 5 | 7 |
            +---+---+---+---+
        

        Here, the leftmost key of the page is stored in the parent page, thus 1 is also stored in the parent page.
        Now, we have to inject the value 6. The underlying page will be split in two pages :

            +---+---+   +---+---+---+
            | 1 | 3 |   | 5 | 6 | 7 |
            +---+---+   +---+---+---+
        

        and we have to add a new key (6 here) in the parent page. The result will be :

            +---+---+---+---+
            | A | 1 | 5 | C |
            +---+---+---+---+
                |   |   |   |
                |   |   |   +--->...
                |   |   |
         ...<---+   |   +-------+
                    V           V 
                    +---+---+   +---+---+---+
                    | 1 | 3 |   | 5 | 6 | 7 |
                    +---+---+   +---+---+---+
        

        Here, we took the leftmost key of the right page, as it's done in the code :

        K pivot = rightLeaf.keys[0].getKey();
        

        Now, why do we do :

                if ( pivot == null )
                {
                    pivot = rightLeaf.keys[0].getKey();
                }
        

        when the pivot obviously won't be null ? (and we take the exact same value !)

        This is typically a debug piece of code that remained in the code. The (debug) idea was to put a conditional breakpoint when pivot was null, and redo the same action, but allowing us to know what was going on. This piece of code has been added here : http://svn.apache.org/viewvc/directory/mavibot/trunk/mavibot/src/main/java/org/apache/directory/mavibot/btree/PersistedLeaf.java?r1=1536070&r2=1538151&diff_format=h

        I'll remove this useless piece of code, thanks for point it out !

        Show
        elecharny Emmanuel Lecharny added a comment - - edited Ah ! This is a good catch, although the code is correct. The idea is that a page is split, and we get the new key that has to be moved to the parent from the right page. Let me explain how it works : Before the split, the page is full, and we have to inject a new value in it : +---+---+---+ | A | B | C | +---+---+---+ | | | ...<---+ | +--->... V +---+---+---+---+ | 1 | 3 | 5 | 7 | +---+---+---+---+ Here, the leftmost key of the page is stored in the parent page, thus 1 is also stored in the parent page. Now, we have to inject the value 6 . The underlying page will be split in two pages : +---+---+ +---+---+---+ | 1 | 3 | | 5 | 6 | 7 | +---+---+ +---+---+---+ and we have to add a new key ( 6 here) in the parent page. The result will be : +---+---+---+---+ | A | 1 | 5 | C | +---+---+---+---+ | | | | | | | +--->... | | | ...<---+ | +-------+ V V +---+---+ +---+---+---+ | 1 | 3 | | 5 | 6 | 7 | +---+---+ +---+---+---+ Here, we took the leftmost key of the right page, as it's done in the code : K pivot = rightLeaf.keys[0].getKey(); Now, why do we do : if ( pivot == null ) { pivot = rightLeaf.keys[0].getKey(); } when the pivot obviously won't be null ? (and we take the exact same value !) This is typically a debug piece of code that remained in the code. The (debug) idea was to put a conditional breakpoint when pivot was null, and redo the same action, but allowing us to know what was going on. This piece of code has been added here : http://svn.apache.org/viewvc/directory/mavibot/trunk/mavibot/src/main/java/org/apache/directory/mavibot/btree/PersistedLeaf.java?r1=1536070&r2=1538151&diff_format=h I'll remove this useless piece of code, thanks for point it out !
        Hide
        wayswssb linzhao added a comment -

        So this is not the point for this issue, we need to do more debug to find the root cause of this bug together. Thank you.

        Show
        wayswssb linzhao added a comment - So this is not the point for this issue, we need to do more debug to find the root cause of this bug together. Thank you.
        Hide
        elecharny Emmanuel Lecharny added a comment -

        Yeah :/

        Sorry, I have been sucked into day job yesterday and partly today too...

        Show
        elecharny Emmanuel Lecharny added a comment - Yeah :/ Sorry, I have been sucked into day job yesterday and partly today too...
        Hide
        wayswssb linzhao added a comment -

        I changed line 809 in PersistedLeaf.java. My code is "for ( int i = depth - 1; i >= 0; i-- )" and your code is "for ( int i = depth - 2; i >= 0; i-- )". I will retest the case and let you know the result.

        Show
        wayswssb linzhao added a comment - I changed line 809 in PersistedLeaf.java. My code is "for ( int i = depth - 1; i >= 0; i-- )" and your code is "for ( int i = depth - 2; i >= 0; i-- )". I will retest the case and let you know the result.
        Hide
        wayswssb linzhao added a comment -

        Hi Emmanuel,

        Please help to review my code change .

        Show
        wayswssb linzhao added a comment - Hi Emmanuel, Please help to review my code change .
        Hide
        ali.yang Lifan Yang added a comment -

        Hi Emmanuel,

        I have a question about your description of Mavibot algorithm. I think after the split, the `6` should be move up to be a parent and will not be existed in the leaf. The result will be like following

            +---+---+---+---+
            | A | B | 6 | C |
            +---+---+---+---+
                |   |   |   |
                |   |   |   +--->...
                |   |   |
         ...<---+   |   +------+
                    V          V 
                    +---+---+  +---+---+---+
                    | 1 | 3 |  | 5 | 7 |
                    +---+---+  +---+---+---+
        

        Is it right? Or Mavibot has a little difference with the common B-Tree algorithm.

        Show
        ali.yang Lifan Yang added a comment - Hi Emmanuel, I have a question about your description of Mavibot algorithm. I think after the split, the `6` should be move up to be a parent and will not be existed in the leaf. The result will be like following +---+---+---+---+ | A | B | 6 | C | +---+---+---+---+ | | | | | | | +--->... | | | ...<---+ | +------+ V V +---+---+ +---+---+---+ | 1 | 3 | | 5 | 7 | +---+---+ +---+---+---+ Is it right? Or Mavibot has a little difference with the common B-Tree algorithm.
        Hide
        ali.yang Lifan Yang added a comment -

        So more accurate say that Mavibot is a B+ Tree, not a B Tree.

        Show
        ali.yang Lifan Yang added a comment - So more accurate say that Mavibot is a B+ Tree, not a B Tree.
        Hide
        elecharny Emmanuel Lecharny added a comment -

        No, the key is present in the parent and in the leaf. Nodes don't contain any values, only keys. The 6 key can therefore be present in many nodes (not in this exemple though) but the value associated with this key is only present in the leaf.

        Keys are routers in some way...

        Show
        elecharny Emmanuel Lecharny added a comment - No, the key is present in the parent and in the leaf. Nodes don't contain any values, only keys. The 6 key can therefore be present in many nodes (not in this exemple though) but the value associated with this key is only present in the leaf. Keys are routers in some way...
        Hide
        elecharny Emmanuel Lecharny added a comment -

        Indeed, but not exactly. Let's say it's a flavor of B+Tree : B+tree has links between leaves to speedup the tree browsing, we don't have such link because it would break the MVCC algorithm (we would be forced to copy the whol btree everytime we update one key).

        We chose this flavor of B-tree in order to speedup the bulk-loading of values : it's easy to construct a B-tree if only the leaves contain values. One more reason is that copying values when we go up the tree after an update can be a very expensive operation, plus would eat a hell lot of space (much more than when we copy the keys only)

        It's kind of explained on http://directory.apache.org/mavibot/user-guide/2-btree-types.html, where it's said that Mavibot is based on MVCC B+Tree.

        Show
        elecharny Emmanuel Lecharny added a comment - Indeed, but not exactly. Let's say it's a flavor of B+Tree : B+tree has links between leaves to speedup the tree browsing, we don't have such link because it would break the MVCC algorithm (we would be forced to copy the whol btree everytime we update one key). We chose this flavor of B-tree in order to speedup the bulk-loading of values : it's easy to construct a B-tree if only the leaves contain values. One more reason is that copying values when we go up the tree after an update can be a very expensive operation, plus would eat a hell lot of space (much more than when we copy the keys only) It's kind of explained on http://directory.apache.org/mavibot/user-guide/2-btree-types.html , where it's said that Mavibot is based on MVCC B+Tree.
        Hide
        wayswssb linzhao added a comment - - edited

        Hi Emmanuel,
        I changed line 809 in PersistedLeaf.java.
        My code is:
        for ( int i = depth - 1; i >= 0; i-- )

        and your code is :
        for ( int i = depth - 2; i >= 0; i-- )

        My test result is that my code will never reproduce the issue, I tried the scenario more than 30 times.
        Please help to review it. Thanks.

        Show
        wayswssb linzhao added a comment - - edited Hi Emmanuel, I changed line 809 in PersistedLeaf.java. My code is: for ( int i = depth - 1; i >= 0; i-- ) and your code is : for ( int i = depth - 2; i >= 0; i-- ) My test result is that my code will never reproduce the issue, I tried the scenario more than 30 times. Please help to review it. Thanks.
        Hide
        wayswssb linzhao added a comment -

        Hi Emmanuel,

        I can reproduce the issue with your code:

        Below is the debug info for the issue:
        The array stack is 3 length long. The stack info list below:
        [<3,Node[r2693, nbElems:3, offset:10426368] ->

        {r2691|<PersistedKeyHolder[ParentIdAndRdn<2c9ea107-be73-401f-9a76-a126b07ef5a4, 'siteUid=9f4781ed-3337-4fd1-808e-d122e9dc9d26'>[nbC:0, nbD:0], 374]>|r2633|<PersistedKeyHolder[ParentIdAndRdn<2e7cd647-f11f-4ffd-a10b-23f46014c878, 'networkUid=1d879967-d598-46e2-b47a-664ac407d89b'>[nbC:0, nbD:0], 383]>|r2683|<PersistedKeyHolder[ParentIdAndRdn<2e7cd647-f11f-4ffd-a10b-23f46014c878, 'networkUid=a129a53d-313d-4dc2-b684-c41d8d282296'>[nbC:0, nbD:0], 383]>|r2689}

        >, <10,Node[r2689, nbElems:11, offset:8612352] ->

        {r2266|<PersistedKeyHolder[ParentIdAndRdn<2e7cd647-f11f-4ffd-a10b-23f46014c878, 'networkUid=aa7f3ff5-8b72-4b44-950e-8c92d8b20416'>[nbC:0, nbD:0], 465]>|r2561|<PersistedKeyHolder[ParentIdAndRdn<2e7cd647-f11f-4ffd-a10b-23f46014c878, 'networkUid=b989c411-f6c0-4e87-bbe0-3f59b4c938ce'>[nbC:0, nbD:0], 465]>|r2501|<PersistedKeyHolder[ParentIdAndRdn<2e7cd647-f11f-4ffd-a10b-23f46014c878, 'networkUid=c3568dd9-cf76-44bf-8353-76e6062920b4'>[nbC:0, nbD:0], 465]>|r2421|<PersistedKeyHolder[ParentIdAndRdn<2e7cd647-f11f-4ffd-a10b-23f46014c878, 'networkUid=cb6c3bdd-da5d-4298-bb5d-0be8fe04a9d7'>[nbC:0, nbD:0], 465]>|r2621|<PersistedKeyHolder[ParentIdAndRdn<2e7cd647-f11f-4ffd-a10b-23f46014c878, 'networkUid=d713cec0-99f2-4b92-8df8-b2d97495fe3d'>[nbC:0, nbD:0], 465]>|r1886|<PersistedKeyHolder[ParentIdAndRdn<2e7cd647-f11f-4ffd-a10b-23f46014c878, 'networkUid=e17b1bb4-d8cf-4be3-9407-fd71867e3b82'>[nbC:0, nbD:0], 465]>|r1686|<PersistedKeyHolder[ParentIdAndRdn<2e7cd647-f11f-4ffd-a10b-23f46014c878, 'networkUid=ee77edbf-0b6b-4548-a65f-83f3f117f9a6'>[nbC:0, nbD:0], 465]>|r2566|<PersistedKeyHolder[ParentIdAndRdn<2e7cd647-f11f-4ffd-a10b-23f46014c878, 'networkUid=f53e6384-8050-4072-9a6f-b3a563842553'>[nbC:0, nbD:0], 465]>|r2616|<PersistedKeyHolder[ParentIdAndRdn<2e7cd647-f11f-4ffd-a10b-23f46014c878, 'networkUid=fad6196f-e37f-4f27-b58d-a27df44998f2'>[nbC:0, nbD:0], 465]>|r2651|<PersistedKeyHolder[ParentIdAndRdn<8d99a724-8878-4f6e-ab9c-ea606dbf0018, 'siteLinkUid=949a5fa8-35e9-4be0-867d-91f72bad63f8'>[nbC:0, nbD:0], 468]>|r2206|<PersistedKeyHolder[ParentIdAndRdn<ebe2a436-3251-4ee1-98a4-2e146b9c0732, 'ou=Configuration'>[nbC:1, nbD:1], 236]>|r2686}

        >, <11,Leaf[r2206, nbElems:11, offset:9030144] ->

        {<PersistedKeyHolder[ParentIdAndRdn<8d99a724-8878-4f6e-ab9c-ea606dbf0018, 'siteLinkUid=949a5fa8-35e9-4be0-867d-91f72bad63f8'>[nbC:0, nbD:0], 386],ValueHolder[StringSerializer, isRaw[44]]>, <PersistedKeyHolder[ParentIdAndRdn<8d99a724-8878-4f6e-ab9c-ea606dbf0018, 'siteLinkUid=9ad18f38-bb0f-4a92-9d89-a4079edeeaef'>[nbC:0, nbD:0], 386],ValueHolder[StringSerializer, isRaw[44]]>, <PersistedKeyHolder[ParentIdAndRdn<8d99a724-8878-4f6e-ab9c-ea606dbf0018, 'siteLinkUid=9b706ed8-62d5-47d7-bb18-1693bb2d59fc'>[nbC:0, nbD:0], 386],ValueHolder[StringSerializer, isRaw[44]]>, <PersistedKeyHolder[ParentIdAndRdn<8d99a724-8878-4f6e-ab9c-ea606dbf0018, 'siteLinkUid=aebdc544-850c-4ba7-897d-bc08e7016919'>[nbC:0, nbD:0], 386],ValueHolder[StringSerializer, isRaw[44]]>, <PersistedKeyHolder[ParentIdAndRdn<9e6ca5cd-e710-4d5c-955b-f10fa6d9ada7, 'mpeUid=492f90fd-8f6b-45ee-b3b3-5bd465cda021'>[nbC:0, nbD:0], 371],ValueHolder[StringSerializer, isRaw[44]]>, <PersistedKeyHolder[ParentIdAndRdn<9e6ca5cd-e710-4d5c-955b-f10fa6d9ada7, 'mpeUid=58f3e6a2-fbe8-4786-afca-a123d011e518'>[nbC:0, nbD:0], 371],ValueHolder[StringSerializer, isRaw[44]]>, <PersistedKeyHolder[ParentIdAndRdn<9e6ca5cd-e710-4d5c-955b-f10fa6d9ada7, 'mpeUid=99d4df21-fc98-4338-8ad8-0282ec600ded'>[nbC:0, nbD:0], 371],ValueHolder[StringSerializer, isRaw[44]]>, <PersistedKeyHolder[ParentIdAndRdn<9e6ca5cd-e710-4d5c-955b-f10fa6d9ada7, 'mpeUid=a50f3b17-9ea4-448a-ba7c-4c0a185c6ebd'>[nbC:0, nbD:0], 371],ValueHolder[StringSerializer, isRaw[44]]>, <PersistedKeyHolder[ParentIdAndRdn<ae2b074b-456c-4b4a-b56c-0179659791d9, 'territoryId=c1f78fce-3e3a-49fb-a749-9c9705067cdd'>[nbC:0, nbD:0], 386],ValueHolder[StringSerializer, isRaw[44]]>, <PersistedKeyHolder[ParentIdAndRdn<cc1dc4be-c341-4353-95c1-749f5ab43e05, 'objectClass=siteTopologyServerConfigurationObject'>[nbC:0, nbD:0], 345],ValueHolder[StringSerializer, isRaw[44]]>, <PersistedKeyHolder[ParentIdAndRdn<cdd2edcd-89ce-47aa-ae76-1faf64001b11, 'territoryUid=c1f78fce-3e3a-49fb-a749-9c9705067cdd'>[nbC:0, nbD:0], 389],ValueHolder[StringSerializer, isRaw[44]]>}

        >, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null]

        Currently, the depth is 2. So, you can see the detail of the second Node in the stack array. The second node's pos is 10 and nbElems is 11. So if you change the code to for ( int i = depth - 1; i >= 0; i-- ). The value isLast will be false, so the TupleCursor will not move to the last. So what is your thought? Is it correct?

        Show
        wayswssb linzhao added a comment - Hi Emmanuel, I can reproduce the issue with your code: Below is the debug info for the issue: The array stack is 3 length long. The stack info list below: [<3,Node [r2693, nbElems:3, offset:10426368] -> {r2691|<PersistedKeyHolder[ParentIdAndRdn<2c9ea107-be73-401f-9a76-a126b07ef5a4, 'siteUid=9f4781ed-3337-4fd1-808e-d122e9dc9d26'>[nbC:0, nbD:0], 374]>|r2633|<PersistedKeyHolder[ParentIdAndRdn<2e7cd647-f11f-4ffd-a10b-23f46014c878, 'networkUid=1d879967-d598-46e2-b47a-664ac407d89b'>[nbC:0, nbD:0], 383]>|r2683|<PersistedKeyHolder[ParentIdAndRdn<2e7cd647-f11f-4ffd-a10b-23f46014c878, 'networkUid=a129a53d-313d-4dc2-b684-c41d8d282296'>[nbC:0, nbD:0], 383]>|r2689} >, <10,Node [r2689, nbElems:11, offset:8612352] -> {r2266|<PersistedKeyHolder[ParentIdAndRdn<2e7cd647-f11f-4ffd-a10b-23f46014c878, 'networkUid=aa7f3ff5-8b72-4b44-950e-8c92d8b20416'>[nbC:0, nbD:0], 465]>|r2561|<PersistedKeyHolder[ParentIdAndRdn<2e7cd647-f11f-4ffd-a10b-23f46014c878, 'networkUid=b989c411-f6c0-4e87-bbe0-3f59b4c938ce'>[nbC:0, nbD:0], 465]>|r2501|<PersistedKeyHolder[ParentIdAndRdn<2e7cd647-f11f-4ffd-a10b-23f46014c878, 'networkUid=c3568dd9-cf76-44bf-8353-76e6062920b4'>[nbC:0, nbD:0], 465]>|r2421|<PersistedKeyHolder[ParentIdAndRdn<2e7cd647-f11f-4ffd-a10b-23f46014c878, 'networkUid=cb6c3bdd-da5d-4298-bb5d-0be8fe04a9d7'>[nbC:0, nbD:0], 465]>|r2621|<PersistedKeyHolder[ParentIdAndRdn<2e7cd647-f11f-4ffd-a10b-23f46014c878, 'networkUid=d713cec0-99f2-4b92-8df8-b2d97495fe3d'>[nbC:0, nbD:0], 465]>|r1886|<PersistedKeyHolder[ParentIdAndRdn<2e7cd647-f11f-4ffd-a10b-23f46014c878, 'networkUid=e17b1bb4-d8cf-4be3-9407-fd71867e3b82'>[nbC:0, nbD:0], 465]>|r1686|<PersistedKeyHolder[ParentIdAndRdn<2e7cd647-f11f-4ffd-a10b-23f46014c878, 'networkUid=ee77edbf-0b6b-4548-a65f-83f3f117f9a6'>[nbC:0, nbD:0], 465]>|r2566|<PersistedKeyHolder[ParentIdAndRdn<2e7cd647-f11f-4ffd-a10b-23f46014c878, 'networkUid=f53e6384-8050-4072-9a6f-b3a563842553'>[nbC:0, nbD:0], 465]>|r2616|<PersistedKeyHolder[ParentIdAndRdn<2e7cd647-f11f-4ffd-a10b-23f46014c878, 'networkUid=fad6196f-e37f-4f27-b58d-a27df44998f2'>[nbC:0, nbD:0], 465]>|r2651|<PersistedKeyHolder[ParentIdAndRdn<8d99a724-8878-4f6e-ab9c-ea606dbf0018, 'siteLinkUid=949a5fa8-35e9-4be0-867d-91f72bad63f8'>[nbC:0, nbD:0], 468]>|r2206|<PersistedKeyHolder[ParentIdAndRdn<ebe2a436-3251-4ee1-98a4-2e146b9c0732, 'ou=Configuration'>[nbC:1, nbD:1], 236]>|r2686} >, <11,Leaf [r2206, nbElems:11, offset:9030144] -> {<PersistedKeyHolder[ParentIdAndRdn<8d99a724-8878-4f6e-ab9c-ea606dbf0018, 'siteLinkUid=949a5fa8-35e9-4be0-867d-91f72bad63f8'>[nbC:0, nbD:0], 386],ValueHolder[StringSerializer, isRaw[44]]>, <PersistedKeyHolder[ParentIdAndRdn<8d99a724-8878-4f6e-ab9c-ea606dbf0018, 'siteLinkUid=9ad18f38-bb0f-4a92-9d89-a4079edeeaef'>[nbC:0, nbD:0], 386],ValueHolder[StringSerializer, isRaw[44]]>, <PersistedKeyHolder[ParentIdAndRdn<8d99a724-8878-4f6e-ab9c-ea606dbf0018, 'siteLinkUid=9b706ed8-62d5-47d7-bb18-1693bb2d59fc'>[nbC:0, nbD:0], 386],ValueHolder[StringSerializer, isRaw[44]]>, <PersistedKeyHolder[ParentIdAndRdn<8d99a724-8878-4f6e-ab9c-ea606dbf0018, 'siteLinkUid=aebdc544-850c-4ba7-897d-bc08e7016919'>[nbC:0, nbD:0], 386],ValueHolder[StringSerializer, isRaw[44]]>, <PersistedKeyHolder[ParentIdAndRdn<9e6ca5cd-e710-4d5c-955b-f10fa6d9ada7, 'mpeUid=492f90fd-8f6b-45ee-b3b3-5bd465cda021'>[nbC:0, nbD:0], 371],ValueHolder[StringSerializer, isRaw[44]]>, <PersistedKeyHolder[ParentIdAndRdn<9e6ca5cd-e710-4d5c-955b-f10fa6d9ada7, 'mpeUid=58f3e6a2-fbe8-4786-afca-a123d011e518'>[nbC:0, nbD:0], 371],ValueHolder[StringSerializer, isRaw[44]]>, <PersistedKeyHolder[ParentIdAndRdn<9e6ca5cd-e710-4d5c-955b-f10fa6d9ada7, 'mpeUid=99d4df21-fc98-4338-8ad8-0282ec600ded'>[nbC:0, nbD:0], 371],ValueHolder[StringSerializer, isRaw[44]]>, <PersistedKeyHolder[ParentIdAndRdn<9e6ca5cd-e710-4d5c-955b-f10fa6d9ada7, 'mpeUid=a50f3b17-9ea4-448a-ba7c-4c0a185c6ebd'>[nbC:0, nbD:0], 371],ValueHolder[StringSerializer, isRaw[44]]>, <PersistedKeyHolder[ParentIdAndRdn<ae2b074b-456c-4b4a-b56c-0179659791d9, 'territoryId=c1f78fce-3e3a-49fb-a749-9c9705067cdd'>[nbC:0, nbD:0], 386],ValueHolder[StringSerializer, isRaw[44]]>, <PersistedKeyHolder[ParentIdAndRdn<cc1dc4be-c341-4353-95c1-749f5ab43e05, 'objectClass=siteTopologyServerConfigurationObject'>[nbC:0, nbD:0], 345],ValueHolder[StringSerializer, isRaw[44]]>, <PersistedKeyHolder[ParentIdAndRdn<cdd2edcd-89ce-47aa-ae76-1faf64001b11, 'territoryUid=c1f78fce-3e3a-49fb-a749-9c9705067cdd'>[nbC:0, nbD:0], 389],ValueHolder[StringSerializer, isRaw[44]]>} >, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null] Currently, the depth is 2. So, you can see the detail of the second Node in the stack array. The second node's pos is 10 and nbElems is 11. So if you change the code to for ( int i = depth - 1; i >= 0; i-- ) . The value isLast will be false, so the TupleCursor will not move to the last. So what is your thought? Is it correct?
        Hide
        ali.yang Lifan Yang added a comment -

        Hi Emmanuel,
        Is the cause about the key comparator used by apacheds. Since if the key is existed, the execution should not go to the else branch of browse method. Change from -2 to -1 just make the TupleCursor not point to the afterLast. So let the app has the chance to find the key with cursor.next.

        Show
        ali.yang Lifan Yang added a comment - Hi Emmanuel, Is the cause about the key comparator used by apacheds. Since if the key is existed, the execution should not go to the else branch of browse method. Change from -2 to -1 just make the TupleCursor not point to the afterLast . So let the app has the chance to find the key with cursor.next .
        Hide
        elecharny Emmanuel Lecharny added a comment -

        Note that there was an error in my initial diagram, where I inverted the 6 and 5 keys. In your copy of my diagram, the 6 key in the parent node should be 5

        Show
        elecharny Emmanuel Lecharny added a comment - Note that there was an error in my initial diagram, where I inverted the 6 and 5 keys. In your copy of my diagram, the 6 key in the parent node should be 5
        Hide
        ali.yang Lifan Yang added a comment -

        Yes it is an obvious error .

        About the lost data issue. After change from depth - 2 to depth - 1, the lost data did not happen again. (After Lin 30 round test). But I think it is not the root cause.

        Show
        ali.yang Lifan Yang added a comment - Yes it is an obvious error . About the lost data issue. After change from depth - 2 to depth - 1 , the lost data did not happen again. (After Lin 30 round test). But I think it is not the root cause.
        Hide
        elecharny Emmanuel Lecharny added a comment -

        Ok, beside the clear error (-2 instead of -1), I think the algorithm was not careful enough.

        I've committed a refactoring that now move immediately before the next key in the B-tree when we are at the end of a page. This time, I go up in the B-tree looking for the page on which we are not at the end, and when found, I increment the position, then go down to the leaf selecting the leftmost key of each level. That should work.

        Can you give it a try ?

        Thanks !

        Show
        elecharny Emmanuel Lecharny added a comment - Ok, beside the clear error ( -2 instead of -1 ), I think the algorithm was not careful enough. I've committed a refactoring that now move immediately before the next key in the B-tree when we are at the end of a page. This time, I go up in the B-tree looking for the page on which we are not at the end, and when found, I increment the position, then go down to the leaf selecting the leftmost key of each level. That should work. Can you give it a try ? Thanks !
        Hide
        wayswssb linzhao added a comment - - edited

        NullPinterException happened at line 260 in AbstractPage when calling method TupleCursor.next. I found that child is null in that page. Please help to check.

        Show
        wayswssb linzhao added a comment - - edited NullPinterException happened at line 260 in AbstractPage when calling method TupleCursor.next. I found that child is null in that page. Please help to check.
        Hide
        elecharny Emmanuel Lecharny added a comment -

        Line 260 in AbstractPage can't throw a NPE, it's a method declaration. Be sure to use the latest trunk revision.

        Show
        elecharny Emmanuel Lecharny added a comment - Line 260 in AbstractPage can't throw a NPE, it's a method declaration. Be sure to use the latest trunk revision.
        Hide
        wayswssb linzhao added a comment - - edited

        I am sure I am using the latest build. This is another NullPointerException. I found that why NullPointerException happened during my testing. The problem happened at line 823 in TupleCursor. java. Sometimes, The page is a PersistedLeaf, so the leaf has no children. When calling get page method, it will throw NullPointerException.

        Show
        wayswssb linzhao added a comment - - edited I am sure I am using the latest build. This is another NullPointerException. I found that why NullPointerException happened during my testing. The problem happened at line 823 in TupleCursor. java. Sometimes, The page is a PersistedLeaf, so the leaf has no children. When calling get page method, it will throw NullPointerException.
        Hide
        elecharny Emmanuel Lecharny added a comment - - edited

        Ok, now that's different.

        I wonder if the firs test : if ( depth == 0 ) is correct. If we only have a leaf, then depth should be 1, not 0. In this case, we could have the NPE you see.

        Can you check with if ( depth == 1 ) instead ?

        Show
        elecharny Emmanuel Lecharny added a comment - - edited Ok, now that's different. I wonder if the firs test : if ( depth == 0 ) is correct. If we only have a leaf, then depth should be 1, not 0. In this case, we could have the NPE you see. Can you check with if ( depth == 1 ) instead ?
        Hide
        wayswssb linzhao added a comment -

        I just remove your logic for browse method, I think in method hasNextParentPos in TupleCursor can handle some condition. So we don't need to move position in browse method. I will let you know the results. Thank you.

        Show
        wayswssb linzhao added a comment - I just remove your logic for browse method, I think in method hasNextParentPos in TupleCursor can handle some condition. So we don't need to move position in browse method. I will let you know the results. Thank you.
        Hide
        wayswssb linzhao added a comment -

        I can't reproduce the issue when remove you new logic....

        Show
        wayswssb linzhao added a comment - I can't reproduce the issue when remove you new logic....
        Hide
        elecharny Emmanuel Lecharny added a comment -

        Do you have a patch ?

        Show
        elecharny Emmanuel Lecharny added a comment - Do you have a patch ?
        Hide
        wayswssb linzhao added a comment -

        I only have the rpm file, we don't have a patch for apacheds and mavibot. I only remove your logic from line 845 to line 865 in PersistedLeaf class.

        Show
        wayswssb linzhao added a comment - I only have the rpm file, we don't have a patch for apacheds and mavibot. I only remove your logic from line 845 to line 865 in PersistedLeaf class.
        Hide
        elecharny Emmanuel Lecharny added a comment -

        That's fine ! I just wanted what was the modification you have done.

        Looking at the code, I agree that I was probably overdoing.

        Do you confirm that removing those lines fix your issue ?

        Show
        elecharny Emmanuel Lecharny added a comment - That's fine ! I just wanted what was the modification you have done. Looking at the code, I agree that I was probably overdoing. Do you confirm that removing those lines fix your issue ?
        Hide
        wayswssb linzhao added a comment -

        The issue seems fixed, but I find another exception may be not related to this issue. The problem occurred when apacheds started. Exception list below:
        INFO | jvm 1 | 2015/03/03 23:45:19 | [23:45:19] INFO [org.apache.directory.server.core.partition.impl.btree.jdbm.JdbmPartition] - Setting CacheRecondManager's cache size to 100
        INFO | jvm 1 | 2015/03/03 23:45:19 | [23:45:19] INFO [org.apache.directory.server.core.api.CacheService] - fetching the cache named root
        INFO | jvm 1 | 2015/03/03 23:46:25 | Error in WrapperListener.start callback. java.lang.OutOfMemoryError: GC overhead limit exceeded
        INFO | jvm 1 | 2015/03/03 23:46:25 | java.lang.OutOfMemoryError: GC overhead limit exceeded
        INFO | jvm 1 | 2015/03/03 23:46:25 | at org.apache.directory.mavibot.btree.RecordManager.fetchPage(RecordManager.java:3049)
        INFO | jvm 1 | 2015/03/03 23:46:25 | at org.apache.directory.mavibot.btree.RecordManager.readPageIOs(RecordManager.java:797)
        INFO | jvm 1 | 2015/03/03 23:46:25 | at org.apache.directory.mavibot.btree.RecordManager.deserialize(RecordManager.java:987)
        INFO | jvm 1 | 2015/03/03 23:46:25 | at org.apache.directory.mavibot.btree.PersistedPageHolder.fetchElement(PersistedPageHolder.java:133)
        INFO | jvm 1 | 2015/03/03 23:46:25 | at org.apache.directory.mavibot.btree.PersistedPageHolder.getValue(PersistedPageHolder.java:113)
        INFO | jvm 1 | 2015/03/03 23:46:25 | at org.apache.directory.mavibot.btree.AbstractPage.get(AbstractPage.java:252)
        INFO | jvm 1 | 2015/03/03 23:46:25 | at org.apache.directory.mavibot.btree.AbstractBTree.get(AbstractBTree.java:505)
        INFO | jvm 1 | 2015/03/03 23:46:25 | at org.apache.directory.mavibot.btree.PersistedBTree.get(PersistedBTree.java:43)
        INFO | jvm 1 | 2015/03/03 23:46:25 | at org.apache.directory.server.core.partition.impl.btree.mavibot.MavibotTable.get(MavibotTable.java:317)
        INFO | jvm 1 | 2015/03/03 23:46:25 | at org.apache.directory.server.core.partition.impl.btree.mavibot.MavibotIndex.forwardLookup(MavibotIndex.java:305)
        INFO | jvm 1 | 2015/03/03 23:46:25 | at org.apache.directory.server.core.partition.impl.btree.mavibot.MavibotIndex.forwardLookup(MavibotIndex.java:58)
        INFO | jvm 1 | 2015/03/03 23:46:25 | at org.apache.directory.server.core.partition.impl.btree.AbstractBTreePartition.getEntryId(AbstractBTreePartition.java:2473)
        INFO | jvm 1 | 2015/03/03 23:46:25 | at org.apache.directory.server.xdbm.search.impl.DefaultSearchEngine.computeResult(DefaultSearchEngine.java:123)
        INFO | jvm 1 | 2015/03/03 23:46:25 | at org.apache.directory.server.core.partition.impl.btree.AbstractBTreePartition.search(AbstractBTreePartition.java:1141)
        INFO | jvm 1 | 2015/03/03 23:46:25 | at org.apache.directory.server.core.shared.partition.DefaultPartitionNexus.search(DefaultPartitionNexus.java:624)
        INFO | jvm 1 | 2015/03/03 23:46:25 | at org.apache.directory.server.core.shared.ReferralManagerImpl.init(ReferralManagerImpl.java:178)
        INFO | jvm 1 | 2015/03/03 23:46:25 | at org.apache.directory.server.core.shared.ReferralManagerImpl.<init>(ReferralManagerImpl.java:86)
        INFO | jvm 1 | 2015/03/03 23:46:25 | at org.apache.directory.server.core.referral.ReferralInterceptor.init(ReferralInterceptor.java:213)
        INFO | jvm 1 | 2015/03/03 23:46:25 | at org.apache.directory.server.core.DefaultDirectoryService.initInterceptors(DefaultDirectoryService.java:685)
        INFO | jvm 1 | 2015/03/03 23:46:25 | at org.apache.directory.server.core.DefaultDirectoryService.initialize(DefaultDirectoryService.java:1818)
        INFO | jvm 1 | 2015/03/03 23:46:25 | at org.apache.directory.server.core.DefaultDirectoryService.startup(DefaultDirectoryService.java:1244)
        INFO | jvm 1 | 2015/03/03 23:46:25 | at org.apache.directory.server.ApacheDsService.initDirectoryService(ApacheDsService.java:323)
        INFO | jvm 1 | 2015/03/03 23:46:25 | at org.apache.directory.server.ApacheDsService.start(ApacheDsService.java:182)
        INFO | jvm 1 | 2015/03/03 23:46:25 | at org.apache.directory.server.wrapper.ApacheDsTanukiWrapper.start(ApacheDsTanukiWrapper.java:72)
        INFO | jvm 1 | 2015/03/03 23:46:25 | at org.tanukisoftware.wrapper.WrapperManager$12.run(WrapperManager.java:2788)
        STATUS | wrapper | 2015/03/03 23:46:26 | <-- Wrapper Stopped
        STATUS | wrapper | 2015/03/03 23:47:17 | --> Wrapper Started as Daemon
        STATUS | wrapper | 2015/03/03 23:47:17 | Launching a JVM...

        I did't change anything about JVM memory, so what is the default value for wrapper.java.initmemory=1024 and wrapper.java.maxmemory=2048 in wrapper.conf?

        Show
        wayswssb linzhao added a comment - The issue seems fixed, but I find another exception may be not related to this issue. The problem occurred when apacheds started. Exception list below: INFO | jvm 1 | 2015/03/03 23:45:19 | [23:45:19] INFO [org.apache.directory.server.core.partition.impl.btree.jdbm.JdbmPartition] - Setting CacheRecondManager's cache size to 100 INFO | jvm 1 | 2015/03/03 23:45:19 | [23:45:19] INFO [org.apache.directory.server.core.api.CacheService] - fetching the cache named root INFO | jvm 1 | 2015/03/03 23:46:25 | Error in WrapperListener.start callback. java.lang.OutOfMemoryError: GC overhead limit exceeded INFO | jvm 1 | 2015/03/03 23:46:25 | java.lang.OutOfMemoryError: GC overhead limit exceeded INFO | jvm 1 | 2015/03/03 23:46:25 | at org.apache.directory.mavibot.btree.RecordManager.fetchPage(RecordManager.java:3049) INFO | jvm 1 | 2015/03/03 23:46:25 | at org.apache.directory.mavibot.btree.RecordManager.readPageIOs(RecordManager.java:797) INFO | jvm 1 | 2015/03/03 23:46:25 | at org.apache.directory.mavibot.btree.RecordManager.deserialize(RecordManager.java:987) INFO | jvm 1 | 2015/03/03 23:46:25 | at org.apache.directory.mavibot.btree.PersistedPageHolder.fetchElement(PersistedPageHolder.java:133) INFO | jvm 1 | 2015/03/03 23:46:25 | at org.apache.directory.mavibot.btree.PersistedPageHolder.getValue(PersistedPageHolder.java:113) INFO | jvm 1 | 2015/03/03 23:46:25 | at org.apache.directory.mavibot.btree.AbstractPage.get(AbstractPage.java:252) INFO | jvm 1 | 2015/03/03 23:46:25 | at org.apache.directory.mavibot.btree.AbstractBTree.get(AbstractBTree.java:505) INFO | jvm 1 | 2015/03/03 23:46:25 | at org.apache.directory.mavibot.btree.PersistedBTree.get(PersistedBTree.java:43) INFO | jvm 1 | 2015/03/03 23:46:25 | at org.apache.directory.server.core.partition.impl.btree.mavibot.MavibotTable.get(MavibotTable.java:317) INFO | jvm 1 | 2015/03/03 23:46:25 | at org.apache.directory.server.core.partition.impl.btree.mavibot.MavibotIndex.forwardLookup(MavibotIndex.java:305) INFO | jvm 1 | 2015/03/03 23:46:25 | at org.apache.directory.server.core.partition.impl.btree.mavibot.MavibotIndex.forwardLookup(MavibotIndex.java:58) INFO | jvm 1 | 2015/03/03 23:46:25 | at org.apache.directory.server.core.partition.impl.btree.AbstractBTreePartition.getEntryId(AbstractBTreePartition.java:2473) INFO | jvm 1 | 2015/03/03 23:46:25 | at org.apache.directory.server.xdbm.search.impl.DefaultSearchEngine.computeResult(DefaultSearchEngine.java:123) INFO | jvm 1 | 2015/03/03 23:46:25 | at org.apache.directory.server.core.partition.impl.btree.AbstractBTreePartition.search(AbstractBTreePartition.java:1141) INFO | jvm 1 | 2015/03/03 23:46:25 | at org.apache.directory.server.core.shared.partition.DefaultPartitionNexus.search(DefaultPartitionNexus.java:624) INFO | jvm 1 | 2015/03/03 23:46:25 | at org.apache.directory.server.core.shared.ReferralManagerImpl.init(ReferralManagerImpl.java:178) INFO | jvm 1 | 2015/03/03 23:46:25 | at org.apache.directory.server.core.shared.ReferralManagerImpl.<init>(ReferralManagerImpl.java:86) INFO | jvm 1 | 2015/03/03 23:46:25 | at org.apache.directory.server.core.referral.ReferralInterceptor.init(ReferralInterceptor.java:213) INFO | jvm 1 | 2015/03/03 23:46:25 | at org.apache.directory.server.core.DefaultDirectoryService.initInterceptors(DefaultDirectoryService.java:685) INFO | jvm 1 | 2015/03/03 23:46:25 | at org.apache.directory.server.core.DefaultDirectoryService.initialize(DefaultDirectoryService.java:1818) INFO | jvm 1 | 2015/03/03 23:46:25 | at org.apache.directory.server.core.DefaultDirectoryService.startup(DefaultDirectoryService.java:1244) INFO | jvm 1 | 2015/03/03 23:46:25 | at org.apache.directory.server.ApacheDsService.initDirectoryService(ApacheDsService.java:323) INFO | jvm 1 | 2015/03/03 23:46:25 | at org.apache.directory.server.ApacheDsService.start(ApacheDsService.java:182) INFO | jvm 1 | 2015/03/03 23:46:25 | at org.apache.directory.server.wrapper.ApacheDsTanukiWrapper.start(ApacheDsTanukiWrapper.java:72) INFO | jvm 1 | 2015/03/03 23:46:25 | at org.tanukisoftware.wrapper.WrapperManager$12.run(WrapperManager.java:2788) STATUS | wrapper | 2015/03/03 23:46:26 | <-- Wrapper Stopped STATUS | wrapper | 2015/03/03 23:47:17 | --> Wrapper Started as Daemon STATUS | wrapper | 2015/03/03 23:47:17 | Launching a JVM... I did't change anything about JVM memory, so what is the default value for wrapper.java.initmemory=1024 and wrapper.java.maxmemory=2048 in wrapper.conf?
        Hide
        elecharny Emmanuel Lecharny added a comment -

        Increase the memory. You are clearly running out of it.

        There are a lot of work to be injected in the way Mavibot manage memory : we have a lot of cache everywhere.

        How many entries are you injecting in the server ?

        Show
        elecharny Emmanuel Lecharny added a comment - Increase the memory. You are clearly running out of it. There are a lot of work to be injected in the way Mavibot manage memory : we have a lot of cache everywhere. How many entries are you injecting in the server ?
        Hide
        akiran Kiran Ayyagari added a comment -

        Did you set the cache size of Mavibot to -1 by any chance?

        Show
        akiran Kiran Ayyagari added a comment - Did you set the cache size of Mavibot to -1 by any chance?
        Hide
        wayswssb linzhao added a comment -

        BTW, now I set wrapper.java.initmemory=1024 and wrapper.java.maxmemory=2048 in wrapper.conf and the server still has OutOfMemoryException during the process of startup. Please give me some advise.

        Show
        wayswssb linzhao added a comment - BTW, now I set wrapper.java.initmemory=1024 and wrapper.java.maxmemory=2048 in wrapper.conf and the server still has OutOfMemoryException during the process of startup. Please give me some advise.
        Hide
        wayswssb linzhao added a comment - - edited

        I didn't set cache size when creating mavibot partition. I only inject less than 600 entries in my server. I only changed the wrapper.java.initmemory and wrapper.java.maxmemory in apacheds warpper.conf. Is there some other way to set the memory in apacheds?

        Show
        wayswssb linzhao added a comment - - edited I didn't set cache size when creating mavibot partition. I only inject less than 600 entries in my server. I only changed the wrapper.java.initmemory and wrapper.java.maxmemory in apacheds warpper.conf. Is there some other way to set the memory in apacheds?
        Hide
        elecharny Emmanuel Lecharny added a comment -

        Can you tell us which schema and data file are you using so that we can replay it ? (there are many versions, it may be a good idea just to keep the one that are causing the OOM problem)

        Show
        elecharny Emmanuel Lecharny added a comment - Can you tell us which schema and data file are you using so that we can replay it ? (there are many versions, it may be a good idea just to keep the one that are causing the OOM problem)
        Hide
        wayswssb linzhao added a comment - - edited

        OOM problem happened when fetching entry Supercluster. I already attached the supercluster partition (supercluster_partition.ldif), data (site.ldif) and schema (site_topology_schema.ldif) file in this bug several days ago. If you want any other thing please let me know. Please add some more values under Ou=Supercluster node.

        Show
        wayswssb linzhao added a comment - - edited OOM problem happened when fetching entry Supercluster. I already attached the supercluster partition (supercluster_partition.ldif), data (site.ldif) and schema (site_topology_schema.ldif) file in this bug several days ago. If you want any other thing please let me know. Please add some more values under Ou=Supercluster node.
        Hide
        elecharny Emmanuel Lecharny added a comment -

        That's fine, I just wanted to be sure that I was using the same data. Could you delete the attached ldif files that are obsolete ?

        FTR, I don't get an OOM when I run the server with those data, but I'm running it from the IDE. My memory settings are larger, I guess. Can you test with 2Gb ?

        Show
        elecharny Emmanuel Lecharny added a comment - That's fine, I just wanted to be sure that I was using the same data. Could you delete the attached ldif files that are obsolete ? FTR, I don't get an OOM when I run the server with those data, but I'm running it from the IDE. My memory settings are larger, I guess. Can you test with 2Gb ?
        Hide
        wayswssb linzhao added a comment -

        How to set memory in apacheds? Only changed the wrapper.java.initmemory and wrapper.java.maxmemory? OK, I will delete obsoleted data.

        Show
        wayswssb linzhao added a comment - How to set memory in apacheds? Only changed the wrapper.java.initmemory and wrapper.java.maxmemory? OK, I will delete obsoleted data.
        Hide
        wayswssb linzhao added a comment -

        Sometimes I found that fetching the cache named example can also have OOM errors,

        Show
        wayswssb linzhao added a comment - Sometimes I found that fetching the cache named example can also have OOM errors,
        Hide
        elecharny Emmanuel Lecharny added a comment -

        Which JVM/Version are you using ?

        Show
        elecharny Emmanuel Lecharny added a comment - Which JVM/Version are you using ?
        Hide
        wayswssb linzhao added a comment - - edited

        openjdk version "1.8.0_31"
        OpenJDK Runtime Environment (build 1.8.0_31-b13)
        OpenJDK 64-Bit Server VM (build 25.31-b07, mixed mode)

        Show
        wayswssb linzhao added a comment - - edited openjdk version "1.8.0_31" OpenJDK Runtime Environment (build 1.8.0_31-b13) OpenJDK 64-Bit Server VM (build 25.31-b07, mixed mode)
        Hide
        wayswssb linzhao added a comment -

        I should say I find another exception during the testing. when delete entry from apacheds. Exception list below, may be this exception is not related to the OOM exception.
        [06:22:04] WARN [org.apache.directory.server.core.api.interceptor.context.FilteringOperationContext] - Requested attribute javaRemoteLocation does not exist in the schema, it will be ignored
        [06:25:40] ERROR [org.apache.directory.server.core.partition.impl.btree.mavibot.MavibotTable] - ERR_133 Exception while removing 6c7dd69c-0f53-4b49-8ce6-be84b165938b from index 1.3.6.1.4.1.18060.0.4.1.2.50_reverse
        java.lang.ArrayIndexOutOfBoundsException: 6
        at org.apache.directory.mavibot.btree.RecordManager.readBytes(RecordManager.java:1209)
        at org.apache.directory.mavibot.btree.RecordManager.readPage(RecordManager.java:1021)
        at org.apache.directory.mavibot.btree.RecordManager.deserialize(RecordManager.java:989)
        at org.apache.directory.mavibot.btree.PersistedPageHolder.fetchElement(PersistedPageHolder.java:133)
        at org.apache.directory.mavibot.btree.PersistedPageHolder.getValue(PersistedPageHolder.java:113)
        at org.apache.directory.mavibot.btree.AbstractPage.getReference(AbstractPage.java:155)
        at org.apache.directory.mavibot.btree.RecordManager.serializeNodeValue(RecordManager.java:1708)
        at org.apache.directory.mavibot.btree.RecordManager.serializePage(RecordManager.java:1653)
        at org.apache.directory.mavibot.btree.RecordManager.writePage(RecordManager.java:2808)
        at org.apache.directory.mavibot.btree.PersistedNode.createHolder(PersistedNode.java:906)
        at org.apache.directory.mavibot.btree.PersistedNode.handleRemoveResult(PersistedNode.java:207)
        at org.apache.directory.mavibot.btree.PersistedNode.delete(PersistedNode.java:607)
        at org.apache.directory.mavibot.btree.AbstractPage.delete(AbstractPage.java:217)
        at org.apache.directory.mavibot.btree.PersistedBTree.processDelete(PersistedBTree.java:332)
        at org.apache.directory.mavibot.btree.PersistedBTree.delete(PersistedBTree.java:292)
        at org.apache.directory.mavibot.btree.AbstractBTree.delete(AbstractBTree.java:390)
        at org.apache.directory.mavibot.btree.AbstractBTree.delete(AbstractBTree.java:324)
        at org.apache.directory.mavibot.btree.PersistedBTree.delete(PersistedBTree.java:43)
        at org.apache.directory.server.core.partition.impl.btree.mavibot.MavibotTable.remove(MavibotTable.java:389)
        at org.apache.directory.server.core.partition.impl.btree.mavibot.MavibotIndex.drop(MavibotIndex.java:391)
        at org.apache.directory.server.core.partition.impl.btree.mavibot.MavibotIndex.drop(MavibotIndex.java:58)
        at org.apache.directory.server.core.partition.impl.btree.AbstractBTreePartition.delete(AbstractBTreePartition.java:1078)
        at org.apache.directory.server.core.partition.impl.btree.AbstractBTreePartition.delete(AbstractBTreePartition.java:917)
        at org.apache.directory.server.core.shared.partition.DefaultPartitionNexus.delete(DefaultPartitionNexus.java:405)
        at org.apache.directory.server.core.api.interceptor.BaseInterceptor$1.delete(BaseInterceptor.java:192)
        at org.apache.directory.server.core.api.interceptor.BaseInterceptor.next(BaseInterceptor.java:490)
        at org.apache.directory.server.core.journal.JournalInterceptor.delete(JournalInterceptor.java:181)
        at org.apache.directory.server.core.api.interceptor.BaseInterceptor.next(BaseInterceptor.java:490)
        at org.apache.directory.server.core.trigger.TriggerInterceptor.delete(TriggerInterceptor.java:336)
        at org.apache.directory.server.core.api.interceptor.BaseInterceptor.next(BaseInterceptor.java:490)
        at org.apache.directory.server.core.event.EventInterceptor.delete(EventInterceptor.java:250)
        at org.apache.directory.server.core.api.interceptor.BaseInterceptor.next(BaseInterceptor.java:490)
        at org.apache.directory.server.core.subtree.SubentryInterceptor.delete(SubentryInterceptor.java:1058)
        at org.apache.directory.server.core.api.interceptor.BaseInterceptor.next(BaseInterceptor.java:490)
        at org.apache.directory.server.core.operational.OperationalAttributeInterceptor.delete(OperationalAttributeInterceptor.java:462)
        at org.apache.directory.server.core.api.interceptor.BaseInterceptor.next(BaseInterceptor.java:490)
        at org.apache.directory.server.core.exception.ExceptionInterceptor.delete(ExceptionInterceptor.java:207)
        at org.apache.directory.server.core.api.interceptor.BaseInterceptor.next(BaseInterceptor.java:490)
        at org.apache.directory.server.core.admin.AdministrativePointInterceptor.delete(AdministrativePointInterceptor.java:1261)
        at org.apache.directory.server.core.api.interceptor.BaseInterceptor.next(BaseInterceptor.java:490)
        at org.apache.directory.server.core.authz.DefaultAuthorizationInterceptor.delete(DefaultAuthorizationInterceptor.java:225)
        at org.apache.directory.server.core.api.interceptor.BaseInterceptor.next(BaseInterceptor.java:490)
        at org.apache.directory.server.core.authz.AciAuthorizationInterceptor.delete(AciAuthorizationInterceptor.java:663)
        at org.apache.directory.server.core.api.interceptor.BaseInterceptor.next(BaseInterceptor.java:490)
        at org.apache.directory.server.core.referral.ReferralInterceptor.delete(ReferralInterceptor.java:288)
        at org.apache.directory.server.core.api.interceptor.BaseInterceptor.next(BaseInterceptor.java:490)
        at org.apache.directory.server.core.authn.AuthenticationInterceptor.delete(AuthenticationInterceptor.java:754)
        at org.apache.directory.server.core.api.interceptor.BaseInterceptor.next(BaseInterceptor.java:490)
        at org.apache.directory.server.core.normalization.NormalizationInterceptor.delete(NormalizationInterceptor.java:174)
        at org.apache.directory.server.core.DefaultOperationManager.delete(DefaultOperationManager.java:641)
        at org.apache.directory.server.core.shared.DefaultCoreSession.delete(DefaultCoreSession.java:924)
        at org.apache.directory.server.core.shared.DefaultCoreSession.delete(DefaultCoreSession.java:907)
        at org.apache.directory.server.ldap.handlers.request.DeleteRequestHandler.handle(DeleteRequestHandler.java:55)
        at org.apache.directory.server.ldap.handlers.request.DeleteRequestHandler.handle(DeleteRequestHandler.java:39)
        at org.apache.directory.server.ldap.handlers.LdapRequestHandler.handleMessage(LdapRequestHandler.java:207)
        at org.apache.directory.server.ldap.handlers.LdapRequestHandler.handleMessage(LdapRequestHandler.java:56)
        at org.apache.mina.handler.demux.DemuxingIoHandler.messageReceived(DemuxingIoHandler.java:221)
        at org.apache.directory.server.ldap.LdapProtocolHandler.messageReceived(LdapProtocolHandler.java:217)
        at org.apache.mina.core.filterchain.DefaultIoFilterChain$TailFilter.messageReceived(DefaultIoFilterChain.java:854)
        at org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextMessageReceived(DefaultIoFilterChain.java:542)
        at org.apache.mina.core.filterchain.DefaultIoFilterChain.access$1300(DefaultIoFilterChain.java:48)
        at org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.messageReceived(DefaultIoFilterChain.java:943)
        at org.apache.mina.core.filterchain.IoFilterEvent.fire(IoFilterEvent.java:74)
        at org.apache.mina.core.session.IoEvent.run(IoEvent.java:63)
        at org.apache.mina.filter.executor.UnorderedThreadPoolExecutor$Worker.runTask(UnorderedThreadPoolExecutor.java:475)
        at org.apache.mina.filter.executor.UnorderedThreadPoolExecutor$Worker.run(UnorderedThreadPoolExecutor.java:429)
        at java.lang.Thread.run(Thread.java:745)

        Show
        wayswssb linzhao added a comment - I should say I find another exception during the testing. when delete entry from apacheds. Exception list below, may be this exception is not related to the OOM exception. [06:22:04] WARN [org.apache.directory.server.core.api.interceptor.context.FilteringOperationContext] - Requested attribute javaRemoteLocation does not exist in the schema, it will be ignored [06:25:40] ERROR [org.apache.directory.server.core.partition.impl.btree.mavibot.MavibotTable] - ERR_133 Exception while removing 6c7dd69c-0f53-4b49-8ce6-be84b165938b from index 1.3.6.1.4.1.18060.0.4.1.2.50_reverse java.lang.ArrayIndexOutOfBoundsException: 6 at org.apache.directory.mavibot.btree.RecordManager.readBytes(RecordManager.java:1209) at org.apache.directory.mavibot.btree.RecordManager.readPage(RecordManager.java:1021) at org.apache.directory.mavibot.btree.RecordManager.deserialize(RecordManager.java:989) at org.apache.directory.mavibot.btree.PersistedPageHolder.fetchElement(PersistedPageHolder.java:133) at org.apache.directory.mavibot.btree.PersistedPageHolder.getValue(PersistedPageHolder.java:113) at org.apache.directory.mavibot.btree.AbstractPage.getReference(AbstractPage.java:155) at org.apache.directory.mavibot.btree.RecordManager.serializeNodeValue(RecordManager.java:1708) at org.apache.directory.mavibot.btree.RecordManager.serializePage(RecordManager.java:1653) at org.apache.directory.mavibot.btree.RecordManager.writePage(RecordManager.java:2808) at org.apache.directory.mavibot.btree.PersistedNode.createHolder(PersistedNode.java:906) at org.apache.directory.mavibot.btree.PersistedNode.handleRemoveResult(PersistedNode.java:207) at org.apache.directory.mavibot.btree.PersistedNode.delete(PersistedNode.java:607) at org.apache.directory.mavibot.btree.AbstractPage.delete(AbstractPage.java:217) at org.apache.directory.mavibot.btree.PersistedBTree.processDelete(PersistedBTree.java:332) at org.apache.directory.mavibot.btree.PersistedBTree.delete(PersistedBTree.java:292) at org.apache.directory.mavibot.btree.AbstractBTree.delete(AbstractBTree.java:390) at org.apache.directory.mavibot.btree.AbstractBTree.delete(AbstractBTree.java:324) at org.apache.directory.mavibot.btree.PersistedBTree.delete(PersistedBTree.java:43) at org.apache.directory.server.core.partition.impl.btree.mavibot.MavibotTable.remove(MavibotTable.java:389) at org.apache.directory.server.core.partition.impl.btree.mavibot.MavibotIndex.drop(MavibotIndex.java:391) at org.apache.directory.server.core.partition.impl.btree.mavibot.MavibotIndex.drop(MavibotIndex.java:58) at org.apache.directory.server.core.partition.impl.btree.AbstractBTreePartition.delete(AbstractBTreePartition.java:1078) at org.apache.directory.server.core.partition.impl.btree.AbstractBTreePartition.delete(AbstractBTreePartition.java:917) at org.apache.directory.server.core.shared.partition.DefaultPartitionNexus.delete(DefaultPartitionNexus.java:405) at org.apache.directory.server.core.api.interceptor.BaseInterceptor$1.delete(BaseInterceptor.java:192) at org.apache.directory.server.core.api.interceptor.BaseInterceptor.next(BaseInterceptor.java:490) at org.apache.directory.server.core.journal.JournalInterceptor.delete(JournalInterceptor.java:181) at org.apache.directory.server.core.api.interceptor.BaseInterceptor.next(BaseInterceptor.java:490) at org.apache.directory.server.core.trigger.TriggerInterceptor.delete(TriggerInterceptor.java:336) at org.apache.directory.server.core.api.interceptor.BaseInterceptor.next(BaseInterceptor.java:490) at org.apache.directory.server.core.event.EventInterceptor.delete(EventInterceptor.java:250) at org.apache.directory.server.core.api.interceptor.BaseInterceptor.next(BaseInterceptor.java:490) at org.apache.directory.server.core.subtree.SubentryInterceptor.delete(SubentryInterceptor.java:1058) at org.apache.directory.server.core.api.interceptor.BaseInterceptor.next(BaseInterceptor.java:490) at org.apache.directory.server.core.operational.OperationalAttributeInterceptor.delete(OperationalAttributeInterceptor.java:462) at org.apache.directory.server.core.api.interceptor.BaseInterceptor.next(BaseInterceptor.java:490) at org.apache.directory.server.core.exception.ExceptionInterceptor.delete(ExceptionInterceptor.java:207) at org.apache.directory.server.core.api.interceptor.BaseInterceptor.next(BaseInterceptor.java:490) at org.apache.directory.server.core.admin.AdministrativePointInterceptor.delete(AdministrativePointInterceptor.java:1261) at org.apache.directory.server.core.api.interceptor.BaseInterceptor.next(BaseInterceptor.java:490) at org.apache.directory.server.core.authz.DefaultAuthorizationInterceptor.delete(DefaultAuthorizationInterceptor.java:225) at org.apache.directory.server.core.api.interceptor.BaseInterceptor.next(BaseInterceptor.java:490) at org.apache.directory.server.core.authz.AciAuthorizationInterceptor.delete(AciAuthorizationInterceptor.java:663) at org.apache.directory.server.core.api.interceptor.BaseInterceptor.next(BaseInterceptor.java:490) at org.apache.directory.server.core.referral.ReferralInterceptor.delete(ReferralInterceptor.java:288) at org.apache.directory.server.core.api.interceptor.BaseInterceptor.next(BaseInterceptor.java:490) at org.apache.directory.server.core.authn.AuthenticationInterceptor.delete(AuthenticationInterceptor.java:754) at org.apache.directory.server.core.api.interceptor.BaseInterceptor.next(BaseInterceptor.java:490) at org.apache.directory.server.core.normalization.NormalizationInterceptor.delete(NormalizationInterceptor.java:174) at org.apache.directory.server.core.DefaultOperationManager.delete(DefaultOperationManager.java:641) at org.apache.directory.server.core.shared.DefaultCoreSession.delete(DefaultCoreSession.java:924) at org.apache.directory.server.core.shared.DefaultCoreSession.delete(DefaultCoreSession.java:907) at org.apache.directory.server.ldap.handlers.request.DeleteRequestHandler.handle(DeleteRequestHandler.java:55) at org.apache.directory.server.ldap.handlers.request.DeleteRequestHandler.handle(DeleteRequestHandler.java:39) at org.apache.directory.server.ldap.handlers.LdapRequestHandler.handleMessage(LdapRequestHandler.java:207) at org.apache.directory.server.ldap.handlers.LdapRequestHandler.handleMessage(LdapRequestHandler.java:56) at org.apache.mina.handler.demux.DemuxingIoHandler.messageReceived(DemuxingIoHandler.java:221) at org.apache.directory.server.ldap.LdapProtocolHandler.messageReceived(LdapProtocolHandler.java:217) at org.apache.mina.core.filterchain.DefaultIoFilterChain$TailFilter.messageReceived(DefaultIoFilterChain.java:854) at org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextMessageReceived(DefaultIoFilterChain.java:542) at org.apache.mina.core.filterchain.DefaultIoFilterChain.access$1300(DefaultIoFilterChain.java:48) at org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.messageReceived(DefaultIoFilterChain.java:943) at org.apache.mina.core.filterchain.IoFilterEvent.fire(IoFilterEvent.java:74) at org.apache.mina.core.session.IoEvent.run(IoEvent.java:63) at org.apache.mina.filter.executor.UnorderedThreadPoolExecutor$Worker.runTask(UnorderedThreadPoolExecutor.java:475) at org.apache.mina.filter.executor.UnorderedThreadPoolExecutor$Worker.run(UnorderedThreadPoolExecutor.java:429) at java.lang.Thread.run(Thread.java:745)
        Hide
        wayswssb linzhao added a comment -

        Is this OOM related to this issue https://issues.apache.org/jira/browse/DIRSHARED-108?

        Show
        wayswssb linzhao added a comment - Is this OOM related to this issue https://issues.apache.org/jira/browse/DIRSHARED-108?
        Hide
        akiran Kiran Ayyagari added a comment -

        Is this OOM related to this issue...

        No, it is not.

        Show
        akiran Kiran Ayyagari added a comment - Is this OOM related to this issue... No, it is not.
        Hide
        akiran Kiran Ayyagari added a comment -

        Lin Zhao Can you test this with Oracle VM, I haven't tested it using OpenJDK and also I don't think anyone of us in the Directory
        team tested it with OpenJDK.

        Show
        akiran Kiran Ayyagari added a comment - Lin Zhao Can you test this with Oracle VM, I haven't tested it using OpenJDK and also I don't think anyone of us in the Directory team tested it with OpenJDK.
        Hide
        wayswssb linzhao added a comment -

        For the ArrayIndexOutOfBoundsException issue, our program will access apacheds with multi-thread. It means that sometimes we do search, add, edit and delete action together. Is it the multi-thread issue to cause this problem?

        Show
        wayswssb linzhao added a comment - For the ArrayIndexOutOfBoundsException issue, our program will access apacheds with multi-thread. It means that sometimes we do search, add, edit and delete action together. Is it the multi-thread issue to cause this problem?
        Hide
        elecharny Emmanuel Lecharny added a comment -

        Can we please open new JIRA for new probelms ?

        It helps to get some track on each different issue.

        Thanks !

        PS : I'll close this issue.

        Show
        elecharny Emmanuel Lecharny added a comment - Can we please open new JIRA for new probelms ? It helps to get some track on each different issue. Thanks ! PS : I'll close this issue.
        Hide
        elecharny Emmanuel Lecharny added a comment -

        The browseFrom method was broken. It has been fixed.

        Show
        elecharny Emmanuel Lecharny added a comment - The browseFrom method was broken. It has been fixed.
        Hide
        wayswssb linzhao added a comment - - edited

        Hi Emmanuel Lecharny,

        I found that there is another partition LdifPartition in apacheds, can we use this partition in apacheds? Because we found this partition is very simple and it stored data in to a file. If we can use this kind of this partition, do you have some doc about how to add this partition into apacheds?

        Thanks,
        Lin

        Show
        wayswssb linzhao added a comment - - edited Hi Emmanuel Lecharny, I found that there is another partition LdifPartition in apacheds, can we use this partition in apacheds? Because we found this partition is very simple and it stored data in to a file. If we can use this kind of this partition, do you have some doc about how to add this partition into apacheds? Thanks, Lin

          People

          • Assignee:
            Unassigned
            Reporter:
            wayswssb linzhao
          • Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development