[DIRSERVER-2172] ApacheDS 2.0.0M12 in production, getting ERR_554 double get for block block 3,474

• Type: Bug
• Status: Open
• Priority: Critical
• Resolution: Unresolved
• Component/s: jdbm
• Environment: GNU/Linux x86_64
• Assignee: Unassigned
• Reporter: sourabh sihare
• Created: 09/Nov/16 10:57
• Updated: 26/Oct/20 16:30

Description

We have upgraded apacheds from 2.0.0-M23 assuming that will fix the issue with data getting corrupt but we are still facing the same issue.The apacheds server keeps running but data does not load fully from front end like 400 users instead of actual 3000+ users.
-> As a workaround we take backup of partition daily and when the issue occurs we replace the partition from backed-up partition and restart the server.
> From apacheds.log we get below error trace

[16:13:21] DEBUG [org.apache.directory.server.OPERATION_LOG] - << CompareOperation successful
[16:13:21] DEBUG [org.apache.directory.server.OPERATION_TIME] - Compare operation took 183000 ns
[16:13:21] DEBUG [org.apache.directory.server.OPERATION_LOG] - >> CompareOperation : CompareContext for Dn 'uid=GA1346,ou=users,o=sccm', oid : <objectClass>, value :'groupOfNames'
[16:13:21] DEBUG [org.apache.directory.server.OPERATION_LOG] - >> LookupOperation : FilteringOperationContext for Dn 'uid=GA1346,ou=users,o=sccm', +, *
[16:13:21] WARN [org.apache.directory.server.ldap.LdapProtocolHandler] - Unexpected exception forcing session to close: sending disconnect notice to client.
java.lang.Error: ERR_554 double get for block 3,474
at jdbm.recman.RecordFile.get(RecordFile.java:185)
at jdbm.recman.PhysicalRowIdManager.allocNew(PhysicalRowIdManager.java:202)
at jdbm.recman.PhysicalRowIdManager.alloc(PhysicalRowIdManager.java:177)
at jdbm.recman.PhysicalRowIdManager.update(PhysicalRowIdManager.java:101)
at jdbm.recman.BaseRecordManager.update(BaseRecordManager.java:281)
at jdbm.recman.CacheRecordManager$CacheListener.cacheObjectEvicted(CacheRecordManager.java:459)
at jdbm.recman.CacheRecordManager$CacheListener.cacheObjectEvicted(CacheRecordManager.java:444)
at jdbm.helper.MRU.purgeEntry(MRU.java:310)
at jdbm.helper.MRU.put(MRU.java:128)
at jdbm.recman.CacheRecordManager.fetch(CacheRecordManager.java:268)
at jdbm.btree.BPage.loadBPage(BPage.java:949)
at jdbm.btree.BPage.find(BPage.java:280)
at jdbm.btree.BTree.find(BTree.java:413)
at org.apache.directory.server.core.partition.impl.btree.jdbm.JdbmTable.get(JdbmTable.java:343)
at org.apache.directory.server.core.partition.impl.btree.AbstractBTreePartition.fetch(AbstractBTreePartition.java:1274)
at org.apache.directory.server.core.partition.impl.btree.AbstractBTreePartition.lookup(AbstractBTreePartition.java:1202)
at org.apache.directory.server.core.shared.partition.DefaultPartitionNexus.lookup(DefaultPartitionNexus.java:464)
at org.apache.directory.server.core.api.interceptor.BaseInterceptor$1.lookup(BaseInterceptor.java:161)
at org.apache.directory.server.core.api.interceptor.BaseInterceptor.next(BaseInterceptor.java:483)
at org.apache.directory.server.core.collective.CollectiveAttributeInterceptor.lookup(CollectiveAttributeInterceptor.java:143)
at org.apache.directory.server.core.api.interceptor.BaseInterceptor.next(BaseInterceptor.java:483)
at org.apache.directory.server.core.operational.OperationalAttributeInterceptor.lookup(OperationalAttributeInterceptor.java:329)
at org.apache.directory.server.core.api.interceptor.BaseInterceptor.next(BaseInterceptor.java:483)
at org.apache.directory.server.core.schema.SchemaInterceptor.lookup(SchemaInterceptor.java:1142)
at org.apache.directory.server.core.api.interceptor.BaseInterceptor.next(BaseInterceptor.java:483)
at org.apache.directory.server.core.authz.DefaultAuthorizationInterceptor.lookup(DefaultAuthorizationInterceptor.java:231)
at org.apache.directory.server.core.api.interceptor.BaseInterceptor.next(BaseInterceptor.java:483)
at org.apache.directory.server.core.authz.AciAuthorizationInterceptor.lookup(AciAuthorizationInterceptor.java:779)
at org.apache.directory.server.core.api.interceptor.BaseInterceptor.next(BaseInterceptor.java:483)
at org.apache.directory.server.core.authn.AuthenticationInterceptor.lookup(AuthenticationInterceptor.java:852)
at org.apache.directory.server.core.api.interceptor.BaseInterceptor.next(BaseInterceptor.java:483)
at org.apache.directory.server.core.normalization.NormalizationInterceptor.lookup(NormalizationInterceptor.java:196)
at org.apache.directory.server.core.DefaultOperationManager.lookup(DefaultOperationManager.java:775)
at org.apache.directory.server.core.shared.DefaultCoreSession.lookup(DefaultCoreSession.java:560)
at org.apache.directory.server.core.DefaultOperationManager.getOriginalEntry(DefaultOperationManager.java:194)
at org.apache.directory.server.core.DefaultOperationManager.compare(DefaultOperationManager.java:531)
at org.apache.directory.server.core.shared.DefaultCoreSession.compare(DefaultCoreSession.java:885)
at org.apache.directory.server.ldap.handlers.request.CompareRequestHandler.handle(CompareRequestHandler.java:52)
at org.apache.directory.server.ldap.handlers.request.CompareRequestHandler.handle(CompareRequestHandler.java:38)
at org.apache.directory.server.ldap.handlers.LdapRequestHandler.handleMessage(LdapRequestHandler.java:207)
at org.apache.directory.server.ldap.handlers.LdapRequestHandler.handleMessage(LdapRequestHandler.java:56)
at org.apache.mina.handler.demux.DemuxingIoHandler.messageReceived(DemuxingIoHandler.java:243)
at org.apache.directory.server.ldap.LdapProtocolHandler.messageReceived(LdapProtocolHandler.java:216)
at org.apache.mina.core.filterchain.DefaultIoFilterChain$TailFilter.messageReceived(DefaultIoFilterChain.java:854)
at org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextMessageReceived(DefaultIoFilterChain.java:542)
at org.apache.mina.core.filterchain.DefaultIoFilterChain.access$1300(DefaultIoFilterChain.java:48)
at org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.messageReceived(DefaultIoFilterChain.java:943)
at org.apache.mina.core.filterchain.IoFilterEvent.fire(IoFilterEvent.java:74)
at org.apache.mina.core.session.IoEvent.run(IoEvent.java:63)
at org.apache.mina.filter.executor.UnorderedThreadPoolExecutor$Worker.runTask(UnorderedThreadPoolExecutor.java:476)
at org.apache.mina.filter.executor.UnorderedThreadPoolExecutor$Worker.run(UnorderedThreadPoolExecutor.java:430)
at java.lang.Thread.run(Thread.java:745)