Uploaded image for project: 'Ignite'
  1. Ignite
  2. IGNITE-11796

WAL recovery stopped with 'Partition consistency failure: newPageId'

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.7.5
    • Component/s: None
    • Labels:
      None

      Description

      https://repository.apache.org/content/repositories/orgapacheignite-1436/ test using TC bot showed that there can be a situation when DB contains an incorrect record. This record probably does not lead to corruption (because 2.7.0 works well with this DB).

      But validation introduced in https://issues.apache.org/jira/browse/IGNITE-11030 stops WAL recovery.

      java.util.concurrent.ExecutionException: java.lang.AssertionError: Partition consistency failure: newPageId=100000000f3d3 (newPartId: 0) pageId=101001a0000f3d3 (partId: 26)
      	at java.base/java.util.concurrent.FutureTask.report(FutureTask.java:122)
      	at java.base/java.util.concurrent.FutureTask.get(FutureTask.java:205)
      	at org.apache.ignite.ci.di.IgniteTcBotModule.lambda$configure$0(IgniteTcBotModule.java:61)
      	at com.google.inject.internal.ProviderInternalFactory.provision(ProviderInternalFactory.java:85)
      	at com.google.inject.internal.InternalFactoryToInitializableAdapter.provision(InternalFactoryToInitializableAdapter.java:57)
      	at com.google.inject.internal.ProviderInternalFactory.circularGet(ProviderInternalFactory.java:59)
      	at com.google.inject.internal.InternalFactoryToInitializableAdapter.get(InternalFactoryToInitializableAdapter.java:47)
      	at com.google.inject.internal.InjectorImpl$1.get(InjectorImpl.java:1050)
      	at com.google.inject.internal.InjectorImpl.getInstance(InjectorImpl.java:1086)
      	at org.apache.ignite.ci.web.auth.AuthenticationFilter.filter(AuthenticationFilter.java:127)
      	at org.glassfish.jersey.server.ContainerFilteringStage.apply(ContainerFilteringStage.java:108)
      	at org.glassfish.jersey.server.ContainerFilteringStage.apply(ContainerFilteringStage.java:44)
      	at org.glassfish.jersey.process.internal.Stages.process(Stages.java:173)
      	at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:245)
      	at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248)
      	at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244)
      	at org.glassfish.jersey.internal.Errors.process(Errors.java:292)
      	at org.glassfish.jersey.internal.Errors.process(Errors.java:274)
      	at org.glassfish.jersey.internal.Errors.process(Errors.java:244)
      	at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265)
      	at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:232)
      	at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:679)
      	at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:392)
      	at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346)
      	at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:365)
      	at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:318)
      	at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205)
      	at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:857)
      	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:535)
      	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146)
      	at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
      	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
      	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257)
      	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)
      	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
      	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1340)
      	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
      	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)
      	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)
      	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
      	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1242)
      	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
      	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
      	at org.eclipse.jetty.server.Server.handle(Server.java:503)
      	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:364)
      	at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260)
      	at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305)
      	at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
      	at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118)
      	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333)
      	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310)
      	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
      	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:132)
      	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:765)
      	at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:683)
      	at java.base/java.lang.Thread.run(Thread.java:834)
      Caused by: java.lang.AssertionError: Partition consistency failure: newPageId=100000000f3d3 (newPartId: 0) pageId=101001a0000f3d3 (partId: 26)
      	at org.apache.ignite.internal.pagemem.wal.record.delta.InitNewPageRecord.<init>(InitNewPageRecord.java:62)
      	at org.apache.ignite.internal.processors.cache.persistence.wal.serializer.RecordDataV1Serializer.readPlainRecord(RecordDataV1Serializer.java:796)
      	at org.apache.ignite.internal.processors.cache.persistence.wal.serializer.RecordDataV2Serializer.readPlainRecord(RecordDataV2Serializer.java:197)
      	at org.apache.ignite.internal.processors.cache.persistence.wal.serializer.RecordDataV1Serializer.readRecord(RecordDataV1Serializer.java:208)
      	at org.apache.ignite.internal.processors.cache.persistence.wal.serializer.RecordV2Serializer$2.readWithHeaders(RecordV2Serializer.java:162)
      	at org.apache.ignite.internal.processors.cache.persistence.wal.serializer.RecordV1Serializer.readWithCrc(RecordV1Serializer.java:372)
      	at org.apache.ignite.internal.processors.cache.persistence.wal.serializer.RecordV2Serializer.readRecord(RecordV2Serializer.java:235)
      	at org.apache.ignite.internal.processors.cache.persistence.wal.AbstractWalRecordsIterator.advanceRecord(AbstractWalRecordsIterator.java:243)
      	at org.apache.ignite.internal.processors.cache.persistence.wal.AbstractWalRecordsIterator.advance(AbstractWalRecordsIterator.java:154)
      	at org.apache.ignite.internal.processors.cache.persistence.wal.AbstractWalRecordsIterator.onNext(AbstractWalRecordsIterator.java:123)
      	at org.apache.ignite.internal.processors.cache.persistence.wal.AbstractWalRecordsIterator.onNext(AbstractWalRecordsIterator.java:52)
      	at org.apache.ignite.internal.util.GridCloseableIteratorAdapter.nextX(GridCloseableIteratorAdapter.java:41)
      	at org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager$RestoreStateContext.next(GridCacheDatabaseSharedManager.java:4781)
      	at org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager$RestoreBinaryState.next(GridCacheDatabaseSharedManager.java:4858)
      	at org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.restoreMemory(GridCacheDatabaseSharedManager.java:2016)
      	at org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.readMetastore(GridCacheDatabaseSharedManager.java:653)
      	at org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.notifyMetaStorageSubscribersOnReadyForRead(GridCacheDatabaseSharedManager.java:4611)
      	at org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:1048)
      	at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:2038)
      	at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1730)
      	at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1158)
      	at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:678)
      	at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:603)
      	at org.apache.ignite.Ignition.start(Ignition.java:323)
      	at org.apache.ignite.ci.db.Ignite1Init.ignitionStart(Ignite1Init.java:83)
      	at org.apache.ignite.ci.di.MonitoredTaskInterceptor.invoke(MonitoredTaskInterceptor.java:111)
      	at org.apache.ignite.ci.di.AutoProfilingInterceptor.invoke(AutoProfilingInterceptor.java:72)
      	at org.apache.ignite.ci.db.Ignite1Init.startIgnite(Ignite1Init.java:62)
      	at org.apache.ignite.ci.db.Ignite1Init.init(Ignite1Init.java:139)
      	at org.apache.ignite.ci.di.AutoProfilingInterceptor.invoke(AutoProfilingInterceptor.java:72)
      	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
      	... 1 more
      

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                dpavlov Dmitry Pavlov
                Reporter:
                dpavlov Dmitry Pavlov
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 20m
                  20m