Uploaded image for project: 'Ignite'
  1. Ignite
  2. IGNITE-12850

Ignite node cannot be started (metastorage history loading fails)

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Blocker
    • Resolution: Unresolved
    • Affects Version/s: 2.8, 2.7.6
    • Fix Version/s: None
    • Component/s: cache
    • Labels:
      None
    • Ignite Flags:
      Docs Required, Release Notes Required

      Description

      1. metastorage is using persistence
      2. when a node is ready to write, writeBaselineTopology is called with null history item, and generates base line topology history with gaps
      3. from that point, it is impossible to start the node as `restoreHistory` throws an exception when it is processing the gap

      tested on 2.7.6, but it seems that ignite 2.8.0 would suffer from the same issue - by looking at the code

      2020-03-21_00:00:03.867 [fapi-main-0] INFO  o.a.i.i.p.c.GridClusterStateProcessor:117 <> - Restoring history for BaselineTopology[id=9]
      2020-03-21_00:00:03.904 [fapi-main-0] ERROR o.a.ignite.internal.IgniteKernal:137 <> - Exception during start processors, node will be stopped and close connections
      org.apache.ignite.IgniteCheckedException: Restoring of BaselineTopology history has failed, expected history item not found for id=8
              at org.apache.ignite.internal.processors.cluster.BaselineTopologyHistory.restoreHistory(BaselineTopologyHistory.java:54)
              at org.apache.ignite.internal.processors.cluster.GridClusterStateProcessor.onReadyForRead(GridClusterStateProcessor.java:223)
              at org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.notifyMetastorageReadyForRead(GridCacheDatabaseSharedManager.java:409)
              at org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.readMetastore(GridCacheDatabaseSharedManager.java:675)
              at org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.notifyMetaStorageSubscribersOnReadyForRead(GridCacheDatabaseSharedManager.java:4730)
              at org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:1048)
              at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:2038)
              at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1730)
              at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1158)
              at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:678)
       

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              sarunas Sarunas Valaskevicius
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated: