Uploaded image for project: 'Geode'
  1. Geode
  2. GEODE-7945

Cluster restart recovery from disk blocked by waiting replies of CreateRegionMessage

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.13.0
    • Component/s: None
    • Labels:

      Description

      A cluster restart recovering from disk has unexpected delays in some of the members. The logs show that those delayed members are waiting for replies of CreateRegionMessage, before loading the krf files. And the reason for waiting the replies of CreateRegionMessage is likely because other members are holding some lock while busy loading the krf files. Therefore the replies are delayed.

      Once those delayed members get the replies of CreateRegionMessage, it starts loading the krf files. If the delayed members contain the latest data, this could block other members waiting for the latest data.

      Because the cluster members are blocking each other at different stages during the cluster restart recovery process, the whole process is unexpectedly long when the disk store contains large amount of data.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                jchen21 Jianxia Chen
                Reporter:
                jchen21 Jianxia Chen
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 1.5h
                  1.5h