Apache S4
  1. Apache S4
  2. S4-44

optional backoff upon multiple consecutive failed checkpoint fetches

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.4, 0.5.0
    • Fix Version/s: 0.5.0
    • Labels:
      None

      Description

      if a checkpointing backend system becomes unresponsive (e.g. stalled NFS), and that a series of recoveries is to proceed (for instance, startup or failover), then each checkpoint fetching operation will block, wait for a timeout or another kind of exception, and the system will then continue without recovering this PE.

      We should provide a way to detect this pattern (multiple backend fetches failures in a short amount of time) and temporarily disable fetching from the backend, in order to reduce blocking when backend becomes unresponsive.

        Issue Links

          Activity

          Hide
          Matthieu Morel added a comment -

          implemented in branch S4-44 https://git-wip-us.apache.org/repos/asf?p=incubator-s4.git;a=tree;h=refs/heads/S4-44;hb=S4-44

          I also refactored the checkpointing mechanism so that time-based checkpointing does not generate extra events.

          Show
          Matthieu Morel added a comment - implemented in branch S4-44 https://git-wip-us.apache.org/repos/asf?p=incubator-s4.git;a=tree;h=refs/heads/S4-44;hb=S4-44 I also refactored the checkpointing mechanism so that time-based checkpointing does not generate extra events.
          Hide
          Matthieu Morel added a comment -

          patch available for S4 piper in S4-11

          Show
          Matthieu Morel added a comment - patch available for S4 piper in S4-11
          Hide
          Matthieu Morel added a comment -

          Fixed in S4-11

          Show
          Matthieu Morel added a comment - Fixed in S4-11

            People

            • Assignee:
              Matthieu Morel
              Reporter:
              Matthieu Morel
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development