Uploaded image for project: 'Kudu'
  1. Kudu
  2. KUDU-1078

Under heavy load, log cache reads return "Op in future"

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • Private Beta
    • 1.6.0
    • consensus
    • None

    Description

      JD accidentally configured botl80 so that all nodes were writing to a single tablet (talk about a stress test!) I see the following warning occasionally in the leader logs:

      W0827 11:34:04.051275 55950 consensus_queue.cc:272] T d6ff74fb04454712873abd0d2328e59b P 668314723deb4a818cc9a43eba51073c [LEADER]: The logs necessary to catch up peer 600e62435ca24425b29c00d9726b78be have been garbage collected. The follower will never be able to catch up (Not found: op in future). Instructing remote peer to remotely bootstrap.

      Calling this a blocker because, if we couple this with the "automatically re-bootstrap nodes if they fall behind", we'll be accidentally deleting tablets when this happens (no good!)

      Attachments

        Activity

          People

            tlipcon Todd Lipcon
            tlipcon Todd Lipcon
            Votes:
            0 Vote for this issue
            Watchers:
            8 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: