Uploaded image for project: 'ZooKeeper'
  1. ZooKeeper
  2. ZOOKEEPER-2325

Data inconsistency if all snapshots empty or missing

VotersStop watchingWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    Description

      When loading state from snapshots on startup, FileTxnSnapLog.java ignores the result of FileSnap.deserialize, which is -1L if no valid snapshots are found. Recovery proceeds with dt.lastProcessed == 0, its initial value.

      The result is that Zookeeper will process the transaction logs and then begin serving requests with a different state than the rest of the ensemble.

      To reproduce:
      In a healthy zookeeper cluster of size >= 3, shut down one node.
      Either delete all snapshots for this node or change all to be empty files.
      Restart the node.

      We believe this can happen organically if a node runs out of disk space.

      Attachments

        1. ZOOKEEPER-2325.001.patch
          0.8 kB
          Andrew Grasso
        2. ZOOKEEPER-2325-test.patch
          5 kB
          Andrew Grasso
        3. zk.patch
          1 kB
          Benjamin Reed

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            agrasso Andrew Grasso
            agrasso Andrew Grasso
            Votes:
            1 Vote for this issue
            Watchers:
            15 Stop watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Time Tracking

                Estimated:
                Original Estimate - 1h
                1h
                Remaining:
                Time Spent - 10m Remaining Estimate - 50m
                50m
                Logged:
                Time Spent - 10m Remaining Estimate - 50m
                10m

                Slack

                  Issue deployment