Uploaded image for project: 'ZooKeeper'
  1. ZooKeeper
  2. ZOOKEEPER-2325

Data inconsistency if all snapshots empty or missing

    XMLWordPrintableJSON

    Details

      Description

      When loading state from snapshots on startup, FileTxnSnapLog.java ignores the result of FileSnap.deserialize, which is -1L if no valid snapshots are found. Recovery proceeds with dt.lastProcessed == 0, its initial value.

      The result is that Zookeeper will process the transaction logs and then begin serving requests with a different state than the rest of the ensemble.

      To reproduce:
      In a healthy zookeeper cluster of size >= 3, shut down one node.
      Either delete all snapshots for this node or change all to be empty files.
      Restart the node.

      We believe this can happen organically if a node runs out of disk space.

        Attachments

        1. zk.patch
          1 kB
          Benjamin Reed
        2. ZOOKEEPER-2325-test.patch
          5 kB
          Andrew Grasso
        3. ZOOKEEPER-2325.001.patch
          0.8 kB
          Andrew Grasso

          Issue Links

            Activity

              People

              • Assignee:
                agrasso Andrew Grasso
                Reporter:
                agrasso Andrew Grasso
              • Votes:
                1 Vote for this issue
                Watchers:
                15 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - 1h
                  1h
                  Remaining:
                  Time Spent - 10m Remaining Estimate - 50m
                  50m
                  Logged:
                  Time Spent - 10m Remaining Estimate - 50m
                  10m