ZooKeeper
  1. ZooKeeper
  2. ZOOKEEPER-2003

Missing fsync() on the logs parent directory

    Details

    • Type: Bug Bug
    • Status: Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: 3.4.6
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      After studying the steps ZooKeeper takes to update the logs we found the following bug. The bug may not manifest in the current file system implementations, but it violates the POSIX recommendations and may be an issue in some file systems.

      Looking at the strace of zookeeper we see the following:
      mkdir(v)
      create(v/log)
      append(v/log)
      trunk(v/log)
      write(v/log)
      fdatasync(v/log)

      Although the data is fdatasynced to the log, the parent directory was never fsynced, consequently in case of a crash, the parent directory or the log file may be lost, as the parent directory and file metadata were never persisted on disk.
      To be safe, both the log directory, and parent directory of the log directory should be fsynced as well.

        Activity

        Samer Al-Kiswany created issue -
        Samer Al-Kiswany made changes -
        Field Original Value New Value
        Description After studying the steps ZooKeeper takes to update the logs we found the following bug. The bug may not manifest in the current file system implementations, but it violates the POSIX recommendations and may be an issue in some file systems.

        Looking at the strace of zookeeper we see the following:
        mkdir(v)
        create(v/log)
        append(v/log)
        trunk(v/log) to almost 64MBs
        write(v/log) 1 byte
        fdatasync(v/log)

        Although the data is fdatasynced to the log, the parent directory was never fsynced, consequently in case of a crash, the parent directory or the log file may be lost, as the parent directory and file metadata were never persisted on disk.
        To be safe, both the log directory, and parent directory of the log directory should be fsynced as well.
        After studying the steps ZooKeeper takes to update the logs we found the following bug. The bug may not manifest in the current file system implementations, but it violates the POSIX recommendations and may be an issue in some file systems.

        Looking at the strace of zookeeper we see the following:
        mkdir(v)
        create(v/log)
        append(v/log)
        trunk(v/log)
        write(v/log)
        fdatasync(v/log)

        Although the data is fdatasynced to the log, the parent directory was never fsynced, consequently in case of a crash, the parent directory or the log file may be lost, as the parent directory and file metadata were never persisted on disk.
        To be safe, both the log directory, and parent directory of the log directory should be fsynced as well.

          People

          • Assignee:
            Unassigned
            Reporter:
            Samer Al-Kiswany
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:

              Development