Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-119

logSync() may block NameNode forever.

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.21.0, 1.0.3
    • Component/s: namenode
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      1. FSEditLog.logSync() first waits until isSyncRunning is false and then performs syncing to file streams by calling EditLogOutputStream.flush().
        If an exception is thrown after isSyncRunning is set to true all threads will always wait on this condition.
        An IOException may be thrown by EditLogOutputStream.setReadyToFlush() or a RuntimeException may be thrown by EditLogOutputStream.flush() or by processIOError().
      2. The loop that calls eStream.flush() for multiple EditLogOutputStream-s is not synchronized, which means that another thread may encounter an error and modify editStreams by say calling processIOError(). Then the iterating process in logSync() will break with IndexOutOfBoundException.

        Attachments

        1. HDFS119.branch1.0.patch
          5 kB
          Brandon Li
        2. HDFS-119.patch
          8 kB
          Suresh Srinivas
        3. HDFS-119.patch
          7 kB
          Suresh Srinivas
        4. HDFS-119-branch-1.0.patch
          5 kB
          Konstantin Shvachko
        5. HDFS-119-branch-1.0.patch
          3 kB
          Konstantin Shvachko

          Issue Links

            Activity

              People

              • Assignee:
                sureshms Suresh Srinivas
                Reporter:
                shv Konstantin Shvachko
              • Votes:
                0 Vote for this issue
                Watchers:
                10 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: