Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-119

logSync() may block NameNode forever.

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 0.21.0, 1.0.3
    • namenode
    • None
    • Reviewed

    Description

      1. FSEditLog.logSync() first waits until isSyncRunning is false and then performs syncing to file streams by calling EditLogOutputStream.flush().
        If an exception is thrown after isSyncRunning is set to true all threads will always wait on this condition.
        An IOException may be thrown by EditLogOutputStream.setReadyToFlush() or a RuntimeException may be thrown by EditLogOutputStream.flush() or by processIOError().
      2. The loop that calls eStream.flush() for multiple EditLogOutputStream-s is not synchronized, which means that another thread may encounter an error and modify editStreams by say calling processIOError(). Then the iterating process in logSync() will break with IndexOutOfBoundException.

      Attachments

        1. HDFS119.branch1.0.patch
          5 kB
          Brandon Li
        2. HDFS-119-branch-1.0.patch
          5 kB
          Konstantin Shvachko
        3. HDFS-119-branch-1.0.patch
          3 kB
          Konstantin Shvachko
        4. HDFS-119.patch
          8 kB
          Suresh Srinivas
        5. HDFS-119.patch
          7 kB
          Suresh Srinivas

        Issue Links

          Activity

            People

              sureshms Suresh Srinivas
              shv Konstantin Shvachko
              Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: