Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-4452

getAdditionalBlock() can create multiple blocks if the client times out and retries.

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Critical
    • Resolution: Fixed
    • 2.0.2-alpha
    • 2.0.3-alpha
    • namenode
    • None

    Description

      HDFS client tries to addBlock() to a file. If NameNode is busy the client can timeout and will reissue the same request again. The two requests will race with each other in FSNamesystem.getAdditionalBlock(), which can result in creating two new blocks on the NameNode while the client will know of only one of them. This eventually results in NotReplicatedYetException because the extra block is never reported by any DataNode, which stalls file creation and puts it in invalid state with an empty block in the middle.

      Attachments

        1. TestAddBlockRetry.java
          3 kB
          Konstantin Shvachko
        2. getAdditionalBlock.patch
          15 kB
          Konstantin Shvachko
        3. getAdditionalBlock.patch
          20 kB
          Konstantin Shvachko
        4. getAdditionalBlock-branch2.patch
          20 kB
          Konstantin Shvachko
        5. getAdditionalBlock.patch
          20 kB
          Konstantin Shvachko

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            shv Konstantin Shvachko
            shv Konstantin Shvachko
            Votes:
            0 Vote for this issue
            Watchers:
            11 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment