Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-8966

Separate the lock used in namespace and block management layer

Add voteVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Open
    • Major
    • Resolution: Unresolved
    • None
    • None
    • None
    • None

    Description

      Currently the namespace and the block management layer share one giant lock. One consequence that we have seen more and more often is that the namespace hangs due to excessive activities from the block management layer. For example, the NN might take a couple hundred milliseconds to handle a large block report. Because the NN holds the write lock during processing the block report, all namespace requests are paused. In production we have seen these lock contentions cause long latencies and instabilities in the cluster.

      This umbrella jira proposes to separate the lock used by namespace and the block management layer.

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            wheat9 Haohui Mai
            wheat9 Haohui Mai

            Dates

              Created:
              Updated:

              Slack

                Issue deployment