VotersWatch issueWatchersLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.0.2-alpha
    • Fix Version/s: 2.8.0, 2.9.0, 3.0.0-alpha1
    • Component/s: namenode
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      Edit logging is a major source of contention within the NN. LogEdit is called within the namespace write log, while logSync is called outside of the lock to allow greater concurrency. The handler thread remains busy until logSync returns to provide the client with a durability guarantee for the response.

      Write heavy RPC load and/or slow IO causes handlers to stall in logSync. Although the write lock is not held, readers are limited/starved and the call queue fills. Combining an edit log thread with postponed RPC responses from HADOOP-10300 will provide the same durability guarantee but immediately free up the handlers.

        Attachments

        1. HDFS-7964-rebase.patch
          86 kB
          Jing Zhao
        2. HDFS-7964-branch-2.8.0.patch
          87 kB
          Zhe Zhang
        3. HDFS-7964-branch-2.7.patch
          87 kB
          Zhe Zhang
        4. HDFS-7964.patch
          79 kB
          Daryn Sharp
        5. HDFS-7964.patch
          84 kB
          Daryn Sharp
        6. HDFS-7964.patch
          86 kB
          Daryn Sharp
        7. HDFS-7964.patch
          86 kB
          Jing Zhao

        Issue Links

          Activity

            People

            • Assignee:
              daryn Daryn Sharp
              Reporter:
              daryn Daryn Sharp

              Dates

              • Created:
                Updated:
                Resolved:

                Issue deployment