Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-12157

Do fsyncDirectory(..) outside of FSDataset lock

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Critical
    • Resolution: Fixed
    • None
    • 2.9.0, 3.0.0-beta1, 2.8.2, 2.7.5
    • datanode
    • None

    Description

      HDFS-5042 introduced fsyncDirectory(..) to save blocks from power failure.
      Do it outside of FSDataset lock to avoid overall performance degradation if disk takes more time to sync.

      Attachments

        1. HDFS-12157-01.patch
          5 kB
          Vinayakumar B
        2. HDFS-12157-branch-2-01.patch
          5 kB
          Vinayakumar B
        3. HDFS-12157-branch-2.7-01.patch
          6 kB
          Vinayakumar B
        4. HDFS-12157-branch-2.7-01.patch
          6 kB
          Vinayakumar B

        Issue Links

        is broken by

        Bug - A problem which impairs or prevents the functions of the product. HDFS-5042 Completed files lost after power failure

        • Critical - Crashes, loss of data, severe memory leak.
        • Resolved

        Activity
          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            vinayakumarb Vinayakumar B
            vinayakumarb Vinayakumar B
            Votes:
            0 Vote for this issue
            Watchers:
            9 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment