Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-15450

Avoid fsync storm triggered by DiskChecker and handle disk full situation

Log workAgile BoardRank to TopRank to BottomAttach filesAttach ScreenshotBulk Copy AttachmentsBulk Move AttachmentsVotersWatch issueWatchersCreate sub-taskConvert to sub-taskMoveLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Blocker
    • Resolution: Fixed
    • None
    • 3.1.0, 2.10.0, 3.2.0, 3.1.1, 2.9.2, 3.0.3, 2.8.5
    • None
    • None

    Description

      Fix disk checker issues reported by Kihwal Lee in HADOOP-13738

      There are non-hdfs users of DiskChecker, who use it proactively, not just on failures. This was fine before, but now it incurs heavy I/O due to introduction of fsync() in the code.

      Attachments

        Issue Links

        blocks

        Bug - A problem which impairs or prevents the functions of the product. HDFS-13538 HDFS DiskChecker should handle disk full situation

        • Critical - Crashes, loss of data, severe memory leak.
        • Open
        is broken by

        Improvement - An improvement or enhancement to an existing feature or task. HADOOP-13738 DiskChecker should perform some disk IO

        • Major - Major loss of function.
        • Resolved

        Activity
          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            arp Arpit Agarwal Assign to me
            kihwal Kihwal Lee
            Votes:
            0 Vote for this issue
            Watchers:
            17 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment