Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-6115

flush() should be called for every append on block scan verification log

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Minor
    • Resolution: Fixed
    • 2.3.0, 2.4.0
    • 2.4.0
    • datanode
    • None
    • Reviewed

    Description

      RollingLogsImpl#out is PrintWriter and it have the default buffersize of 8kB.

      So until the 8kB buffer is filled all scan verification entries will not be flushed, hence there will be chance loosing this scan information if the datanode gets restarted. And One more scan will happen for these blocks.

      8kB will have ~80 entries, so 80 blocks needs to be re-scanned again.

      Attachments

        1. HDFS-6115.patch
          0.6 kB
          Vinayakumar B

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            vinayakumarb Vinayakumar B
            vinayakumarb Vinayakumar B
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment