Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-11118 Block Storage for HDFS
  3. HDFS-12099

Block Storage: Add flushid in place of timestamp in DirtyLog file signatures

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Open
    • Major
    • Resolution: Unresolved
    • HDFS-7240
    • HDFS-7240
    • ozone
    • None

    Description

      If the config "dfs.cblock.block.buffer.flush.interval.seconds" is set to an extremely low value i.e 1 sec. Then multiple dirty logs files are generated during block buffer flush with the same timestamp signature.

      This can be avoided by keeping a notion of flush id in BlockBuffer manager, which can be incremented in BlockBufferManager#triggerBlockBufferFlush.

      When block is restarted, then the current flush id can easily be reconstructed by comparing the timestamp of all the dirty log files choosing the max flushid + 1 as the next flush id.

      Attachments

        Activity

          People

            msingh Mukul Kumar Singh
            msingh Mukul Kumar Singh
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated: