Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-6109

let sync_file_range() system call run in background

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 2.3.0, 3.0.0-alpha1
    • 2.5.0
    • datanode
    • None
    • Reviewed

    Description

      Through we passed SYNC_FILE_RANGE_WRITE to sync_file_range, to make it as asynchronous as possible, it still could be blocked, e.g. the os io request queue is full.
      Since we use sync_file_range just as a page cache advisor role it doesn't decide or guarantee the real durability, it would be nice if we could run it in backgroud. At least my test log showed, a few sync_file_range calls still cost tens of ms or more, due to the happened location is in the critical write path(BlockReceiver class), from a upper view, like HBase application, will "hung" tens of ms as well during Hlog syncing.
      Generally speaking, the patch could not improve too much, but, better than before, right ?

      Attachments

        1. HDFS-6109.txt
          9 kB
          Liang Xie
        2. HDFS-6109-v2.txt
          9 kB
          Liang Xie
        3. HDFS-6109-v3.txt
          12 kB
          Liang Xie
        4. HDFS-6109-v4.txt
          9 kB
          Liang Xie

        Activity

          People

            xieliang007 Liang Xie
            xieliang007 Liang Xie
            Votes:
            0 Vote for this issue
            Watchers:
            10 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: