Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-15000

Improve FsDatasetImpl to avoid IO operation in datasetLock

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Patch Available
    • Major
    • Resolution: Unresolved
    • None
    • None
    • datanode
    • None

    Description

      As HDFS-14997 mentioned, some methods in #FsDatasetImpl such as #finalizeBlock, #finalizeReplica, #createRbw includes IO operation in the datasetLock, It will block some logic when IO load is very high. We should reduce grain fineness or move IO operation out of datasetLock.

      Attachments

        1. HDFS-15000.001.patch
          8 kB
          Mingxiang Li

        Issue Links

          is related to

          Improvement - An improvement or enhancement to an existing feature or task. HDFS-15160 ReplicaMap, Disk Balancer, Directory Scanner and various FsDatasetImpl methods should use datanode readlock

          • Major - Major loss of function.
          • Resolved
          relates to

          Bug - A problem which impairs or prevents the functions of the product. HDFS-8496 Calling stopWriter() with FSDatasetImpl lock held may block other threads

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. HDFS-11187 Optimize disk access for last partial chunk checksum of Finalized replica

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. HDFS-9668 Optimize the locking in FsDatasetImpl

          • Major - Major loss of function.
          • Patch Available

          Activity

            People

              Aiphag0 Mingxiang Li
              hexiaoqiao Xiaoqiao He
              Votes:
              0 Vote for this issue
              Watchers:
              20 Start watching this issue

              Dates

                Created:
                Updated: