Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-15382

Split one FsDatasetImpl lock to volume grain locks.

    XMLWordPrintableJSON

Details

    • Reviewed
    • Hide
      Throughput is one of the core performance evaluation for DataNode instance. However it does not reach the best performance especially for Federation deploy all the time although there are different improvement, because of the global coarse-grain lock. These series issues (include HDFS-16534, HDFS-16511, HDFS-15382 and HDFS-16429.) try to split the global coarse-grain lock to fine-grain lock which is double level lock for blockpool and volume, to improve the throughput and avoid lock impacts between blockpools and volumes.
      Show
      Throughput is one of the core performance evaluation for DataNode instance. However it does not reach the best performance especially for Federation deploy all the time although there are different improvement, because of the global coarse-grain lock. These series issues (include HDFS-16534 , HDFS-16511 , HDFS-15382 and HDFS-16429 .) try to split the global coarse-grain lock to fine-grain lock which is double level lock for blockpool and volume, to improve the throughput and avoid lock impacts between blockpools and volumes.

    Description

      In HDFS-15180 we split lock to blockpool grain size.But when one volume is in heavy load and will block other request which in same blockpool but different volume.So we split lock to two leval to avoid this happend.And to improve datanode performance.

      Attachments

        1. HDFS-15382-sample.patch
          119 kB
          Mingxiang Li
        2. image-2020-06-02-1.png
          236 kB
          Mingxiang Li
        3. image-2020-06-03-1.png
          53 kB
          Mingxiang Li

        Issue Links

          Activity

            People

              Aiphag0 Mingxiang Li
              Aiphag0 Mingxiang Li
              Votes:
              0 Vote for this issue
              Watchers:
              22 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 24h 10m
                  24h 10m