Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-14988

HDFS should avoid read/write data from slow disks.

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 3.1.1, 3.2.1
    • Fix Version/s: None
    • Component/s: block placement, datanode
    • Labels:
      None
    • Target Version/s:

      Description

      Slow disk causes real-time service(such as HBase ) become slowdown.

      The slow disk detection is added to the HDFS-11461, but only the slow disk is recorded in the Metric. I hope to further handle the detected slow disk. In my view, the slow disk can be added to the read data policy. If the block is on the slow disk of the DataNode, the block of other DataNodes is selected. For write data, slow disks can be added to the data write policy. We can remove the slow disk from all disks and then select a disk to write data based on dfs.datanode.fsdataset.volume.choosing.policy.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              jj336013 yimeng
            • Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

              • Created:
                Updated: