Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-9666

Enable hdfs-client to read even remote SSD/RAM prior to local disk replica to improve random read

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Patch Available
    • Major
    • Resolution: Unresolved
    • 2.6.0, 2.7.0
    • None
    • hdfs-client
    • None
    • hdfs-client, heterogeneous storage

    Description

      We want to improve random read performance of HDFS for HBase, so enabled the heterogeneous storage in our cluster. But there are only ~50% of datanode & regionserver hosts with SSD. we can set hfile with only ONE_SSD not ALL_SSD storagepolicy and the regionserver on none-SSD host can only read the local disk replica . So we developed this feature in hdfs client to read even remote SSD/RAM prior to local disk replica.

      Attachments

        1. HDFS-9666.0.patch
          8 kB
          chunde
        2. HDFS-9666.001.patch
          16 kB
          Jiandan Yang
        3. HDFS-9666.002.patch
          17 kB
          Jiandan Yang
        4. HDFS-9666.003.patch
          18 kB
          Jiandan Yang
        5. HDFS-9666.004.patch
          19 kB
          Jiandan Yang

        Issue Links

          is related to

          Sub-task - The sub-task of the issue HBASE-14061 Support CF-level Storage Policy

          • Major - Major loss of function.
          • Closed

          Activity

            People

              yangjiandan Jiandan Yang
              aderen chunde
              Votes:
              1 Vote for this issue
              Watchers:
              15 Start watching this issue

              Dates

                Created:
                Updated: