Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-7343

HDFS smart storage management

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      As discussed in HDFS-7285, it would be better to have a comprehensive and flexible storage policy engine considering file attributes, metadata, data temperature, storage type, EC codec, available hardware capabilities, user/application preference and etc.

      Modified the title for re-purpose.
      We'd extend this effort some bit and aim to work on a comprehensive solution to provide smart storage management service in order for convenient, intelligent and effective utilizing of erasure coding or replicas, HDFS cache facility, HSM offering, and all kinds of tools (balancer, mover, disk balancer and so on) in a large cluster.

        Attachments

        1. HDFS-Smart-Storage-Management.pdf
          290 kB
          Wei Zhou
        2. HDFS-Smart-Storage-Management-update.pdf
          301 kB
          Wei Zhou
        3. move.jpg
          26 kB
          Wei Zhou
        4. HDFSSmartStorageManagement-General-20170315.pdf
          202 kB
          Wei Zhou
        5. HDFSSmartStorageManagement-Phase1-20170315.pdf
          285 kB
          Wei Zhou
        6. tables_in_ssm.xlsx
          30 kB
          Wei Zhou
        7. access_count_tables.jpg
          119 kB
          Wei Zhou

          Issue Links

            Activity

              People

              • Assignee:
                zhouwei Wei Zhou
                Reporter:
                drankye Kai Zheng
              • Votes:
                5 Vote for this issue
                Watchers:
                79 Start watching this issue

                Dates

                • Created:
                  Updated: