Details

      Description

      Currently all the volumes on datanode is reported as a single storage. This change proposes reporting them as individual storage. This requires:

      1. A unique storage ID for each storage
        • This needs to be generated during formatting
      2. There should be an option to allow existing disks to be reported as single storage unit for backward compatibility.
      3. A functionality is also needed to split the existing all volumes as single storage unit to to individual storage units.
      4. Configuration must allow for each storage unit a storage type attribute. (Now HDFS-5000)
      5. Block reports must be sent on a per storage basis. In some cases (such memory tier) block reports may need to be sent more frequently. That means block reporting period must be on a per storage type basis.

      My proposal is for new clusters to configure volumes by default as separate storage unit. Lets discuss.

        Attachments

        1. HDFS-4988.09.patch
          53 kB
          Arpit Agarwal
        2. HDFS-4988.08.patch
          54 kB
          Arpit Agarwal
        3. HDFS-4988.07.patch
          56 kB
          Arpit Agarwal
        4. HDFS-4988.06.patch
          61 kB
          Arpit Agarwal
        5. HDFS-4988.05.patch
          101 kB
          Arpit Agarwal
        6. HDFS-4988.02.patch
          147 kB
          Arpit Agarwal
        7. HDFS-4988.01.patch
          141 kB
          Arpit Agarwal

          Issue Links

            Activity

              People

              • Assignee:
                arp Arpit Agarwal
                Reporter:
                sureshms Suresh Srinivas
              • Votes:
                0 Vote for this issue
                Watchers:
                8 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: