Currently all the volumes on datanode is reported as a single storage. This change proposes reporting them as individual storage. This requires:
- A unique storage ID for each storage
- This needs to be generated during formatting
- There should be an option to allow existing disks to be reported as single storage unit for backward compatibility.
- A functionality is also needed to split the existing all volumes as single storage unit to to individual storage units.
Configuration must allow for each storage unit a storage type attribute. (Now HDFS-5000)
- Block reports must be sent on a per storage basis. In some cases (such memory tier) block reports may need to be sent more frequently. That means block reporting period must be on a per storage type basis.
My proposal is for new clusters to configure volumes by default as separate storage unit. Lets discuss.