[HDFS-8833] Erasure coding: store EC schema and cell size in INodeFile and eliminate notion of EC zones

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: HDFS-7285
• Fix Version/s: HDFS-7285
• Component/s: namenode
• Target Version/s: HDFS-7285
• Hadoop Flags: Reviewed
• Assignee: Zhe Zhang
• Reporter: Zhe Zhang
• Created: 28/Jul/15 23:15
• Updated: 12/Dec/17 23:49
• Resolved: 10/Sep/15 06:13

Description

We have discussed storing EC schema with files instead of EC zones and recently revisited the discussion under HDFS-8059.

As a recap, the zone concept has severe limitations including renaming and nested configuration. Those limitations are valid in encryption for security reasons and it doesn't make sense to carry them over in EC.

This JIRA aims to store EC schema and cell size on INodeFile level. For simplicity, we should first implement it as an xattr and consider memory optimizations (such as moving it to file header) as a follow-on. We should also disable changing EC policy on a non-empty file / dir in the first phase.

Issue Links

• is duplicated by: HDFS-8059 Erasure coding: revisit how to store EC schema and cellSize in NameNode
• relates to: HDFS-12919 RBF: Support erasure coding methods in RouterRpcServer
• relates to: HDFS-8373 Ec files can't be deleted into Trash because of that Trash isn't EC zone.
• requires: HDFS-8854 Erasure coding: add ECPolicy to replace schema+cellSize in hadoop-hdfs