[HDFS-9809] Abstract implementation-specific details from the datanode

• Type: Task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 3.0.0-alpha2
• Component/s: datanode, fs
• Hadoop Flags: Reviewed
• Assignee: Virajith Jalaparti
• Reporter: Virajith Jalaparti
• Created: 13/Feb/16 01:58
• Updated: 29/Mar/17 17:44
• Resolved: 29/Mar/17 17:44

Description

Multiple parts of the Datanode (FsVolumeSpi, ReplicaInfo, FSVolumeImpl etc.) implicitly assume that blocks are stored in java.io.File(s) and that volumes are divided into directories. We propose to abstract these details, which would help in supporting other storages.

Issue Links

• is depended upon by: HDFS-10675 [READ] Datanode support to read from external stores.
• is related to: HDFS-9806 Allow HDFS block replicas to be provided by an external storage system

Sub-Tasks

• Modify ReplicaInfo to remove the assumption that replica metadata and data are stored in java.io.File.: Patch Available
• Modifications to remove the assumption that FsVolumes are backed by java.io.File.: Resolved
• Modifications to remove the assumption that StorageLocation is associated with java.io.File in Datanode.: Resolved
• Modify LOG statements to use sl4j templates.: Resolved