[HDFS-12090] Handling writes from HDFS to Provided storages

• Type: New Feature
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Labels: pull-request-available
• Assignee: Unassigned
• Reporter: Virajith Jalaparti
• Created: 05/Jul/17 17:47
• Updated: 08/Dec/20 01:57

Description

HDFS-9806 introduces the concept of PROVIDED storage, which makes data in external storage systems accessible through HDFS. However, HDFS-9806 is limited to data being read through HDFS. This JIRA will deal with how data can be written to such PROVIDED storages from HDFS.

Issue Links

• is part of: HDFS-15714 HDFS Provided Storage Read/Write Mount Support On-the-fly
• is related to: HDFS-9806 Allow HDFS block replicas to be provided by an external storage system
• is related to: HDFS-14805 Mounting external stores in HDFS on-the-fly
• links to: GitHub Pull Request #1726

Sub-Tasks

• [PROVIDED Phase 2] Refactor FsDatasetImpl to use the BlockAlias from ClientProtocol for PROVIDED blocks.: Open
• [DISCUSS] Provided Storage BlockAlias Refactoring: Open
• [PROVIDED Phase 2] Provided Storage Mount Manager (PSMM) mount: Patch Available
• [PROVIDED Phase 2] Add a pluggable policy for selecting locations for Provided files.: Open
• [PROVIDED Phase 2] Implement pause/resume for syncservice: Open
• [PROVIDED Phase 2] Implement full resync command for syncservice: Open
• [PROVIDED Phase 2] Provided ReplicaMap should be LRU with separate lookup from normal Replicas: Open
• Implement Multipart-aware distcp: Open
• remove META-INF/services/org.apache.hadoop.fs.MultipartUploader file: Open