[HADOOP-12666] Support Microsoft Azure Data Lake - as a file system in Hadoop

• Type: New Feature
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 3.0.0-alpha1
• Component/s: fs/adl, tools
• Target Version/s: 3.0.0-alpha1
• Hadoop Flags: Reviewed
• Release Note: Hadoop now supports integration with Azure Data Lake as an alternative Hadoop-compatible file system. Please refer to the Hadoop site documentation of Azure Data Lake for details on usage and configuration.
• Assignee: Vishwajeet Dusane
• Reporter: Vishwajeet Dusane
• Created: 22/Dec/15 10:46
• Updated: 05/Feb/18 23:06
• Resolved: 09/Jun/16 21:54

Description

Description

This JIRA describes a new file system implementation for accessing Microsoft Azure Data Lake Store (ADL) from within Hadoop. This would enable existing Hadoop applications such has MR, HIVE, Hbase etc.., to use ADL store as input or output.

ADL is ultra-high capacity, Optimized for massive throughput with rich management and security features. More details available at https://azure.microsoft.com/en-us/services/data-lake-store/

Issue Links

• Dependent: HADOOP-13037 Refactor Azure Data Lake Store as an independent FileSystem
• Dependent: HADOOP-12875 [Azure Data Lake] Support for contract test and unit test cases
• is blocked by: HDFS-9938 Support extensions to WebHdfsFileSystem
• is depended upon by: HADOOP-13242 Authenticate to Azure Data Lake using client ID and keys