[HADOOP-15616] Incorporate Tencent Cloud COS File System Implementation

• Type: New Feature
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 3.3.0
• Component/s: fs/cos

Release Note

Tencent cloud is top 2 cloud vendors in China market and the object store COS is widely used among China’s cloud users. This task implements a COSN filesytem to support Tencent cloud COS natively in Hadoop. With simple configuration, Hadoop applications, like Spark and Hive, can read/write data from COS without any code change.

• Assignee: Yang Yu
• Reporter: Junping Du
• Created: 18/Jul/18 08:16
• Updated: 20/Apr/20 10:16
• Resolved: 27/Sep/19 09:12

Description

Tencent cloud is top 2 cloud vendors in China market and the object store COS （https://intl.cloud.tencent.com/product/cos） is widely used among China’s cloud users but now it is hard for hadoop user to access data laid on COS storage as no native support for COS in Hadoop.

This work aims to integrate Tencent cloud COS with Hadoop/Spark/Hive, just like what we do before for S3, ADL, OSS, etc. With simple configuration, Hadoop applications can read/write data from COS without any code change.

Issue Links

• relates to: HADOOP-16955 Umbrella Jira for improving the Hadoop-cos support in Hadoop