[HADOOP-19236] Integration of Volcano Engine TOS in Hadoop.

• Type: New Feature
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 3.4.0
• Component/s: fs, tools
• Labels: pull-request-available
• Assignee: Zheng Hu
• Reporter: Jinglun
• Created: 25/Jul/24 02:49
• Updated: 11/Dec/24 12:26

Description

Volcano Engine is a fast growing cloud vendor launched by ByteDance, and TOS is the object storage service of Volcano Engine. A common way is to store data into TOS and run Hadoop/Spark/Flink applications to access TOS. But there is no original support for TOS in hadoop, thus it is not easy for users to build their Big Data System based on TOS.
 
This work aims to integrate TOS with Hadoop to help users run their applications on TOS. Users only need to do some simple configuration, then their applications can read/write TOS without any code change. This work is similar to AWS S3, AzureBlob, AliyunOSS, Tencnet COS and HuaweiCloud Object Storage in Hadoop.

 

 Please see the attached document "Integration of Volcano Engine TOS in Hadoop" for more details.

Issue Links

• links to: GitHub Pull Request #7194