Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-15616

Incorporate Tencent Cloud COS File System Implementation

    XMLWordPrintableJSON

    Details

    • Type: New Feature
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.3.0
    • Component/s: fs/cos
    • Labels:
      None
    • Release Note:
      Hide
      Tencent cloud is top 2 cloud vendors in China market and the object store COS is widely used among China’s cloud users. This task implements a COSN filesytem to support Tencent cloud COS natively in Hadoop. With simple configuration, Hadoop applications, like Spark and Hive, can read/write data from COS without any code change.
      Show
      Tencent cloud is top 2 cloud vendors in China market and the object store COS is widely used among China’s cloud users. This task implements a COSN filesytem to support Tencent cloud COS natively in Hadoop. With simple configuration, Hadoop applications, like Spark and Hive, can read/write data from COS without any code change.

      Description

      Tencent cloud is top 2 cloud vendors in China market and the object store COS (https://intl.cloud.tencent.com/product/cos) is widely used among China’s cloud users but now it is hard for hadoop user to access data laid on COS storage as no native support for COS in Hadoop.

      This work aims to integrate Tencent cloud COS with Hadoop/Spark/Hive, just like what we do before for S3, ADL, OSS, etc. With simple configuration, Hadoop applications can read/write data from COS without any code change.

        Attachments

        1. HADOOP-15616.011.patch
          213 kB
          Yang Yu
        2. HADOOP-15616.010.patch
          213 kB
          Yang Yu
        3. HADOOP-15616.009.patch
          204 kB
          Yang Yu
        4. Tencent-COS-Integrated-v2.pdf
          2.66 MB
          Yang Yu
        5. HADOOP-15616.008.patch
          204 kB
          Yang Yu
        6. HADOOP-15616.007.patch
          182 kB
          Yang Yu
        7. HADOOP-15616.006.patch
          181 kB
          Yang Yu
        8. HADOOP-15616.005.patch
          154 kB
          Yang Yu
        9. HADOOP-15616.004.patch
          152 kB
          Yang Yu
        10. HADOOP-15616.003.patch
          151 kB
          Yang Yu
        11. HADOOP-15616.002.patch
          148 kB
          Yang Yu
        12. HADOOP-15616.001.patch
          144 kB
          Yang Yu
        13. Tencent-COS-Integrated.pdf
          533 kB
          Yang Yu

          Issue Links

            Activity

              People

              • Assignee:
                yuyang733 Yang Yu
                Reporter:
                junping_du Junping Du
              • Votes:
                0 Vote for this issue
                Watchers:
                25 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: