Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-12756

Incorporate Aliyun OSS file system implementation

    Details

    • Type: New Feature
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: HADOOP-12756
    • Fix Version/s: HADOOP-12756, 3.0.0-alpha2, 2.9.1
    • Component/s: fs, fs/oss
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Hide
      Aliyun OSS is widely used among China’s cloud users and this work implemented a new Hadoop compatible filesystem AliyunOSSFileSystem with oss scheme, similar to the s3a and azure support.
      Show
      Aliyun OSS is widely used among China’s cloud users and this work implemented a new Hadoop compatible filesystem AliyunOSSFileSystem with oss scheme, similar to the s3a and azure support.

      Description

      Aliyun OSS is widely used among China’s cloud users, but currently it is not easy to access data laid on OSS storage from user’s Hadoop/Spark application, because of no original support for OSS in Hadoop.

      This work aims to integrate Aliyun OSS with Hadoop. By simple configuration, Spark/Hadoop applications can read/write data from OSS without any code change. Narrowing the gap between user’s APP and data storage, like what have been done for S3 in Hadoop

        Attachments

        1. Aliyun-OSS-integration.pdf
          144 kB
          shimingfei
        2. Aliyun-OSS-integration-v2.pdf
          79 kB
          Genmao Yu
        3. HADOOP-12756.003.patch
          95 kB
          Kai Zheng
        4. HADOOP-12756.004.patch
          95 kB
          Kai Zheng
        5. HADOOP-12756.005.patch
          96 kB
          shimingfei
        6. HADOOP-12756.006.patch
          102 kB
          shimingfei
        7. HADOOP-12756.007.patch
          103 kB
          shimingfei
        8. HADOOP-12756.008.patch
          103 kB
          Kai Zheng
        9. HADOOP-12756.009.patch
          103 kB
          Kai Zheng
        10. HADOOP-12756.010.patch
          147 kB
          shimingfei
        11. HADOOP-12756-v02.patch
          92 kB
          shimingfei
        12. HCFS User manual.md
          10 kB
          Ling Zhou
        13. OSS integration.pdf
          225 kB
          Ling Zhou

          Issue Links

            Activity

              People

              • Assignee:
                kakagou mingfei.shi
                Reporter:
                shimingfei shimingfei
              • Votes:
                2 Vote for this issue
                Watchers:
                53 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: