Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-13377 AliyunOSS: improvements for stabilization and optimization
  3. HADOOP-15323

AliyunOSS: Improve copy file performance for AliyunOSSFileSystemStore

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 2.10.0, 3.2.0, 3.1.1, 2.9.2, 3.0.3, 3.3.0
    • 2.10.0, 3.0.4, 3.3.0, 3.1.2, 3.2.1, 2.9.3
    • fs/oss
    • None
    • Reviewed

    Description

      Aliyun OSS will support shallow copy which means server will only copy metadata when copy object operation occurs. 

      With shallow copy, we can use copyObject api instead of multi-part copy api if we do not change object storage type & encryption type & source object  is uploaded by Put / Multipart upload api.

      We will try to use copyObject api, and check result. If shallow copy disabled for this object, then we will use multipart copy. So, I will remove fs.oss.multipart.upload.threshold configuration.

      Attachments

        1. HADOOP-15323.003.patch
          14 kB
          wujinhu
        2. HADOOP-15323.002.patch
          15 kB
          wujinhu
        3. HADOOP-15323.001.patch
          12 kB
          wujinhu

        Activity

          People

            wujinhu wujinhu
            wujinhu wujinhu
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: