Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-11687

Ignore x-* and response headers when copying an Amazon S3 object

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.7.0
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: fs/s3
    • Labels:
      None

      Description

      The EMC ViPR/ECS object storage platform uses proprietary headers starting by x-emc-* (like Amazon does with x-amz-*).
      Headers starting by x-emc-* should be included in the signature computation, but it's not done by the Amazon S3 Java SDK (it's done by the EMC S3 SDK).

      When s3a copy an object it copies all the headers, but when the object includes x-emc-* headers, it generates a signature mismatch.

      Removing the x-emc-* headers from the copy would allow s3a to be compatible with the EMC ViPR/ECS object storage platform.

      Removing the x-* which aren't x-amz-* headers from the copy would allow s3a to be compatible with any object storage platform which is using proprietary headers

        Attachments

        1. HADOOP-11687.001.patch
          2 kB
          Aaron Peterson
        2. HADOOP-11687.002.patch
          3 kB
          Harsh J
        3. HADOOP-11687.003.patch
          4 kB
          Harsh J
        4. HADOOP-11687.004.patch
          5 kB
          Harsh J

          Issue Links

            Activity

              People

              • Assignee:
                qwertymaniac Harsh J
                Reporter:
                djannot Denis Jannot
              • Votes:
                0 Vote for this issue
                Watchers:
                11 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: