Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-17643

WASB : Make metadata checks case insensitive

    XMLWordPrintableJSON

Details

    Description

      WASB driver uses meta data on blobs to denote permission, whether its a place holder 0 sized blob for dir etc.
      For storage migration users uses Azcopy, it copies the blobs but will cause the metadata keys to get changed to camel case. As per discussion with MSFT Azcopy team, this is a known issue and technical limitation. This is what Azcopy team explained
      "For context, blob metadata is implemented with HTTP headers. They are case insensitive but case preserving.
      There is a known issue with the Go language. The HTTP client that it provides does this case modification to the response headers before we can read the raw values, so the destination metadata keys have a different casing than the source. We’ve reached out to the Go Team in the past but weren’t successful in convincing them to change the behaviour. We don’t have a short term solution right now"

      So propose to change the metadata key checks to do case insensitive checks. May be make case insensitive check configurable with defaults to false for compatibility.

      Attachments

        Activity

          People

            anoop.hbase Anoop Sam John
            anoop.hbase Anoop Sam John
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 3h 50m
                3h 50m