Uploaded image for project: 'Libcloud'
  1. Libcloud
  2. LIBCLOUD-390

driver.create_container gets a HTTP Error 411 with Azure Blobs driver

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.12.3, 0.13.0
    • Fix Version/s: 0.13.1
    • Component/s: Storage
    • Labels:
      None

      Description

      On current Windows Azure, the Content-Length header is required for container creation requests. Currently we get the following error:

       [...]
        File "/Users/ogrisel/code/libcloud/libcloud/storage/drivers/azure_blobs.py", line 499, in create_container
          method='PUT', headers=headers)
        File "/Users/ogrisel/code/libcloud/libcloud/common/base.py", line 617, in request
          connection=self)
        File "/Users/ogrisel/code/libcloud/libcloud/common/base.py", line 93, in __init__
          raise Exception(self.parse_error())
        File "/Users/ogrisel/code/libcloud/libcloud/common/azure.py", line 74, in parse_error
          driver=self)
      LibcloudError: <LibcloudError in <libcloud.common.azure.AzureResponse object at 0x10cf8dc90> 'Unknown error Status code: 411.'>
      

      I will soon push a fix in a branch on my github fork.

        Activity

        Hide
        ogrisel Olivier Grisel added a comment -
        Show
        ogrisel Olivier Grisel added a comment - Here is the fix: https://github.com/apache/libcloud/pull/143/files
        Hide
        kami Tomaz Muraus added a comment -

        This seems to be the same problem as in LIBCLOUD-362. We have introduced a regression in 0.13.0 release which caused this issue, but as noted in LIBCLOUD-362 the issue been fixed in trunk.

        P.S. Your "Affects Versions" says 0.12.3, can you confirm that is in fact the version you are running (import libcloud; libcloud._version_)?

        In any case thanks for your report.

        Show
        kami Tomaz Muraus added a comment - This seems to be the same problem as in LIBCLOUD-362 . We have introduced a regression in 0.13.0 release which caused this issue, but as noted in LIBCLOUD-362 the issue been fixed in trunk. P.S. Your "Affects Versions" says 0.12.3, can you confirm that is in fact the version you are running (import libcloud; libcloud._ version _)? In any case thanks for your report.
        Hide
        ogrisel Olivier Grisel added a comment -

        I thought I was running trunk ('0.14.0-dev') but I just noticed I had several versions installed on my box. I will check now.

        Show
        ogrisel Olivier Grisel added a comment - I thought I was running trunk ('0.14.0-dev') but I just noticed I had several versions installed on my box. I will check now.
        Hide
        kami Tomaz Muraus added a comment -

        Edit - it would be great if you can confirm that this issue has been fixed in trunk.

        If that is indeed the case, I will backport a bugfix to 0.13.x branch and get the 0.13.1 release with a fix out asap.

        Show
        kami Tomaz Muraus added a comment - Edit - it would be great if you can confirm that this issue has been fixed in trunk. If that is indeed the case, I will backport a bugfix to 0.13.x branch and get the 0.13.1 release with a fix out asap.
        Hide
        ogrisel Olivier Grisel added a comment -

        I confirm that this error can be reproduced on today's trunk and hence that LIBCLOUD-362 does not fix it. Maybe the Azure Blob storage driver does not leverage features of the base class correctly but I am not very familiar with the class hierarchy so I cannot really tell.

        Show
        ogrisel Olivier Grisel added a comment - I confirm that this error can be reproduced on today's trunk and hence that LIBCLOUD-362 does not fix it. Maybe the Azure Blob storage driver does not leverage features of the base class correctly but I am not very familiar with the class hierarchy so I cannot really tell.
        Hide
        kami Tomaz Muraus added a comment -

        OK, thanks, I'll have a more detailed look tomorrow.

        Would also appreciate if you can paste the output using debug mode (http://ci.apache.org/projects/libcloud/docs/troubleshooting.html#debugging) when creating a container using trunk version.

        Show
        kami Tomaz Muraus added a comment - OK, thanks, I'll have a more detailed look tomorrow. Would also appreciate if you can paste the output using debug mode ( http://ci.apache.org/projects/libcloud/docs/troubleshooting.html#debugging ) when creating a container using trunk version.
        Hide
        ogrisel Olivier Grisel added a comment -

        I checked and it correctly inherits from the base class and reuses the `libcloud/common/base.py`'s `request` method. But as data is None in this case, the fix for LIBCLOUD-362 does not apply here.

        Show
        ogrisel Olivier Grisel added a comment - I checked and it correctly inherits from the base class and reuses the `libcloud/common/base.py`'s `request` method. But as data is None in this case, the fix for LIBCLOUD-362 does not apply here.
        Hide
        kami Tomaz Muraus added a comment -

        You are correct - the fix in trunk needs more love.

        It should still send Content-Length 0 with PUT and POST requests even if data is None. It should just skip encoding data if data is None.

        I will work on pushing a fix to trunk tomorrow.

        Show
        kami Tomaz Muraus added a comment - You are correct - the fix in trunk needs more love. It should still send Content-Length 0 with PUT and POST requests even if data is None. It should just skip encoding data if data is None. I will work on pushing a fix to trunk tomorrow.
        Hide
        ogrisel Olivier Grisel added a comment -

        Here is the debugging info:

        # -------- begin 4412025888 request ----------
        curl -i -X PUT -H 'x-ms-version: 2012-02-12' -H 'Accept-Encoding: gzip,deflate' -H 'X-LC-Request-ID: 4412025888' -H 'Content-Length: 0' -H 'User-Agent: libcloud/0.14.0-dev (Microsoft Azure (blobs)) ' -H 'Authorization: SharedKey XXXX:YYYYY=' -H 'x-ms-date: Tue, 03 Sep 2013 22:50:19 GMT' --compress 'https://XXXX.blob.core.windows.net:443/container3?restype=container'
        # -------- begin 4412025888:4412060160 response ----------
        HTTP/1.1 201 Created
        X-Ms-Version: 2012-02-12
        Transfer-Encoding: chunked
        X-Ms-Request-Id: b7472e27-0146-4c80-be04-da61ac746c8e
        Server: Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0
        Last-Modified: Tue, 03 Sep 2013 22:50:15 GMT
        Etag: "0x8D0770F172EBBCE"
        Date: Tue, 03 Sep 2013 22:50:15 GMT
        
        0
        
        0
        
        # -------- end 4412025888:4412060160 response ----------
        
        
        Show
        ogrisel Olivier Grisel added a comment - Here is the debugging info: # -------- begin 4412025888 request ---------- curl -i -X PUT -H 'x-ms-version: 2012-02-12' -H 'Accept-Encoding: gzip,deflate' -H 'X-LC-Request-ID: 4412025888' -H 'Content-Length: 0' -H 'User-Agent: libcloud/0.14.0-dev (Microsoft Azure (blobs)) ' -H 'Authorization: SharedKey XXXX:YYYYY=' -H 'x-ms-date: Tue, 03 Sep 2013 22:50:19 GMT' --compress 'https: //XXXX.blob.core.windows.net:443/container3?restype=container' # -------- begin 4412025888:4412060160 response ---------- HTTP/1.1 201 Created X-Ms-Version: 2012-02-12 Transfer-Encoding: chunked X-Ms-Request-Id: b7472e27-0146-4c80-be04-da61ac746c8e Server: Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0 Last-Modified: Tue, 03 Sep 2013 22:50:15 GMT Etag: "0x8D0770F172EBBCE" Date: Tue, 03 Sep 2013 22:50:15 GMT 0 0 # -------- end 4412025888:4412060160 response ----------
        Hide
        ogrisel Olivier Grisel added a comment -

        Sorry this was with my fixed applied. Let me redo it from the trunk:

        # -------- begin 4467220776 request ----------
        curl -i -X PUT -H 'x-ms-version: 2012-02-12' -H 'Accept-Encoding: gzip,deflate' -H 'X-LC-Request-ID: 4467220776' -H 'User-Agent: libcloud/0.14.0-dev (Microsoft Azure (blobs)) ' -H 'Authorization: SharedKey XXXXXXX:YYYYY=' -H 'x-ms-date: Tue, 03 Sep 2013 22:53:35 GMT' --compress 'https://XXXXXXX.blob.core.windows.net:443/container4?restype=container'
        # -------- begin 4467220776:4467221712 response ----------
        HTTP/1.1 411 Length Required
        Date: Tue, 03 Sep 2013 22:53:32 GMT
        Connection: close
        Content-Type: text/html; charset=us-ascii
        Content-Length: 344
        Server: Microsoft-HTTPAPI/2.0
        
        <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN""http://www.w3.org/TR/html4/strict.dtd">
        <HTML><HEAD><TITLE>Length Required</TITLE>
        <META HTTP-EQUIV="Content-Type" Content="text/html; charset=us-ascii"></HEAD>
        <BODY><h2>Length Required</h2>
        <hr><p>HTTP Error 411. The request must be chunked or have a content length.</p>
        </BODY></HTML>
        
        # -------- end 4467220776:4467221712 response ----------
        
        Traceback (most recent call last):
          File "/tmp/create_container.py", line 9, in <module>
            driver.create_container('container4')
          File "/Users/ogrisel/code/libcloud/libcloud/storage/drivers/azure_blobs.py", line 497, in create_container
            method='PUT')
          File "/Users/ogrisel/code/libcloud/libcloud/common/base.py", line 617, in request
            connection=self)
          File "/Users/ogrisel/code/libcloud/libcloud/common/base.py", line 93, in __init__
            raise Exception(self.parse_error())
          File "/Users/ogrisel/code/libcloud/libcloud/common/azure.py", line 74, in parse_error
            driver=self)
        libcloud.common.types.LibcloudError: <LibcloudError in <libcloud.common.azure.AzureResponse object at 0x10a43ad50> 'Unknown error Status code: 411.'>
        
        Show
        ogrisel Olivier Grisel added a comment - Sorry this was with my fixed applied. Let me redo it from the trunk: # -------- begin 4467220776 request ---------- curl -i -X PUT -H 'x-ms-version: 2012-02-12' -H 'Accept-Encoding: gzip,deflate' -H 'X-LC-Request-ID: 4467220776' -H 'User-Agent: libcloud/0.14.0-dev (Microsoft Azure (blobs)) ' -H 'Authorization: SharedKey XXXXXXX:YYYYY=' -H 'x-ms-date: Tue, 03 Sep 2013 22:53:35 GMT' --compress 'https: //XXXXXXX.blob.core.windows.net:443/container4?restype=container' # -------- begin 4467220776:4467221712 response ---------- HTTP/1.1 411 Length Required Date: Tue, 03 Sep 2013 22:53:32 GMT Connection: close Content-Type: text/html; charset=us-ascii Content-Length: 344 Server: Microsoft-HTTPAPI/2.0 <!DOCTYPE HTML PUBLIC "- //W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd" > <HTML><HEAD><TITLE>Length Required</TITLE> <META HTTP-EQUIV= "Content-Type" Content= "text/html; charset=us-ascii" ></HEAD> <BODY><h2>Length Required</h2> <hr><p>HTTP Error 411. The request must be chunked or have a content length.</p> </BODY></HTML> # -------- end 4467220776:4467221712 response ---------- Traceback (most recent call last): File "/tmp/create_container.py" , line 9, in <module> driver.create_container('container4') File "/Users/ogrisel/code/libcloud/libcloud/storage/drivers/azure_blobs.py" , line 497, in create_container method='PUT') File "/Users/ogrisel/code/libcloud/libcloud/common/base.py" , line 617, in request connection=self) File "/Users/ogrisel/code/libcloud/libcloud/common/base.py" , line 93, in __init__ raise Exception(self.parse_error()) File "/Users/ogrisel/code/libcloud/libcloud/common/azure.py" , line 74, in parse_error driver=self) libcloud.common.types.LibcloudError: <LibcloudError in <libcloud.common.azure.AzureResponse object at 0x10a43ad50> 'Unknown error Status code: 411.'>
        Hide
        ogrisel Olivier Grisel added a comment -

        Alright, please feel free to close this as duplicate and close my PR once your fix is pushed to trunk. Thanks!

        Show
        ogrisel Olivier Grisel added a comment - Alright, please feel free to close this as duplicate and close my PR once your fix is pushed to trunk. Thanks!
        Hide
        ogrisel Olivier Grisel added a comment -

        BTW, I think the mock test is still useful.

        Show
        ogrisel Olivier Grisel added a comment - BTW, I think the mock test is still useful.
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit 08a79e4d17d694477194d2c957db0ea5f67e8648 in branch refs/heads/trunk from Tomaz Muraus
        [ https://git-wip-us.apache.org/repos/asf?p=libcloud.git;h=08a79e4 ]

        Only send Content-Length: 0 even if "data" is None.

        Fixes LIBCLOUD-390.

        Show
        jira-bot ASF subversion and git services added a comment - Commit 08a79e4d17d694477194d2c957db0ea5f67e8648 in branch refs/heads/trunk from Tomaz Muraus [ https://git-wip-us.apache.org/repos/asf?p=libcloud.git;h=08a79e4 ] Only send Content-Length: 0 even if "data" is None. Fixes LIBCLOUD-390 .
        Hide
        kami Tomaz Muraus added a comment -

        OK, I've pushed a commit to trunk which should fix this issue.

        I'll work on preparing a new release with this bug fix as soon as you confirm this fixes your issue.

        Show
        kami Tomaz Muraus added a comment - OK, I've pushed a commit to trunk which should fix this issue. I'll work on preparing a new release with this bug fix as soon as you confirm this fixes your issue.
        Hide
        ogrisel Olivier Grisel added a comment - - edited

        I just pulled your changes in trunk and I confirm that I can now create Azure blob containers successfully. Thanks!

        Show
        ogrisel Olivier Grisel added a comment - - edited I just pulled your changes in trunk and I confirm that I can now create Azure blob containers successfully. Thanks!
        Hide
        kami Tomaz Muraus added a comment -

        Great, thanks!

        I will work on preparing a new release and starting a voting thread for it, hopefully today.

        Show
        kami Tomaz Muraus added a comment - Great, thanks! I will work on preparing a new release and starting a voting thread for it, hopefully today.

          People

          • Assignee:
            kami Tomaz Muraus
            Reporter:
            ogrisel Olivier Grisel
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development