Uploaded image for project: 'Libcloud'
  1. Libcloud
  2. LIBCLOUD-257

S3 get_object hangs when object doesn't exist but only on python 2.6

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.12.1
    • Component/s: Storage
    • Labels:
      None
    • Environment:

      Python2.6

      Description

      A colleague wrote a proof-of-concept script against S3 with libcloud and python 2.7. When I tried to run it on a 2.6 machine it failed. It hung whilst calling get_object. I traced this back to a change between python 2.6 and 2.7 and how it handles 'HEAD' requests.

        Activity

        Hide
        jc2k John Carr added a comment -

        This is how I fixed it.

        Show
        jc2k John Carr added a comment - This is how I fixed it.
        Hide
        kami Tomaz Muraus added a comment -

        Interesting, thanks for the patch.

        I'll try to dig into and replicate the problem with and without the patch.

        Show
        kami Tomaz Muraus added a comment - Interesting, thanks for the patch. I'll try to dig into and replicate the problem with and without the patch.
        Hide
        kami Tomaz Muraus added a comment -

        I've checked this patch and noticed some issues:

        1. Indentation in the HTTPResponse class is messed up

        2. Calling self.close if method is HEAD every time doesn't seem like a right this to do. If this issue is specific to Python 2.6, we should only do it in this specific case.

        Show
        kami Tomaz Muraus added a comment - I've checked this patch and noticed some issues: 1. Indentation in the HTTPResponse class is messed up 2. Calling self.close if method is HEAD every time doesn't seem like a right this to do. If this issue is specific to Python 2.6, we should only do it in this specific case.
        Hide
        jc2k John Carr added a comment -

        Sorry about the formatting, it was a quick refactor of the monkey patch I ended up using to get it working for me to show the nature of the problem. While I tested the original monkey patch quite a bit, in my haste I didn't test the monkey-patch-less version. I can attach a new patch if you like, but you have probably already fixed it locally if you were able to recreate the problem and prove this fixes it?

        While I agree that making this patch only active on 2.6 is probably a good idea, I just wanted to stress that this isn't some weird hack to make python 2.6 work. Calling self.close() if the method is HEAD is the right thing to do. Or at least, thats what python 2.7 and later do - this patch it is essentially a backport of the behaviour found on all later pythons.

        What is the preferred way to hide something like this from python 2.7 and later in libcloud?

        Show
        jc2k John Carr added a comment - Sorry about the formatting, it was a quick refactor of the monkey patch I ended up using to get it working for me to show the nature of the problem. While I tested the original monkey patch quite a bit, in my haste I didn't test the monkey-patch-less version. I can attach a new patch if you like, but you have probably already fixed it locally if you were able to recreate the problem and prove this fixes it? While I agree that making this patch only active on 2.6 is probably a good idea, I just wanted to stress that this isn't some weird hack to make python 2.6 work. Calling self.close() if the method is HEAD is the right thing to do. Or at least, thats what python 2.7 and later do - this patch it is essentially a backport of the behaviour found on all later pythons. What is the preferred way to hide something like this from python 2.7 and later in libcloud?
        Hide
        kami Tomaz Muraus added a comment -

        Ah, thanks for the explanation. I'll have a look at Python source code and see how HEAD requests are handled in 2.6 and 2.7.

        As far as only running the code in some versions, we use sys.version_info.

        Show
        kami Tomaz Muraus added a comment - Ah, thanks for the explanation. I'll have a look at Python source code and see how HEAD requests are handled in 2.6 and 2.7. As far as only running the code in some versions, we use sys.version_info.
        Hide
        kami Tomaz Muraus added a comment -

        I've made the changes I mentioned above and merged the patch into trunk - http://svn.apache.org/viewvc?rev=1408567&view=rev.

        Thanks again for tracking down this issue and submitting patch.

        Show
        kami Tomaz Muraus added a comment - I've made the changes I mentioned above and merged the patch into trunk - http://svn.apache.org/viewvc?rev=1408567&view=rev . Thanks again for tracking down this issue and submitting patch.

          People

          • Assignee:
            kami Tomaz Muraus
            Reporter:
            jc2k John Carr
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development