Uploaded image for project: 'Libcloud'
  1. Libcloud
  2. LIBCLOUD-886

GoogleStorageDriver.get_container does not catch 400 errors, and invalid Container instances are created as a result



    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Storage
    • Labels:
    • Environment:

      Linux mint, python2.7.10, libcloud 1.5.0


      This bug concerns calls made to libcloud.storage.drivers.GoogleStorageDriver.get_container method.

      GoogleStorageDriver calls the parent method in libcloud.storage.s3.BaseS3StorageDriver. This method is not catching the 400 response status from a bad request to google. It only checks for 404 status. The bad request is caused by searching for buckets with names that violate Google's bucket name conventions.

      As a result, an apparently valid libcloud.storage.base.Container instance is returned, unknown to the client.

      I would fix this, but I would like some direction as to where libcloud's maintainers would prefer the change happen. It seems strange to me that 400 responses are allowed to percolate up, but I don't want to create side effects that deep in the code base. The easiest fix would be to add an additional check in the libcloud.storage.s3.BaseS3StorageDriver.get_container method. However, I'm not sure if it's appropriate there since I don't know what AWS' response is from a similarly malformatted bucket name (i.e. does it always return 404?). Thus, if it makes sense, perhaps the get_container method needs to be overridden in the child GoogleStorageDriver class.

      An ipython session showing this is pasted below:

      ipython session
      # show my existing buckets
      # driver is an instance of libcloud.storage.drivers.GoogleStorageDriver
      In [5]: driver.list_containers()
      [<Container: name=cccb-scripts, provider=Google Cloud Storage>,
       <Container: name=cccb-seq, provider=Google Cloud Storage>,
       <Container: name=circ_rna, provider=Google Cloud Storage>,
       <Container: name=dfci-demo-bucket, provider=Google Cloud Storage>,
       <Container: name=dfci-mev-testbucket, provider=Google Cloud Storage>,
       <Container: name=www.lung-genomics.org, provider=Google Cloud Storage>]
      # try to get a bucket that does not exist, but has a valid name
      # this works as expected, raising the proper exception
      In [6]: driver.get_container('some-missing-bucket')
      ContainerDoesNotExistError                Traceback (most recent call last)
      <ipython-input-6-57fd97369629> in <module>()
      ----> 1 driver.get_container('some-missing-bucket')
      /home/brian/Python/gcloud_venv/lib/python2.7/site-packages/libcloud/storage/drivers/s3.pyc in get_container(self, container_name)
          298             if response.status == httplib.NOT_FOUND:
          299                 raise ContainerDoesNotExistError(value=None, driver=self,
      --> 300                                                  container_name=container_name)
          301         except InvalidCredsError:
          302             # This just means the user doesn't have IAM permissions to do a
      ContainerDoesNotExistError: <ContainerDoesNotExistError in <libcloud.storage.drivers.google_storage.GoogleStorageDriver object at 0x7fd5ceddcf90>, container=some-missing-bucket, value=None>
      # Now, call for a missing bucket that violates Google's bucket naming
      # convention (e.g. has a capital letter)
      # this returns a Container object!
      In [7]: driver.get_container('some-missing-bucketA')
      Out[7]: <Container: name=some-missing-bucketA, provider=Google Cloud Storage>
      In [8]: libcloud.__version__
      Out[8]: '1.5.0'




            • Assignee:
              brianlawney Brian Lawney
            • Votes:
              0 Vote for this issue
              1 Start watching this issue


              • Created:

                Time Tracking

                Original Estimate - 168h
                Remaining Estimate - 168h
                Time Spent - Not Specified
                Not Specified