Uploaded image for project: 'Libcloud'
  1. Libcloud
  2. LIBCLOUD-800

GoogleStorageDriver marking everything as application/json MIME-type.

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Storage
    • Labels:
      None

      Description

      This started when I changed the GoogleStorageConnection to inherit from GoogleBaseConnection, which always uses application/json.

        Activity

        Hide
        deanputney Dean Putney added a comment -

        I see this issue as well, and it's blocking our progression to using service accounts with Google. We also have a problem with ACLs not being updated properly, which may be related.

        Show
        deanputney Dean Putney added a comment - I see this issue as well, and it's blocking our progression to using service accounts with Google. We also have a problem with ACLs not being updated properly, which may be related.
        Hide
        deanputney Dean Putney added a comment -

        In `add_default_headers` in google_storage.py, the `GoogleStorageConnection` class checks for the S3 auth, otherwise applies the super class's default headers, which overwrites the `Content-Type`. Would it make sense to remove this else condition, or extend the logic to include service accounts or other auth types?

        Show
        deanputney Dean Putney added a comment - In `add_default_headers` in google_storage.py, the `GoogleStorageConnection` class checks for the S3 auth, otherwise applies the super class's default headers, which overwrites the `Content-Type`. Would it make sense to remove this else condition, or extend the logic to include service accounts or other auth types?
        Hide
        crunk1 Scott Crunkleton added a comment -

        I'm looking into it now. But yes, that is where it is occurring.

        Show
        crunk1 Scott Crunkleton added a comment - I'm looking into it now. But yes, that is where it is occurring.
        Hide
        deanputney Dean Putney added a comment -

        Awesome! If I can be of help somehow, let me know. Any idea if this is related to ACLs not going through?

        Show
        deanputney Dean Putney added a comment - Awesome! If I can be of help somehow, let me know. Any idea if this is related to ACLs not going through?
        Hide
        crunk1 Scott Crunkleton added a comment -

        Can you provide an example of ACLs not being updated? I'll try to knock both issues out at once.

        Show
        crunk1 Scott Crunkleton added a comment - Can you provide an example of ACLs not being updated? I'll try to knock both issues out at once.
        Hide
        deanputney Dean Putney added a comment -

        I sure can. Here's a quick example you should be able to use to reproduce the problem: https://gist.github.com/deanputney/5c2df8b85c299ded3a3d

        Show
        deanputney Dean Putney added a comment - I sure can. Here's a quick example you should be able to use to reproduce the problem: https://gist.github.com/deanputney/5c2df8b85c299ded3a3d
        Hide
        deanputney Dean Putney added a comment -

        I'll note here that if you use the HMAC keys to authenticate with this example, the test will succeed and both the Content-Type and ACL will be correct.

        Show
        deanputney Dean Putney added a comment - I'll note here that if you use the HMAC keys to authenticate with this example, the test will succeed and both the Content-Type and ACL will be correct.
        Hide
        crunk1 Scott Crunkleton added a comment -

        Right. The change I made is trying to straddle the line between the HMAC authentication and Google OAuth2 authentication models. The problem is I picked up too much logic from GoogleBaseConnection when I changed GoogleStorageConnection to inherit from it instead of BaseS3Connection.

        Show
        crunk1 Scott Crunkleton added a comment - Right. The change I made is trying to straddle the line between the HMAC authentication and Google OAuth2 authentication models. The problem is I picked up too much logic from GoogleBaseConnection when I changed GoogleStorageConnection to inherit from it instead of BaseS3Connection.
        Hide
        crunk1 Scott Crunkleton added a comment -

        I'm currently working on separating the Google OAuth2 stuff from GoogleBaseConnection. Then GoogleStorageConnection can inherit from BaseS3Connection, but still optionally use OAuth2.

        Show
        crunk1 Scott Crunkleton added a comment - I'm currently working on separating the Google OAuth2 stuff from GoogleBaseConnection. Then GoogleStorageConnection can inherit from BaseS3Connection, but still optionally use OAuth2.
        Hide
        githubbot ASF GitHub Bot added a comment -

        GitHub user crunk1 opened a pull request:

        https://github.com/apache/libcloud/pull/689

        Addressing https://issues.apache.org/jira/browse/LIBCLOUD-800. Changed

        GoogleStorageConnection to not inherit from GoogleBaseConnection. Separated OAuth2 credential management from GoogleBaseConnection. This allows both GoogleBaseConnection and GoogleStorageConnection to use OAuth2. Modified and ran tests to validate changes.

        Also ran a few manual tests using GoogleStorageDriver and GCENodeDriver. GoogleStorageDriver works with either OAuth2 credentials or S3 Interoperability credentials. Tested setting the mime type on an uploaded GCS object.

        You can merge this pull request into a Git repository by running:

        $ git pull https://github.com/crunk1/libcloud trunk

        Alternatively you can review and apply these changes as the patch at:

        https://github.com/apache/libcloud/pull/689.patch

        To close this pull request, make a commit to your master/trunk branch
        with (at least) the following in the commit message:

        This closes #689


        commit 536d027a08b0a72a63d8ef8af58f84325c352054
        Author: Scott Crunkleton <crunkleton@google.com>
        Date: 2016-02-02T00:03:44Z

        Addressing https://issues.apache.org/jira/browse/LIBCLOUD-800. Changed
        GoogleStorageConnection to not inherit from GoogleBaseConnection.
        Separated OAuth2 credential management from GoogleBaseConnection. This
        allows both GoogleBaseConnection and GoogleStorageConnection to use
        OAuth2. Modified and ran tests to validate changes.

        Also ran a few manual tests.


        Show
        githubbot ASF GitHub Bot added a comment - GitHub user crunk1 opened a pull request: https://github.com/apache/libcloud/pull/689 Addressing https://issues.apache.org/jira/browse/LIBCLOUD-800 . Changed GoogleStorageConnection to not inherit from GoogleBaseConnection. Separated OAuth2 credential management from GoogleBaseConnection. This allows both GoogleBaseConnection and GoogleStorageConnection to use OAuth2. Modified and ran tests to validate changes. Also ran a few manual tests using GoogleStorageDriver and GCENodeDriver. GoogleStorageDriver works with either OAuth2 credentials or S3 Interoperability credentials. Tested setting the mime type on an uploaded GCS object. You can merge this pull request into a Git repository by running: $ git pull https://github.com/crunk1/libcloud trunk Alternatively you can review and apply these changes as the patch at: https://github.com/apache/libcloud/pull/689.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #689 commit 536d027a08b0a72a63d8ef8af58f84325c352054 Author: Scott Crunkleton <crunkleton@google.com> Date: 2016-02-02T00:03:44Z Addressing https://issues.apache.org/jira/browse/LIBCLOUD-800 . Changed GoogleStorageConnection to not inherit from GoogleBaseConnection. Separated OAuth2 credential management from GoogleBaseConnection. This allows both GoogleBaseConnection and GoogleStorageConnection to use OAuth2. Modified and ran tests to validate changes. Also ran a few manual tests.
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user crunk1 closed the pull request at:

        https://github.com/apache/libcloud/pull/689

        Show
        githubbot ASF GitHub Bot added a comment - Github user crunk1 closed the pull request at: https://github.com/apache/libcloud/pull/689
        Show
        crunk1 Scott Crunkleton added a comment - Fixed in https://github.com/apache/libcloud/pull/689

          People

          • Assignee:
            Unassigned
            Reporter:
            crunk1 Scott Crunkleton
          • Votes:
            2 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development