Uploaded image for project: 'jclouds'
  1. jclouds
  2. JCLOUDS-1358

putBlob with zero length InputSteam is failing for aws s3

    Details

      Description

      Right now there is a method payload(InputStream stream) in BlobBuilder, for uploading streamed bytes to blob stores. However, when I open InputStream from empty file I am getting following error :

      Caused by: org.jclouds.http.HttpException: unable to reset unrepeatable payload stream after calculating AWS4 signature.
      at org.jclouds.s3.filters.Aws4SignerForAuthorizationHeader.closeOrResetPayloadStream(Aws4SignerForAuthorizationHeader.java:215)
      at org.jclouds.s3.filters.Aws4SignerForAuthorizationHeader.calculatePayloadContentHash(Aws4SignerForAuthorizationHeader.java:200)
      at org.jclouds.s3.filters.Aws4SignerForAuthorizationHeader.getPayloadHash(Aws4SignerForAuthorizationHeader.java:176)
      at org.jclouds.s3.filters.Aws4SignerForAuthorizationHeader.sign(Aws4SignerForAuthorizationHeader.java:144)
      at org.jclouds.s3.filters.RequestAuthorizeSignatureV4.signForAuthorizationHeader(RequestAuthorizeSignatureV4.java:91)
      at org.jclouds.aws.s3.filters.AWSRequestAuthorizeSignatureV4.signForAuthorizationHeader(AWSRequestAuthorizeSignatureV4.java:60)
      at org.jclouds.s3.filters.RequestAuthorizeSignatureV4.filter(RequestAuthorizeSignatureV4.java:55)
      at org.jclouds.http.internal.BaseHttpCommandExecutorService.invoke(BaseHttpCommandExecutorService.java:92)
      ... 92 more
      

        Activity

        Hide
        gaul Andrew Gaul added a comment -

        BaseBlobIntegrationTest.testPutZeroLengthInputStream exercises zero-length InputStream and mvn integration-test -pl :aws-s3 -Plive -Dtest.aws-s3.identity="$JCLOUDS_IDENTITY" -Dtest.aws-s3.credential="$JCLOUDS_CREDENTIAL" -Dtest=AWSS3BlobIntegrationLiveTest#testPutZeroLengthInputStream succeeds for me against 2.0.x and master. Could you investigate the differences between your test and the jclouds test?

        Show
        gaul Andrew Gaul added a comment - BaseBlobIntegrationTest.testPutZeroLengthInputStream exercises zero-length InputStream and mvn integration-test -pl :aws-s3 -Plive -Dtest.aws-s3.identity="$JCLOUDS_IDENTITY" -Dtest.aws-s3.credential="$JCLOUDS_CREDENTIAL" -Dtest=AWSS3BlobIntegrationLiveTest#testPutZeroLengthInputStream succeeds for me against 2.0.x and master. Could you investigate the differences between your test and the jclouds test?
        Hide
        dgyurdzhekliev Deyan added a comment -

        It seems that underlying ByteArrayInputStream's stream supports mark/reset (used in the test case). In my case I am using InputStream implementation which does not support it and the above exception is being thrown. Signing the request cause stream resetting.

        Show
        dgyurdzhekliev Deyan added a comment - It seems that underlying ByteArrayInputStream's stream supports mark/reset (used in the test case). In my case I am using InputStream implementation which does not support it and the above exception is being thrown. Signing the request cause stream resetting.
        Hide
        gaul Andrew Gaul added a comment - - edited

        The jclouds test input, RandomByteSource.RandomInputStream, also does not support mark or reset. Please investigate why the jclouds test passes.

        Show
        gaul Andrew Gaul added a comment - - edited The jclouds test input, RandomByteSource.RandomInputStream , also does not support mark or reset. Please investigate why the jclouds test passes.
        Hide
        dgyurdzhekliev Deyan added a comment -

        The test we are talking about 'testPutZeroLengthInputStream()' is using ByteSource.empty(). The underlying stream for empty ByteSource is ByteArrayInputStream which supports marking/resetting.

        Show
        dgyurdzhekliev Deyan added a comment - The test we are talking about ' testPutZeroLengthInputStream() ' is using ByteSource.empty(). The underlying stream for empty ByteSource is ByteArrayInputStream which supports marking/resetting.
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit d29b79674d9ebd31f5a7ed83d908ed0d3081ab6a in jclouds's branch refs/heads/master from Andrew Gaul
        [ https://git-wip-us.apache.org/repos/asf?p=jclouds.git;h=d29b796 ]

        JCLOUDS-1358: Fix zero byte InputStream test input

        Do not use ByteSource.empty() since it is backed by a
        ByteArrayInputStream which supports reset.

        Show
        jira-bot ASF subversion and git services added a comment - Commit d29b79674d9ebd31f5a7ed83d908ed0d3081ab6a in jclouds's branch refs/heads/master from Andrew Gaul [ https://git-wip-us.apache.org/repos/asf?p=jclouds.git;h=d29b796 ] JCLOUDS-1358 : Fix zero byte InputStream test input Do not use ByteSource.empty() since it is backed by a ByteArrayInputStream which supports reset.
        Hide
        dgyurdzhekliev Deyan added a comment -

        The test is now accurate. But why is resetting needed when empty stream is being used? When the stream is not empty there is no resetting either.

        Show
        dgyurdzhekliev Deyan added a comment - The test is now accurate. But why is resetting needed when empty stream is being used? When the stream is not empty there is no resetting either.
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit 9b59d099d18bb2f1894f81b0f52a9c853aa5e599 in jclouds's branch refs/heads/master from Andrew Gaul
        [ https://git-wip-us.apache.org/repos/asf?p=jclouds.git;h=9b59d09 ]

        JCLOUDS-1358: Handle 0-byte input with v4 signature

        Previously this failed due to use of mark and reset on the
        InputStream.

        Show
        jira-bot ASF subversion and git services added a comment - Commit 9b59d099d18bb2f1894f81b0f52a9c853aa5e599 in jclouds's branch refs/heads/master from Andrew Gaul [ https://git-wip-us.apache.org/repos/asf?p=jclouds.git;h=9b59d09 ] JCLOUDS-1358 : Handle 0-byte input with v4 signature Previously this failed due to use of mark and reset on the InputStream.
        Hide
        dgyurdzhekliev Deyan added a comment -

        Any thoughts on "why is resetting needed when empty stream is being used? When the stream is not empty there is no resetting either."

        Show
        dgyurdzhekliev Deyan added a comment - Any thoughts on "why is resetting needed when empty stream is being used? When the stream is not empty there is no resetting either."
        Hide
        gaul Andrew Gaul added a comment -

        Sorry I don't. Please look at the code and report back.

        Show
        gaul Andrew Gaul added a comment - Sorry I don't. Please look at the code and report back.
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit 9a302453e01c3dfbd2279b3fa7e49d9a83b11ff3 in jclouds's branch refs/heads/2.0.x from Andrew Gaul
        [ https://git-wip-us.apache.org/repos/asf?p=jclouds.git;h=9a30245 ]

        JCLOUDS-1358: Fix zero byte InputStream test input

        Do not use ByteSource.empty() since it is backed by a
        ByteArrayInputStream which supports reset.

        Show
        jira-bot ASF subversion and git services added a comment - Commit 9a302453e01c3dfbd2279b3fa7e49d9a83b11ff3 in jclouds's branch refs/heads/2.0.x from Andrew Gaul [ https://git-wip-us.apache.org/repos/asf?p=jclouds.git;h=9a30245 ] JCLOUDS-1358 : Fix zero byte InputStream test input Do not use ByteSource.empty() since it is backed by a ByteArrayInputStream which supports reset.
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit b7b67a0f6ab5d4dd0266abc3a3d59961c29e0267 in jclouds's branch refs/heads/2.0.x from Andrew Gaul
        [ https://git-wip-us.apache.org/repos/asf?p=jclouds.git;h=b7b67a0 ]

        JCLOUDS-1358: Handle 0-byte input with v4 signature

        Previously this failed due to use of mark and reset on the
        InputStream.

        Show
        jira-bot ASF subversion and git services added a comment - Commit b7b67a0f6ab5d4dd0266abc3a3d59961c29e0267 in jclouds's branch refs/heads/2.0.x from Andrew Gaul [ https://git-wip-us.apache.org/repos/asf?p=jclouds.git;h=b7b67a0 ] JCLOUDS-1358 : Handle 0-byte input with v4 signature Previously this failed due to use of mark and reset on the InputStream.

          People

          • Assignee:
            gaul Andrew Gaul
            Reporter:
            dgyurdzhekliev Deyan
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development