[NIFI-8608] PutS3Object processor missing SHA256 Hash on https endpoint. Enhancement

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 1.12.1, 1.13.1, 1.13.2
• Component/s: Core Framework
• Assignee: Unassigned
• Reporter: Denis Jakupovic
• Created: 17/May/21 11:48
• Updated: 17/May/21 11:53

Description

Here is the HTTP PUT Header which is send by the PutS3Object processor if the endpoint url of the s3 backend uses the https protocol:

PUT HTTP/1.1 https://xxxxxxxx/bucket/file

Host: xxxxxxxxx

User-Agent: NiFi, aws-sdk-java/1.11.880 Linux/3.10.0-1160.11.1.el7.x86_64 OpenJDK_64-Bit_Server_VM/25.282-b08 java/1.8.0_282 vendor/Oracle_Corporation

Content-Length: 25600

Amz-Sdk-Invocation-Id: 8a531ba6-7aa2-e679-0c5b-84779636e71e

Amz-Sdk-Request: attempt=1;max=1

Amz-Sdk-Retry: 0/0/500

Authorization: AWS4-HMAC-SHA256 Credential=xxxxxxxxxxxxxxxxxxxx/20210517/us-east-1/s3/aws4_request, SignedHeaders=amz-sdk-invocation-id;amz-sdk-request;amz-sdk-retry;content-disposition;content-length;content-type;host;user-agent;x-amz-content-sha256;x-amz-date;x-amz-storage-class, Signature=fa16cf4bxxxxxxxxxx

Content-Disposition: 3004856f-6327-460d-b8ab-8ce0490f7a9d

Content-Type: application/octet-stream

Expect: 100-continue

X-Amz-Content-Sha256: UNSIGNED-PAYLOAD

X-Amz-Date: 20210517T112524Z

X-Amz-Storage-Class: STANDARD

Accept-Encoding: gzip

 

If the endpoint is set to http scheme instead the sha256 hash is being send. Tested with "Use Chunked encoding" true and false.

It would be great if the processor had a property where signed headers could be put dynamically seperated by a semicolon ; and inserted in the AWSV4 signature at signedheaders and in the calculation of the signature.