Upload content via S3 gateway with "Content-Type: plain/text; charset=utf-8" header will fail with 500 code. At the same time, "Content-Type: plain/text" (without charset) and "Content-Type: plain/text; charset=UTF-8" (in uppercase) were accepted.
We're running the official Ozone v1.1.0 release.
These lines shown as below are known behavior:
- 'text/plain' ==> OK
- 'text/plain; charset=UTF-8' ==> OK
- 'text/plain; charset=utf-8' ==> 500
- 'text/html; charset=UTF-8' ==> OK (same as text/plain case)
- 'text/html; charset=utf-8' ==> 500 (same as text/plain case)
- 'text/markdown; charset=UTF-8' ==> OK
- 'text/markdown; charset=utf-8' ==> OK
I've attached a code snippet (upload.go) and a log from S3 gateway (s3g.log). This code can run easily with following command lines.
HTTP request from upload.go is here:
We thought the issue is similar to
HDDS-4856 but this issue will happen when empty Content-Type header was given. Our facing problem will appear when Content-Type defined together with the charset directive.
The stack trace of S3 gateway indicates the request proceeded to remote invocation and failed with authentication error with org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken). There are many parts to preprocess a request before reaching RPC Client, so I suspect some Java servlet filters like ozone.s3.HeaderPreprocessor and hdds.server.http.NoCacheFilter. But as far as I found out, these are not a cause of this problem directly (because these are almost for generating response header). I also suspected some components like OzoneClientProducer, Signature Processor and HeaderPreprocessor. I think this is not direct cause. How do I investigate this problem in more detail?