[DRILL-7289] fs.s3a.path.style.access does not seem to work

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 1.16.0
• Component/s: Functions - Drill

Environment

Running on Kubernetes

• Assignee: Unassigned
• Reporter: Gururajesh Elango
• Created: 10/Jun/19 08:42
• Updated: 24/Jun/19 11:11

Description

fs.s3a.path.style.access does not seem to work. Please see https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingBucket.html#access-bucket-intro to know about path style access

How to reproduce:

1. Create a bucket in minio(S3 simulator)

2. Define a storage like below

"storage": {
s3: {
type: "file",
connection: "s3a://new-bucket",
"config":

{ "fs.s3a.access.key": "minio-user", "fs.s3a.secret.key": "minio-password", "fs.s3a.endpoint": "http://<minio-url>:9000", "fs.s3a.connection.ssl.enabled": "false", "fs.s3a.path.style.access": "true", "fs.s3a.connection.timeout": "5000", "fs.s3a.connection.maximum": "100" }

,
"workspaces": {
"tmp":

{ "location": "/tmp/drill", "writable": "true", "defaultInputFormat": "", "allowAccessOutsideWorkspace": "false" }

,
"root":

{ "location": "/", "writable": "false", "defaultInputFormat": "", "allowAccessOutsideWorkspace": "false" }

},
"formats": {
"parquet":

{ "type": "parquet" }

,
"json":

{ "type": "json", "extensions": [ "json" ] }

,
"avro":

{ "type": "avro" }

},
"enabled": "true"
}
}

3. In the logs, Expect an error which states

new-bucket.<minio-url>:9000 is not reachable but what is expected is that Drill tries to reach

http://<minio-url>:9000/impact-enable-bucket