[DRILL-7197] Drill attempts to create table on Amazon when configured for on premise S3A

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Not A Problem
• Affects Version/s: 1.15.0
• Component/s: Server

Environment

On premises 4 node Minio S3A storage cluster:

{
 "type": "file",
 "connection": "s3a://logs",
 "config": {
  "fs.s3a.endpoint": "x.x.x.x:9001",
  "fs.s3a.connection.ssl.enabled": "false",
  "fs.s3a.access.key": "xxx",
  "fs.s3a.secret.key": "xxx"
 },
 "workspaces": {
  "tmp": {
   "location": "/tmp",
   "writable": true,
   "defaultInputFormat": null,
   "allowAccessOutsideWorkspace": false
  },
  "logs": {
  "location": "/",
  "writable": true,
  "defaultInputFormat": null,
  "allowAccessOutsideWorkspace": false 
 }

},

• Assignee: Unassigned
• Reporter: Matt Keranen
• Created: 23/Apr/19 19:03
• Updated: 23/Apr/19 19:30
• Resolved: 23/Apr/19 19:30

Description

Attempting to create Parquet tables from JSON data both within an on premises Minio S3A cluster.

Able to SELECT from local S3 storage (Minio), but CTAS seems to be trying to go to Amazon S3:

    SELECT * FROM s3.logs LIMIT 1;

works, but

    CREATE TABLE s3.logs.`test` SELECT ... FROM s3.logs

results in the following error. Drill appears to attempt a connection to Amazon on a CTAS but not a SELECT alone:

 

Error: SYSTEM ERROR: AmazonS3Exception: Status Code: 403, AWS Service: Amazon S3, AWS Request ID: XXX, AWS Error Code: InvalidAccessKeyId, AWS Error Message: The AWS Access Key Id you provided does not exist in our records.