[HIVE-20133] Cannot set fs.s3a.access.key and fs.s3a.secret.key from HiveMetastoreClient

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 3.0.0
• Component/s: Standalone Metastore
• Assignee: Unassigned
• Reporter: Sandhya Agarwal
• Due: 11/Jul/18
• Created: 10/Jul/18 12:14
• Updated: 10/Jul/18 12:22

Description

Hello,

We want to leverage standalone metastore for our project, for which I want to enable access to multiple S3 buckets, each with its own access key and secret key. I am trying to access the metastore operations from a Java client using HiveMetastoreClient and using the thrift metastore URI to connect to the metastore service. For the database location and table location, I am using the S3A file system scheme. I am trying to set the access key and secret key for the S3 bucket location in the HiveMetastoreClient, but the client side settings are never honoured and I cannot figure out a way to do this. One way is to provide the per-bucket access key and secret key on the server side in metastore-site.xml. However, I want this to be dynamic as I want the metastore to be multi tenant enabled and based on the current tenant user, I wish to provide these keys through my client. I tried looking through the metastore source code, but did not find a way to do this. Can this be done ?

Thank you,

Sandhya