Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-16913

Support per-session S3 credentials

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      Currently, the credentials needed to support Hive-on-S3 (or any other cloud-storage) need to be to the hive-site.xml. Either using a hadoop credential provider or by adding the keys in the hive-site.xml in plain text (unsecure)

      This limits the usecase to using a single S3 key. If we configure per bucket s3 keys like described here it exposes the access to all the buckets to all the hive users.

      It is possible that there are different sets of users who would not like to share there buckets and still be able to process the data using Hive. Enabling session level credentials will help solve such use-cases. For example, currently this doesn't work

      set fs.s3a.secret.key=my_secret_key;
      set fs.s3a.access.key=my_access.key;
      

      Because metastore is unaware of the the keys. This doesn't work either

      set fs.s3a.secret.key=my_secret_key;
      set fs.s3a.access.key=my_access.key;
      set metaconf:fs.s3a.secret.key=my_secret_key;
      set metaconf:fs.s3a.access.key=my_access_key;
      

      This is because only a certain metastore configurations defined in HiveConf.MetaVars are allowed to be set by the user. If we enable the above approaches we could potentially allow multiple S3 credentials on a per-session level basis.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                vihangk1 Vihang Karajgaonkar
                Reporter:
                vihangk1 Vihang Karajgaonkar
              • Votes:
                1 Vote for this issue
                Watchers:
                9 Start watching this issue

                Dates

                • Created:
                  Updated: