Details

    • Type: Bug Bug
    • Status: Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: 0.13.0
    • Fix Version/s: None
    • Component/s: Security
    • Labels:
      None

      Description

      HDFS Background

      • When a file or directory is created, its owner is the user identity of the client process, and its group is inherited from parent (the BSD rule). Permissions are taken from default umask. Extended Acl's are taken from parent unless they are set explicitly.

      Goals
      To reduce need to set fine-grain file security props after every operation, users may want the following Hive warehouse file/dir to auto-inherit security properties from their directory parents:

      • Directories created by new database/table/partition/bucket
      • Files added to tables via load/insert
      • Table directories exported/imported (open question of whether exported table inheriting perm from new parent needs another flag)

      What may be inherited:

      • Basic file permission
      • Groups (already done by HDFS for new directories)
      • Extended ACL's (already done by HDFS for new directories)

      Behavior

      • When "hive.warehouse.subdir.inherit.perms" flag is enabled in Hive, Hive will try to do all above inheritances. In the future, we can add more flags for more finer-grained control.
      • Failure by Hive to inherit will not cause operation to fail. Rule of thumb of when security-prop inheritance will happen is the following:
        • To run chmod, a user must be the owner of the file, or else a super-user.
        • To run chgrp, a user must be the owner of files, or else a super-user.
        • Hence, user that hive runs as (either 'hive' or the logged-in user in case of impersonation), must be super-user or owner of the file whose security properties are going to be changed.

        Issue Links

          Activity

          Hide
          Szehon Ho added a comment -

          The second point might be a valid change, to mimic the HDFS way instead of cloning the extended ACL's. I dont have bandwidth to make the change at the moment, someone else can feel free to take a stab (looks like HIVE-11481). It would be more complex, we would have to traverse the tree and essentially copy the HDFS logic for extended ACL for 'default' group.

          I have not investigated enough to comment on the first point.

          Show
          Szehon Ho added a comment - The second point might be a valid change, to mimic the HDFS way instead of cloning the extended ACL's. I dont have bandwidth to make the change at the moment, someone else can feel free to take a stab (looks like HIVE-11481 ). It would be more complex, we would have to traverse the tree and essentially copy the HDFS logic for extended ACL for 'default' group. I have not investigated enough to comment on the first point.
          Hide
          Andrés Cordero added a comment -

          Can some changes be made to Permission Inheritance in Hive?
          I've seen some behavior that doesn't match what the doc claims.
          Namely:

          • Group isn't inherited when the flag is off, "already done by HDFS for new directories" implies that it shouldn't matter.
          • Extended ACLs are not "inherited" they are "cloned", which means that default ACLs don't propagate down as default+access (the HDFS way), but default only (which means default for directories and nothing for files). "Extended Acl's are taken from parent" in the first paragraph already implies this, but it's still rather ambiguous (especially with below containing the same "already done by HDFS" text).
          Show
          Andrés Cordero added a comment - Can some changes be made to Permission Inheritance in Hive ? I've seen some behavior that doesn't match what the doc claims. Namely: Group isn't inherited when the flag is off, "already done by HDFS for new directories" implies that it shouldn't matter. Extended ACLs are not "inherited" they are "cloned", which means that default ACLs don't propagate down as default+access (the HDFS way), but default only (which means default for directories and nothing for files). "Extended Acl's are taken from parent" in the first paragraph already implies this, but it's still rather ambiguous (especially with below containing the same "already done by HDFS" text).
          Hide
          Lefty Leverenz added a comment -

          Thanks for finding that link, Szehon Ho. I changed it to an internal link so now it shows up in the page information as an incoming link.

          Show
          Lefty Leverenz added a comment - Thanks for finding that link, Szehon Ho . I changed it to an internal link so now it shows up in the page information as an incoming link. Page information for Permission Inheritance in Hive
          Hide
          Szehon Ho added a comment -

          Ah I found it. I added a link from HCatalog Authorization where it discusses file permissions. That is in turn is referred from Storage Based Authorization.

          There is also a link from : Configuration Properties. I will remove the label then.

          Show
          Szehon Ho added a comment - Ah I found it. I added a link from HCatalog Authorization where it discusses file permissions. That is in turn is referred from Storage Based Authorization . There is also a link from : Configuration Properties . I will remove the label then.
          Hide
          Szehon Ho added a comment -

          Strange, I thought I added a link from "Storage Based Authorization", but I must have forgotten to save it. I'll try to add it and remove the label.

          Show
          Szehon Ho added a comment - Strange, I thought I added a link from "Storage Based Authorization", but I must have forgotten to save it. I'll try to add it and remove the label.
          Hide
          Lefty Leverenz added a comment -

          Can we remove the TODOC14 label now?

          Also, should any other docs have links to Permission Inheritance in Hive? For example, Authorization or Storage Based Authorization:

          Show
          Lefty Leverenz added a comment - Can we remove the TODOC14 label now? Also, should any other docs have links to Permission Inheritance in Hive? For example, Authorization or Storage Based Authorization: Authorization Storage Based Authorization
          Hide
          Lefty Leverenz added a comment -

          I don't have any experience with that, Szehon Ho. Have you taken a look at how it's done in the Open Issues section of HBase Integration?

          It was added by Carl Steinbach in version 8 of the doc, so maybe Carl can help you. Or maybe all you need is this:

          You might already know, but it's a JIRA Issue box which can be inserted in edit mode from the "+" drop-down list (Insert More Content). Did you do that, then hit a snag with the query?

          Show
          Lefty Leverenz added a comment - I don't have any experience with that, Szehon Ho . Have you taken a look at how it's done in the Open Issues section of HBase Integration? HBase Integration – Open Issues It was added by Carl Steinbach in version 8 of the doc, so maybe Carl can help you. Or maybe all you need is this: Versions Compared – 7 to 8 You might already know, but it's a JIRA Issue box which can be inserted in edit mode from the "+" drop-down list (Insert More Content). Did you do that, then hit a snag with the query?
          Hide
          Szehon Ho added a comment -

          Thanks Lefty, I think its a lower level than Storage Based Authorization, because if the flag is on then permissions will be inherited regardless of which authorization is configured. I updated Storage Based Authorization to add the link according to this understanding.

          Question for you, I had a JQL I wanted to in Permission Inheritance in Hive page to display the full list of patches:
          project = HIVE and issue in linkedIssues(HIVE-6892)
          but its giving me some wiki runtimeError when I try. Do you know how to make that work? Thanks.

          Show
          Szehon Ho added a comment - Thanks Lefty, I think its a lower level than Storage Based Authorization, because if the flag is on then permissions will be inherited regardless of which authorization is configured. I updated Storage Based Authorization to add the link according to this understanding. Question for you, I had a JQL I wanted to in Permission Inheritance in Hive page to display the full list of patches: project = HIVE and issue in linkedIssues( HIVE-6892 ) but its giving me some wiki runtimeError when I try. Do you know how to make that work? Thanks.
          Hide
          Lefty Leverenz added a comment -

          That sounds good, Szehon Ho, although additional links from other docs could increase visibility for this issue. Perhaps we need more documentation about hive.warehouse.subdir.inherit.perms somewhere besides Configuration Properties.

          How does this relate to storage-based authorization? When storage-based authorization is not being used, is this still relevant to create/load/insert/export/import commands?

          Oh, cool, you've already added the new page. Reference links:

          Show
          Lefty Leverenz added a comment - That sounds good, Szehon Ho , although additional links from other docs could increase visibility for this issue. Perhaps we need more documentation about hive.warehouse.subdir.inherit.perms somewhere besides Configuration Properties. How does this relate to storage-based authorization? When storage-based authorization is not being used, is this still relevant to create/load/insert/export/import commands? Oh, cool, you've already added the new page. Reference links: Configuration Properties – hive.warehouse.subdir.inherit.perms Permission Inheritance in Hive Storage Based Authorization
          Hide
          Szehon Ho added a comment -

          I am thinking to create a new wiki child page and add the information of this JIRA directly there, and link from https://cwiki.apache.org/confluence/display/Hive/Configuration+Properties

          + Lefty Leverenz do you have any better thoughts? Thanks

          Show
          Szehon Ho added a comment - I am thinking to create a new wiki child page and add the information of this JIRA directly there, and link from https://cwiki.apache.org/confluence/display/Hive/Configuration+Properties + Lefty Leverenz do you have any better thoughts? Thanks
          Hide
          Szehon Ho added a comment -

          Adding some specs here.

          Show
          Szehon Ho added a comment - Adding some specs here.

            People

            • Assignee:
              Szehon Ho
              Reporter:
              Szehon Ho
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:

                Development