Details

    • Type: Bug Bug
    • Status: Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: 0.13.0
    • Fix Version/s: None
    • Component/s: Security
    • Labels:
      None

      Description

      HDFS Background

      • When a file or directory is created, its owner is the user identity of the client process, and its group is inherited from parent (the BSD rule). Permissions are taken from default umask. Extended Acl's are taken from parent unless they are set explicitly.

      Goals
      To reduce need to set fine-grain file security props after every operation, users may want the following Hive warehouse file/dir to auto-inherit security properties from their directory parents:

      • Directories created by new database/table/partition/bucket
      • Files added to tables via load/insert
      • Table directories exported/imported (open question of whether exported table inheriting perm from new parent needs another flag)

      What may be inherited:

      • Basic file permission
      • Groups (already done by HDFS for new directories)
      • Extended ACL's (already done by HDFS for new directories)

      Behavior

      • When "hive.warehouse.subdir.inherit.perms" flag is enabled in Hive, Hive will try to do all above inheritances. In the future, we can add more flags for more finer-grained control.
      • Failure by Hive to inherit will not cause operation to fail. Rule of thumb of when security-prop inheritance will happen is the following:
        • To run chmod, a user must be the owner of the file, or else a super-user.
        • To run chgrp, a user must be the owner of files, or else a super-user.
        • Hence, user that hive runs as (either 'hive' or the logged-in user in case of impersonation), must be super-user or owner of the file whose security properties are going to be changed.

        Issue Links

          Activity

          Szehon Ho created issue -
          Szehon Ho made changes -
          Field Original Value New Value
          Link This issue incorporates HIVE-6648 [ HIVE-6648 ]
          Szehon Ho made changes -
          Link This issue incorporates HIVE-6792 [ HIVE-6792 ]
          Szehon Ho made changes -
          Link This issue incorporates HIVE-3756 [ HIVE-3756 ]
          Szehon Ho made changes -
          Link This issue incorporates HIVE-6891 [ HIVE-6891 ]
          Szehon Ho made changes -
          Assignee Szehon Ho [ szehon ]
          Affects Version/s 0.13.0 [ 12324986 ]
          Component/s Security [ 12313866 ]
          Szehon Ho made changes -
          Link This issue incorporates HIVE-6916 [ HIVE-6916 ]
          Szehon Ho made changes -
          Link This issue incorporates HIVE-7015 [ HIVE-7015 ]
          Szehon Ho made changes -
          Link This issue relates to HIVE-7092 [ HIVE-7092 ]
          Szehon Ho made changes -
          Link This issue relates to HIVE-7092 [ HIVE-7092 ]
          Szehon Ho made changes -
          Link This issue incorporates HIVE-7092 [ HIVE-7092 ]
          Szehon Ho made changes -
          Link This issue incorporates HIVE-7117 [ HIVE-7117 ]
          Szehon Ho made changes -
          Description Making an umbrealla JIRA to track the smaller sub-issues. *HDFS Background*
          * When a file or directory is created, its owner is the user identity of the client process, and its group is inherited from parent (the BSD rule). Permissions are taken from default umask. Extended Acl's are inherited from parent (? TBD)

          *Goals*
          Following are file/dir of Hive that are user might want to inherit security properties from parent:
          * Directories created by new table/partition/bucket should inherit from parent (groups already inherited by HDFS, extended ACL's TBD)
          * Files added to tables via load/insert should inherit from parent
          * Tables both exported/imported should inherit from parent (open question of whether exported table inheriting perm needs another flag)


          Following are the security properties that user might want to inherit for the above cases
          * Basic permission
          * Groups (already done in some cases by HDFS for new table/partition/bucket directories)
          * Extended ACL's (TBD)


          *Behavior*
          * When "hive.warehouse.subdir.inherit.perms" flag is enabled in Hive, Hive will try to do all above inheritances. In the future, we can add more flags for more finer-grained control.
          * Failure by Hive to inherit will not cause operation to fail. Rule of thumb of when it will fail is the following:
          ** To run chmod, a user must be the owner of the file, or else a super-user.
          ** To run chgrp, a user must be the owner of files, or else a super-user.
          ** Hence, user that hive runs as (either 'hive' or the logged-in user in case of impersonation), must be super-user or owner of the file whose security properties are going to be changed.
          Szehon Ho made changes -
          Description *HDFS Background*
          * When a file or directory is created, its owner is the user identity of the client process, and its group is inherited from parent (the BSD rule). Permissions are taken from default umask. Extended Acl's are inherited from parent (? TBD)

          *Goals*
          Following are file/dir of Hive that are user might want to inherit security properties from parent:
          * Directories created by new table/partition/bucket should inherit from parent (groups already inherited by HDFS, extended ACL's TBD)
          * Files added to tables via load/insert should inherit from parent
          * Tables both exported/imported should inherit from parent (open question of whether exported table inheriting perm needs another flag)


          Following are the security properties that user might want to inherit for the above cases
          * Basic permission
          * Groups (already done in some cases by HDFS for new table/partition/bucket directories)
          * Extended ACL's (TBD)


          *Behavior*
          * When "hive.warehouse.subdir.inherit.perms" flag is enabled in Hive, Hive will try to do all above inheritances. In the future, we can add more flags for more finer-grained control.
          * Failure by Hive to inherit will not cause operation to fail. Rule of thumb of when it will fail is the following:
          ** To run chmod, a user must be the owner of the file, or else a super-user.
          ** To run chgrp, a user must be the owner of files, or else a super-user.
          ** Hence, user that hive runs as (either 'hive' or the logged-in user in case of impersonation), must be super-user or owner of the file whose security properties are going to be changed.
          *HDFS Background*
          * When a file or directory is created, its owner is the user identity of the client process, and its group is inherited from parent (the BSD rule). Permissions are taken from default umask. Extended Acl's are inherited from parent (? TBD)

          *Goals*
          Following are file/dir of Hive that user might want to inherit security properties from parent:
          * Directories created by new table/partition/bucket should inherit from parent (groups already inherited by HDFS, extended ACL's TBD)
          * Files added to tables via load/insert should inherit from parent
          * Tables both exported/imported should inherit from parent (open question of whether exported table inheriting perm needs another flag)


          Following are the security properties that user might want to inherit for the above cases
          * Basic permission
          * Groups (already done in some cases by HDFS for new table/partition/bucket directories)
          * Extended ACL's (TBD)


          *Behavior*
          * When "hive.warehouse.subdir.inherit.perms" flag is enabled in Hive, Hive will try to do all above inheritances. In the future, we can add more flags for more finer-grained control.
          * Failure by Hive to inherit will not cause operation to fail. Rule of thumb of when security-prop inheritance will fail is the following:
          ** To run chmod, a user must be the owner of the file, or else a super-user.
          ** To run chgrp, a user must be the owner of files, or else a super-user.
          ** Hence, user that hive runs as (either 'hive' or the logged-in user in case of impersonation), must be super-user or owner of the file whose security properties are going to be changed.
          Szehon Ho made changes -
          Description *HDFS Background*
          * When a file or directory is created, its owner is the user identity of the client process, and its group is inherited from parent (the BSD rule). Permissions are taken from default umask. Extended Acl's are inherited from parent (? TBD)

          *Goals*
          Following are file/dir of Hive that user might want to inherit security properties from parent:
          * Directories created by new table/partition/bucket should inherit from parent (groups already inherited by HDFS, extended ACL's TBD)
          * Files added to tables via load/insert should inherit from parent
          * Tables both exported/imported should inherit from parent (open question of whether exported table inheriting perm needs another flag)


          Following are the security properties that user might want to inherit for the above cases
          * Basic permission
          * Groups (already done in some cases by HDFS for new table/partition/bucket directories)
          * Extended ACL's (TBD)


          *Behavior*
          * When "hive.warehouse.subdir.inherit.perms" flag is enabled in Hive, Hive will try to do all above inheritances. In the future, we can add more flags for more finer-grained control.
          * Failure by Hive to inherit will not cause operation to fail. Rule of thumb of when security-prop inheritance will fail is the following:
          ** To run chmod, a user must be the owner of the file, or else a super-user.
          ** To run chgrp, a user must be the owner of files, or else a super-user.
          ** Hence, user that hive runs as (either 'hive' or the logged-in user in case of impersonation), must be super-user or owner of the file whose security properties are going to be changed.
          *HDFS Background*
          * When a file or directory is created, its owner is the user identity of the client process, and its group is inherited from parent (the BSD rule). Permissions are taken from default umask. Extended Acl's are inherited from parent (? TBD)

          *Goals*
          Users may want the following Hive warehouse file/dir to inherit security properties from their directory parents:
          * Directories created by new table/partition/bucket
          * Files added to tables via load/insert
          * Table directories exported/imported (open question of whether exported table inheriting perm from new parent needs another flag)


          What may be inherited:
          * Basic file permission
          * Groups (already done in some cases by HDFS for new table/partition/bucket directories)
          * Extended ACL's (TBD)


          *Behavior*
          * When "hive.warehouse.subdir.inherit.perms" flag is enabled in Hive, Hive will try to do all above inheritances. In the future, we can add more flags for more finer-grained control.
          * Failure by Hive to inherit will not cause operation to fail. Rule of thumb of when security-prop inheritance will happen is the following:
          ** To run chmod, a user must be the owner of the file, or else a super-user.
          ** To run chgrp, a user must be the owner of files, or else a super-user.
          ** Hence, user that hive runs as (either 'hive' or the logged-in user in case of impersonation), must be super-user or owner of the file whose security properties are going to be changed.
          Szehon Ho made changes -
          Link This issue incorporates HIVE-7119 [ HIVE-7119 ]
          Szehon Ho made changes -
          Description *HDFS Background*
          * When a file or directory is created, its owner is the user identity of the client process, and its group is inherited from parent (the BSD rule). Permissions are taken from default umask. Extended Acl's are inherited from parent (? TBD)

          *Goals*
          Users may want the following Hive warehouse file/dir to inherit security properties from their directory parents:
          * Directories created by new table/partition/bucket
          * Files added to tables via load/insert
          * Table directories exported/imported (open question of whether exported table inheriting perm from new parent needs another flag)


          What may be inherited:
          * Basic file permission
          * Groups (already done in some cases by HDFS for new table/partition/bucket directories)
          * Extended ACL's (TBD)


          *Behavior*
          * When "hive.warehouse.subdir.inherit.perms" flag is enabled in Hive, Hive will try to do all above inheritances. In the future, we can add more flags for more finer-grained control.
          * Failure by Hive to inherit will not cause operation to fail. Rule of thumb of when security-prop inheritance will happen is the following:
          ** To run chmod, a user must be the owner of the file, or else a super-user.
          ** To run chgrp, a user must be the owner of files, or else a super-user.
          ** Hence, user that hive runs as (either 'hive' or the logged-in user in case of impersonation), must be super-user or owner of the file whose security properties are going to be changed.
          *HDFS Background*
          * When a file or directory is created, its owner is the user identity of the client process, and its group is inherited from parent (the BSD rule). Permissions are taken from default umask. Extended Acl's are taken from parent unless they are set explicitly.

          *Goals*
          To reduce need to set fine-grain file security props after every operation, users may want the following Hive warehouse file/dir to auto-inherit security properties from their directory parents:
          * Directories created by new table/partition/bucket
          * Files added to tables via load/insert
          * Table directories exported/imported (open question of whether exported table inheriting perm from new parent needs another flag)


          What may be inherited:
          * Basic file permission
          * Groups (already done by HDFS for new directories)
          * Extended ACL's (already done by HDFS for new directories)


          *Behavior*
          * When "hive.warehouse.subdir.inherit.perms" flag is enabled in Hive, Hive will try to do all above inheritances. In the future, we can add more flags for more finer-grained control.
          * Failure by Hive to inherit will not cause operation to fail. Rule of thumb of when security-prop inheritance will happen is the following:
          ** To run chmod, a user must be the owner of the file, or else a super-user.
          ** To run chgrp, a user must be the owner of files, or else a super-user.
          ** Hence, user that hive runs as (either 'hive' or the logged-in user in case of impersonation), must be super-user or owner of the file whose security properties are going to be changed.
          Szehon Ho made changes -
          Labels TODOC14
          Szehon Ho made changes -
          Link This issue incorporates HIVE-7450 [ HIVE-7450 ]
          Szehon Ho made changes -
          Link This issue incorporates HIVE-8791 [ HIVE-8791 ]
          Szehon Ho made changes -
          Description *HDFS Background*
          * When a file or directory is created, its owner is the user identity of the client process, and its group is inherited from parent (the BSD rule). Permissions are taken from default umask. Extended Acl's are taken from parent unless they are set explicitly.

          *Goals*
          To reduce need to set fine-grain file security props after every operation, users may want the following Hive warehouse file/dir to auto-inherit security properties from their directory parents:
          * Directories created by new table/partition/bucket
          * Files added to tables via load/insert
          * Table directories exported/imported (open question of whether exported table inheriting perm from new parent needs another flag)


          What may be inherited:
          * Basic file permission
          * Groups (already done by HDFS for new directories)
          * Extended ACL's (already done by HDFS for new directories)


          *Behavior*
          * When "hive.warehouse.subdir.inherit.perms" flag is enabled in Hive, Hive will try to do all above inheritances. In the future, we can add more flags for more finer-grained control.
          * Failure by Hive to inherit will not cause operation to fail. Rule of thumb of when security-prop inheritance will happen is the following:
          ** To run chmod, a user must be the owner of the file, or else a super-user.
          ** To run chgrp, a user must be the owner of files, or else a super-user.
          ** Hence, user that hive runs as (either 'hive' or the logged-in user in case of impersonation), must be super-user or owner of the file whose security properties are going to be changed.
          *HDFS Background*
          * When a file or directory is created, its owner is the user identity of the client process, and its group is inherited from parent (the BSD rule). Permissions are taken from default umask. Extended Acl's are taken from parent unless they are set explicitly.

          *Goals*
          To reduce need to set fine-grain file security props after every operation, users may want the following Hive warehouse file/dir to auto-inherit security properties from their directory parents:
          * Directories created by new database/table/partition/bucket
          * Files added to tables via load/insert
          * Table directories exported/imported (open question of whether exported table inheriting perm from new parent needs another flag)


          What may be inherited:
          * Basic file permission
          * Groups (already done by HDFS for new directories)
          * Extended ACL's (already done by HDFS for new directories)


          *Behavior*
          * When "hive.warehouse.subdir.inherit.perms" flag is enabled in Hive, Hive will try to do all above inheritances. In the future, we can add more flags for more finer-grained control.
          * Failure by Hive to inherit will not cause operation to fail. Rule of thumb of when security-prop inheritance will happen is the following:
          ** To run chmod, a user must be the owner of the file, or else a super-user.
          ** To run chgrp, a user must be the owner of files, or else a super-user.
          ** Hence, user that hive runs as (either 'hive' or the logged-in user in case of impersonation), must be super-user or owner of the file whose security properties are going to be changed.
          Szehon Ho made changes -
          Link This issue incorporates HIVE-8864 [ HIVE-8864 ]
          Szehon Ho made changes -
          Labels TODOC14

            People

            • Assignee:
              Szehon Ho
              Reporter:
              Szehon Ho
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:

                Development