Hive
  1. Hive
  2. HIVE-6792

hive.warehouse.subdir.inherit.perms doesn't work correctly in CTAS

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.14.0
    • Fix Version/s: 0.14.0
    • Component/s: Authorization, Security
    • Labels:
      None

      Description

      hive.warehouse.subdir.inherit.perms doesn't work correctly in CTAS. When it is set to true, the table created using create table .. as select.. does not inherit its parent directory's group and permission mode. It can be easily reproduced:
      ==
      hive> dfs -ls -R /user/hive/warehouse;
      drwxrwx--T - hive hive 0 2014-03-30 17:44 /user/hive/warehouse/ctas.db
      drwxr-xr-x - hive hive 0 2014-03-30 17:20 /user/hive/warehouse/ctas_src_tbl
      rw-rr- 3 hive hive 46059 2014-03-30 17:20 /user/hive/warehouse/ctas_src_tbl/000000_0

      hive> create table ctas.test_perm as select * from ctas_src_tbl;

      hive> dfs -ls -R /user/hive/warehouse;
      drwxrwx--T - hive hive 0 2014-03-30 17:46 /user/hive/warehouse/ctas.db
      drwxr-xr-x - hive supergroup 0 2014-03-30 17:46 /user/hive/warehouse/ctas.db/test_perm
      rw-rr- 3 hive supergroup 46059 2014-03-30 17:46 /user/hive/warehouse/ctas.db/test_perm/000000_0
      drwxr-xr-x - hive hive 0 2014-03-30 17:20 /user/hive/warehouse/ctas_src_tbl
      rw-rr- 3 hive hive 46059 2014-03-30 17:20 /user/hive/warehouse/ctas_src_tbl/000000_0
      ==
      The created table does not inherit its database ctas's group hive and permission mode 770, instead it takes the default group (supergroup) and permission mode (755) in hdfs

      1. HIVE-6792.1.patch
        3 kB
        Brock Noland
      2. HIVE-6792-1.patch
        3 kB
        Chaoyu Tang
      3. HIVE-6792.patch
        4 kB
        Chaoyu Tang

        Issue Links

          Activity

          Hide
          Chaoyu Tang added a comment -

          Please review attached patch. With this, the new created table (test_perm) with its data file inherit group and permission mode from their parent cats.db.
          ==
          hive> dfs -ls -R /user/hive/warehouse;
          drwxrwx--T - hive hive 0 2014-03-30 17:56 /user/hive/warehouse/ctas.db
          drwxrwx--T - hive hive 0 2014-03-30 17:56 /user/hive/warehouse/ctas.db/test_perm
          rw-rw--- 3 hive hive 46059 2014-03-30 17:56 /user/hive/warehouse/ctas.db/test_perm/000000_0
          drwxr-xr-x - hive hive 0 2014-03-30 17:20 /user/hive/warehouse/ctas_src_tbl
          rw-rr- 3 hive hive 46059 2014-03-30 17:20 /user/hive/warehouse/ctas_src_tbl/000000_0

          Show
          Chaoyu Tang added a comment - Please review attached patch. With this, the new created table (test_perm) with its data file inherit group and permission mode from their parent cats.db. == hive> dfs -ls -R /user/hive/warehouse; drwxrwx--T - hive hive 0 2014-03-30 17:56 /user/hive/warehouse/ctas.db drwxrwx--T - hive hive 0 2014-03-30 17:56 /user/hive/warehouse/ctas.db/test_perm rw-rw --- 3 hive hive 46059 2014-03-30 17:56 /user/hive/warehouse/ctas.db/test_perm/000000_0 drwxr-xr-x - hive hive 0 2014-03-30 17:20 /user/hive/warehouse/ctas_src_tbl rw-r r - 3 hive hive 46059 2014-03-30 17:20 /user/hive/warehouse/ctas_src_tbl/000000_0
          Hide
          Szehon Ho added a comment -

          Mostly looks good on first pass, one nit: is it necessary to pass in 'conf' variable to 'MoveTask.createTargetPath'? It is already a member variable.

          Also, do you want to submit the patch to trigger the pre-commit test?

          Show
          Szehon Ho added a comment - Mostly looks good on first pass, one nit: is it necessary to pass in 'conf' variable to 'MoveTask.createTargetPath'? It is already a member variable. Also, do you want to submit the patch to trigger the pre-commit test?
          Hide
          Chaoyu Tang added a comment -

          Thanks, Szehon, for pointing out. I changed to use member variable conf instead and please see attached HIVE-6792-1.patch

          Show
          Chaoyu Tang added a comment - Thanks, Szehon, for pointing out. I changed to use member variable conf instead and please see attached HIVE-6792 -1.patch
          Hide
          Szehon Ho added a comment -

          Please upload in the format HIVE-6792.1.patch to trigger the pre commit test.

          Show
          Szehon Ho added a comment - Please upload in the format HIVE-6792 .1.patch to trigger the pre commit test.
          Hide
          Brock Noland added a comment -

          +1 pending tests.

          Show
          Brock Noland added a comment - +1 pending tests.
          Hide
          Brock Noland added a comment -

          Uploading the same patch with a name precommits like.

          Show
          Brock Noland added a comment - Uploading the same patch with a name precommits like.
          Hide
          Hive QA added a comment -

          Overall: +1 all checks pass

          Here are the results of testing the latest attachment:
          https://issues.apache.org/jira/secure/attachment/12638621/HIVE-6792.1.patch

          SUCCESS: +1 5547 tests passed

          Test results: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/2129/testReport
          Console output: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/2129/console

          Messages:

          Executing org.apache.hive.ptest.execution.PrepPhase
          Executing org.apache.hive.ptest.execution.ExecutionPhase
          Executing org.apache.hive.ptest.execution.ReportingPhase
          

          This message is automatically generated.

          ATTACHMENT ID: 12638621

          Show
          Hive QA added a comment - Overall : +1 all checks pass Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12638621/HIVE-6792.1.patch SUCCESS: +1 5547 tests passed Test results: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/2129/testReport Console output: http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/2129/console Messages: Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase This message is automatically generated. ATTACHMENT ID: 12638621
          Hide
          Brock Noland added a comment -

          Thank you Chaoyu! I have committed this to trunk.

          Show
          Brock Noland added a comment - Thank you Chaoyu! I have committed this to trunk.
          Hide
          Thejas M Nair added a comment -

          This has been fixed in 0.14 release. Please open new jira if you see any issues.

          Show
          Thejas M Nair added a comment - This has been fixed in 0.14 release. Please open new jira if you see any issues.

            People

            • Assignee:
              Chaoyu Tang
              Reporter:
              Chaoyu Tang
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development