Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-17791

Temp dirs under the staging directory should honour `inheritPerms`

Log workAgile BoardRank to TopRank to BottomVotersWatch issueWatchersCreate sub-taskConvert to sub-taskMoveLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.2.0, 2.4.0
    • Fix Version/s: 2.2.1, 2.4.0
    • Component/s: Authorization
    • Labels:
      None

      Description

      For Chris Drome:

      CLI creates two levels of staging directories but calls setPermissions on the top-level directory only if hive.warehouse.subdir.inherit.perms=true.

      The top-level directory, /user/cdrome/hive/words_text_dist/dt=c/.hive-staging_hive_2016-07-15_08-44-22_082_5534649671389063929-1 is created the first time Context.getExternalTmpPath is called.

      The child directory, /user/cdrome/hive/words_text_dist/dt=c/.hive-staging_hive_2016-07-15_08-44-22_082_5534649671389063929-1/_tmp.-ext-10000 is created when TezTask.execute is called at line 164:

      DAG dag = build(jobConf, work, scratchDir, appJarLr, additionalLr, ctx);
      

      This calls DagUtils.createVertex, which calls Utilities.createTmpDirs:

      3770   private static void createTmpDirs(Configuration conf,
      3771       List<Operator<? extends OperatorDesc>> ops) throws IOException {
      3772 
      3773     while (!ops.isEmpty()) {
      3774       Operator<? extends OperatorDesc> op = ops.remove(0);
      3775 
      3776       if (op instanceof FileSinkOperator) {
      3777         FileSinkDesc fdesc = ((FileSinkOperator) op).getConf();
      3778         Path tempDir = fdesc.getDirName();
      3779 
      3780         if (tempDir != null) {
      3781           Path tempPath = Utilities.toTempPath(tempDir);
      3782           FileSystem fs = tempPath.getFileSystem(conf);
      3783           fs.mkdirs(tempPath); // <------ HERE!
      3784         }
      3785       }
      3786 
      3787       if (op.getChildOperators() != null) {
      3788         ops.addAll(op.getChildOperators());
      3789       }
      3790     }
      3791   }
      

      It turns out that inheritPerms is no longer part of master. I'll rebase this for branch-2, and branch-2.2. master will have to wait till the issues around StorageBasedAuthProvider, directory permissions, etc. are sorted out.

      (Note to self: YHIVE-857)

        Attachments

        1. HIVE-17791.2-branch-2.patch
          1 kB
          Mithun Radhakrishnan
        2. HIVE-17791.1-branch-2.patch
          1 kB
          Mithun Radhakrishnan

          Activity

          $i18n.getText('security.level.explanation', $currentSelection) Viewable by All Users
          Cancel

            People

            • Assignee:
              cdrome Chris Drome Assign to me
              Reporter:
              mithun Mithun Radhakrishnan

              Dates

              • Created:
                Updated:
                Resolved:

                Issue deployment