[HIVE-17791] Temp dirs under the staging directory should honour `inheritPerms`

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.2.0, 2.4.0
• Fix Version/s: 2.2.1, 2.4.0
• Component/s: Authorization
• Assignee: Chris Drome
• Reporter: Mithun Radhakrishnan
• Created: 12/Oct/17 18:52
• Updated: 26/Oct/17 22:00
• Resolved: 26/Oct/17 22:00

Description

For cdrome:

CLI creates two levels of staging directories but calls setPermissions on the top-level directory only if hive.warehouse.subdir.inherit.perms=true.

The top-level directory, /user/cdrome/hive/words_text_dist/dt=c/.hive-staging_hive_2016-07-15_08-44-22_082_5534649671389063929-1 is created the first time Context.getExternalTmpPath is called.

The child directory, /user/cdrome/hive/words_text_dist/dt=c/.hive-staging_hive_2016-07-15_08-44-22_082_5534649671389063929-1/_tmp.-ext-10000 is created when TezTask.execute is called at line 164:

DAG dag = build(jobConf, work, scratchDir, appJarLr, additionalLr, ctx);

This calls DagUtils.createVertex, which calls Utilities.createTmpDirs:

3770   private static void createTmpDirs(Configuration conf,
3771       List<Operator<? extends OperatorDesc>> ops) throws IOException {
3772 
3773     while (!ops.isEmpty()) {
3774       Operator<? extends OperatorDesc> op = ops.remove(0);
3775 
3776       if (op instanceof FileSinkOperator) {
3777         FileSinkDesc fdesc = ((FileSinkOperator) op).getConf();
3778         Path tempDir = fdesc.getDirName();
3779 
3780         if (tempDir != null) {
3781           Path tempPath = Utilities.toTempPath(tempDir);
3782           FileSystem fs = tempPath.getFileSystem(conf);
3783           fs.mkdirs(tempPath); // <------ HERE!
3784         }
3785       }
3786 
3787       if (op.getChildOperators() != null) {
3788         ops.addAll(op.getChildOperators());
3789       }
3790     }
3791   }

It turns out that inheritPerms is no longer part of master. I'll rebase this for branch-2, and branch-2.2. master will have to wait till the issues around StorageBasedAuthProvider, directory permissions, etc. are sorted out.

(Note to self: YHIVE-857)