Details
-
Bug
-
Status: Closed
-
Major
-
Resolution: Invalid
-
0.16.1
-
None
-
None
-
None
Description
There is a static method FileSystem.mkdirs() that create the directory first and then sets permissions in it. Instead, it should create the directory with the correct permissions.
Otherwise, even if dfs.permission is set to off, one might see job submission failures with the following stack trace:
at org.apache.hadoop.dfs.PermissionChecker.check(PermissionChecker.java:173)
at
org.apache.hadoop.dfs.PermissionChecker.checkTraverse(PermissionChecker.java:129)
at
org.apache.hadoop.dfs.PermissionChecker.checkPermission(PermissionChecker.java:99)
at org.apache.hadoop.dfs.FSNamesystem.checkPermission(FSNamesystem.java:4031)
at org.apache.hadoop.dfs.FSNamesystem.checkOwner(FSNamesystem.java:3986)
at org.apache.hadoop.dfs.FSNamesystem.setPermission(FSNamesystem.java:715)
at org.apache.hadoop.dfs.NameNode.setPermission(NameNode.java:297)
at sun.reflect.GeneratedMethodAccessor21.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:409)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:899)
at org.apache.hadoop.ipc.Client.call(Client.java:512)
at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:198)
at org.apache.hadoop.dfs.$Proxy4.setPermission(Unknown Source)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
at
org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
at org.apache.hadoop.dfs.$Proxy4.setPermission(Unknown Source)
at
org.apache.hadoop.dfs.DistributedFileSystem.setPermission(DistributedFileSystem.java:351)
at org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:218)
at org.apache.hadoop.mapred.JobTracker.<init>(JobTracker.java:688)
at org.apache.hadoop.mapred.JobTracker.startTracker(JobTracker.java:124)
at org.apache.hadoop.mapred.JobTracker.main(JobTracker.java:2114)
Attachments
Issue Links
- is related to
-
HADOOP-1873 User permissions for Map/Reduce
- Closed
The static method is required to avoid umask, as is sometimes required, yet still have mkdirs implement POSIX conventions. So calling mkdirs(path, permissions) won't do what's desired here, will it?
I don't see the exception in your stack trace. How does it fail?