[MAPREDUCE-3668] AccessControlException when running mapred job -list command

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Duplicate
• Affects Version/s: 0.23.1
• Component/s: client, mrv2, security
• Assignee: Vinod Kumar Vavilapalli
• Reporter: Jason Darrell Lowe
• Created: 13/Jan/12 19:25
• Updated: 30/Jan/12 23:52
• Resolved: 30/Jan/12 23:51

Description

If a user tries to examine the status of all jobs running on a secure cluster the mapred client can fail with an AccessControlException. For example, submitting two jobs each from a different user then trying to query the status as the second user can fail like this:

$ mapred job -list all
12/01/12 20:01:12 WARN conf.Configuration: mapred.used.genericoptionsparser is deprecated. Instead, use
mapreduce.client.genericoptionsparser.used
Total jobs:2
JobId State StartTime UserName Queue Priority Maps Reduces UsedContainers RsvdContainers UsedMem RsvdMem NeededMem AM info
12/01/12 20:01:14 INFO mapred.ClientServiceDelegate: Application state is completed. FinalApplicationStatus=SUCCEEDED. Redirecting to job history server
job_1326396427223_0002 SUCCEEDED 1326398424244 user2 default NORMAL 2 2 0 0 0M 0M 0M
hostremoved:8088/proxy/application_1326396427223_0002/jobhistory/job/job_1326396427223_2_2
12/01/12 20:01:14 INFO mapred.ClientServiceDelegate: Application state is completed. FinalApplicationStatus=SUCCEEDED. Redirecting to job history server
12/01/12 20:01:14 WARN mapred.ClientServiceDelegate: Error from remote end: User user2 cannot perform operation VIEW_JOB on job_1326396427223_0001
Exception in thread "main" RemoteTrace:
java.security.AccessControlException: User user2 cannot perform operation VIEW_JOB on job_1326396427223_0001
at org.apache.hadoop.mapreduce.v2.hs.HistoryClientService$MRClientProtocolHandler.checkAccess(HistoryClientService.java:293)
at org.apache.hadoop.mapreduce.v2.hs.HistoryClientService$MRClientProtocolHandler.verifyAndGetJob(HistoryClientService.java:184)
at org.apache.hadoop.mapreduce.v2.hs.HistoryClientService$MRClientProtocolHandler.getJobReport(HistoryClientService.java:200)
at org.apache.hadoop.mapreduce.v2.api.impl.pb.service.MRClientProtocolPBServiceImpl.getJobReport(MRClientProtocolPBServiceImpl.java:106)
at org.apache.hadoop.yarn.proto.MRClientProtocol$MRClientProtocolService$2.callBlockingMethod(MRClientProtocol.java:187)
at org.apache.hadoop.yarn.ipc.ProtoOverHadoopRpcEngine$Server.call(ProtoOverHadoopRpcEngine.java:344)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1490)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1486)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1157)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1484)
at Local Trace:
org.apache.hadoop.yarn.exceptions.impl.pb.YarnRemoteExceptionPBImpl: User user2 cannot perform operation VIEW_JOB on job_1326396427223_0001
at org.apache.hadoop.yarn.ipc.ProtoOverHadoopRpcEngine$Invoker.invoke(ProtoOverHadoopRpcEngine.java:151)
at $Proxy10.getJobReport(Unknown Source)
at org.apache.hadoop.mapreduce.v2.api.impl.pb.client.MRClientProtocolPBClientImpl.getJobReport(MRClientProtocolPBClientImpl.java:104)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.mapred.ClientServiceDelegate.invoke(ClientServiceDelegate.java:328)
at org.apache.hadoop.mapred.ClientServiceDelegate.getJobStatus(ClientServiceDelegate.java:405)
at org.apache.hadoop.mapred.YARNRunner.getJobStatus(YARNRunner.java:431)
at org.apache.hadoop.mapreduce.Cluster.getJob(Cluster.java:186)
at org.apache.hadoop.mapreduce.tools.CLI.displayJobList(CLI.java:571)
at org.apache.hadoop.mapreduce.tools.CLI.listAllJobs(CLI.java:500)
at org.apache.hadoop.mapreduce.tools.CLI.run(CLI.java:298)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:69)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:83)
at org.apache.hadoop.mapred.JobClient.main(JobClient.java:1209)

The information provided by the command is similar to what is presented on the ResourceManager web UI, and that page has no security.

Marking this as a blocker since many of our automated acceptance tests use this command to obtain the status of jobs running in the cluster.

Issue Links

• is duplicated by: MAPREDUCE-3720 Command line listJobs should not visit each AM