Details
-
Bug
-
Status: Patch Available
-
Major
-
Resolution: Unresolved
-
2.7.1
-
None
-
None
-
None
Description
I am running the test suite against a dynamic race detector called RV-Predict. Here is a race report that I got:
Data race on field org.apache.hadoop.mapred.JobConf.credentials: {{{ Concurrent read in thread T327 (locks held: {}) ----> at org.apache.hadoop.mapred.JobConf.getCredentials(JobConf.java:505) at org.apache.hadoop.mapreduce.task.JobContextImpl.<init>(JobContextImpl.java:70) at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:524) T327 is created by T22 at org.apache.hadoop.mapred.LocalJobRunner$Job.<init>(LocalJobRunner.java:218) Concurrent write in thread T22 (locks held: {Monitor@496c673a, Monitor@496319b0}) ----> at org.apache.hadoop.mapred.JobConf.setCredentials(JobConf.java:510) at org.apache.hadoop.mapred.LocalJobRunner.submitJob(LocalJobRunner.java:787) at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:241) at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1290) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1669) at org.apache.hadoop.mapreduce.Job.submit(Job.java:1287) at org.apache.hadoop.mapreduce.lib.jobcontrol.ControlledJob.submit(ControlledJob.java:335) locked Monitor@496319b0 at org.apache.hadoop.mapreduce.lib.jobcontrol.ControlledJob.submit(ControlledJob.java:n/a) at org.apache.hadoop.mapreduce.lib.jobcontrol.JobControl.run(JobControl.java:245) locked Monitor@496c673a at org.apache.hadoop.mapreduce.lib.jobcontrol.JobControl.run(JobControl.java:229) T22 is created by T1 at org.apache.hadoop.mapred.jobcontrol.TestJobControl.doJobControlTest(TestJobControl.java:111) }}}
In the source code of org.apache.hadoop.mapreduce.JobStatus.submitJob function, we have the following lines:
Job job = new Job(JobID.downgrade(jobid), jobSubmitDir);
job.job.setCredentials(credentials);
It looks a bit suspicious: Job extends thread and at the end of its constructor it starts a new thread which creates a new instance of JobContextImpl which reads credentials. However, the first thread concurrently sets credentials after a creating the Job instance.