Description
When execute "mvn test -Dtest=TestRMContainerAllocator#testReportedAppProgress", It failed with message:
Caused by: java.io.FileNotFoundException: File /home/yuling.sh/hadoop-common/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/target/org.apache.hadoop.mapreduce.v2.app.TestRMContainerAllocator/appattempt_1392009213299_0001_000001/.staging/job_1392009213299_0001/job.xml does not exist
But in fact, the job.xml exits:
-rw-rw-r-- 1 yuling.sh yuling.sh 65791 2月 10 13:13 /home/yuling.sh/hadoop-common/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/target/org.apache.hadoop.mapreduce.v2.app.TestRMContainerAllocator/yuling.sh/.staging/job_1392009213299_0001/job.xml
See the following code:
public Job submit(Configuration conf, boolean mapSpeculative, boolean reduceSpeculative) throws Exception { String user = conf.get(MRJobConfig.USER_NAME, UserGroupInformation .getCurrentUser().getShortUserName()); conf.set(MRJobConfig.USER_NAME, user); conf.set(MRJobConfig.MR_AM_STAGING_DIR, testAbsPath.toString()); conf.setBoolean(MRJobConfig.MR_AM_CREATE_JH_INTERMEDIATE_BASE_DIR, true); // TODO: fix the bug where the speculator gets events with // not-fully-constructed objects. For now, disable speculative exec conf.setBoolean(MRJobConfig.MAP_SPECULATIVE, mapSpeculative); conf.setBoolean(MRJobConfig.REDUCE_SPECULATIVE, reduceSpeculative); init(conf); start(); DefaultMetricsSystem.shutdown(); Job job = getContext().getAllJobs().values().iterator().next(); if (assignedQueue != null) { job.setQueueName(assignedQueue); } // Write job.xml String jobFile = MRApps.getJobFile(conf, user, TypeConverter.fromYarn(job.getID())); LOG.info("Writing job conf to " + jobFile); new File(jobFile).getParentFile().mkdirs(); conf.writeXml(new FileOutputStream(jobFile)); return job; }
At first, user is "yuling.sh", but the UGI is setted to attemptId at "start();", after that, job.xml write to yuling.sh/.staging/job_1392009213299_0001/job.xml. But when the job is running, MRAppMaster can't find the job.xml at appattempt_1392009213299_0001_000001/.staging/job_1392009213299_0001.