Details
-
Type:
Bug
-
Status: Closed
-
Priority:
Major
-
Resolution: Fixed
-
Affects Version/s: 2.3.0
-
Fix Version/s: 2.4.0
-
Component/s: jobhistoryserver
-
Labels:None
-
Target Version/s:
-
Hadoop Flags:Reviewed
Description
when job complete, there are WARN complains in the log:
2014-03-19 13:31:10,036 WARN org.apache.hadoop.mapreduce.v2.jobhistory.FileNameIndexUtils: Unable to parse launch time from job history file job_1395204058904_0003-1395206473646-root-test_one_word-1395206966214-4-2-SUCCEEDED-root.test-queue-1395206480070.jhist : java.lang.NumberFormatException: For input string: "queue"
because there is 
in the queue name 'test-queue', we split the job history file name by , and get the ninth item as job start time.
FileNameIndexUtils.java
private static final int JOB_START_TIME_INDEX = 9;
but there is another potential issue:
if I also include '-' in the job name(test_one_world in this case), there are all misunderstand.
Issue Links
- breaks
-
MAPREDUCE-5813
YarnChild does not load job.xml with mapreduce.job.classloader=true
-
- Closed
-
- is related to
-
MAPREDUCE-5815
Fix NPE in TestMRAppMaster
-
- Closed
-
Activity
- All
- Comments
- Work Log
- History
- Activity
- Transitions
Moved to MapReduce project.
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12635609/MAPREDUCE-5805.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 1 new or modified test files.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 javadoc. There were no new javadoc warning messages.
+1 eclipse:eclipse. The patch built with eclipse:eclipse.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed unit tests in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common.
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4443//testReport/
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4443//console
This message is automatically generated.
Looks good, checking this in.
Committed this to trunk, branch-2 and branch-2.4. Thanks Akira!
SUCCESS: Integrated in Hadoop-trunk-Commit #5414 (See https://builds.apache.org/job/Hadoop-trunk-Commit/5414/)
MAPREDUCE-5805. Fixed MapReduce JobHistory encoding of queue-name to escape hyphens and thus avoid parsing errors. Contributed by Akira Ajisaka. (vinodkv: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1582125)
- /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
- /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/jobhistory/FileNameIndexUtils.java
- /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/test/java/org/apache/hadoop/mapreduce/v2/jobhistory/TestFileNameIndexUtils.java
SUCCESS: Integrated in Hadoop-Yarn-trunk #522 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/522/)
MAPREDUCE-5805. Fixed MapReduce JobHistory encoding of queue-name to escape hyphens and thus avoid parsing errors. Contributed by Akira Ajisaka. (vinodkv: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1582125)
- /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
- /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/jobhistory/FileNameIndexUtils.java
- /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/test/java/org/apache/hadoop/mapreduce/v2/jobhistory/TestFileNameIndexUtils.java
FAILURE: Integrated in Hadoop-Mapreduce-trunk #1739 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1739/)
MAPREDUCE-5805. Fixed MapReduce JobHistory encoding of queue-name to escape hyphens and thus avoid parsing errors. Contributed by Akira Ajisaka. (vinodkv: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1582125)
- /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
- /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/jobhistory/FileNameIndexUtils.java
- /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/test/java/org/apache/hadoop/mapreduce/v2/jobhistory/TestFileNameIndexUtils.java
SUCCESS: Integrated in Hadoop-Hdfs-trunk #1714 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1714/)
MAPREDUCE-5805. Fixed MapReduce JobHistory encoding of queue-name to escape hyphens and thus avoid parsing errors. Contributed by Akira Ajisaka. (vinodkv: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1582125)
- /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
- /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/jobhistory/FileNameIndexUtils.java
- /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/test/java/org/apache/hadoop/mapreduce/v2/jobhistory/TestFileNameIndexUtils.java
I looked around the code and found that user name and job name are escaped, but queue name is not escaped. I'll create a patch to escape queue name shortly.