[HADOOP-11549] flaky test detection tool failed to handle special control characters in test result

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.7.0
• Fix Version/s: 2.7.0
• Component/s: tools
• Target Version/s: 2.7.0
• Hadoop Flags: Reviewed
• Assignee: Yongjun Zhang
• Reporter: Yongjun Zhang
• Created: 04/Feb/15 19:33
• Updated: 10/Apr/15 20:04
• Resolved: 04/Feb/15 22:20

Description

When running the tool from HADOOP-11045 on latest Hadoop-hdfs-trunk job, I'm seeing a problem here:

[yzhang@localhost jenkinsftf]$ ./determine-flaky-tests-hadoop.py -j Hadoop-hdfs-trunk 
****Recently FAILED builds in url: https://builds.apache.org/job/Hadoop-hdfs-trunk
    THERE ARE 5 builds (out of 6) that have failed tests in the past 14 days, as listed below:

===>https://builds.apache.org/job/Hadoop-hdfs-trunk/2026/testReport (2015-02-04 03:30:00)
    Could not open testReport, check https://builds.apache.org/job/Hadoop-hdfs-trunk/2026/Console for why it was reported failed
...

I saw that the testReport can actually be opened from browser. After looking, I found that HDFS-7287 fix added the following test code:

//Create a directory whose name should be escaped in XML
Path invalidXMLDir = new Path("/dirContainingInvalidXMLChar\u0000here");
hdfs.mkdirs(invalidXMLDir);
...

And the output from this code caused the tool to choke.

I found a solution here and I'm attaching a patch.

Issue Links

• is related to: HADOOP-11045 Introducing a tool to detect flaky tests of hadoop jenkins test job