[HDFS-4235] when outputting XML, OfflineEditsViewer can't handle some edits containing non-ASCII strings

• Type: Bug
• Status: Closed
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 2.0.3-alpha
• Fix Version/s: 2.1.0-beta
• Target Version/s: 2.1.0-beta
• Hadoop Flags: Reviewed
• Assignee: Colin McCabe
• Reporter: Colin McCabe
• Created: 29/Nov/12 01:21
• Updated: 27/Aug/13 22:08
• Resolved: 26/Feb/13 00:53

Description

It seems that when outputting XML, OfflineEditsViewer can't handle some edits containing non-ASCII strings.

Example:

cmccabe@keter:/h> ./bin/hdfs oev -i ~/Downloads/current2/edits -o /tmp/u.xml                                         
17:11:24,662 ERROR OfflineEditsBinaryLoader:82 - Got IOException at position 10593
Encountered exception. Exiting: SAX error: The character '�' is an invalid XML character
java.io.IOException: SAX error: The character '�' is an invalid XML character
        at org.apache.hadoop.hdfs.tools.offlineEditsViewer.XmlEditsVisitor.visitOp(XmlEditsVisitor.java:119)
        at org.apache.hadoop.hdfs.tools.offlineEditsViewer.OfflineEditsBinaryLoader.loadEdits(OfflineEditsBinaryLoader.java:78)
        at org.apache.hadoop.hdfs.tools.offlineEditsViewer.OfflineEditsViewer.go(OfflineEditsViewer.java:142)
        at org.apache.hadoop.hdfs.tools.offlineEditsViewer.OfflineEditsViewer.run(OfflineEditsViewer.java:228)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
        at org.apache.hadoop.hdfs.tools.offlineEditsViewer.OfflineEditsViewer.main(OfflineEditsViewer.java:237)

Probably, we forgot to properly escape and/or re-encode a filename before putting it into the XML. The other processors (stats, binary) don't have this problem, so it is purely an XML encoding issue.