[HADOOP-4099] HFTP interface compatibility with older releases broken

• Type: Bug
• Status: Closed
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 0.19.0
• Fix Version/s: 0.19.0
• Component/s: fs
• Hadoop Flags: Reviewed
• Assignee: Dhruba Borthakur
• Reporter: Kan Zhang
• Created: 06/Sep/08 02:05
• Updated: 20/Nov/08 23:38
• Resolved: 13/Sep/08 08:55

Description

Using current trunk (Revision 692556) build and trying to distcp from a 0.18.0 cluster via HFTP, got the following NullPointerException. Seems like on line 165 in HftpFileSystem.java, a potential null return value isn't checked before using.

coursemud-lm:bin kan$ ./hadoop distcp hftp://ucdev18.inktomisearch.com:50070/output/part-00000 /copied/part-00001
08/09/05 19:00:10 INFO tools.DistCp: srcPaths=[hftp://ucdev18.inktomisearch.com:50070/output/part-00000]
08/09/05 19:00:10 INFO tools.DistCp: destPath=/copied/part-00001
With failures, global counters are inaccurate; consider running with -i
Copy failed: java.lang.NullPointerException
at java.text.SimpleDateFormat.parse(SimpleDateFormat.java:1215)
at java.text.DateFormat.parse(DateFormat.java:335)
at org.apache.hadoop.hdfs.HftpFileSystem$LsParser.startElement(HftpFileSystem.java:165)
at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.startElement(AbstractSAXParser.java:501)
at com.sun.org.apache.xerces.internal.parsers.AbstractXMLDocumentParser.emptyElement(AbstractXMLDocumentParser.java:179)
at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.scanStartElement(XMLNSDocumentScannerImpl.java:377)
at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(XMLDocumentFragmentScannerImpl.java:2740)
at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:647)
at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next(XMLNSDocumentScannerImpl.java:140)
at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:508)
at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:807)
at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:737)
at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:107)
at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1205)
at org.apache.hadoop.hdfs.HftpFileSystem$LsParser.fetchList(HftpFileSystem.java:194)
at org.apache.hadoop.hdfs.HftpFileSystem$LsParser.getFileStatus(HftpFileSystem.java:205)
at org.apache.hadoop.hdfs.HftpFileSystem.getFileStatus(HftpFileSystem.java:234)
at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:649)
at org.apache.hadoop.tools.DistCp.checkSrcPath(DistCp.java:614)
at org.apache.hadoop.tools.DistCp.copy(DistCp.java:631)
at org.apache.hadoop.tools.DistCp.run(DistCp.java:838)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
at org.apache.hadoop.tools.DistCp.main(DistCp.java:865)

Issue Links

• relates to: HADOOP-1869 access times of HDFS files