Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-4099

HFTP interface compatibility with older releases broken

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: 0.19.0
    • Fix Version/s: 0.19.0
    • Component/s: fs
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Using current trunk (Revision 692556) build and trying to distcp from a 0.18.0 cluster via HFTP, got the following NullPointerException. Seems like on line 165 in HftpFileSystem.java, a potential null return value isn't checked before using.

      coursemud-lm:bin kan$ ./hadoop distcp hftp://ucdev18.inktomisearch.com:50070/output/part-00000 /copied/part-00001
      08/09/05 19:00:10 INFO tools.DistCp: srcPaths=[hftp://ucdev18.inktomisearch.com:50070/output/part-00000]
      08/09/05 19:00:10 INFO tools.DistCp: destPath=/copied/part-00001
      With failures, global counters are inaccurate; consider running with -i
      Copy failed: java.lang.NullPointerException
      at java.text.SimpleDateFormat.parse(SimpleDateFormat.java:1215)
      at java.text.DateFormat.parse(DateFormat.java:335)
      at org.apache.hadoop.hdfs.HftpFileSystem$LsParser.startElement(HftpFileSystem.java:165)
      at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.startElement(AbstractSAXParser.java:501)
      at com.sun.org.apache.xerces.internal.parsers.AbstractXMLDocumentParser.emptyElement(AbstractXMLDocumentParser.java:179)
      at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.scanStartElement(XMLNSDocumentScannerImpl.java:377)
      at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(XMLDocumentFragmentScannerImpl.java:2740)
      at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:647)
      at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next(XMLNSDocumentScannerImpl.java:140)
      at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:508)
      at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:807)
      at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:737)
      at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:107)
      at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1205)
      at org.apache.hadoop.hdfs.HftpFileSystem$LsParser.fetchList(HftpFileSystem.java:194)
      at org.apache.hadoop.hdfs.HftpFileSystem$LsParser.getFileStatus(HftpFileSystem.java:205)
      at org.apache.hadoop.hdfs.HftpFileSystem.getFileStatus(HftpFileSystem.java:234)
      at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:649)
      at org.apache.hadoop.tools.DistCp.checkSrcPath(DistCp.java:614)
      at org.apache.hadoop.tools.DistCp.copy(DistCp.java:631)
      at org.apache.hadoop.tools.DistCp.run(DistCp.java:838)
      at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
      at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
      at org.apache.hadoop.tools.DistCp.main(DistCp.java:865)

        Attachments

        1. hftpAccessTime.patch
          0.7 kB
          Dhruba Borthakur

          Issue Links

            Activity

              People

              • Assignee:
                dhruba Dhruba Borthakur
                Reporter:
                kzhang Kan Zhang
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: