Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-3889

distcp: Better Error Message should be thrown when accessing source files/directory with no read permission

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Minor
    • Resolution: Fixed
    • 0.17.1
    • 0.19.0
    • None
    • None
    • Incompatible change, Reviewed
    • Changed DistCp error messages when there is a RemoteException. Changed the corresponding return value from -999 to -3.

    Description

      Distcp: Better Error Message should be thrown when accessing/copying source files/directory with no read permission . So that it would help user to somehow avoid copying that subdirectory or get the owner to change the permissions.

      $ hadoop dfs -lsr /user/scoopy/
      Found 1 items
      /user/scoopy/string1 <r 3> 2054687 2008-08-01 10:41 rwx------ scoopy data

      --> Trying to copy the file which tom don't have permission to read

      $ logname
      tom

      $ hadoop distcp -i hftp://namenode1:8121/user/scoopy/string1 hdfs://namenode2:8121/user/tom/
      08/07/31 17:36:00 INFO util.CopyFiles: srcPaths=[hftp://namenode1:8121/user/scoopy/string1]
      08/07/31 17:36:00 INFO util.CopyFiles: destPath=hdfs://namenode2:8121/user/tom/
      With failures, global counters are inaccurate; consider running with -i
      Copy failed: java.io.IOException: invalid xml directory content
      at org.apache.hadoop.dfs.HftpFileSystem$LsParser.fetchList(HftpFileSystem.java:184)
      at org.apache.hadoop.dfs.HftpFileSystem$LsParser.listStatus(HftpFileSystem.java:199)
      at org.apache.hadoop.dfs.HftpFileSystem$LsParser.listStatus(HftpFileSystem.java:207)
      at org.apache.hadoop.dfs.HftpFileSystem.listStatus(HftpFileSystem.java:214)
      at org.apache.hadoop.util.CopyFiles.setup(CopyFiles.java:933)
      at org.apache.hadoop.util.CopyFiles.copy(CopyFiles.java:603)
      at org.apache.hadoop.util.CopyFiles.run(CopyFiles.java:743)
      at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
      at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
      at org.apache.hadoop.util.CopyFiles.main(CopyFiles.java:763)
      Caused by: org.xml.sax.SAXException: Unrecognized entry: RemoteException
      at org.apache.hadoop.dfs.HftpFileSystem$LsParser.startElement(HftpFileSystem.java:151)
      at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.startElement(AbstractSAXParser.java:501)
      at
      com.sun.org.apache.xerces.internal.parsers.AbstractXMLDocumentParser.emptyElement(AbstractXMLDocumentParser.java:179)
      at
      com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.scanStartElement(XMLNSDocumentScannerImpl.java:377)
      at
      com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(XMLDocumentFragmentScannerImpl.java:2740)
      at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:647)
      at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next(XMLNSDocumentScannerImpl.java:140)
      at
      com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:508)
      at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:807)
      at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:737)
      at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:107)
      at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1205)
      at org.apache.hadoop.dfs.HftpFileSystem$LsParser.fetchList(HftpFileSystem.java:182)
      ... 9 more


      Attachments

        1. 3889_20080813.patch
          6 kB
          Tsz-wo Sze
        2. 3889_20080813b.patch
          13 kB
          Tsz-wo Sze

        Issue Links

          Activity

            People

              szetszwo Tsz-wo Sze
              peeyushb Peeyush Bishnoi
              Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: