Hadoop Common
  1. Hadoop Common
  2. HADOOP-3889

distcp: Better Error Message should be thrown when accessing source files/directory with no read permission

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 0.17.1
    • Fix Version/s: 0.19.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Incompatible change, Reviewed
    • Release Note:
      Changed DistCp error messages when there is a RemoteException. Changed the corresponding return value from -999 to -3.

      Description

      Distcp: Better Error Message should be thrown when accessing/copying source files/directory with no read permission . So that it would help user to somehow avoid copying that subdirectory or get the owner to change the permissions.

      $ hadoop dfs -lsr /user/scoopy/
      Found 1 items
      /user/scoopy/string1 <r 3> 2054687 2008-08-01 10:41 rwx------ scoopy data

      --> Trying to copy the file which tom don't have permission to read

      $ logname
      tom

      $ hadoop distcp -i hftp://namenode1:8121/user/scoopy/string1 hdfs://namenode2:8121/user/tom/
      08/07/31 17:36:00 INFO util.CopyFiles: srcPaths=[hftp://namenode1:8121/user/scoopy/string1]
      08/07/31 17:36:00 INFO util.CopyFiles: destPath=hdfs://namenode2:8121/user/tom/
      With failures, global counters are inaccurate; consider running with -i
      Copy failed: java.io.IOException: invalid xml directory content
      at org.apache.hadoop.dfs.HftpFileSystem$LsParser.fetchList(HftpFileSystem.java:184)
      at org.apache.hadoop.dfs.HftpFileSystem$LsParser.listStatus(HftpFileSystem.java:199)
      at org.apache.hadoop.dfs.HftpFileSystem$LsParser.listStatus(HftpFileSystem.java:207)
      at org.apache.hadoop.dfs.HftpFileSystem.listStatus(HftpFileSystem.java:214)
      at org.apache.hadoop.util.CopyFiles.setup(CopyFiles.java:933)
      at org.apache.hadoop.util.CopyFiles.copy(CopyFiles.java:603)
      at org.apache.hadoop.util.CopyFiles.run(CopyFiles.java:743)
      at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
      at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
      at org.apache.hadoop.util.CopyFiles.main(CopyFiles.java:763)
      Caused by: org.xml.sax.SAXException: Unrecognized entry: RemoteException
      at org.apache.hadoop.dfs.HftpFileSystem$LsParser.startElement(HftpFileSystem.java:151)
      at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.startElement(AbstractSAXParser.java:501)
      at
      com.sun.org.apache.xerces.internal.parsers.AbstractXMLDocumentParser.emptyElement(AbstractXMLDocumentParser.java:179)
      at
      com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.scanStartElement(XMLNSDocumentScannerImpl.java:377)
      at
      com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(XMLDocumentFragmentScannerImpl.java:2740)
      at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:647)
      at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next(XMLNSDocumentScannerImpl.java:140)
      at
      com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:508)
      at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:807)
      at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:737)
      at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:107)
      at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1205)
      at org.apache.hadoop.dfs.HftpFileSystem$LsParser.fetchList(HftpFileSystem.java:182)
      ... 9 more


      1. 3889_20080813b.patch
        13 kB
        Tsz Wo Nicholas Sze
      2. 3889_20080813.patch
        6 kB
        Tsz Wo Nicholas Sze

        Issue Links

          Activity

          Hide
          Tsz Wo Nicholas Sze added a comment -

          3889_20080813.patch: shows "Permission denied". Need some tests.

          Show
          Tsz Wo Nicholas Sze added a comment - 3889_20080813.patch: shows "Permission denied". Need some tests.
          Hide
          Tsz Wo Nicholas Sze added a comment -

          3889_20080813b.patch: added a test

          Show
          Tsz Wo Nicholas Sze added a comment - 3889_20080813b.patch: added a test
          Hide
          Tsz Wo Nicholas Sze added a comment -

          Passed tests locally except the ones in HADOOP-3950. Try Hudson ...

          Show
          Tsz Wo Nicholas Sze added a comment - Passed tests locally except the ones in HADOOP-3950 . Try Hudson ...
          Hide
          Chris Douglas added a comment -

          +1

          I just committed this. Thanks, Nicholas

          Show
          Chris Douglas added a comment - +1 I just committed this. Thanks, Nicholas
          Hide
          Hudson added a comment -
          Show
          Hudson added a comment - Integrated in Hadoop-trunk #581 (See http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/581/ )

            People

            • Assignee:
              Tsz Wo Nicholas Sze
              Reporter:
              Peeyush Bishnoi
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development