Hadoop Common
  1. Hadoop Common
  2. HADOOP-3889

distcp: Better Error Message should be thrown when accessing source files/directory with no read permission

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 0.17.1
    • Fix Version/s: 0.19.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Incompatible change, Reviewed
    • Release Note:
      Changed DistCp error messages when there is a RemoteException. Changed the corresponding return value from -999 to -3.

      Description

      Distcp: Better Error Message should be thrown when accessing/copying source files/directory with no read permission . So that it would help user to somehow avoid copying that subdirectory or get the owner to change the permissions.

      $ hadoop dfs -lsr /user/scoopy/
      Found 1 items
      /user/scoopy/string1 <r 3> 2054687 2008-08-01 10:41 rwx------ scoopy data

      --> Trying to copy the file which tom don't have permission to read

      $ logname
      tom

      $ hadoop distcp -i hftp://namenode1:8121/user/scoopy/string1 hdfs://namenode2:8121/user/tom/
      08/07/31 17:36:00 INFO util.CopyFiles: srcPaths=[hftp://namenode1:8121/user/scoopy/string1]
      08/07/31 17:36:00 INFO util.CopyFiles: destPath=hdfs://namenode2:8121/user/tom/
      With failures, global counters are inaccurate; consider running with -i
      Copy failed: java.io.IOException: invalid xml directory content
      at org.apache.hadoop.dfs.HftpFileSystem$LsParser.fetchList(HftpFileSystem.java:184)
      at org.apache.hadoop.dfs.HftpFileSystem$LsParser.listStatus(HftpFileSystem.java:199)
      at org.apache.hadoop.dfs.HftpFileSystem$LsParser.listStatus(HftpFileSystem.java:207)
      at org.apache.hadoop.dfs.HftpFileSystem.listStatus(HftpFileSystem.java:214)
      at org.apache.hadoop.util.CopyFiles.setup(CopyFiles.java:933)
      at org.apache.hadoop.util.CopyFiles.copy(CopyFiles.java:603)
      at org.apache.hadoop.util.CopyFiles.run(CopyFiles.java:743)
      at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
      at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
      at org.apache.hadoop.util.CopyFiles.main(CopyFiles.java:763)
      Caused by: org.xml.sax.SAXException: Unrecognized entry: RemoteException
      at org.apache.hadoop.dfs.HftpFileSystem$LsParser.startElement(HftpFileSystem.java:151)
      at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.startElement(AbstractSAXParser.java:501)
      at
      com.sun.org.apache.xerces.internal.parsers.AbstractXMLDocumentParser.emptyElement(AbstractXMLDocumentParser.java:179)
      at
      com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.scanStartElement(XMLNSDocumentScannerImpl.java:377)
      at
      com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(XMLDocumentFragmentScannerImpl.java:2740)
      at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:647)
      at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next(XMLNSDocumentScannerImpl.java:140)
      at
      com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:508)
      at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:807)
      at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:737)
      at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:107)
      at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1205)
      at org.apache.hadoop.dfs.HftpFileSystem$LsParser.fetchList(HftpFileSystem.java:182)
      ... 9 more


      1. 3889_20080813b.patch
        13 kB
        Tsz Wo Nicholas Sze
      2. 3889_20080813.patch
        6 kB
        Tsz Wo Nicholas Sze

        Issue Links

          Activity

          Peeyush Bishnoi created issue -
          Tsz Wo Nicholas Sze made changes -
          Field Original Value New Value
          Link This issue is related to HADOOP-3913 [ HADOOP-3913 ]
          Tsz Wo Nicholas Sze made changes -
          Attachment 3889_20080813.patch [ 12388185 ]
          Tsz Wo Nicholas Sze made changes -
          Attachment 3889_20080813b.patch [ 12388197 ]
          Tsz Wo Nicholas Sze made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Assignee Tsz Wo (Nicholas), SZE [ szetszwo ]
          Tsz Wo Nicholas Sze made changes -
          Link This issue blocks HADOOP-3942 [ HADOOP-3942 ]
          Tsz Wo Nicholas Sze made changes -
          Release Note Changed DistCp error messages when there is a RemoteException. Also changed the corresponding return value from -999 to -3.
          Hadoop Flags [Incompatible change]
          Component/s tools/distcp [ 12312387 ]
          Component/s dfs [ 12310710 ]
          Chris Douglas made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Resolution Fixed [ 1 ]
          Hadoop Flags [Incompatible change] [Incompatible change, Reviewed]
          Fix Version/s 0.19.0 [ 12313211 ]
          Robert Chansler made changes -
          Hadoop Flags [Reviewed, Incompatible change] [Incompatible change, Reviewed]
          Release Note Changed DistCp error messages when there is a RemoteException. Also changed the corresponding return value from -999 to -3. Changed DistCp error messages when there is a RemoteException. Changed the corresponding return value from -999 to -3.
          Nigel Daley made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Owen O'Malley made changes -
          Component/s tools/distcp [ 12312387 ]
          Gavin made changes -
          Link This issue blocks HADOOP-3942 [ HADOOP-3942 ]
          Gavin made changes -
          Link This issue is depended upon by HADOOP-3942 [ HADOOP-3942 ]

            People

            • Assignee:
              Tsz Wo Nicholas Sze
              Reporter:
              Peeyush Bishnoi
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development