[HADOOP-3889] distcp: Better Error Message should be thrown when accessing source files/directory with no read permission

• Type: Bug
• Status: Closed
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 0.17.1
• Fix Version/s: 0.19.0
• Hadoop Flags: Incompatible change, Reviewed
• Release Note: Changed DistCp error messages when there is a RemoteException. Changed the corresponding return value from -999 to -3.
• Assignee: Tsz-wo Sze
• Reporter: Peeyush Bishnoi
• Created: 01/Aug/08 19:43
• Updated: 02/May/13 02:29
• Resolved: 14/Aug/08 23:07

Description

Distcp: Better Error Message should be thrown when accessing/copying source files/directory with no read permission . So that it would help user to somehow avoid copying that subdirectory or get the owner to change the permissions.

$ hadoop dfs -lsr /user/scoopy/
Found 1 items
/user/scoopy/string1 <r 3> 2054687 2008-08-01 10:41 rwx------ scoopy data

--> Trying to copy the file which tom don't have permission to read

$ logname
tom

$ hadoop distcp -i hftp://namenode1:8121/user/scoopy/string1 hdfs://namenode2:8121/user/tom/
08/07/31 17:36:00 INFO util.CopyFiles: srcPaths=[hftp://namenode1:8121/user/scoopy/string1]
08/07/31 17:36:00 INFO util.CopyFiles: destPath=hdfs://namenode2:8121/user/tom/
With failures, global counters are inaccurate; consider running with -i
Copy failed: java.io.IOException: invalid xml directory content
at org.apache.hadoop.dfs.HftpFileSystem$LsParser.fetchList(HftpFileSystem.java:184)
at org.apache.hadoop.dfs.HftpFileSystem$LsParser.listStatus(HftpFileSystem.java:199)
at org.apache.hadoop.dfs.HftpFileSystem$LsParser.listStatus(HftpFileSystem.java:207)
at org.apache.hadoop.dfs.HftpFileSystem.listStatus(HftpFileSystem.java:214)
at org.apache.hadoop.util.CopyFiles.setup(CopyFiles.java:933)
at org.apache.hadoop.util.CopyFiles.copy(CopyFiles.java:603)
at org.apache.hadoop.util.CopyFiles.run(CopyFiles.java:743)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
at org.apache.hadoop.util.CopyFiles.main(CopyFiles.java:763)
Caused by: org.xml.sax.SAXException: Unrecognized entry: RemoteException
at org.apache.hadoop.dfs.HftpFileSystem$LsParser.startElement(HftpFileSystem.java:151)
at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.startElement(AbstractSAXParser.java:501)
at
com.sun.org.apache.xerces.internal.parsers.AbstractXMLDocumentParser.emptyElement(AbstractXMLDocumentParser.java:179)
at
com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.scanStartElement(XMLNSDocumentScannerImpl.java:377)
at
com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(XMLDocumentFragmentScannerImpl.java:2740)
at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:647)
at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next(XMLNSDocumentScannerImpl.java:140)
at
com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:508)
at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:807)
at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:737)
at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:107)
at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1205)
at org.apache.hadoop.dfs.HftpFileSystem$LsParser.fetchList(HftpFileSystem.java:182)
... 9 more

---

Issue Links

• is depended upon by: HADOOP-3942 Update DistCp documentation
• is related to: HADOOP-3913 Distcp allows copy even if directory permissions are 000.