[HADOOP-5836] Bug in S3N handling of directory markers using an object with a trailing "/" causes jobs to fail

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.18.3
• Fix Version/s: 1.1.0, 0.21.0
• Component/s: fs/s3
• Target Version/s: 1.1.0
• Hadoop Flags: Reviewed
• Assignee: Ian Nowland
• Reporter: Ian Nowland
• Created: 14/May/09 17:56
• Updated: 15/Mar/12 23:55
• Resolved: 03/Jun/09 10:43

Description

Some tools which upload to S3 and use a object terminated with a "/" as a directory marker, for instance "s3n://mybucket/mydir/". If asked to iterate that "directory" via listStatus(), then the current code will return an empty file "", which the InputFormatter happily assigns to a split, and which later causes a task to fail, and probably the job to fail.