[HDFS-12831] HDFS throws FileNotFoundException on getFileBlockLocations(path-to-directory)

• Type: Bug
• Status: Patch Available
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 3.1.2
• Component/s: namenode
• Assignee: Hemanth Boyina
• Reporter: Steve Loughran
• Created: 17/Nov/17 10:17
• Updated: 23/Sep/19 17:35

Description

The HDFS implementation of getFileBlockLocations(path, offset, len) throws an exception if the path references a directory.

The base implementation (and all other filesystems) just return an empty array, something implemented in getFileBlockLocations(filestatsus, offset, len); something written up in filesystem.md as the correct behaviour.

1. has been shown to break things: SPARK-14959
2. there's no contract tests for these APIs; shows up in HADOOP-15044.
3. even if this is considered a wontfix, it should raise something like PathIsDirectoryException rather than FNFE

Issue Links

• breaks: SPARK-14959 ​Problem Reading partitioned ORC or Parquet files
• is related to: HADOOP-14943 Add common getFileBlockLocations() emulation for object stores, including S3A