[HDFS-11156] Add new op GETFILEBLOCKLOCATIONS to WebHDFS REST API

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.7.3
• Fix Version/s: 3.0.0-alpha2
• Component/s: webhdfs
• Target Version/s: 2.8.0, 3.0.0-alpha2
• Assignee: Weiwei Yang
• Reporter: Weiwei Yang
• Created: 18/Nov/16 12:59
• Updated: 17/Feb/20 21:52
• Resolved: 15/Sep/17 01:56

Description

Following webhdfs REST API

http://<HOST>:<PORT>/webhdfs/v1/<PATH>?op=GET_BLOCK_LOCATIONS&offset=0&length=1

will get a response like

{
  "LocatedBlocks" : {
    "fileLength" : 1073741824,
    "isLastBlockComplete" : true,
    "isUnderConstruction" : false,
    "lastLocatedBlock" : { ... },
    "locatedBlocks" : [ {...} ]
  }
}

This represents for o.a.h.h.p.LocatedBlocks. However according to FileSystem API,

public BlockLocation[] getFileBlockLocations(Path p, long start, long len)

clients would expect an array of BlockLocation. This mismatch should be fixed. Marked as Incompatible change as this will change the output of the GET_BLOCK_LOCATIONS API.

Issue Links

• breaks: HDFS-12442 WebHdfsFileSystem#getFileBlockLocations will always return BlockLocation#corrupt as false
• breaks: HDFS-12459 Fix revert: Add new op GETFILEBLOCKLOCATIONS to WebHDFS REST API
• breaks: HDFS-13351 Revert HDFS-11156 from branch-2/branch-2.8
• contains: HDFS-11166 Add webhdfs GETFILEBLOCKLOCATIONS document
• is related to: HDFS-12457 Revert HDFS-11156 Add new op GETFILEBLOCKLOCATIONS to WebHDFS REST API
• is related to: HDFS-6874 Add GETFILEBLOCKLOCATIONS operation to HttpFS
• relates to: HDFS-14477 No enum constant Operation.GET_BLOCK_LOCATIONS