[HDFS-8512] WebHDFS : GETFILESTATUS should return LocatedBlock with storage type info

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 2.8.0, 3.0.0-alpha1
• Component/s: webhdfs
• Hadoop Flags: Reviewed
• Assignee: Xiaoyu Yao
• Reporter: Sumana Sathish
• Created: 01/Jun/15 23:39
• Updated: 30/Aug/16 01:33
• Resolved: 25/Nov/15 21:58

Description

Storage type inside LocatedBlock object is not fully exposed for GETFILESTATUS

$ curl -i "http://127.0.0.1:50070/webhdfs/v1/HOT/FILE1?user.name=xyao&op=GETFILESTATUS"
HTTP/1.1 200 OK
Cache-Control: no-cache
Expires: Wed, 27 May 2015 18:04:13 GMT
Date: Wed, 27 May 2015 18:04:13 GMT
Pragma: no-cache
Expires: Wed, 27 May 2015 18:04:13 GMT
Date: Wed, 27 May 2015 18:04:13 GMT
Pragma: no-cache
Content-Type: application/json
Set-Cookie: hadoop.auth="u=xyao&p=xyao&t=simple&e=1432785853423&s=W4O5kKiYHmzzey4h7I9J9eL9EMY="; Path=/; Expires=Thu, 28-May-2015 04:04:13 GMT; HttpOnly
Transfer-Encoding: chunked
Server: Jetty(6.1.26)
 
{"FileStatus":{"accessTime":1432683737985,"blockSize":134217728,"childrenNum":0,"fileId":16405,"group":"hadoop","length":150318178,"modificationTime":1432683738427,"owner":"xyao","pathSuffix":"","permission":"644","replication":1,"storagePolicy":7,"type":"FILE"}}
 $ curl -i "http://127.0.0.1:50070/webhdfs/v1/HOT/FILE1?user.name=xyao&op=GET_BLOCK_LOCATIONS&offset=0&length=150318178"
HTTP/1.1 200 OK
Cache-Control: no-cache
Expires: Wed, 27 May 2015 18:04:55 GMT
Date: Wed, 27 May 2015 18:04:55 GMT
Pragma: no-cache
Expires: Wed, 27 May 2015 18:04:55 GMT
Date: Wed, 27 May 2015 18:04:55 GMT
Pragma: no-cache
Content-Type: application/json
Set-Cookie: hadoop.auth="u=xyao&p=xyao&t=simple&e=1432785895031&s=TUiaNsCrARAPKz6xrddoQ1eHOXA="; Path=/; Expires=Thu, 28-May-2015 04:04:55 GMT; HttpOnly
Transfer-Encoding: chunked
Server: Jetty(6.1.26)
 
{"LocatedBlocks":{"fileLength":150318178,"isLastBlockComplete":true,"isUnderConstruction":false,"lastLocatedBlock":{"block":{"blockId":1073741847,"blockPoolId":"BP-474445704-192.168.70.1-1432674221011","generationStamp":1023,"numBytes":16100450},"blockToken":{"urlString":"AAAAAA"},"cachedLocations":[],"isCorrupt":false,"locations":[{"adminState":"NORMAL","blockPoolUsed":300670976,"cacheCapacity":0,"cacheUsed":0,"capacity":1996329943040,"dfsUsed":300670976,"hostName":"192.168.70.1","infoPort":50075,"infoSecurePort":0,"ipAddr":"192.168.70.1","ipcPort":50020,"lastUpdate":1432749892058,"lastUpdateMonotonic":1432749892058,"name":"192.168.70.1:50010","networkLocation":"/default-rack","remaining":782138327040,"storageID":"49a30d0f-99f8-4b87-b986-502fe926271a","xceiverCount":1,"xferPort":50010}],"startOffset":134217728},"locatedBlocks":[{"block":{"blockId":1073741846,"blockPoolId":"BP-474445704-192.168.70.1-1432674221011","generationStamp":1022,"numBytes":134217728},"blockToken":{"urlString":"AAAAAA"},"cachedLocations":[],"isCorrupt":false,"locations":[{"adminState":"NORMAL","blockPoolUsed":300670976,"cacheCapacity":0,"cacheUsed":0,"capacity":1996329943040,"dfsUsed":300670976,"hostName":"192.168.70.1","infoPort":50075,"infoSecurePort":0,"ipAddr":"192.168.70.1","ipcPort":50020,"lastUpdate":1432749892058,"lastUpdateMonotonic":1432749892058,"name":"192.168.70.1:50010","networkLocation":"/default-rack","remaining":782138327040,"storageID":"49a30d0f-99f8-4b87-b986-502fe926271a","xceiverCount":1,"xferPort":50010}],"startOffset":0},{"block":{"blockId":1073741847,"blockPoolId":"BP-474445704-192.168.70.1-1432674221011","generationStamp":1023,"numBytes":16100450},"blockToken":{"urlString":"AAAAAA"},"cachedLocations":[],"isCorrupt":false,"locations":[{"adminState":"NORMAL","blockPoolUsed":300670976,"cacheCapacity":0,"cacheUsed":0,"capacity":1996329943040,"dfsUsed":300670976,"hostName":"192.168.70.1","infoPort":50075,"infoSecurePort":0,"ipAddr":"192.168.70.1","ipcPort":50020,"lastUpdate":1432749892058,"lastUpdateMonotonic":1432749892058,"name":"192.168.70.1:50010","networkLocation":"/default-rack","remaining":782138327040,"storageID":"49a30d0f-99f8-4b87-b986-502fe926271a","xceiverCount":1,"xferPort":50010}],"startOffset":134217728}]}}

Issue Links

• is related to: HDFS-7590 Stabilize and document getBlockLocation API in WebHDFS