Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-8512

WebHDFS : GETFILESTATUS should return LocatedBlock with storage type info

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: webhdfs
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Storage type inside LocatedBlock object is not fully exposed for GETFILESTATUS

      $ curl -i "http://127.0.0.1:50070/webhdfs/v1/HOT/FILE1?user.name=xyao&op=GETFILESTATUS"
      HTTP/1.1 200 OK
      Cache-Control: no-cache
      Expires: Wed, 27 May 2015 18:04:13 GMT
      Date: Wed, 27 May 2015 18:04:13 GMT
      Pragma: no-cache
      Expires: Wed, 27 May 2015 18:04:13 GMT
      Date: Wed, 27 May 2015 18:04:13 GMT
      Pragma: no-cache
      Content-Type: application/json
      Set-Cookie: hadoop.auth="u=xyao&p=xyao&t=simple&e=1432785853423&s=W4O5kKiYHmzzey4h7I9J9eL9EMY="; Path=/; Expires=Thu, 28-May-2015 04:04:13 GMT; HttpOnly
      Transfer-Encoding: chunked
      Server: Jetty(6.1.26)
       
      {"FileStatus":{"accessTime":1432683737985,"blockSize":134217728,"childrenNum":0,"fileId":16405,"group":"hadoop","length":150318178,"modificationTime":1432683738427,"owner":"xyao","pathSuffix":"","permission":"644","replication":1,"storagePolicy":7,"type":"FILE"}}
       $ curl -i "http://127.0.0.1:50070/webhdfs/v1/HOT/FILE1?user.name=xyao&op=GET_BLOCK_LOCATIONS&offset=0&length=150318178"
      HTTP/1.1 200 OK
      Cache-Control: no-cache
      Expires: Wed, 27 May 2015 18:04:55 GMT
      Date: Wed, 27 May 2015 18:04:55 GMT
      Pragma: no-cache
      Expires: Wed, 27 May 2015 18:04:55 GMT
      Date: Wed, 27 May 2015 18:04:55 GMT
      Pragma: no-cache
      Content-Type: application/json
      Set-Cookie: hadoop.auth="u=xyao&p=xyao&t=simple&e=1432785895031&s=TUiaNsCrARAPKz6xrddoQ1eHOXA="; Path=/; Expires=Thu, 28-May-2015 04:04:55 GMT; HttpOnly
      Transfer-Encoding: chunked
      Server: Jetty(6.1.26)
       
      {"LocatedBlocks":{"fileLength":150318178,"isLastBlockComplete":true,"isUnderConstruction":false,"lastLocatedBlock":{"block":{"blockId":1073741847,"blockPoolId":"BP-474445704-192.168.70.1-1432674221011","generationStamp":1023,"numBytes":16100450},"blockToken":{"urlString":"AAAAAA"},"cachedLocations":[],"isCorrupt":false,"locations":[{"adminState":"NORMAL","blockPoolUsed":300670976,"cacheCapacity":0,"cacheUsed":0,"capacity":1996329943040,"dfsUsed":300670976,"hostName":"192.168.70.1","infoPort":50075,"infoSecurePort":0,"ipAddr":"192.168.70.1","ipcPort":50020,"lastUpdate":1432749892058,"lastUpdateMonotonic":1432749892058,"name":"192.168.70.1:50010","networkLocation":"/default-rack","remaining":782138327040,"storageID":"49a30d0f-99f8-4b87-b986-502fe926271a","xceiverCount":1,"xferPort":50010}],"startOffset":134217728},"locatedBlocks":[{"block":{"blockId":1073741846,"blockPoolId":"BP-474445704-192.168.70.1-1432674221011","generationStamp":1022,"numBytes":134217728},"blockToken":{"urlString":"AAAAAA"},"cachedLocations":[],"isCorrupt":false,"locations":[{"adminState":"NORMAL","blockPoolUsed":300670976,"cacheCapacity":0,"cacheUsed":0,"capacity":1996329943040,"dfsUsed":300670976,"hostName":"192.168.70.1","infoPort":50075,"infoSecurePort":0,"ipAddr":"192.168.70.1","ipcPort":50020,"lastUpdate":1432749892058,"lastUpdateMonotonic":1432749892058,"name":"192.168.70.1:50010","networkLocation":"/default-rack","remaining":782138327040,"storageID":"49a30d0f-99f8-4b87-b986-502fe926271a","xceiverCount":1,"xferPort":50010}],"startOffset":0},{"block":{"blockId":1073741847,"blockPoolId":"BP-474445704-192.168.70.1-1432674221011","generationStamp":1023,"numBytes":16100450},"blockToken":{"urlString":"AAAAAA"},"cachedLocations":[],"isCorrupt":false,"locations":[{"adminState":"NORMAL","blockPoolUsed":300670976,"cacheCapacity":0,"cacheUsed":0,"capacity":1996329943040,"dfsUsed":300670976,"hostName":"192.168.70.1","infoPort":50075,"infoSecurePort":0,"ipAddr":"192.168.70.1","ipcPort":50020,"lastUpdate":1432749892058,"lastUpdateMonotonic":1432749892058,"name":"192.168.70.1:50010","networkLocation":"/default-rack","remaining":782138327040,"storageID":"49a30d0f-99f8-4b87-b986-502fe926271a","xceiverCount":1,"xferPort":50010}],"startOffset":134217728}]}}
      

        Attachments

        1. HDFS-8512.01.patch
          5 kB
          Xiaoyu Yao
        2. HDFS-8512.00.patch
          6 kB
          Xiaoyu Yao

          Issue Links

            Activity

              People

              • Assignee:
                xyao Xiaoyu Yao
                Reporter:
                ssathish@hortonworks.com Sumana Sathish
              • Votes:
                0 Vote for this issue
                Watchers:
                6 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: