Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-11156

Add new op GETFILEBLOCKLOCATIONS to WebHDFS REST API

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 2.7.3
    • 3.0.0-alpha2
    • webhdfs
    • None

    Description

      Following webhdfs REST API

      http://<HOST>:<PORT>/webhdfs/v1/<PATH>?op=GET_BLOCK_LOCATIONS&offset=0&length=1
      

      will get a response like

      {
        "LocatedBlocks" : {
          "fileLength" : 1073741824,
          "isLastBlockComplete" : true,
          "isUnderConstruction" : false,
          "lastLocatedBlock" : { ... },
          "locatedBlocks" : [ {...} ]
        }
      }
      

      This represents for o.a.h.h.p.LocatedBlocks. However according to FileSystem API,

      public BlockLocation[] getFileBlockLocations(Path p, long start, long len)
      

      clients would expect an array of BlockLocation. This mismatch should be fixed. Marked as Incompatible change as this will change the output of the GET_BLOCK_LOCATIONS API.

      Attachments

        1. BlockLocationProperties_JSON_Schema.jpg
          283 kB
          Weiwei Yang
        2. BlockLocations_JSON_Schema.jpg
          167 kB
          Weiwei Yang
        3. FileStatuses_JSON_Schema.jpg
          147 kB
          Weiwei Yang
        4. HDFS-11156.01.patch
          15 kB
          Weiwei Yang
        5. HDFS-11156.02.patch
          16 kB
          Weiwei Yang
        6. HDFS-11156.03.patch
          16 kB
          Weiwei Yang
        7. HDFS-11156.04.patch
          15 kB
          Weiwei Yang
        8. HDFS-11156.05.patch
          11 kB
          Weiwei Yang
        9. HDFS-11156.06.patch
          12 kB
          Weiwei Yang
        10. HDFS-11156.07.patch
          20 kB
          Weiwei Yang
        11. HDFS-11156.08.patch
          29 kB
          Weiwei Yang
        12. HDFS-11156.09.patch
          29 kB
          Weiwei Yang
        13. HDFS-11156.10.patch
          30 kB
          Weiwei Yang
        14. HDFS-11156.11.patch
          30 kB
          Weiwei Yang
        15. HDFS-11156.12.patch
          30 kB
          Weiwei Yang
        16. HDFS-11156.13.patch
          31 kB
          Weiwei Yang
        17. HDFS-11156.14.patch
          31 kB
          Weiwei Yang
        18. HDFS-11156.15.patch
          33 kB
          Weiwei Yang
        19. HDFS-11156.16.patch
          33 kB
          Weiwei Yang
        20. HDFS-11156-branch-2.01.patch
          33 kB
          Weiwei Yang
        21. Output_JSON_format_v10.jpg
          220 kB
          Weiwei Yang
        22. SampleResponse_JSON.jpg
          248 kB
          Weiwei Yang

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            cheersyang Weiwei Yang
            cheersyang Weiwei Yang
            Votes:
            0 Vote for this issue
            Watchers:
            12 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Issue deployment