Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-2316

[umbrella] WebHDFS: a complete FileSystem implementation for accessing HDFS over HTTP

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Type: New Feature
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.23.1, 1.0.0
    • Component/s: webhdfs
    • Labels:
    • Release Note:
      Hide
      Provide WebHDFS as a complete FileSystem implementation for accessing HDFS over HTTP.
      Previous hftp feature was a read-only FileSystem and does not provide "write" accesses.
      Show
      Provide WebHDFS as a complete FileSystem implementation for accessing HDFS over HTTP. Previous hftp feature was a read-only FileSystem and does not provide "write" accesses.

      Description

      We current have hftp for accessing HDFS over HTTP. However, hftp is a read-only FileSystem and does not provide "write" accesses.

      In HDFS-2284, we propose to have WebHDFS for providing a complete FileSystem implementation for accessing HDFS over HTTP. The is the umbrella JIRA for the tasks.

        Attachments

        1. test-webhdfs-0.20s
          0.5 kB
          Tsz-wo Sze
        2. test-webhdfs
          0.4 kB
          Tsz-wo Sze
        3. WebHdfsAPI20111111.pdf
          211 kB
          Tsz-wo Sze
        4. WebHdfsAPI20111103.pdf
          183 kB
          Tsz-wo Sze
        5. WebHdfsAPI20111020.pdf
          141 kB
          Tsz-wo Sze

        Issue Links

        1.
        Write Http access to HDFS Sub-task Closed Tsz-wo Sze Actions
        2.
        Read access to HDFS using HTTP REST Sub-task Closed Tsz-wo Sze Actions
        3.
        Provide authentication to webhdfs using SPNEGO Sub-task Closed Tsz-wo Sze Actions
        4.
        Configuration option to enable/disable webhdfs. Sub-task Closed Jitendra Nath Pandey Actions
        5.
        Support getFileBlockLocations and getDelegationToken in webhdfs Sub-task Closed Tsz-wo Sze Actions
        6.
        Support getContentSummary and getFileChecksum in webhdfs Sub-task Closed Tsz-wo Sze Actions
        7.
        webhdfs: support case insensitive query parameter names Sub-task Closed Tsz-wo Sze Actions
        8.
        Support delegation token renewal in webhdfs Sub-task Closed Tsz-wo Sze Actions
        9.
        webhdfs api parameter validation should be better Sub-task Closed Tsz-wo Sze Actions
        10.
        webhdfs returns two content-type headers Sub-task Closed Tsz-wo Sze Actions
        11.
        webhdfs open an invalid path leads to a 500 which states a npe, we should return a 404 with appropriate error message Sub-task Closed Tsz-wo Sze Actions
        12.
        webhdfs liststatus json does not convert to a valid xml document Sub-task Closed Tsz-wo Sze Actions
        13.
        webhdfs setpermission,settimes,setowner api does not return any content upon success Sub-task Resolved Unassigned Actions
        14.
        webhdfs GETFILESTATUS calls shows the localName as empty Sub-task Resolved Unassigned Actions
        15.
        tail using a webhdfs uri throws an error Sub-task Closed Tsz-wo Sze Actions
        16.
        webhdfs mkdirs api call creates path with 777 permission, we should default it to 755 Sub-task Closed Tsz-wo Sze Actions
        17.
        distcp with a webhdfs uri on a secure cluster fails Sub-task Closed Jitendra Nath Pandey Actions
        18.
        [webhdfs] When Getting the file using OP=OPEN with DN http address, ESTABLISHED sockets are growing. Sub-task Closed Uma Maheswara Rao G Actions
        19.
        add version prefix and root methods to webhdfs Sub-task Closed Tsz-wo Sze Actions
        20.
        webhdfs api's should return a root element in the json response Sub-task Closed Tsz-wo Sze Actions
        21.
        webhdfs liststatus json response is not correct Sub-task Closed Suresh Srinivas Actions
        22.
        webhdfs setreplication api should return a 403 when called on a directory Sub-task Closed Tsz-wo Sze Actions
        23.
        The renewer in NamenodeWebHdfsMethods.generateDelegationToken(..) is not used Sub-task Closed Tsz-wo Sze Actions
        24.
        webhdfs throws a npe when ugi is null from getDelegationToken Sub-task Closed Tsz-wo Sze Actions
        25.
        Remove the use of Range header from webhdfs Sub-task Closed Tsz-wo Sze Actions
        26.
        webhdfs rest call to a secure dn fails when a token is sent Sub-task Closed Tsz-wo Sze Actions
        27.
        Support doAs and GETHOMEDIRECTORY in webhdfs Sub-task Closed Tsz-wo Sze Actions
        28.
        Change WebHdfsFileSystem to two-step create/append Sub-task Closed Tsz-wo Sze Actions

          Activity

            People

            • Assignee:
              szetszwo Tsz-wo Sze
              Reporter:
              szetszwo Tsz-wo Sze

              Dates

              • Created:
                Updated:
                Resolved:

                Issue deployment