Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-2316

[umbrella] WebHDFS: a complete FileSystem implementation for accessing HDFS over HTTP

    Details

    • Type: New Feature New Feature
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.23.1, 1.0.0
    • Component/s: webhdfs
    • Labels:
    • Release Note:
      Hide
      Provide WebHDFS as a complete FileSystem implementation for accessing HDFS over HTTP.
      Previous hftp feature was a read-only FileSystem and does not provide "write" accesses.
      Show
      Provide WebHDFS as a complete FileSystem implementation for accessing HDFS over HTTP. Previous hftp feature was a read-only FileSystem and does not provide "write" accesses.

      Description

      We current have hftp for accessing HDFS over HTTP. However, hftp is a read-only FileSystem and does not provide "write" accesses.

      In HDFS-2284, we propose to have WebHDFS for providing a complete FileSystem implementation for accessing HDFS over HTTP. The is the umbrella JIRA for the tasks.

      1. WebHdfsAPI20111020.pdf
        141 kB
        Tsz Wo Nicholas Sze
      2. WebHdfsAPI20111103.pdf
        183 kB
        Tsz Wo Nicholas Sze
      3. WebHdfsAPI20111111.pdf
        211 kB
        Tsz Wo Nicholas Sze
      4. test-webhdfs
        0.4 kB
        Tsz Wo Nicholas Sze
      5. test-webhdfs-0.20s
        0.5 kB
        Tsz Wo Nicholas Sze

        Issue Links

        1.
        Write Http access to HDFS Sub-task Closed Tsz Wo Nicholas Sze
         
        2.
        Read access to HDFS using HTTP REST Sub-task Closed Tsz Wo Nicholas Sze
         
        3.
        Provide authentication to webhdfs using SPNEGO Sub-task Closed Tsz Wo Nicholas Sze
         
        4.
        Configuration option to enable/disable webhdfs. Sub-task Closed Jitendra Nath Pandey
         
        5.
        Support getFileBlockLocations and getDelegationToken in webhdfs Sub-task Closed Tsz Wo Nicholas Sze
         
        6.
        Support getContentSummary and getFileChecksum in webhdfs Sub-task Closed Tsz Wo Nicholas Sze
         
        7.
        webhdfs: support case insensitive query parameter names Sub-task Closed Tsz Wo Nicholas Sze
         
        8.
        Support delegation token renewal in webhdfs Sub-task Closed Tsz Wo Nicholas Sze
         
        9.
        webhdfs api parameter validation should be better Sub-task Closed Tsz Wo Nicholas Sze
         
        10.
        webhdfs returns two content-type headers Sub-task Closed Tsz Wo Nicholas Sze
         
        11.
        webhdfs open an invalid path leads to a 500 which states a npe, we should return a 404 with appropriate error message Sub-task Closed Tsz Wo Nicholas Sze
         
        12.
        webhdfs liststatus json does not convert to a valid xml document Sub-task Closed Tsz Wo Nicholas Sze
         
        13.
        webhdfs setpermission,settimes,setowner api does not return any content upon success Sub-task Resolved Unassigned
         
        14.
        webhdfs GETFILESTATUS calls shows the localName as empty Sub-task Resolved Unassigned
         
        15.
        tail using a webhdfs uri throws an error Sub-task Closed Tsz Wo Nicholas Sze
         
        16.
        webhdfs mkdirs api call creates path with 777 permission, we should default it to 755 Sub-task Closed Tsz Wo Nicholas Sze
         
        17.
        distcp with a webhdfs uri on a secure cluster fails Sub-task Closed Jitendra Nath Pandey
         
        18.
        [webhdfs] When Getting the file using OP=OPEN with DN http address, ESTABLISHED sockets are growing. Sub-task Closed Uma Maheswara Rao G
         
        19.
        add version prefix and root methods to webhdfs Sub-task Closed Tsz Wo Nicholas Sze
         
        20.
        webhdfs api's should return a root element in the json response Sub-task Closed Tsz Wo Nicholas Sze
         
        21.
        webhdfs liststatus json response is not correct Sub-task Closed Suresh Srinivas
         
        22.
        webhdfs setreplication api should return a 403 when called on a directory Sub-task Closed Tsz Wo Nicholas Sze
         
        23.
        The renewer in NamenodeWebHdfsMethods.generateDelegationToken(..) is not used Sub-task Closed Tsz Wo Nicholas Sze
         
        24.
        webhdfs throws a npe when ugi is null from getDelegationToken Sub-task Closed Tsz Wo Nicholas Sze
         
        25.
        Remove the use of Range header from webhdfs Sub-task Closed Tsz Wo Nicholas Sze
         
        26.
        webhdfs rest call to a secure dn fails when a token is sent Sub-task Closed Tsz Wo Nicholas Sze
         
        27.
        Support doAs and GETHOMEDIRECTORY in webhdfs Sub-task Closed Tsz Wo Nicholas Sze
         
        28.
        Change WebHdfsFileSystem to two-step create/append Sub-task Closed Tsz Wo Nicholas Sze
         

          Activity

          No work has yet been logged on this issue.

            People

            • Assignee:
              Tsz Wo Nicholas Sze
              Reporter:
              Tsz Wo Nicholas Sze
            • Votes:
              0 Vote for this issue
              Watchers:
              21 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development