Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-2316

[umbrella] WebHDFS: a complete FileSystem implementation for accessing HDFS over HTTP

    Details

    • Type: New Feature
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.23.1, 1.0.0
    • Component/s: webhdfs
    • Labels:
    • Release Note:
      Hide
      Provide WebHDFS as a complete FileSystem implementation for accessing HDFS over HTTP.
      Previous hftp feature was a read-only FileSystem and does not provide "write" accesses.
      Show
      Provide WebHDFS as a complete FileSystem implementation for accessing HDFS over HTTP. Previous hftp feature was a read-only FileSystem and does not provide "write" accesses.

      Description

      We current have hftp for accessing HDFS over HTTP. However, hftp is a read-only FileSystem and does not provide "write" accesses.

      In HDFS-2284, we propose to have WebHDFS for providing a complete FileSystem implementation for accessing HDFS over HTTP. The is the umbrella JIRA for the tasks.

        Attachments

        1. WebHdfsAPI20111111.pdf
          211 kB
          Tsz Wo Nicholas Sze
        2. WebHdfsAPI20111103.pdf
          183 kB
          Tsz Wo Nicholas Sze
        3. WebHdfsAPI20111020.pdf
          141 kB
          Tsz Wo Nicholas Sze
        4. test-webhdfs-0.20s
          0.5 kB
          Tsz Wo Nicholas Sze
        5. test-webhdfs
          0.4 kB
          Tsz Wo Nicholas Sze

          Issue Links

          1.
          Write Http access to HDFS Sub-task Closed Tsz Wo Nicholas Sze
          2.
          Read access to HDFS using HTTP REST Sub-task Closed Tsz Wo Nicholas Sze
          3.
          Provide authentication to webhdfs using SPNEGO Sub-task Closed Tsz Wo Nicholas Sze
          4.
          Configuration option to enable/disable webhdfs. Sub-task Closed Jitendra Nath Pandey
          5.
          Support getFileBlockLocations and getDelegationToken in webhdfs Sub-task Closed Tsz Wo Nicholas Sze
          6.
          Support getContentSummary and getFileChecksum in webhdfs Sub-task Closed Tsz Wo Nicholas Sze
          7.
          webhdfs: support case insensitive query parameter names Sub-task Closed Tsz Wo Nicholas Sze
          8.
          Support delegation token renewal in webhdfs Sub-task Closed Tsz Wo Nicholas Sze
          9.
          webhdfs api parameter validation should be better Sub-task Closed Tsz Wo Nicholas Sze
          10.
          webhdfs returns two content-type headers Sub-task Closed Tsz Wo Nicholas Sze
          11.
          webhdfs open an invalid path leads to a 500 which states a npe, we should return a 404 with appropriate error message Sub-task Closed Tsz Wo Nicholas Sze
          12.
          webhdfs liststatus json does not convert to a valid xml document Sub-task Closed Tsz Wo Nicholas Sze
          13.
          webhdfs setpermission,settimes,setowner api does not return any content upon success Sub-task Resolved Unassigned
          14.
          webhdfs GETFILESTATUS calls shows the localName as empty Sub-task Resolved Unassigned
          15.
          tail using a webhdfs uri throws an error Sub-task Closed Tsz Wo Nicholas Sze
          16.
          webhdfs mkdirs api call creates path with 777 permission, we should default it to 755 Sub-task Closed Tsz Wo Nicholas Sze
          17.
          distcp with a webhdfs uri on a secure cluster fails Sub-task Closed Jitendra Nath Pandey
          18.
          [webhdfs] When Getting the file using OP=OPEN with DN http address, ESTABLISHED sockets are growing. Sub-task Closed Uma Maheswara Rao G
          19.
          add version prefix and root methods to webhdfs Sub-task Closed Tsz Wo Nicholas Sze
          20.
          webhdfs api's should return a root element in the json response Sub-task Closed Tsz Wo Nicholas Sze
          21.
          webhdfs liststatus json response is not correct Sub-task Closed Suresh Srinivas
          22.
          webhdfs setreplication api should return a 403 when called on a directory Sub-task Closed Tsz Wo Nicholas Sze
          23.
          The renewer in NamenodeWebHdfsMethods.generateDelegationToken(..) is not used Sub-task Closed Tsz Wo Nicholas Sze
          24.
          webhdfs throws a npe when ugi is null from getDelegationToken Sub-task Closed Tsz Wo Nicholas Sze
          25.
          Remove the use of Range header from webhdfs Sub-task Closed Tsz Wo Nicholas Sze
          26.
          webhdfs rest call to a secure dn fails when a token is sent Sub-task Closed Tsz Wo Nicholas Sze
          27.
          Support doAs and GETHOMEDIRECTORY in webhdfs Sub-task Closed Tsz Wo Nicholas Sze
          28.
          Change WebHdfsFileSystem to two-step create/append Sub-task Closed Tsz Wo Nicholas Sze

            Activity

              People

              • Assignee:
                szetszwo Tsz Wo Nicholas Sze
                Reporter:
                szetszwo Tsz Wo Nicholas Sze
              • Votes:
                0 Vote for this issue
                Watchers:
                22 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: