Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-2178

HttpFS - a read/write Hadoop file system proxy


    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.23.0
    • Fix Version/s: 0.23.1
    • Component/s: None
    • Labels:
    • Hadoop Flags:


      We'd like to contribute Hoop to Hadoop HDFS as a replacement (an improvement) for HDFS Proxy.

      Hoop provides access to all Hadoop Distributed File System (HDFS) operations (read and write) over HTTP/S.

      The Hoop server component is a REST HTTP gateway to HDFS supporting all file system operations. It can be accessed using standard HTTP tools (i.e. curl and wget), HTTP libraries from different programing languages (i.e. Perl, Java Script) as well as using the Hoop client. The Hoop server component is a standard Java web-application and it has been implemented using Jersey (JAX-RS).

      The Hoop client component is an implementation of Hadoop FileSystem client that allows using the familiar Hadoop filesystem API to access HDFS data through a Hoop server.

      Repo: https://github.com/cloudera/hoop

      Docs: http://cloudera.github.com/hoop

      Blog: http://www.cloudera.com/blog/2011/07/hoop-hadoop-hdfs-over-http/

      Hoop is a Maven based project that depends on Hadoop HDFS and Alfredo (for Kerberos HTTP SPNEGO authentication).

      To make the integration easy, HDFS Mavenization (HDFS-2096) would have to be done first, as well as the Alfredo contribution (HADOOP-7119).

      1. HDFS-2178.patch
        554 kB
        Alejandro Abdelnur
      2. HDFS-2178.patch
        555 kB
        Alejandro Abdelnur
      3. HDFS-2178.patch
        558 kB
        Alejandro Abdelnur
      4. HDFS-2178.patch
        608 kB
        Alejandro Abdelnur
      5. HDFS-2178.patch
        608 kB
        Alejandro Abdelnur
      6. HDFS-2178.patch
        627 kB
        Alejandro Abdelnur
      7. HDFS-2178.patch
        626 kB
        Alejandro Abdelnur
      8. HDFS-2178.patch
        609 kB
        Alejandro Abdelnur
      9. HDFSoverHTTP-API.html
        15 kB
        Alejandro Abdelnur
      10. HdfsHttpAPI.pdf
        316 kB
        Alejandro Abdelnur

        Issue Links



            • Assignee:
              Alejandro Abdelnur
              Alejandro Abdelnur
            • Votes:
              1 Vote for this issue
              22 Start watching this issue


              • Created: