[HADOOP-5010] Document HTTP/HTTPS methods to read directory and file data

• Type: Improvement
• Status: Resolved
• Priority: Trivial
• Resolution: Fixed
• Affects Version/s: 0.18.0
• Component/s: documentation
• Assignee: Unassigned
• Reporter: Marco Nicosia
• Created: 10/Jan/09 05:19
• Updated: 11/Aug/11 18:17
• Resolved: 11/Aug/11 18:17

Description

In HADOOP-1563, cutting wrote:

The URI for this should be something like hftp://host:port/a/b/c, since, while HTTP will be used as the transport, this will not be a FileSystem for arbitrary HTTP urls.

Recently, we've been talking about implementing an HDFS proxy (HADOOP-4575) which would be a secure way to make HFTP/HSFTP available. In so doing, we may even remove HFTP/HSFTP from being offered on the HDFS itself (that's another discussion).

In the case of the HDFS proxy, does it make sense to do away with the artificial HFTP/HSFTP protocols, and instead simply offer standard HTTP and HTTPS? That would allow non-HDFS-specific clients, as well as using various standard HTTP infrastructure, such as load balancers, etc.

NB, to the best of my knowledge, HFTP is only documented on the distcp page, and HSFTP is not documented at all?