Description
The current Data Transfer Protocol (DTP) implements a rich set of features that span across multiple layers, including:
- Connection pooling and authentication (session layer)
- Encryption (presentation layer)
- Data writing pipeline (application layer)
All these features are HDFS-specific and defined by implementation. As a result it requires non-trivial amount of work to implement HDFS clients and servers.
This jira explores to delegate the responsibilities of the session and presentation layers to the HTTP/2 protocol. Particularly, HTTP/2 handles connection multiplexing, QoS, authentication and encryption, reducing the scope of DTP to the application layer only. By leveraging the existing HTTP/2 library, it should simplify the implementation of both HDFS clients and servers.
Attachments
Attachments
Issue Links
- supercedes
-
HDFS-5270 Use thread pools in the datenode daemons
- Patch Available
1.
|
Abstract an application layer in DataNode WebHdfs implementation | Resolved | Duo Zhang | |
2.
|
Support HTTP/2 in datanode | Resolved | Duo Zhang | |
3.
|
Implement HTTP/2 stream channels | Resolved | Duo Zhang | |
4.
|
Add read block support for DataNode HTTP/2 server | Patch Available | Duo Zhang | |
5.
|
Implement a new block reader that reads data over HTTP/2 | Open | Unassigned | |
6.
|
Add client support for HTTP/2 stream channels | Resolved | Duo Zhang | |
7.
|
Implement flow control | Open | Unassigned |